Rasterflow, Earth Intelligence & inference engine now in public preview Learn More

SedonaSnow: Apache Sedona On Snowflake, Accelerating Your GIS Pipeline, Exploring Global Fishing Watch Data With GeoParquet, and Apache Sedona 1.5.1 Release

Welcome to This Month In Wherobots the monthly developer newsletter for the Wherobots & Apache Sedona community! In this edition we cover SedonaSnow: Apache Sedona on Snowflake, accelerating your GIS pipeline with Apache Sedona, exploring Global Fishing Watch public data with SedonaDB and GeoParquet, and a look at new features and updates in the 1.5.1 release of Apache Sedona.

Apache Sedona Now Available In Snowflake Marketplace: SedonaSnow

SedonaSnow

Apache Sedona, the scalable open-source geospatial compute engine is now available on Snowflake via the Snowflake Marketplace or via manual installation. The SedonaSnow plugin brings Apache Sedona’s Spatial SQL functionality to Snowflake via 130+ Sedona SQL “ST” SQL functions that can be used alongside Snowflake SQL.

Read More About Using SedonaSnow In Snowflake In This Tutorial

This month’s Wherobots & Apache Sedona featured community members are Alihan Zihna, Lead Data Scientist at CKDelta and Fernando Palacios, Director of Data Science & Data Engineering also at CKDelta. Alihan and Fernando presented “GIS Pipeline Acceleration With Apache Sedona” at the Data + AI Summit where they share how they were able to improve the performance and innovation of their geospatial analysis pipelines, going from a pipeline that took 48 hours to complete down to 10 minutes using Apache Sedona. Thank you Fernando and Alihan for being a part of the community and sharing your work!

GIS Pipeline Acceleration With Apache Sedona

GIS pipeline accelerations with Apache Sedona

In this presentation from Data + AI Summit, Fernando and Alihan discuss some of the various usecases for working with large-scale geospatial data at conglomerate CKDelta, part of the Hutchinson Group which operates ports, utility networks, retail stores and mobile telecom networks with hundreds of millions of users across dozens of countries. They discuss how geospatial analytics at scale is important for identifying water leakage in their utility network, understanding customer satisfaction, identifying sites for electric vehicle charging station installation, and forecasting the supply and demand of energy. They provide a technical overview of Apache Sedona and share the results of improving and extending their geospatial analytics pipelines including one process that reduced running time from 48 hours to 10 minutes using Apache Sedona.

Watch the recording of “GIS Pipeline Acceleration With Apache Sedona”

The Wherobots Notebook Environment – Getting Started With Wherobots Cloud & SedonaDB Part 2

Wherobots Initial Notebook

In Part 2 of our Getting Started With Wherobots Cloud & SedonaDB series we dive into the Wherobots Notebook Environment including how to configure and start notebook runtimes, an overview of the sample notebooks included in Wherobots Cloud, and how to use version control like git with notebooks. If you missed it check out Part 1: An Overview of Wherobots Cloud or sign up for a free Wherobots Cloud account to get started directly.

Read More About The Wherobots Notebooks Environment

Exploring Global Fishing Watch Public Data With SedonaDB & GeoParquet

Matched vs unmatched vessels

This post is a hands-on look at offshore ocean infrastructure and industrial vessel activity with SedonaDB using data from Global Fishing Watch. We also see how GeoParquet can be used with this data to improve the efficiency of data retrieval and enable large-scale geospatial visualization using GeoArrow and the Lonboard Python visualization library.

Read “Exploring Global Fishing Watch Public Data With SedonaDB & GeoParquet”

Apache Sedona 1.5.1 Release

Apache Sedona 1.5.1 Release Notes

The most recent release of Apache Sedona introduces some exciting new updates including support for Spark 3.5, 20+ new raster functions, 7 new vector functions, support for running Sedona in Snowflake with SedonaSnow, updates to Sedona’s GeoParquet reader and writer, and more! The updated raster functions include RS_ZonalStats for computing zonal statistics, RS_Tile and RS_TileExplode to enable tiling large rasters, and updates to RS_MapAlgebra to enable user defined raster functions that can work across multiple rasters. Updated vector functions include ST_IsValidReason which exposes the reason geometries might not be valid, and ST_LineLocatePoint which can be useful for map matching and snapping data to road networks.

Read More About Apache Sedona 1.5.1 In The Release Notes

Hands-On With Havasu and GeoParquet

GeoParquet and Iceberg Havasu

Each month you can find a new livestream tutorial on the Wherobots YouTube channel. January’s livestream was all about working with GeoParquet and Havasu tables in SedonaDB. We dig in to understanding some of the optimizations built into the Apache Parquet format to learn how Parquet delivers efficient data storage and data retrieval before exploring the GeoParquet specification for storing geospatial data in Parquet. We cover loading, analyzing, and creating GeoParquet files using SedonaDB with a focus on comparing performance of various GeoParquet partitioning strategies. Finally, we see how the Havasu extension to the Apache Iceberg table format enables working with both vector and raster geospatial data backed by GeoParquet but with the familiar developer experience of SQL tables.

Watch The Recording: Hands-On With Havasu And GeoParquet

Upcoming Events

  • Apache Sedona Community Office Hour (Online Zoom Call – February 13, 2024) – Join the Apache Sedona community for updates on the state of Apache Sedona, presentation and demo of recent features, and provide your input into the roadmap, future plans, and contribution opportunities.
  • Raster Data Analysis With Spatial SQL & SedonaDB (Online Livestream – February 29th, 2024) – This month our livestream is focused on raster data analysis. We’ll see how to load raster data in SedonaDB and perform raster operations like map algebra and zonal statistics using Spatial SQL. Be sure to subscribe to the Wherobots YouTube channel to keep up to date with more Wherobots livestreams and videos!

Want to receive this monthly update in your inbox? Sign up for the Spatial Intelligence Newsletter:

Key takeaways

  • This February 2024 “This Month In Wherobots” newsletter recaps SedonaSnow, a GIS pipeline talk, a Global Fishing Watch tutorial, notebooks part 2, and Apache Sedona 1.5.1.
  • Apache Sedona is available on the Snowflake Marketplace as SedonaSnow, exposing 130+ Sedona ST_ SQL functions alongside Snowflake SQL.
  • Featured CKDelta speakers Alihan Zihna and Fernando Ayuso Palacios describe a geospatial pipeline that went from 48 hours to 10 minutes with Apache Sedona (Data + AI Summit).
  • Sedona 1.5.1 notes in this issue: Spark 3.5 support, 20+ new raster functions, 7 new vector functions, SedonaSnow, GeoParquet reader/writer updates, RS_ZonalStats, RS_Tile/RS_TileExplode, RS_MapAlgebra across rasters, ST_IsValidReason, and ST_LineLocatePoint.
  • Upcoming: Sedona office hour February 13 and a raster Spatial SQL livestream on February 29.

Frequently Asked Questions

What is SedonaSnow?

Apache Sedona packaged for Snowflake (Marketplace or manual install), providing 130+ Spatial SQL ST_ functions inside Snowflake.

What pipeline speedup does CKDelta report in this issue?

A geospatial analysis pipeline that took 48 hours was reduced to 10 minutes using Apache Sedona, as presented at Data + AI Summit.

What is new in Apache Sedona 1.5.1 according to the newsletter?

Spark 3.5, 20+ raster functions, 7 vector functions, Snowflake/SedonaSnow, GeoParquet updates, RS_ZonalStats, raster tiling, multi-raster map algebra, ST_IsValidReason, and ST_LineLocatePoint.

Is this a standalone product announcement?

No. It is a monthly community newsletter that links to tutorials (Global Fishing Watch, notebooks part 2, Havasu/GeoParquet livestream) and the 1.5.1 release notes.

RELATED POSTS
VIEW MORE

RasterFlow is now available in Public Preview

RasterFlow makes planetary-scale earth intelligence workflows easy and costs predictable. We are excited to announce that RasterFlow is now in Public Preview, opening up the power of planetary scale Earth Intelligence to all Wherobots Professional Edition customers! RasterFlow let’s you solve complex monitoring challenges with vision-language models or tailored models for specific use cases, without […]

Wherobots for QGIS: Cloud-Scale Spatial Data, from Wherobots Labs

Wherobots for QGIS is a new plugin that connects QGIS to Wherobots Cloud. From a panel inside QGIS, an analyst can run spatial SQL against WherobotsDB, load the results as a map layer, push a local layer up to a Wherobots Iceberg table, and pull raster data into the canvas.  Why Wherobots for QGIS: Eliminating […]

How Wherobots builds with NVIDIA to let AI see the physical world

The world and what happens in it is digitized by petabytes of raw and derivative spatial datasets of various data types and scales, and the potential for applying AI to it is immense. But the AI models and agents we use every day need connectivity to tools that turn this data into usable insights and relationships. Wherobots gives AI the ability operate on and understand raw physical-world data, and NVIDIA GPUs are core to it. Architecturally here’s how this works at a high level.

El Niño 2026, atmospheric rivers, and California’s burn scars: one SQL engine, two data models

7,713 mapped building footprints sit inside or within 500 m (1,640 ft) of the 42 Eaton basins USGS rates high hazard for debris flows, and 388 km (241 mi) of mapped road and path cross the low ground below the scar. WherobotsDB finds both in SQL: a join to the USGS basins for the buildings, and a raster vector join over elevation for the roads.