How Wherobots builds with NVIDIA to let AI see the physical world

AI was blind to the physical world, but Wherobots is giving AI the ability to understand it using data, compute, and NVIDIA GPUs

The world and what happens in it is digitized by petabytes of raw and derivative spatial datasets of various data types and scales, and the potential for applying AI to it is immense. But the AI models and agents we use every day need connectivity to tools that turn this data into usable insights and relationships.

Wherobots gives AI the ability operate on and understand raw physical-world data, and NVIDIA GPUs are core to it. Architecturally here’s how this works at a high level. Wherobots:

1. Integrates with your open data architecture: by utilizing data lakes like Amazon S3 and Iceberg catalogs such as Databricks Unity Catalog and AWS Glue Data Catalog as the center of truth, you can integrate Wherobots into your existing data architecture.
2. Brings the spatial capability: data teams and their agents have the capability to solve the most challenging physical world problems by directing Wherobots to extract insights from multi-dimensional, raster, vector, and tabular datasets of various formats and scales.
3. Is plug-and-play: in minutes, you can enable AI coding tools like VS Code, Claude Code, and Cursor to direct Wherobots operations on this data and build innovations with AI.

Under the hood, we deploy NVIDIA GPUs and libraries on tasks they are optimized for.

• NVIDIA GPUs run at the core of RasterFlow, which is the preprocessing and inference solution designed to make insight extraction from satellite and aerial sensor datasets easy. We also leverage CUDA streams to optimize performance. RasterFlow crunches pixels with a distributed fleet of NVIDIA G7e AWS instances, accelerated by RTX PRO 6000 Blackwell GPUs.
• SedonaDB, which WherobotsDB utilizes in a distributed mode, can run spatial joins directly on the ray tracing cores inside NVIDIA GPUs to deliver the best price-performance for a class of spatial join types.

Lets dig in.

RasterFlow on NVIDIA

RasterFlow prepares large scale imagery data for inference and runs computer vision models on a distributed inference architectureIts goal is to give customers the ability to generate insights from this data across physical areas of interest of any scale, enabling real-world applications such as:

• Last-Mile Delivery Routing: Identifies sidewalks (using models like Tile2Net) which can help determine viable navigational routes for both manual and autonomous delivery systems.
• Property Risk Analytics: Can aid in quantifying the number of properties facing flood risk across coastal cities by embedding FEMA flood zone designations into parcel-level data, helping insurers, reinsurers, and real estate developers evaluate policy and property portfolios.

Since its introduction in private preview, we continue to optimize the RasterFlow architecture, reaching a point where mosaicking and inference tasks cost 2–3x less than the next best option at scale. For instance, RasterFlow supports Meta’s Segment Anything Model (SAM3) for general-purpose object detection on Earth Observation imagery. With the efficiencies we’ve achieved, you can use SAM3 to detect objects from high-resolution 30cm imagery for less than $25 per 1,000 square kilometers.

RasterFlow uses CUDA streams to keep GPUs busy

Data movement can kill inference performance, so we use NVIDIA CUDA streams and GPUs to address this bottleneck. Every batch of satellite imagery patches travels from host memory to the GPU before the model can run, and every batch of prediction patches travels back before RasterFlow can merge it into a seamless output mosaic. If we ran these steps serially one after another, our GPUs would idle on PCIe transfers with every forward pass.

CUDA streams remove that idle time. A stream is an ordered queue of GPU work, where different kinds of work in different streams can execute concurrently. NVIDIA GPUs include dedicated copy engines that move data across PCIe independently of the compute cores, which means a transfer in one stream proceeds while a model forward pass runs in another. RasterFlow drives this through PyTorch’s torch.cuda.Stream API.

This inference loop has separate upload and download streams feeding two reusable buffer slots. While the model runs batch N on the first stream, a second stream copies the next batch to the GPU, and a download stream drains the predictions from batch N-1 back to host memory, where the CPU merges patches in the mosaic accumulator. At steady state three batches are in flight at once: one uploading, one computing, one landing in the output. RasterFlow allocates these buffers once and reuses them for every batch. CUDA events sequence the handoff between streams, so data movement never starts before the forward pass that produced it finishes. This allows the GPU to stay busy from the first patch to the last.

What’s next: GPU-native decompression with nvCOMP and Zarr

RasterFlow stores every analysis-ready mosaic as a Zarr store of compressed chunks on Amazon S3, and today those chunks decode on the CPU.

nvCOMP, NVIDIA’s library of GPU-accelerated compression codecs, can move that decode step onto the GPU itself, and we are looking forward to nvCOMP-backed codec support landing in Zarr. The payoff of this feature is three-fold. First, compressed chunks cross PCIe instead of raw pixels, so each transfer carries fewer bytes. Second, decompression runs at GPU memory bandwidth (which is higher than CPU) and the decoded array materializes directly in GPU memory where the model needs it. Finally, CPU cores are free for the work only they can do: assembling patches and merging predictions into the mosaic.

RasterFlow use cases

RasterFlow continues to unblock customers who need to extract insights from global-scale imagery datasets. We are actively working with customers across multiple industries, and utilizing RasterFlow to deliver results like these:

1. The Fields of the World global run produced globally consistent agricultural field-boundary data for land-use monitoring, food-system analysis, and model development. We executed this run using RasterFlow, and led it in partnership with Taylor Geospatial, Microsoft AI for Good Lab, and Nasa Harvest. With RasterFlow you can run this same model with 10m Sentinel-2 imagery, including cloud-free mosaic generation and model inference, for less than $1 per 10,000 square kilometers.

2. We are actively supporting the US Forest Service in their research, and our joint goal is to bring their new FireCon model online in the second half of this calendar year. This initiative is key to delivering far more accurate daily wildfire containment suitability mapping system for active wildfires of the Western US, enabling better planning, management, and response to wildfires.

SedonaDB on NVIDIA

WherobotsDB runs a distributed version of SedonaDB, which customers use to efficiently process and join two dimensional raster and vector data. In SedonaDB 0.4, we shipped a GPU-accelerated spatial join as an extension that runs on NVIDIA GPU ray tracing cores. In practice, almost all spatial analyses are powered by one or more spatial joins, which are computationally intensive and frequently a bottleneck in spatial queries. But SedonaDB with GPU acceleration provides relief, and delivers up to a 5.93x speedup and a 59.02% cost reduction on our standard spatial benchmark. The research behind it, RayBooster: A Ray Tracing Engine to Accelerate SedonaDB, was accepted to VLDB 2026 in the Industry Track, and we developed it with The Ohio State University.

Attempts to accelerate traditional analytical database workflows using GPUs have typically required utmost care and/or rethinking of a workflow to minimize data transfer between the existing workflow (on the CPU) and the accelerated implementation (on the GPU). The spatial join is an ideal candidate for this type of acceleration because it is computationally intensive, and relatively small amounts of data transfer can result in large numbers of computations. Furthermore, the required data transfer is only a single direction; geometries must be transferred to the GPU but need not be transferred back.

Another limitation to GPU acceleration of traditional database workflows has been a requirement for a potentially large amount of GPU memory to avoid completely rewriting a workflow with careful attention to the sizes of the inputs. SedonaDB’s join implementation was built to partition large joins into smaller ones that can be run with limited memory with its CPU-based join, and while this heuristic needs to be tuned differently for GPUs, we did not need to rewrite our GPU implementation to support arbitrarily large joins. This extends the acceleration we observed to a wider range of GPUs that customers may have available.

Deploying AI on physical world data with NVIDIA

High complexity, scale, and costs have limited the utilization of spatial data, resulting in a significant gap between AI and physical world data. This gap is closing fast and as it closes, humans are more capable of directing AI towards solutions that drive top, bottom, and the sustainability lines of their business.

Through our investments in Apache Sedona and Wherobots, and by building on NVIDIA technology, data teams and their agents are becoming increasingly capable of innovating and operating solutions with physical world data, under budget, with the talent and AI tools they already have, regardless of spatial data type and degree of complexity.

Get Started with Wherobots

Try Now