A Hands-On Guide for Working with Large-Scale Spatial Data. Learn more.
Authors
Editor’s note: Raster inference is now part of RasterFlow, the Wherobots engine for running computer vision models on satellite and aerial imagery.
Key takeaways
We are excited to announce that Wherobots Raster Inference is now generally available! Raster Inference is a serverless, planetary-scale computer vision solution that enables data teams to extract meaningful insights from aerial imagery (raster data) sources, such as satellites or drones, and puts these insights at the fingertips of data scientists and developers.
During the preview period, customers analyzed raster data by running inference with a limited set of Wherobots-hosted open-source computer vision models. Now, you can bring your own model into Raster Inference, offload inference pipeline management effort, and apply this capability to a much broader set of use cases. We have also made significant enhancements to our compute service to accelerate inference performance.
Data teams use Wherobots Raster Inference to identify information in complex, large-scale overhead imagery data. Some common use cases include:
We’ve met with many businesses struggling to get critical insights from raster data. Some are manually sifting through imagery. This method doesn’t scale, it’s expensive, error prone, and time intensive. Others are utilizing complex computer vision solutions that are not designed for overhead raster imagery. These computer visions solutions:
With Wherobots Raster Inference, you can:
In the following sections, we’ll provide a brief overview of Wherobots-hosted models, how to bring your own model, recent performance improvements, and how to use this feature effectively.
Wherobots-hosted models are precompiled and optimized for raster inference, enabling them to scale effortlessly and execute inference on large datasets. Below is a brief overview of the initial set of Wherobots-hosted models. We plan to host additional models based on customer feedback. For more information on these models and their performance metrics, see our docs page here.
Below is a code snippet on how to use landcover-eurosat-sentinel2 in Wherobots Raster Inference:
# set the model name model_id = "landcover-eurosat-sentinel2" # call the model in raster inference df_predictions = df_raster_input.withColumn("preds", rs_classify(model_id, "outdb_raster"))
The model import function of Raster Inference is built on the STAC Machine Learning Model (MLM) Extension Specification, an open community standard for model sharing we co-developed with CRIM and other collaborators. The MLM specification enables model portability, making it easier to use models across teams and compute platforms.
Before the MLM specification, data scientists and modelers sharing geospatial computer vision models often had to adapt existing standards, such as HuggingFace model cards, to store relevant information. This involved repurposing the cards to specify details like required raster bands, necessary data preprocessing, and post-processing functions. Without a community standard for organizing this information, these model cards were often inconsistent and documented in varying formats, making model sharing and reproducibility across different compute platforms cumbersome.
The STAC MLM extension introduces a community standard designed to simplify storing and sharing geospatial computer vision models. It achieves this by providing a comprehensive schema to:
The MLM specification has already been adopted in key modeling efforts at Terradue and is proudly supported by Radiant Earth. We’ll share more in a series of blog posts and a panel discussion with our collaborators on the MLM in late January — register here on the interest form to receive an invite when the date is finalized.
The MLM specification enables users to quickly and easily use many geospatial, deep learning-based computer vision models with Raster Inference. We leverage a model’s MLM specification to specify and integrate the required data preprocessing, model, and post processing into the larger raster inference pipeline.
To bring your own model to Raster Inference:
You can find a full walkthrough in our documentation on using the MLM form to bring your own model.
In addition to enabling bring your own model, we’ve accelerated asynchronous data loading in the inference engine to boost performance. Below, you can see how performance has evolved with experiments conducted using Wherobots Raster Inference and a Tiny GPU Runtime.
For a full tutorial example on how to bring your own model to segment solar farms in Sentinel-2 imagery, see our documentation here.
We’ll walk through an example on how to identify solar infrastructure in raster imagery using a Wherobots-hosted model. In our example, we will be using the model:
solar-satlas-sentinel2
The full Python notebook file can be found on our GitHub.
import warnings warnings.filterwarnings('ignore') from wherobots.inference.data.io import read_raster_table from sedona.spark import SedonaContext from pyspark.sql.functions import expr from wherobots.inference.engine.register import create_semantic_segmentation_udfs from pyspark.sql.functions import col config = SedonaContext.builder().appName('segmentation-batch-inference') .getOrCreate() sedona = SedonaContext.create(config)
tif_folder_path = "s3a://wherobots-benchmark-prod/data/ml/satlas" files_df = read_raster_table(tif_folder_path, sedona, limit=400) df_raster_input = files_df.withColumn( "outdb_raster", expr("RS_FromPath(path)") ) df_raster_input.cache().count() df_raster_input.show(truncate=False) df_raster_input.createOrReplaceTempView("df_raster_input")
Specify a Wherobots-hosted model to run inference
model_id = "solar-satlas-sentinel2"
You can run Wherobots Raster Inference using either the Wherobot’s SQL API or Python API.
Using the SQL API
predictions_df = sedona.sql(""" SELECT outdb_raster, segment_result.* FROM ( SELECT outdb_raster, RS_SEGMENT('{model_id}', outdb_raster) AS segment_result FROM df_raster_input ) AS segment_fields """) predictions_df.cache().count() predictions_df.show() predictions_df.createOrReplaceTempView("predictions")
Using the Python API
rs_segment = create_semantic_segmentation_udfs(batch_size = 10, sedona=sedona) df = df_raster_input.withColumn("segment_result", rs_segment(model_id, col("outdb_raster"))).select( "outdb_raster", col("segment_result.confidence_array").alias("confidence_array"), col("segment_result.class_map").alias("class_map") ) df.show(3)
df_multipolys = sedona.sql(""" WITH t AS ( SELECT RS_SEGMENT_TO_GEOMS(outdb_raster, confidence_array, array(1), class_map, 0.65) result FROM predictions ) SELECT result.* FROM t """) df_multipolys.cache().count() df_multipolys.show() df_multipolys.createOrReplaceTempView("multipolygon_predictions") df_merged_predictions = sedona.sql(""" SELECT element_at(class_name, 1) AS class_name, cast(element_at(average_pixel_confidence_score, 1) AS double) AS average_pixel_confidence_score, ST_Collect(geometry) AS merged_geom FROM multipolygon_predictions """) df_filtered_predictions = df_merged_predictions.filter("ST_IsEmpty(merged_geom) = False") df_filtered_predictions.cache().count() df_filtered_predictions.show()
from sedona.maps.SedonaKepler import SedonaKepler config = { 'version': 'v1', 'config': { 'mapStyle': { 'styleType': 'dark', 'topLayerGroups': {}, 'visibleLayerGroups': {}, 'mapStyles': {} }, } } map = SedonaKepler.create_map(config=config) SedonaKepler.add_df(map, df=df_filtered_predictions, name="Solar Farm Detections") map
If you’re a Wherobots Professional Edition or Enterprise user, you have access to all capabilities of Wherobots Raster Inference! If you have access to GPU runtimes, sign in to your account now to launch a Wherobots Notebook and explore the feature. If you don’t yet have access, request it today and start using Raster Inference as soon as tomorrow. Explore how easy it is to bring your own model using our guided example on GitHub or within a Wherobots notebook instance. Start integrating Wherobots Raster Inference into your workflow today!
Although Raster Inference is not available in Wherobots Community Edition, we are currently offering a free trial for the Wherobots Professional Edition. You can either sign up through AWS Marketplace or upgrade your account to get started for free and integrate Raster Inference into your workflow today.
We’re eager to hear about the models you’d like us to support and any features you’d like to see added. For product feedback, feel free to email us at feedback@wherobots.com (no request is too small). To see how others are using Raster Inference, ask questions, and share your own experiences, join the Wherobots community. We look forward to your ideas and creations!
Missed our panel discussion with our collaborators CRIM, Terradue and Radiant Earth on the MLM STAC extension? Watch the recording below.
It is a serverless computer vision capability for running inference on aerial and satellite raster imagery at planetary scale, using Wherobots-hosted models or a model you import.
Yes. Fill out the MLM form to produce a JSON description of the model, upload that file to S3, and pass the S3 URI into the raster inference function at runtime. The pipeline uses the MLM spec for preprocessing, the model artifact, and post-processing.
Professional Edition and Enterprise users have access, including GPU runtimes. It is not available in Community Edition; the post points Community users to a Professional Edition free trial.
The walkthrough covers classification (rs_classify / landcover-eurosat-sentinel2) and semantic segmentation (RS_SEGMENT / solar-satlas-sentinel2), then vectorizes predictions with RS_SEGMENT_TO_GEOMS for mapping in SedonaKepler.
RasterFlow is now available in Public Preview
RasterFlow makes planetary-scale earth intelligence workflows easy and costs predictable. We are excited to announce that RasterFlow is now in Public Preview, opening up the power of planetary scale Earth Intelligence to all Wherobots Professional Edition customers! RasterFlow let’s you solve complex monitoring challenges with vision-language models or tailored models for specific use cases, without […]
Providence, Hartford and Chicago: Best Cities for Halloween
Best cities for Halloween 2026: we mapped haunted houses, Spirit Halloween stores, psychics, graveyards and sunset in 55 big metros. Providence narrowly won.
Most Dangerous Roads for Pedestrians and Cyclists in the US (Part 2)
The Wall Street Journal mapped 1.5 miles of Union Avenue in Memphis. We ran the same test on every U.S. arterial. Westheimer Road in Houston ranks first.
Most Dangerous Highway in the US: 25 Deadliest Stretches
The most dangerous highway in the US is 10 miles of U.S. 19 in Pasco County, Florida. See the 25 deadliest stretches and the worst stretch in each state.
share this article
Awesome that you’d like to share our articles. Where would you like to share it to: