What is Satellite Imagery Analysis? AI, Tasks, Uses

Satellite imagery analysis is the process of extracting information from images of the Earth's surface captured by satellites and aircraft. It turns pixels into answers: which land is forest, where every building and field sits, how tall the trees are, and what changed since last year. The AI in satellite imagery analysis described here is computer vision: models that classify, detect, segment, and measure what the pixels show.

Key takeaways

• Satellite imagery analysis turns pixels into maps, object counts, measurements, and change alerts.
• Five computer vision tasks cover the examples here: classification, object detection, segmentation, regression, and change detection.
• Resolution sets what is visible. Sentinel-2 at 10 m maps fields, and 30 cm aerial imagery maps roofs and sidewalks.
• At scale, the hard part is the pipeline around the model: mosaics, cloud masking, patching, stitching, and vectorization.
• In Marion County, Oregon, one run of SAM3, a segmentation model, on Wherobots RasterFlow returned 1,006,232 detections from eight text prompts.

What satellite imagery analysis is

Methods range from an analyst tracing a shoreline by hand to a neural network labeling every pixel on a continent. They developed in this order:

• Visual interpretation. A trained analyst reads tone, texture, shape, size, shadow, and context.
• Pixel and spectral analysis. Band math and classifiers work on the pixel values of Landsat and Sentinel-2, and band ratios such as NDVI (Normalized Difference Vegetation Index) measure vegetation.
• Object-based image analysis. Pixels are grouped into segments first, then the segments are classified by shape and context.
• Deep learning. Neural networks are trained on labeled examples of roofs, sidewalks, or field boundaries and then label every new image.

The input is always a raster: a grid of pixels with one or more spectral bands. It is stored as a GeoTIFF, a Cloud Optimized GeoTIFF, or a cloud-native array format such as Zarr. The Cloud Optimized GeoTIFF has been a standard of the Open Geospatial Consortium (OGC) since 2023, and it lets a reader fetch only the tiles it needs over HTTP. The output is often vector data: polygons and points that join to parcels, roads, and other tables. Remote sensing satellite imagery analysis covers optical, radar, and thermal sensors, and the same methods apply to aerial photographs.

How satellite imagery analysis works

The model is one step in a longer pipeline, and the output is usable only when the surrounding steps work.

1. Select imagery. Pick the sensor, date range, and area of interest, then filter scenes by cloud cover.
2. Build a mosaic. Combine many scenes into one georeferenced raster on a common grid. The result shows no visible joins between scenes. Georeferenced means tied to map coordinates.
3. Patch. Split the mosaic into fixed-size patches that fit in the memory of a graphics processing unit (GPU), with overlap so objects on a patch edge are seen whole.
4. Run inference. Apply the model to every patch, often thousands to millions of them in parallel.
5. Merge. Stitch predictions back into one raster, blending the overlaps to remove seams.
6. Vectorize. Threshold the predictions and convert pixels to polygons, or keep the model's polygons directly.
7. Join and validate. Join the polygons to other tables and compare a sample against reference data.

Steps 2, 3, 5, and 6 grow with the area. Mosaicking a county is manageable; mosaicking a continent means managing tens of thousands of scenes, projections, and seams.

Validation borrows its measures from computer vision. Intersection over union (IoU) divides the area two shapes share by the area they cover together, so 1.0 is a perfect match. The PASCAL VOC challenge (Everingham et al., International Journal of Computer Vision, 2010) counted a detection as correct when its IoU with the reference exceeded 0.5. VOC stands for Visual Object Classes. That 0.5 threshold is now the usual cutoff. Precision is the share of detections that are real, and recall is the share of real objects found. Classified maps use an error matrix, covered under measuring accuracy below.

Computer vision tasks in satellite imagery analysis

The examples in this article fall into classification, object detection, semantic segmentation, regression, and change detection.

Task	Question it answers	Output	Example
Classification	What is this pixel or tile?	A class per pixel or per tile	Land cover classification
Object detection	Where is each object?	Boxes or outlines with confidence scores	Roofs, solar panels, ships, aircraft
Semantic segmentation	Which pixels belong to each class?	A class mask per pixel	Roads, sidewalks, water
Regression	How much, how tall, how dense?	A continuous value per pixel	Canopy height in meters
Change detection	What changed between dates?	A change mask or alert	Clearing, construction, damage

Instance segmentation sits between detection and segmentation: it returns a separate mask for every object, so two adjacent roofs stay two features. Change detection compares two or more dates of the same place, with each date run through the same classification or segmentation step. Foundation models, models pretrained on large general datasets, now cover several of these tasks at once. One promptable model can detect roofs, roads, and solar panels from text prompts with no task-specific training, as in the Wherobots example below.

Resolution of each satellite sensor

Ground sample distance, the ground width of one pixel, sets the smallest object a model can find. A rule of thumb is several pixels across the smallest object of interest.

Source	Resolution	Revisit	Good for
Landsat	30 m	8 days with two satellites	Long-term change since 1972, land cover
Sentinel-2	10 to 60 m	About 5 days	Fields, forests, water, seasonal change
Sentinel-1 SAR	About 10 m	6 days with two satellites	Floods, ships, any weather
USDA NAIP	30 cm to 1 m	About every 2 years per state	Roofs, pools, sidewalks, trees
Commercial optical	Under 1 m	Varies by provider	Vehicles, construction, defense

Landsat 1 launched on July 23, 1972, which makes Landsat the longest continuous record. The Sentinel-2 mission paper (Drusch et al., 2012) describes the 13-band design. Spectral depth matters as much as pixel size: near-infrared separates vegetation from everything else, which is why NAIP's fourth band and Sentinel-2's red-edge bands carry information the eye cannot see. Hyperspectral imagery extends this to hundreds of narrow bands for mineral and crop stress mapping.

Satellite imagery analysis examples

Each of these well-known projects pairs one task with one sensor:

• Global forest loss. Hansen and colleagues mapped forest loss and gain for every 30 m Landsat pixel (Science, 2013) by classifying Landsat time series. They found 2.3 million km² of forest lost from 2000 to 2012. Global Forest Watch now combines optical and radar systems into integrated deforestation alerts.
• Disaster damage. The Open Data Program started by Maxar, now run under the Vantor name, releases before and after imagery of major disasters. Models trained on the xBD benchmark, a labeled set of building damage images, score the damage to each building.
• Agriculture. Fields of the World (Kerner et al., AAAI 2025) is a field boundary benchmark of 70,462 Sentinel-2 samples from 24 countries on four continents. Its models predict both the field interior and its edge, so neighboring fields separate cleanly.
• Forestry and carbon. Meta's canopy height model (Tolan et al., Remote Sensing of Environment, 2024) estimates tree height from sub-meter red, green, and blue (RGB) imagery. Its decoder was trained against aerial lidar, laser height scans taken from aircraft.
• Urban planning. Tile2Net (Hosseini et al., Computers, Environment and Urban Systems, 2023) segments sidewalks, crosswalks, and footpaths from aerial tiles and assembles them into a pedestrian network.
• Poverty mapping. Jean and colleagues combined satellite imagery and machine learning to predict poverty (Science, 2016). A network they trained on nighttime lights produced daytime image features. Those features explained up to 75% of the variation in local economic outcomes.
• Insurance. Roof condition, solar panels, pools, and tree overhang per property across a portfolio, plus damage maps after hurricanes and wildfires.
• Mapping and defense. Building footprints and roads for open basemaps, and monitoring of sites for new structures and vehicles.

Satellite imagery analysis in Wherobots

RasterFlow (Public Preview) builds mosaics from imagery, runs computer vision inference on them, and writes the results as vectorized outputs on the same platform. A Python API drives it. Mosaics and predictions are written as Zarr, and vector results as GeoParquet. Built-in models cover common tasks:

• SAM3 for text-prompted object detection on 30 cm NAIP.
• Tile2Net for sidewalks, crosswalks, and roads on 30 cm NAIP.
• Meta CHM v1, Meta's canopy height model (CHM), on 60 cm NAIP.
• ChesapeakeRSC for rural roads on 1 m NAIP.
• Fields of the World for field boundaries on Sentinel-2 planting and harvest composites.

Custom PyTorch models run through the bring-your-own-model path. Sentinel-2 mosaics skip scenes with 75% or more cloud cover, mask cloud, shadow, and cirrus pixels with the Scene Classification Layer, and take a per-pixel median of what remains.

Pricing is calculated before a task runs, from area, resolution, bands, and time periods. Finding every solar panel array across a 500 km² county with SAM3 on 30 cm NAIP costs $35.00, mosaic and inference included.

# Detect solar panels across a county from a text prompt with SAM3 on 30 cm NAIP
from datetime import datetime
from rasterflow_remote import RasterflowClient
from rasterflow_remote.data_models import GeometryModelRecipes

rf = RasterflowClient()
detections = rf.predict_mosaic_geometries_recipe(
    aoi="s3://your-bucket/county.parquet",
    start=datetime(2022, 1, 1),
    end=datetime(2023, 1, 1),
    model_recipe=GeometryModelRecipes.SAM3_TEXT_GEOMETRY,
    text_prompt="solar panel",
    confidence_threshold=0.5,
)
print(detections.uri)  # GeoParquet containing detected solar-panel polygons

The GeoParquet output loads into WherobotsDB, built by the original creators of Apache Sedona, where a spatial join links each polygon to parcels, buildings, or hazard zones in SQL.

A Marion County walkthrough

In Marion County, Oregon, one RasterFlow pipeline built a 133 GB mosaic from 2022 NAIP imagery. It ran SAM3 with eight text prompts in a single pass and kept detections with a confidence of 0.5 or more. Wherobots publishes the output as wherobots_open_data.rasterflow_output_samples.marion_county_sam3_vector.

Source	Format	Size	What was read
USDA NAIP aerial imagery, 2022	Raster mosaic built by RasterFlow (Zarr)	133 GB	Imagery covering Marion County, as SAM3 input
SAM3 detections in Wherobots open data	WherobotsDB table	1,006,232 rows	label, bbox_score, and geometry, for the query below

Raster and vector stay on one platform: RasterFlow wrote the polygons from the imagery, and WherobotsDB queried them. Most stacks pick raster or vector. This query reads the one detections table and summarizes it per prompt. Areas are computed in Universal Transverse Mercator (UTM) zone 10N, so they come out in square meters. That system's code in the EPSG registry of coordinate systems is EPSG:32610:

SELECT label,
       COUNT(*) AS n,
       ROUND(AVG(bbox_score), 3) AS mean_score,
       ROUND(SUM(ST_Area(ST_Transform(geometry, 'EPSG:4326', 'EPSG:32610'))) / 1e6, 3) AS area_km2
FROM wherobots_open_data.rasterflow_output_samples.marion_county_sam3_vector
GROUP BY label
ORDER BY n DESC

label	n	mean_score	area_km2	mean area per detection
roads	670,625	0.572	54.789	82 m²
roofs	312,245	0.682	41.710	134 m²
solar panels	18,543	0.614	2.101	113 m²
shipping containers	2,554	0.585	0.182	71 m²
swimming pools	1,434	0.725	0.067	47 m²
airports	669	0.571	0.254	380 m²
airplanes	131	0.756	0.005	38 m²
tractors	31	0.550	0.001	32 m²

The table holds 1,006,232 detections, and every row carries the same time stamp, 1 January 2022, the start of the 2022 NAIP window used to build the mosaic.

Reading the confidence scores

• Confidence varies by concept. Airplanes (0.756) and swimming pools (0.725) score highest. Tractors (0.550) and airports (0.571) score lowest. Roofs sit between them at 0.682.
• Rare prompts return few objects. Roads and roofs make up 97.7% of the rows, while the county returned 131 airplanes and 31 tractors. Any precision estimate for a rare class rests on a small sample.
• The cutoff shapes every statistic. Detections under 0.5 were never stored, so the table cannot show the low-confidence half of the precision and recall curve.

Scoring roofs against Overture buildings

The county-wide evaluation of SAM3 roofs against Overture building footprints used three inputs:

Source	What was read	Count inside the county
SAM3 detections, roofs label	Roof polygons	261,414 roofs
Overture Maps buildings	Building footprints	146,642 buildings
Official Marion County boundary	County outline, used to clip both datasets	Not applicable

The evaluation joined SAM3 roofs to Overture buildings where they intersect, in EPSG:32610. It then computed IoU for every intersecting pair. The join returned 207,109 candidate pairs. Total roof area agreed within 7%, and SAM3 matched the shape of three-quarters of known buildings. Many unmatched detections were real rural houses and outbuildings missing from Overture. The authors therefore judged real precision likely higher than 75%.

Research on satellite imagery analysis

Satellite imagery analysis rests on fifty years of research, from texture statistics computed on the first Landsat scenes to promptable segmentation models.

From pixels to objects

The first computer analyses of satellite images classified one pixel at a time from its spectral values. Haralick, Shanmugam, and Dinstein added spatial context in Textural features for image classification (IEEE Transactions on Systems, Man, and Cybernetics, 1973). Their features are statistics of how often gray tones occur together in a neighborhood. They tested them on aerial photographs and on multispectral imagery from the Earth Resources Technology Satellite, later renamed Landsat 1. Compton Tucker's study of red and photographic infrared combinations (Remote Sensing of Environment, 1979) compared linear combinations of red and near-infrared bands for monitoring vegetation. The normalized difference it favored became NDVI.

Comparing dates came next. Ashbindu Singh's review of digital change detection techniques (International Journal of Remote Sensing, 1989) evaluated the procedures developed by then for comparing images of one place from different dates. It remains the starting point for the field. The Hansen forest map above ran that idea globally on Landsat time series.

Sub-meter imagery changed the unit of analysis. Once a house spans hundreds of pixels, a pixel classifier assigns roof texture, shadow, and lawn to separate classes. Thomas Blaschke's Object based image analysis for remote sensing (ISPRS Journal of Photogrammetry and Remote Sensing, 2010) describes the response. Segment the image into objects first, then classify each object by its spectra, shape, and relations to its neighbors. Instance segmentation models such as SAM3 complete that shift, returning one polygon per object straight from the network.

Measuring accuracy

Russell Congalton's review of assessing the accuracy of classifications of remotely sensed data (Remote Sensing of Environment, 1991) set the error matrix as the core tool. An error matrix counts, for each true class, how many sample pixels the map put in each class. Congalton named the choices that decide whether an accuracy number means anything: the classification scheme, the sampling design, the sample size, and spatial autocorrelation between samples. Nearby pixels are similar, so samples drawn close together are not independent. Object detection added the IoU threshold from PASCAL VOC, the measure behind the Marion County comparison.

The open problem is the reference itself. Overture buildings are a curated, multi-source dataset, and they still likely miss buildings in rural areas. For that reason the Marion County evaluation treated many SAM3 "false positives" as real buildings. Roofs also differ from footprints: overhangs and occlusion mean a roof outline and a footprint seldom match. Any accuracy number on overhead imagery compares two datasets. Report the reference's dates, definitions, and gaps beside it.

Benchmarks and deep learning

Zhu, Tuia, Mou, and colleagues reviewed the arrival of neural networks in remote sensing. Their review is Deep learning in remote sensing: a comprehensive review and list of resources (IEEE Geoscience and Remote Sensing Magazine, 2017). Two things brought about the shift: architectures from other fields and labeled benchmarks for overhead imagery.

U-Net (Ronneberger, Fischer, and Brox, MICCAI 2015) was designed for biomedical images. It became a common segmentation network for satellite imagery because of its expanding path. That path scales features back up to full resolution. It recovers the precise localization that building and road edges need. The benchmarks followed, alongside Fields of the World and Tile2Net from the examples above:

• EuroSAT (Helber et al., 2019): 27,000 labeled Sentinel-2 patches in ten land use classes, across all 13 bands.
• xView (Lam et al., 2018): over 1 million objects in 60 classes in 0.3 m WorldView-3 imagery.
• SpaceNet (Van Etten, Lindenbaum, and Bacastow, 2018): building footprint and road network challenges.
• xBD (Gupta et al., 2019): 850,736 building polygons across 19 disasters, labeled before and after.

Foundation models and embeddings

Every benchmark above needs labels for its task. Foundation models move most of the learning to unlabeled imagery or to large general datasets.

Promptable segmentation. Kirillov and colleagues' Segment Anything (ICCV 2023) released over 1 billion masks on 11 million images and segmented an object from a point or a box. Meta's SAM 3 (Carion et al., 2025) replaced the click with a concept: a noun phrase such as "roofs" or an example image. It returns every matching instance. It was trained on a dataset with 4 million unique concept labels. That change is what lets one RasterFlow run detect roofs, roads, and solar panels across a county with no labels.

Pretrained Earth observation encoders. NASA and IBM's Prithvi (Jakubik et al., 2023) was pretrained on more than 1 TB of Harmonized Landsat Sentinel-2 imagery for fine-tuning on downstream tasks with small labeled datasets. Meta's canopy height model in the examples above used a DINOv2 encoder, an image model trained on unlabeled images.

Embeddings as a product. Rolf and colleagues' MOSAIKS (Nature Communications, 2021) computes image encodings once and shares them across tasks, so each new task needs only a linear regression on the user's own labels. Google DeepMind's AlphaEarth Foundations (Brown et al., 2025) takes the same approach with a learned model. It compresses each location's imagery into annual global embedding layers from 2017 through 2024. An embedding is a list of numbers that summarizes a location. With these layers, similarity and change become distance calculations.

Open problems

• Domain shift. A model trained on one region, season, or sensor degrades on another. Tuia, Persello, and Bruzzone's overview of domain adaptation for remote sensing classification (IEEE Geoscience and Remote Sensing Magazine, 2016) frames the problem. Fields of the World spans 24 countries to capture that diversity.
• Reference data. Validation sets are often other models' outputs or volunteer maps with their own gaps, as in the Marion County comparison.
• Time. Imagery, labels, and reference vectors rarely share a date. A 2022 NAIP roof scored against a later building dataset mixes model error with real change.
• Spatial dependence. Random train and test splits of neighboring patches leak information, the same autocorrelation Congalton warned about in 1991.

What scale changes

At county scale the model is the hard part. At continental scale the data movement is. Gorelick and colleagues described one answer in Google Earth Engine: planetary-scale geospatial analysis for everyone (Remote Sensing of Environment, 2017): a catalog of analysis-ready imagery next to parallel compute. The Cloud Optimized GeoTIFF layout defined above serves the same goal on object storage, where each reader fetches only the tiles it needs.

The vector side scales too. A billion model polygons need a spatial engine to join them to parcels, buildings, and boundaries. Jia Yu, Jinxuan Wu, and Mohamed Sarwat introduced GeoSpark (ACM SIGSPATIAL 2015) for distributed spatial queries on Apache Spark. Yu, Zongsi Zhang, and Sarwat detailed its spatial partitioning, indexing, and join design in Spatial data management in Apache Spark: the GeoSpark perspective and beyond (GeoInformatica, 2019). GeoSpark became Apache Sedona. Kanchan Chowdhury and Sarwat's GeoTorch (ACM SIGSPATIAL 2022) brought a spatiotemporal deep learning framework into the same research line.

The numbers at the far end come from the Fields of the World global release. Taylor Geospatial ran PRUE, its latest Fields of the World model, on RasterFlow, generating 348.7 TB across 540,794 objects to produce 8.2 billion field boundaries.

Conclusions

The evidence above shows what satellite imagery analysis needs to produce usable maps and counts: the right resolution, a pipeline around the model, and checks against independent data.

1. In satellite imagery analysis, the object sets the resolution. Fields and forests work at 10 m Sentinel-2. The Marion County run was SAM3 object detection on 30 cm NAIP. Its eight prompts covered roads, roofs, solar panels, shipping containers, swimming pools, airports, airplanes, and tractors. Sidewalks and canopy height come from other RasterFlow models (Tile2Net on 30 cm NAIP, Meta CHM v1 on 60 cm NAIP) that this run did not use.
2. Most of the work in satellite imagery analysis at scale is the pipeline, where the model is one of seven steps. Mosaicking, patching, merging, and vectorizing grow with the area. The Marion County run turned a 133 GB mosaic into about one million polygons.
3. Promptable models let satellite imagery analysis start without labels, but each concept needs its own check. Mean confidence ran from 0.550 for tractors to 0.756 for airplanes. Roads and roofs made up 97.7% of the rows. An accuracy claim for a rare class rests on a small sample.
4. An accuracy number from satellite imagery analysis describes two datasets, so report the reference's gaps beside it. SAM3 roofs matched three-quarters of Overture buildings. Many unmatched detections were real buildings missing from Overture. Counting them as false positives likely understates precision.
5. A satellite imagery analysis result needs its confidence cutoff stored with it. The Marion County table holds no detection under 0.5. It cannot describe the low-confidence half of the precision and recall curve.

Related terms

• Geospatial AI: the wider field of AI on location data and imagery
• Change detection: comparing images over time
• Land cover classification: labeling every pixel by surface type
• Earth observation data: the imagery these models consume
• Building footprint: a common output of imagery models
• NDVI: the band ratio behind most vegetation analysis
• Geospatial intelligence: imagery analysis joined to map data to describe activity at a place

Read more from Wherobots

• Detecting objects from text prompts with RasterFlow and Segment Anything 3
• Evaluating SAM3 roof detections against Overture building footprints
• Fields of the World: a GeoAI pipeline on RasterFlow
• Detect objects with SAM3 in RasterFlow

Run a built-in RasterFlow model from the Model Hub at cloud.wherobots.com.