Planetary-scale answers, unlocked.
A Hands-On Guide for Working with Large-Scale Spatial Data. Learn more.
Authors
A context engine is the system that computes, stores, and serves the context an AI model or agent receives when it runs. It ingests sources and models them as entities, each with a stable ID. It indexes them, computes the relations a question needs, and returns a short result with its sources through an API or a tool call. Jason Cui and Jennifer Li of Andreessen Horowitz made the case for this layer in Your Data Agents Need Context (10 March 2026). For physical world data, a context engine computes spatial relations, such as which county contains a building or how far it sits from the nearest road. Those facts are rarely written down anywhere a model could retrieve them.
A context engine prepares the part of an AI system's input that comes from data. A model's context is everything it receives at inference time, when it runs: instructions, tool definitions, conversation history, and retrieved information. A search index over documents supplies part of the retrieved input. Business definitions, joins across systems, up-to-date values, and access permissions need more work, and a context engine does it.
In their a16z post, Cui and Li argue that data and analytics agents fail without context about how a business defines its terms. They argue that a modern context layer should become a superset of the semantic layer, the shared metric definitions that analytics tools read. Beyond metric definitions, it would cover canonical entities (one agreed record for each customer or product) and identity resolution (matching records for the same entity). It would also cover tribal knowledge (unwritten know-how held by staff) and governance guidance.
Current products use the term in three ways: code context engines, streaming and operational context engines, and context layers for business data.
Context engineering is the practice; a context engine is the system. Anthropic's Applied AI team defines context engineering in Effective context engineering for AI agents (September 2025). The article calls it the set of strategies for curating and maintaining the optimal set of tokens during inference. Tokens are the units of text a model reads. The article describes agents that keep lightweight identifiers and load the data behind them at run time with tools. A context engine is what sits behind those tools: it holds the indexes, runs the joins, and returns the record. Retrieval-augmented generation (RAG) is one retrieval technique an engine can use. The FAQ below compares them.
Sources update on their own schedules, so the engine either recomputes relations on request or updates them as inputs change.
Language models were trained on text, documents, databases, and the internet. Those sources describe places in words. They rarely state the relations a physical world question depends on. Examples are which parcel touches a river, which warehouse sits inside last week's flood warning, and how far a substation is from the nearest paved road. Rohin Manvi and colleagues tested this in GeoLLM (ICLR 2024). Prompting a model with coordinates alone gave poor geographic estimates. Adding nearby OpenStreetMap features to the prompt improved results by 70% over baselines. The map context had to be computed and put in the prompt.
Five properties call for a dedicated engine: computed relations, mixed vector and raster data, coordinate systems, update cycles, and volume.
Context is computed relations. A text context engine retrieves stored passages. A physical world answer is usually a relation computed at query time: containment, distance, overlap, or the raster value under a footprint. Max Egenhofer and Robert Franzosa formalized topological relations such as contains, overlaps, and touches in Point-set topological spatial relations (IJGIS, 1991). The Open Geospatial Consortium (OGC) Simple Feature Access standard defines the geometry types and the predicates, yes-or-no tests between two shapes, that spatial SQL implements. Waldo Tobler's first law of geography (1970), that near things are more related than distant things, is why distance enters almost every physical world question.
Vector and raster mix. Buildings, roads, and boundaries are vectors. Elevation, land cover, and imagery are rasters. A record for one building needs both: polygons joined to points and lines, and a grid cell read under the footprint.
Coordinate systems and scales differ. Overture stores longitude and latitude (EPSG:4326). A degree of longitude shrinks with distance from the equator. Distances therefore belong in a projected coordinate reference system, a flat grid in meters, or on the ellipsoid, a model of the earth's shape. Resolution differs too: a GLO-30 cell, a pixel of the Copernicus 30 m elevation model, spans about 30 m, so neighboring houses can read the same cell.
Update cycles differ. Overture Maps publishes a monthly release. The Census Bureau releases TIGER/Line boundaries once a year. The Copernicus digital elevation model (DEM) on AWS comes from its 2021 release. Weather warnings arrive per event. A record has to carry each source's date.
Volume. The county-sized box in the walkthrough holds 227,564 buildings, and a national portfolio holds millions. That calls for distributed spatial computing across many machines. Jia Yu, Jinxuan Wu, and Mohamed Sarwat addressed that problem with GeoSpark (ACM SIGSPATIAL 2015). Later work covered spatial partitioning and per-partition indexes in Spatial data management in Apache Spark: the GeoSpark perspective and beyond (GeoInformatica, 2019).
Wherobots is the Context Engine for Physical World Data. WherobotsDB runs spatial SQL across vector and raster data on distributed runtimes. RasterFlow (Public Preview) builds mosaics from imagery, runs computer vision inference on them, and writes the results as vectorized outputs on the same platform. The Havasu catalog holds the tables as Apache Iceberg, and the Wherobots MCP server gives AI agents tools to list catalogs and tables, describe a schema, and run SQL.
This walkthrough builds a context record for every building in downtown Asheville, North Carolina, a city in Buncombe County. Hurricane Helene caused catastrophic inland flooding there in September 2024 (National Hurricane Center (NHC) Tropical Cyclone Report). The query reads seven Apache Iceberg tables in Wherobots Open Data:
Building and tile counts are from the run below. The others are COUNT(*) checks of each filter on 9 October 2026. One engine read raster and vector in one query. For each building the query returns:
COUNT(*)
-- One context record per building: county, city, nearest road, places nearby, -- surface height (Copernicus GLO-30) and NWS flash flood warning history -- Distances in meters in UTM zone 17N WITH b AS ( SELECT id, names.primary AS name, geometry, ST_Centroid(geometry) AS c, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS geom_m FROM wherobots_open_data.overture_maps_foundation.buildings_building WHERE bbox.xmin >= -82.575 AND bbox.xmax <= -82.545 AND bbox.ymin >= 35.575 AND bbox.ymax <= 35.605 ), county AS ( -- North Carolina counties, broadcast to every worker SELECT /*+ BROADCAST(c) */ b.id, c.NAME AS county FROM b JOIN wherobots_open_data.us_census.tiger_county c ON ST_Intersects(c.geometry, b.c) WHERE c.STATEFP = '37' ), city AS ( SELECT b.id, d.names.primary AS locality FROM b JOIN wherobots_open_data.overture_maps_foundation.divisions_division_area d ON ST_Intersects(d.geometry, b.c) WHERE d.subtype = 'locality' AND d.class = 'land' AND d.bbox.xmin < -82.545 AND d.bbox.xmax > -82.575 AND d.bbox.ymin < 35.605 AND d.bbox.ymax > 35.575 ), roads AS ( -- Named-network roads in the box plus a 0.01 degree margin SELECT id AS road_id, names.primary AS road_name, class AS road_class, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS g_m FROM wherobots_open_data.overture_maps_foundation.transportation_segment WHERE subtype = 'road' AND class IN ('motorway', 'trunk', 'primary', 'secondary', 'tertiary', 'residential', 'unclassified', 'living_street') AND bbox.xmin < -82.535 AND bbox.xmax > -82.585 AND bbox.ymin < 35.615 AND bbox.ymax > 35.565 ), nearest_road AS ( -- Exact nearest road segment to each footprint (planar distance in meters) SELECT b.id, r.road_id, r.road_name, r.road_class, ST_Distance(b.geom_m, r.g_m) AS road_m FROM b JOIN roads r ON ST_KNN(b.geom_m, r.g_m, 1, false) ), places AS ( SELECT id, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS g_m FROM wherobots_open_data.overture_maps_foundation.places_place WHERE bbox.xmin BETWEEN -82.577 AND -82.543 AND bbox.ymin BETWEEN 35.573 AND 35.607 ), places_100m AS ( SELECT b.id, COUNT(*) AS places_100m FROM b JOIN places p ON ST_DWithin(b.geom_m, p.g_m, 100) GROUP BY b.id ), dem AS ( -- The 256 x 256 GLO-30 tiles that touch the box SELECT name AS tile, rast, footprint FROM wherobots_open_data.copernicus_dem.glo_30m WHERE ST_Intersects(footprint, ST_PolygonFromEnvelope(-82.575, 35.575, -82.545, 35.605)) ), tile_points AS ( -- Group the centroids by tile, so each tile is read once SELECT /*+ BROADCAST(t) */ t.tile, collect_list(b.id) AS ids, collect_list(b.c) AS pts FROM b JOIN dem t ON ST_Intersects(t.footprint, b.c) GROUP BY t.tile ), surface AS ( -- One RS_Values call per tile returns the surface height under every centroid in it SELECT v.id, MAX(v.surface_m) AS surface_m FROM ( SELECT /*+ BROADCAST(p) */ explode(zip_with(p.ids, RS_Values(t.rast, p.pts), (i, z) -> named_struct('id', i, 'surface_m', z))) AS v FROM tile_points p JOIN dem t ON p.tile = t.tile ) GROUP BY v.id ), ffw AS ( -- NWS flash flood warnings (FF.W) issued by the Greenville-Spartanburg office SELECT ISSUED, geometry FROM wherobots_open_data.noaa.nws_watch_warnings WHERE WFO = 'GSP' AND PHENOM = 'FF' AND SIG = 'W' ), ffw_b AS ( SELECT b.id, COUNT(*) AS ffw_count, MAX(w.ISSUED) AS ffw_last, SUM(CASE WHEN w.ISSUED >= '2024-09-25' AND w.ISSUED < '2024-09-29' THEN 1 ELSE 0 END) AS ffw_helene FROM b JOIN ffw w ON ST_Intersects(w.geometry, b.c) GROUP BY b.id ), rec AS ( SELECT b.id, b.name, ROUND(ST_X(b.c), 6) AS lon, ROUND(ST_Y(b.c), 6) AS lat, co.county, ci.locality, r.road_id, r.road_name, r.road_class, ROUND(r.road_m, 1) AS road_m, COALESCE(p.places_100m, 0) AS places_100m, ROUND(s.surface_m, 1) AS surface_m, COALESCE(w.ffw_count, 0) AS ffw_count, w.ffw_last, COALESCE(w.ffw_helene, 0) AS ffw_helene, ST_AsText(b.geometry) AS wkt FROM b LEFT JOIN county co ON b.id = co.id LEFT JOIN city ci ON b.id = ci.id LEFT JOIN nearest_road r ON b.id = r.id LEFT JOIN places_100m p ON b.id = p.id LEFT JOIN surface s ON b.id = s.id LEFT JOIN ffw_b w ON b.id = w.id ) SELECT rec.*, COUNT(*) OVER () AS buildings, SIZE(COLLECT_SET(id) OVER ()) AS distinct_ids, SUM(IF(county IS NOT NULL, 1, 0)) OVER () AS with_county, SUM(IF(locality IS NOT NULL, 1, 0)) OVER () AS in_a_locality, SUM(IF(road_m IS NOT NULL, 1, 0)) OVER () AS with_road, SUM(IF(surface_m IS NOT NULL, 1, 0)) OVER () AS with_surface, ROUND(percentile(road_m, 0.5) OVER (), 1) AS median_road_m, ROUND(percentile(places_100m, 0.5) OVER (), 1) AS median_places_100m, MIN(surface_m) OVER () AS min_surface_m, MAX(surface_m) OVER () AS max_surface_m, SUM(IF(ffw_count > 0, 1, 0)) OVER () AS under_any_ffw, SUM(IF(ffw_helene > 0, 1, 0)) OVER () AS under_helene_ffw, (SELECT COUNT(*) FROM dem) AS dem_tiles FROM rec
The query ran in 49.7 seconds and returned 4,408 records, one per distinct building ID, each with every field filled. All 4,408 centroids fall in Buncombe County and in the city of Asheville. The roads step reads segments up to 0.01° (about 900 m) beyond the building box, so a nearest road closer than that is exact. The median distance from a footprint to its nearest road segment is 13.1 m, and the median building has 3 places within 100 m. Surface heights read from 2 GLO-30 tiles range from 600.3 m to 759.0 m. Every one of the 4,408 buildings sits inside at least one flash flood warning polygon issued from 25 to 28 September 2024. Those were the days of Helene's rainfall in western North Carolina. A warning is a forecast of a hazard, so these counts measure warning history. Flood extent needs observed data such as high-water marks.
The record for the Grove Arcade shows what the model receives:
None of those values sits in one stored row. Each comes from a join computed at query time. The road distance comes from a k nearest neighbor (KNN) join that finds each building's closest road, in meters. The place count comes from a distance join. The surface height comes from a raster read. The warning counts come from a point-in-polygon join against warning polygons. The Copernicus DEM is a surface model that includes buildings and vegetation, so 692.2 m is surface height. Ground elevation needs a terrain model or lidar (see digital elevation model). This version keeps the building and road IDs. A production record also returns the county GEOID (the Census Bureau's numeric code for the county), the warning IDs, the elevation tile name, and each layer's release date. Then every value traces to a source row and a date.
GEOID
Changing only the box coordinates runs the same query over a 0.65° by 0.43° box around Buncombe County, which includes parts of the neighboring counties:
-- Same query; only the box changes. Buildings: WHERE bbox.xmin >= -82.900 AND bbox.xmax <= -82.250 AND bbox.ymin >= 35.400 AND bbox.ymax <= 35.830 -- The divisions, roads, places and DEM filters move by the same offsets as above: -- divisions box -82.900 to -82.250, 35.400 to 35.830; roads -82.910 to -82.240, 35.390 to 35.840; -- places -82.902 to -82.248, 35.398 to 35.832; DEM envelope (-82.900, 35.400, -82.250, 35.830)
It ran in 90.5 seconds and returned 227,564 records for 227,562 distinct building IDs. The road margin stays at 0.01°, so a building whose nearest road is more than about 900 m away needs a wider one. Two IDs appear twice, so a production version adds a tie-break rule, such as the lowest county GEOID, to each one-to-many lookup. The query read 70 GLO-30 tiles, and surface heights range from 328.2 m to 2,019.6 m. The median distance to the nearest road is 24.3 m, and 71,630 records fall inside an Overture locality polygon. The extent grew to about 52 times as many records, and the run time grew 1.8 times. Both query plans showed the county and city lookups as broadcast R-tree joins (BroadcastIndexJoin). These copy the small boundary table to every worker. Both plans also showed the nearest road as a KNN join. Neither plan contained a CartesianProduct, a step that compares every pair of rows. One query covers the full extent, so the record has no tile seams, the edges where separately processed tiles meet. The partitioning tax article shows what a tile loop would cost.
BroadcastIndexJoin
CartesianProduct
The context engines above serve code and business data. Physical world context needs spatial relations computed across vector and raster data. The table compares spatial SQL systems from each product's own documentation.
ST_
GEOGRAPHY
ST_REGIONSTATS
GEOMETRY
RS_
Sources: the PostGIS reference and Citus; the DuckDB spatial function reference; BigQuery raster data; Snowflake geospatial data types; the WherobotsDB reference and raster functions. Function counts are distinct names in each official reference on 6 October 2026, as counted in the partitioning tax article. "Not documented" means the linked documentation describes no such capability.
The idea grew out of retrieval. Patrick Lewis and colleagues paired a language model with a searchable index in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020). Nelson Liu and colleagues then published Lost in the Middle (Transactions of the Association for Computational Linguistics, 2024). It showed that accuracy drops when the relevant passage sits in the middle of a long input. A longer context window therefore does not replace selection. GraphRAG (2024) moved part of the work into a graph built ahead of time. The Model Context Protocol (2024) standardized how agents call data systems, and Anthropic's 2025 article named the practice of context engineering. In March 2026, Cui and Li framed the next step as infrastructure. They traced agent failures to missing business context. They proposed a context layer to supply it. Each step moved more work out of the model and into a system that prepares its input.
Tobler's first law (1970) and Egenhofer and Franzosa's relations (1991) supplied the theory. The OGC standards turned it into shared vocabularies. Simple Feature Access covers geometry and predicates. GeoSPARQL 1.1 (2024) covers geospatial data in Resource Description Framework (RDF) knowledge graphs. Aidan Hogan and colleagues' survey Knowledge Graphs (ACM Computing Surveys, 2021) covers the graph models such engines build on. Krzysztof Janowicz, Song Gao, Grant McKenzie, Yingjie Hu, and Budhendra Bhaduri argued that models for geographic problems need location, distance, and spatial relations built in. They made this argument in GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond (IJGIS, 2020). KnowWhereGraph (2022) serves geographic context as linked data. Dazhou Yu, Riyang Bao, Gengchen Mai, and Liang Zhao's Spatial-RAG (2025) pairs a spatial database filter with semantic ranking to answer geospatial questions. The Spatial Graph RAG post applies the graph approach to Overture data in Wherobots.
The county run repeated one building's handful of joins for 227,564 records. Distributed spatial partitioning and per-partition indexes, the design Yu, Zhang, and Sarwat describe for GeoSpark, keep that to one query per question. Geospatial AI and physical AI workloads add imagery, and with it more raster data.
The Asheville walkthrough shows the job of a context engine for physical world data. It computes each fact at query time from several sources and returns it with IDs a model can cite.
Build context records for your own buildings and assets with a Wherobots free trial at cloud.wherobots.com.
A context engine is the system that prepares the information an AI model or agent receives at run time. It ingests sources and models them as entities with stable IDs. It indexes them, computes the relations a question needs, and serves the result through an API or tool call. It keeps that result current as sources change. Jason Cui and Jennifer Li of Andreessen Horowitz describe the same need for data agents in Your Data Agents Need Context (March 2026).
It gives an agent a compact, sourced answer to the part of a question that lives in data. That part covers which records match, how they relate, and when each source was last updated. The agent calls the engine as a tool, often over the Model Context Protocol. It receives a small record in place of raw tables or documents. The prompt stays short, and every fact traces to an ID.
Context engineering is the practice of choosing which tokens a model receives at each step: instructions, tools, retrieved data, memory, and history. A context engine is the infrastructure that computes, stores, and serves the data part of that context. See What is context engineering for physical AI?
No. Retrieval-augmented generation (RAG) searches an index for passages similar to a question and adds them to the prompt. A context engine can include that retrieval step, and it also joins sources, resolves entities, computes derived values such as distances or containment, and keeps the results current. RAG returns stored text. A context engine can return facts that no stored document contains.
The terms overlap. A context layer usually names the place in an architecture between data sources and AI agents, and a context engine names the system that does the work in that place. Cui and Li of Andreessen Horowitz describe the context layer for data agents as a superset of the semantic layer. It adds canonical entities, identity resolution, tribal knowledge, and governance guidance to metric definitions.
The Model Context Protocol (MCP), which Anthropic introduced in November 2024, is an open standard that lets AI applications call external tools and read resources over JSON-RPC. A context engine exposes its queries as MCP tools, so any MCP client, such as a coding assistant or an agent framework, can request context with one tool call. The Wherobots MCP server, for example, offers tools to list catalogs and tables, describe a table, and run spatial SQL.
Code context engines, such as Augment Code’s Context Engine, index a repository and serve relevant code to coding agents over MCP. Streaming and operational context engines, such as Confluent’s Real-Time Context Engine and Materialize, keep derived business objects current as events arrive. Geo-enrichment services such as KnowWhereGraph return linked facts about a region. Wherobots computes spatial relations across vector and raster data for physical world questions.
Language models were trained on text, documents, databases, and the internet. Those sources rarely state which building sits inside which flood warning, how far it is from the nearest road, or what the surface height is under it. Those facts are relations between shapes and raster cells. They are computed in a known coordinate system from sources that update on different cycles. A context engine for physical world data computes them at query time and returns them with IDs and sources.
Imagery records physical world facts that tables often lack, such as new construction, cleared land, or flood extent. Computer vision inference turns those pixels into features a query can use: building outlines, land cover classes, or detected objects, each with a location. In Wherobots, RasterFlow (Public Preview) builds mosaics from imagery, runs computer vision inference on them, and writes the results as vectorized outputs on the same platform. See AI satellite imagery analysis and land cover classification.
It is how Wherobots describes its platform: a context engine whose sources are physical world data, such as buildings, places, roads, boundaries, elevation models, imagery, and weather warnings. WherobotsDB computes the spatial relations in distributed SQL over vector and raster data. RasterFlow turns imagery into mosaics, inference results, and vector outputs. The Havasu catalog stores the tables in Apache Iceberg. The Wherobots MCP server lets AI agents find tables and run queries.
What is Spatial Intelligence? Definition, AI, Earth Scale
Spatial intelligence: AI that understands 3D space. Open map data holds a height for 96% of San Francisco's buildings and 0.13% of Nairobi's.
What Is PySpark? Definition, Tutorial, and Spatial Data
PySpark is the Python API for Apache Spark. Learn how it runs, with a short tutorial. Then see a spatial join that took 6 minutes without Apache Sedona's join planning and under a second with it.
What is a Semantic Layer? Definition, Examples, Spatial Rules
A semantic layer defines business terms and metrics once, so every tool gives the same answer. For location data it must also set the boundary, the distance and what counts as inside.
share this article
Awesome that you’d like to share our articles. Where would you like to share it to: