--- title: "What is a Context Engine? Why Physical World Data Needs One" url: "https://wherobots.com/discover/context-engine/" description: "A context engine computes, stores, and serves the context AI agents use. How it works, examples, and why physical world data needs spatial relations." published: "2026-10-06" modified: "2026-10-26" --- # What is a Context Engine? Why Physical World Data Needs One A context engine is the system that computes, stores, and serves the context an AI model or agent receives when it runs. It ingests sources and models them as entities, each with a stable ID. It indexes them, computes the relations a question needs, and returns a short result with its sources through an API or a tool call. Jason Cui and Jennifer Li of Andreessen Horowitz made the case for this layer in [Your Data Agents Need Context](https://a16z.com/your-data-agents-need-context/) (10 March 2026). For physical world data, a context engine computes spatial relations, such as which county contains a building or how far it sits from the nearest road. Those facts are rarely written down anywhere a model could retrieve them. ## Key takeaways - A context engine is infrastructure: it ingests, models, indexes, computes relations, serves, and refreshes. Context engineering is the practice of choosing what a model receives. The engine is the system that produces it. - The term covers several product types today: code context engines, streaming and operational context engines, and context layers for business data. - Physical world context is mostly computed at query time: which boundary contains a building, its distance to a road, the raster value under its footprint. Inputs mix vector (points, lines, polygons) and raster (grids of cells) data with different coordinate systems and update schedules. - In Wherobots, one SQL query built a context record for each of 4,408 buildings in downtown Asheville in 49.7 seconds, joining seven sources including a 30 m elevation raster. The same query over a county-sized box returned 227,564 records in 90.5 seconds. ## What is a context engine? A context engine prepares the part of an AI system's input that comes from data. A model's context is everything it receives at inference time, when it runs: instructions, tool definitions, conversation history, and retrieved information. A search index over documents supplies part of the retrieved input. Business definitions, joins across systems, up-to-date values, and access permissions need more work, and a context engine does it. In their a16z post, Cui and Li argue that data and analytics agents fail without context about how a business defines its terms. They argue that a modern context layer should become a superset of the [semantic layer](https://wherobots.com/discover/semantic-layer/), the shared metric definitions that analytics tools read. Beyond metric definitions, it would cover canonical entities (one agreed record for each customer or product) and identity resolution (matching records for the same entity). It would also cover tribal knowledge (unwritten know-how held by staff) and governance guidance. Current products use the term in three ways: code context engines, streaming and operational context engines, and context layers for business data. - **Code context engines** index a code repository and return relevant files and symbols to coding agents. Augment Code serves its engine over the Model Context Protocol (MCP). It made its [Context Engine MCP server](https://www.augmentcode.com/changelog/context-engine-mcp-in-ga) generally available on 6 February 2026. - **Streaming and operational context engines** keep derived business objects current as source systems change. Confluent announced its [Real-Time Context Engine](https://www.businesswire.com/news/home/20251029382595/en/Confluent-Launches-Confluent-Intelligence-to-Solve-the-AI-Context-Gap) on 29 October 2025. It serves context from Apache Kafka and Apache Flink, open source systems for streaming data, through a managed MCP server. Materialize describes [AI context engines](https://materialize.com/blog/ai-context-engines-context-engineering-evolution/) built on incrementally maintained views, query results that update as source data changes. - **Context layers for business data** sit between a company's systems and its agents, as in the a16z post. ## Context engine vs context engineering [Context engineering](https://wherobots.com/discover/context-engineering/) is the practice; a context engine is the system. Anthropic's Applied AI team defines context engineering in [Effective context engineering for AI agents](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) (September 2025). The article calls it the set of strategies for curating and maintaining the optimal set of tokens during inference. Tokens are the units of text a model reads. The article describes agents that keep lightweight identifiers and load the data behind them at run time with tools. A context engine is what sits behind those tools: it holds the indexes, runs the joins, and returns the record. Retrieval-augmented generation (RAG) is one retrieval technique an engine can use. The FAQ below compares them. ## How a context engine works ![Five-column diagram of a context engine for physical world data. Ingest lists buildings, places, roads, boundaries, a 30 m surface height raster, and weather warnings. Ontology lists entity types, relation types, stable IDs, one coordinate reference system for math, and source and date. Index lists spatial partitions, per-partition R-trees, raster tiles, and a catalog. Compute relations lists contains, nearest, within distance, value under, and intersects. Serve lists SQL, MCP tools, the context record, and refresh. A bar underneath shows the model or agent receiving the context record as a tool result](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/context-engine-how-it-works.webp) *The five stages of a context engine, with the examples used in the Asheville walkthrough below. Schematic.* 1. **Ingest.** The engine connects to sources: tables, files, event streams, documents, rasters. It records where each value came from and when. 2. **Model.** An [ontology](https://wherobots.com/discover/ontology/) names the entity types and relations. A building, a place, and a road segment each get a type and a stable ID that an agent can cite and fetch again. 3. **Index.** Text gets keyword indexes or embeddings, numeric vectors that place similar meanings close together. Spatial data is split into partitions by area, each with an R-tree, an index of nested bounding boxes. Rasters are cut into tiles. A query then reads only the pieces it needs. 4. **Compute relations.** At query time the engine joins entities: by key, by meaning, or, for physical data, by geometry. A [spatial join](https://wherobots.com/discover/spatial-join/) answers which polygon contains a point, which segment is nearest, and which warnings covered a site. 5. **Serve.** The result goes back as a compact record through SQL, an API, or a tool. The [Model Context Protocol](https://modelcontextprotocol.io/specification/2025-11-25) (MCP), which Anthropic [open-sourced on 25 November 2024](https://www.anthropic.com/news/model-context-protocol), standardizes that last step. MCP servers offer tools, resources, and prompts over JSON-RPC 2.0, a standard format for remote calls. Any MCP client can call them. Sources update on their own schedules, so the engine either recomputes relations on request or updates them as inputs change. ## Context engine examples - **Coding agents.** A code context engine indexes a repository and returns the functions, call sites, and tests relevant to a change. - **Customer operations.** A streaming context engine joins a customer's account history, current session, and inventory into one object for a support agent. - **Business metrics.** A context layer resolves "revenue" to one definition and one table before an analytics agent writes SQL, the example Cui and Li use. - **Knowledge graphs.** GraphRAG, from Darren Edge and colleagues at Microsoft Research ([From Local to Global](https://arxiv.org/abs/2404.16130), 2024), builds a graph from a text corpus and answers questions from summaries of its communities. - **Geo-enrichment.** KnowWhereGraph, described by Krzysztof Janowicz and colleagues in [Know, Know Where, KnowWhereGraph](https://doi.org/10.1002/aaai.12043) (AI Magazine, 2022), returns linked facts about hazards, soils, climate, and demographics for any region. - **Buildings and hazards.** A context engine for physical world data returns, for one building, its county, nearest road, nearby businesses, surface height, and weather warning history. The walkthrough below builds that record. ## Why physical world data needs a context engine Language models were trained on text, documents, databases, and the internet. Those sources describe places in words. They rarely state the relations a physical world question depends on. Examples are which parcel touches a river, which warehouse sits inside last week's flood warning, and how far a substation is from the nearest paved road. Rohin Manvi and colleagues tested this in [GeoLLM](https://arxiv.org/abs/2310.06213) (ICLR 2024). Prompting a model with coordinates alone gave poor geographic estimates. Adding nearby OpenStreetMap features to the prompt improved results by 70% over baselines. The map context had to be computed and put in the prompt. Five properties call for a dedicated engine: computed relations, mixed [vector and raster data](https://wherobots.com/discover/raster-vs-vector-data/), coordinate systems, update cycles, and volume. **Context is computed relations.** A text context engine retrieves stored passages. A physical world answer is usually a relation computed at query time: containment, distance, overlap, or the raster value under a footprint. Max Egenhofer and Robert Franzosa formalized topological relations such as contains, overlaps, and touches in [Point-set topological spatial relations](https://doi.org/10.1080/02693799108927841) (IJGIS, 1991). The Open Geospatial Consortium (OGC) [Simple Feature Access](https://www.ogc.org/standards/sfa/) standard defines the geometry types and the predicates, yes-or-no tests between two shapes, that spatial SQL implements. Waldo Tobler's [first law of geography](https://doi.org/10.2307/143141) (1970), that near things are more related than distant things, is why distance enters almost every physical world question. **Vector and raster mix.** Buildings, roads, and boundaries are vectors. Elevation, land cover, and imagery are rasters. A record for one building needs both: polygons joined to points and lines, and a grid cell read under the footprint. **Coordinate systems and scales differ.** Overture stores longitude and latitude (EPSG:4326). A degree of longitude shrinks with distance from the equator. Distances therefore belong in a projected [coordinate reference system](https://wherobots.com/discover/coordinate-reference-system/), a flat grid in meters, or on the ellipsoid, a model of the earth's shape. Resolution differs too: a GLO-30 cell, a pixel of the Copernicus 30 m elevation model, spans about 30 m, so neighboring houses can read the same cell. **Update cycles differ.** Overture Maps publishes a [monthly release](https://docs.overturemaps.org/release-calendar/). The Census Bureau releases [TIGER/Line boundaries](https://www.census.gov/geographies/mapping-files/time-series/geo/tiger-line-file.html) once a year. The [Copernicus digital elevation model (DEM)](https://registry.opendata.aws/copernicus-dem/) on AWS comes from its 2021 release. Weather warnings arrive per event. A record has to carry each source's date. **Volume.** The county-sized box in the walkthrough holds 227,564 buildings, and a national portfolio holds millions. That calls for distributed spatial computing across many machines. Jia Yu, Jinxuan Wu, and Mohamed Sarwat addressed that problem with [GeoSpark](https://doi.org/10.1145/2820783.2820860) (ACM SIGSPATIAL 2015). Later work covered spatial partitioning and per-partition indexes in [Spatial data management in Apache Spark: the GeoSpark perspective and beyond](https://doi.org/10.1007/s10707-018-0330-9) (GeoInformatica, 2019). ![Table of seven sources joined in each Asheville context record. Overture buildings, places, roads and divisions are polygons, points and lines with monthly releases. TIGER/Line counties are polygons with a yearly release. Copernicus GLO-30 is a 30 m raster from the 2021 release. National Weather Service (NWS) warnings are polygons added per event. A second panel compares the same SQL on two extents. The downtown box gives 4,408 context records from 2 elevation tiles in 49.7 seconds. The county box gives 227,564 records from 70 tiles in 90.5 seconds](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/context-engine-sources-and-scale.webp) *Each Asheville context record joins seven sources with four geometry types and three update cycles. The same query ran on a downtown box (4,408 records, 49.7 s) and a county box (227,564 records, 90.5 s).* ## Context engine in Wherobots Wherobots is the Context Engine for Physical World Data. WherobotsDB runs spatial SQL across vector and raster data on distributed runtimes. [RasterFlow](https://docs.wherobots.com/develop/rasterflow) (Public Preview) builds mosaics from imagery, runs computer vision inference on them, and writes the results as vectorized outputs on the same platform. The [Havasu catalog](https://wherobots.com/discover/havasu-catalog/) holds the tables as Apache Iceberg, and the [Wherobots MCP server](https://docs.wherobots.com/develop/mcp/mcp-server-overview) gives AI agents tools to list catalogs and tables, describe a schema, and run SQL. This walkthrough builds a context record for every building in downtown Asheville, North Carolina, a city in Buncombe County. Hurricane Helene caused catastrophic inland flooding there in September 2024 ([National Hurricane Center (NHC) Tropical Cyclone Report](https://www.nhc.noaa.gov/data/tcr/AL092024_Helene.pdf)). The query reads seven Apache Iceberg tables in Wherobots Open Data: | Source | Data type | What the query read | Rows read | | --- | --- | --- | --- | | [Overture buildings](https://wherobots.com/havasu/#buildings_building) | Polygons | Footprints inside the downtown box | 4,408 | | [Overture places](https://wherobots.com/havasu/#places_place) | Points | Places in the box plus a small margin | 4,475 | | [Overture road segments](https://wherobots.com/havasu/#transportation_segment) | Lines | Road segments of eight road classes in the box plus 0.01° | 2,157 | | [Overture divisions](https://wherobots.com/havasu/#divisions_division_area) | Polygons | Land locality polygons that overlap the box | 1 | | [TIGER/Line counties](https://wherobots.com/havasu/#tiger_county) | Polygons | North Carolina counties | 100 | | [Copernicus GLO-30 elevation model](https://wherobots.com/havasu/#glo_30m) | Raster, 30 m cells | 256 x 256 tiles that touch the box | 2 tiles | | [NWS watches and warnings](https://wherobots.com/havasu/#nws_watch_warnings) | Polygons | Flash flood warnings from the Greenville-Spartanburg office | 987 | Building and tile counts are from the run below. The others are `COUNT(*)` checks of each filter on 9 October 2026. One engine read raster and vector in one query. For each building the query returns: - the county and the city that contain its centroid - the nearest road segment and its distance from the footprint, in meters - the number of places within 100 m of the footprint - the surface height under the centroid - the number of NWS flash flood warnings whose polygons cover it, the most recent issue time, and how many were issued from 25 to 28 September 2024. Flash flood warnings carry the code FF.W in the Valid Time Event Code (VTEC) system. ``` -- One context record per building: county, city, nearest road, places nearby, -- surface height (Copernicus GLO-30) and NWS flash flood warning history -- Distances in meters in UTM zone 17N WITH b AS ( SELECT id, names.primary AS name, geometry, ST_Centroid(geometry) AS c, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS geom_m FROM wherobots_open_data.overture_maps_foundation.buildings_building WHERE bbox.xmin >= -82.575 AND bbox.xmax <= -82.545 AND bbox.ymin >= 35.575 AND bbox.ymax <= 35.605 ), county AS ( -- North Carolina counties, broadcast to every worker SELECT /*+ BROADCAST(c) */ b.id, c.NAME AS county FROM b JOIN wherobots_open_data.us_census.tiger_county c ON ST_Intersects(c.geometry, b.c) WHERE c.STATEFP = '37' ), city AS ( SELECT b.id, d.names.primary AS locality FROM b JOIN wherobots_open_data.overture_maps_foundation.divisions_division_area d ON ST_Intersects(d.geometry, b.c) WHERE d.subtype = 'locality' AND d.class = 'land' AND d.bbox.xmin < -82.545 AND d.bbox.xmax > -82.575 AND d.bbox.ymin < 35.605 AND d.bbox.ymax > 35.575 ), roads AS ( -- Named-network roads in the box plus a 0.01 degree margin SELECT id AS road_id, names.primary AS road_name, class AS road_class, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS g_m FROM wherobots_open_data.overture_maps_foundation.transportation_segment WHERE subtype = 'road' AND class IN ('motorway', 'trunk', 'primary', 'secondary', 'tertiary', 'residential', 'unclassified', 'living_street') AND bbox.xmin < -82.535 AND bbox.xmax > -82.585 AND bbox.ymin < 35.615 AND bbox.ymax > 35.565 ), nearest_road AS ( -- Exact nearest road segment to each footprint (planar distance in meters) SELECT b.id, r.road_id, r.road_name, r.road_class, ST_Distance(b.geom_m, r.g_m) AS road_m FROM b JOIN roads r ON ST_KNN(b.geom_m, r.g_m, 1, false) ), places AS ( SELECT id, ST_Transform(geometry, 'EPSG:4326', '+proj=utm +zone=17 +datum=WGS84 +units=m +no_defs') AS g_m FROM wherobots_open_data.overture_maps_foundation.places_place WHERE bbox.xmin BETWEEN -82.577 AND -82.543 AND bbox.ymin BETWEEN 35.573 AND 35.607 ), places_100m AS ( SELECT b.id, COUNT(*) AS places_100m FROM b JOIN places p ON ST_DWithin(b.geom_m, p.g_m, 100) GROUP BY b.id ), dem AS ( -- The 256 x 256 GLO-30 tiles that touch the box SELECT name AS tile, rast, footprint FROM wherobots_open_data.copernicus_dem.glo_30m WHERE ST_Intersects(footprint, ST_PolygonFromEnvelope(-82.575, 35.575, -82.545, 35.605)) ), tile_points AS ( -- Group the centroids by tile, so each tile is read once SELECT /*+ BROADCAST(t) */ t.tile, collect_list(b.id) AS ids, collect_list(b.c) AS pts FROM b JOIN dem t ON ST_Intersects(t.footprint, b.c) GROUP BY t.tile ), surface AS ( -- One RS_Values call per tile returns the surface height under every centroid in it SELECT v.id, MAX(v.surface_m) AS surface_m FROM ( SELECT /*+ BROADCAST(p) */ explode(zip_with(p.ids, RS_Values(t.rast, p.pts), (i, z) -> named_struct('id', i, 'surface_m', z))) AS v FROM tile_points p JOIN dem t ON p.tile = t.tile ) GROUP BY v.id ), ffw AS ( -- NWS flash flood warnings (FF.W) issued by the Greenville-Spartanburg office SELECT ISSUED, geometry FROM wherobots_open_data.noaa.nws_watch_warnings WHERE WFO = 'GSP' AND PHENOM = 'FF' AND SIG = 'W' ), ffw_b AS ( SELECT b.id, COUNT(*) AS ffw_count, MAX(w.ISSUED) AS ffw_last, SUM(CASE WHEN w.ISSUED >= '2024-09-25' AND w.ISSUED < '2024-09-29' THEN 1 ELSE 0 END) AS ffw_helene FROM b JOIN ffw w ON ST_Intersects(w.geometry, b.c) GROUP BY b.id ), rec AS ( SELECT b.id, b.name, ROUND(ST_X(b.c), 6) AS lon, ROUND(ST_Y(b.c), 6) AS lat, co.county, ci.locality, r.road_id, r.road_name, r.road_class, ROUND(r.road_m, 1) AS road_m, COALESCE(p.places_100m, 0) AS places_100m, ROUND(s.surface_m, 1) AS surface_m, COALESCE(w.ffw_count, 0) AS ffw_count, w.ffw_last, COALESCE(w.ffw_helene, 0) AS ffw_helene, ST_AsText(b.geometry) AS wkt FROM b LEFT JOIN county co ON b.id = co.id LEFT JOIN city ci ON b.id = ci.id LEFT JOIN nearest_road r ON b.id = r.id LEFT JOIN places_100m p ON b.id = p.id LEFT JOIN surface s ON b.id = s.id LEFT JOIN ffw_b w ON b.id = w.id ) SELECT rec.*, COUNT(*) OVER () AS buildings, SIZE(COLLECT_SET(id) OVER ()) AS distinct_ids, SUM(IF(county IS NOT NULL, 1, 0)) OVER () AS with_county, SUM(IF(locality IS NOT NULL, 1, 0)) OVER () AS in_a_locality, SUM(IF(road_m IS NOT NULL, 1, 0)) OVER () AS with_road, SUM(IF(surface_m IS NOT NULL, 1, 0)) OVER () AS with_surface, ROUND(percentile(road_m, 0.5) OVER (), 1) AS median_road_m, ROUND(percentile(places_100m, 0.5) OVER (), 1) AS median_places_100m, MIN(surface_m) OVER () AS min_surface_m, MAX(surface_m) OVER () AS max_surface_m, SUM(IF(ffw_count > 0, 1, 0)) OVER () AS under_any_ffw, SUM(IF(ffw_helene > 0, 1, 0)) OVER () AS under_helene_ffw, (SELECT COUNT(*) FROM dem) AS dem_tiles FROM rec ``` The query ran in 49.7 seconds and returned 4,408 records, one per distinct building ID, each with every field filled. All 4,408 centroids fall in Buncombe County and in the city of Asheville. The roads step reads segments up to 0.01° (about 900 m) beyond the building box, so a nearest road closer than that is exact. The median distance from a footprint to its nearest road segment is 13.1 m, and the median building has 3 places within 100 m. Surface heights read from 2 GLO-30 tiles range from 600.3 m to 759.0 m. Every one of the 4,408 buildings sits inside at least one flash flood warning polygon issued from 25 to 28 September 2024. Those were the days of Helene's rainfall in western North Carolina. A warning is a forecast of a hazard, so these counts measure warning history. Flood extent needs observed data such as high-water marks. The record for the Grove Arcade shows what the model receives: | Field | Value | | --- | --- | | Overture ID | 2dfd4c8b-0b3f-493e-9b93-0e7452081b14 | | County, city | Buncombe, Asheville | | Nearest road | O Henry Avenue (residential), 10.4 m | | Places within 100 m | 274 | | Surface height | 692.2 m (Copernicus GLO-30) | | NWS flash flood warnings covering it | 16; most recent issued 23 July 2026 | | Issued 25 to 28 September 2024 | 5 | None of those values sits in one stored row. Each comes from a join computed at query time. The road distance comes from a k nearest neighbor (KNN) join that finds each building's closest road, in meters. The place count comes from a distance join. The surface height comes from a raster read. The warning counts come from a point-in-polygon join against warning polygons. The Copernicus DEM is a surface model that includes buildings and vegetation, so 692.2 m is surface height. Ground elevation needs a terrain model or [lidar](https://wherobots.com/discover/lidar-data/) (see [digital elevation model](https://wherobots.com/discover/digital-elevation-model/)). This version keeps the building and road IDs. A production record also returns the county `GEOID` (the Census Bureau's numeric code for the county), the warning IDs, the elevation tile name, and each layer's release date. Then every value traces to a source row and a date. ![Map of 4,408 Overture building footprints in downtown Asheville colored by Copernicus GLO-30 surface height from 600 to 760 meters. The Grove Arcade footprint is highlighted in red, with a dashed line 100 meters around it. A card lists the Grove Arcade context record: its Overture ID, Buncombe County and Asheville, and nearest road O Henry Avenue at 10.4 meters. It also lists 274 places within 100 meters, surface height 692.2 meters, and 16 NWS flash flood warnings covering it, with 5 issued from 25 to 28 September 2024](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/context-engine-asheville-records.webp) *4,408 downtown Asheville buildings colored by surface height, with the Grove Arcade context record. Source: Overture Maps, TIGER/Line, Copernicus GLO-30, and National Oceanic and Atmospheric Administration (NOAA) NWS warnings in Wherobots Open Data; one query, 49.7 s.* ### The same query over a county Changing only the box coordinates runs the same query over a 0.65° by 0.43° box around Buncombe County, which includes parts of the neighboring counties: ``` -- Same query; only the box changes. Buildings: WHERE bbox.xmin >= -82.900 AND bbox.xmax <= -82.250 AND bbox.ymin >= 35.400 AND bbox.ymax <= 35.830 -- The divisions, roads, places and DEM filters move by the same offsets as above: -- divisions box -82.900 to -82.250, 35.400 to 35.830; roads -82.910 to -82.240, 35.390 to 35.840; -- places -82.902 to -82.248, 35.398 to 35.832; DEM envelope (-82.900, 35.400, -82.250, 35.830) ``` It ran in 90.5 seconds and returned 227,564 records for 227,562 distinct building IDs. The road margin stays at 0.01°, so a building whose nearest road is more than about 900 m away needs a wider one. Two IDs appear twice, so a production version adds a tie-break rule, such as the lowest county `GEOID`, to each one-to-many lookup. The query read 70 GLO-30 tiles, and surface heights range from 328.2 m to 2,019.6 m. The median distance to the nearest road is 24.3 m, and 71,630 records fall inside an Overture locality polygon. The extent grew to about 52 times as many records, and the run time grew 1.8 times. Both query plans showed the county and city lookups as broadcast R-tree joins (`BroadcastIndexJoin`). These copy the small boundary table to every worker. Both plans also showed the nearest road as a KNN join. Neither plan contained a `CartesianProduct`, a step that compares every pair of rows. One query covers the full extent, so the record has no tile seams, the edges where separately processed tiles meet. The [partitioning tax](https://wherobots.com/discover/partitioning-tax/) article shows what a tile loop would cost. ## Where physical world context can be computed The context engines above serve code and business data. Physical world context needs spatial relations computed across vector and raster data. The table compares spatial SQL systems from each product's own documentation. | System | What it serves | Vector `ST_` functions | Raster in SQL | Execution | | --- | --- | --- | --- | --- | | PostGIS 3.6 | Rows in a PostgreSQL database | 309 | Yes, PostGIS raster | One PostgreSQL server; Citus shards tables | | DuckDB spatial | In-process tables | 157 | Not documented | In-process, one machine | | BigQuery | `GEOGRAPHY` columns | Not counted | Through Earth Engine with `ST_REGIONSTATS` | Distributed, serverless | | Snowflake | `GEOGRAPHY` and `GEOMETRY` columns | Not counted | Not documented | Virtual warehouses | | WherobotsDB and RasterFlow | Iceberg tables in Havasu, served over SQL and an MCP server | 277 | Yes, 107 `RS_` functions; RasterFlow (Public Preview) for imagery inference | Distributed runtimes, Micro to 4X-Large | Sources: the [PostGIS reference](https://postgis.net/docs/reference.html) and [Citus](https://www.citusdata.com/product/community); the [DuckDB spatial function reference](https://duckdb.org/docs/stable/core_extensions/spatial/functions); [BigQuery raster data](https://docs.cloud.google.com/bigquery/docs/raster-data); [Snowflake geospatial data types](https://docs.snowflake.com/en/sql-reference/data-types-geospatial); the [WherobotsDB reference](https://docs.wherobots.com/reference/wherobots-db/geometry-data/overview) and [raster functions](https://docs.wherobots.com/reference/wherobots-db/raster-data/overview). Function counts are distinct names in each official reference on 6 October 2026, as counted in the partitioning tax article. "Not documented" means the linked documentation describes no such capability. ## Context engine research and what changes at scale ### From retrieval to context engines The idea grew out of retrieval. Patrick Lewis and colleagues paired a language model with a searchable index in [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) (NeurIPS 2020). Nelson Liu and colleagues then published [Lost in the Middle](https://aclanthology.org/2024.tacl-1.9/) (Transactions of the Association for Computational Linguistics, 2024). It showed that accuracy drops when the relevant passage sits in the middle of a long input. A longer context window therefore does not replace selection. GraphRAG (2024) moved part of the work into a graph built ahead of time. The Model Context Protocol (2024) standardized how agents call data systems, and Anthropic's 2025 article named the practice of context engineering. In March 2026, Cui and Li framed the next step as infrastructure. They traced agent failures to missing business context. They proposed a context layer to supply it. Each step moved more work out of the model and into a system that prepares its input. ### Spatial context and knowledge graphs Tobler's first law (1970) and Egenhofer and Franzosa's relations (1991) supplied the theory. The OGC standards turned it into shared vocabularies. Simple Feature Access covers geometry and predicates. [GeoSPARQL 1.1](https://docs.ogc.org/is/22-047r1/22-047r1.html) (2024) covers geospatial data in Resource Description Framework (RDF) knowledge graphs. Aidan Hogan and colleagues' survey [Knowledge Graphs](https://doi.org/10.1145/3447772) (ACM Computing Surveys, 2021) covers the graph models such engines build on. Krzysztof Janowicz, Song Gao, Grant McKenzie, Yingjie Hu, and Budhendra Bhaduri argued that models for geographic problems need location, distance, and spatial relations built in. They made this argument in [GeoAI: spatially explicit artificial intelligence techniques for geographic knowledge discovery and beyond](https://doi.org/10.1080/13658816.2019.1684500) (IJGIS, 2020). KnowWhereGraph (2022) serves geographic context as linked data. Dazhou Yu, Riyang Bao, Gengchen Mai, and Liang Zhao's [Spatial-RAG](https://arxiv.org/abs/2502.18470) (2025) pairs a spatial database filter with semantic ranking to answer geospatial questions. The [Spatial Graph RAG](https://wherobots.com/blog/spatial-graph-rag/) post applies the graph approach to Overture data in Wherobots. ### What scale changes The county run repeated one building's handful of joins for 227,564 records. Distributed spatial partitioning and per-partition indexes, the design Yu, Zhang, and Sarwat describe for GeoSpark, keep that to one query per question. [Geospatial AI](https://wherobots.com/discover/geospatial-ai/) and [physical AI](https://wherobots.com/discover/physical-ai/) workloads add imagery, and with it more raster data. ### Open problems - **Choosing the radius.** The 100 m place count is a parameter of the question, and no source cited here gives a rule for it. - **Conflicting sources.** Overture, county records, and an owner's own data can disagree on a building's footprint or use. A record that keeps each value with its source shows the conflict. - **Time.** Most of the layers above are snapshots. Questions about change need versioned sources and a date on every fact. - **Evaluation.** Benchmarks test models on map questions and spatial puzzles. Few test whether an agent requested the right context for a real place and used it correctly. ## Conclusions The Asheville walkthrough shows the job of a context engine for physical world data. It computes each fact at query time from several sources and returns it with IDs a model can cite. 1. A context engine comes before prompt tuning. In context engineering, a developer chooses what an agent requests at each step. The context engine computes and serves it. 2. For physical world questions, a context engine computes relations at query time. The Asheville record's county, nearest road, place count, surface height, and warning history each came from a join on geometry. None sits in a stored row. 3. A context engine for physical world data does the geometry in one engine, matches coordinate systems within each operation, and passes the model meters and IDs. The downtown query measured distances in Universal Transverse Mercator (UTM) zone 17N. It ran the county, warning, and elevation lookups in longitude and latitude. It kept a stable Overture ID per building, so the building and its nearest road can be fetched again. 4. In this test, the context engine's output grew faster than its run time. The same SQL returned 4,408 records in 49.7 s and 227,564 in 90.5 s: about 52 times the records for 1.8 times the time, with no tile loop. 5. A context engine record carries each source's date and treats warnings as forecasts. The Asheville record counts NWS warnings issued from 25 to 28 September 2024. Flood extent needs observed data such as high-water marks. 6. If the question needs a radius, such as the 100 m place count, the context engine states it in the record. It does so because no source cited here gives a rule for choosing it. ## Related terms - [Context engineering](https://wherobots.com/discover/context-engineering/): the practice of choosing what a model receives at each step - [Physical AI](https://wherobots.com/discover/physical-ai/): AI that perceives, reasons about, and acts in the physical world - [Ontology](https://wherobots.com/discover/ontology/): the entity and relation types a context engine models - [Spatial join](https://wherobots.com/discover/spatial-join/): the operation that computes spatial relations between datasets - [Geospatial AI](https://wherobots.com/discover/geospatial-ai/): machine learning and AI agents applied to spatial data - [Havasu catalog](https://wherobots.com/discover/havasu-catalog/): the Iceberg tables behind the walkthrough - [Partitioning tax](https://wherobots.com/discover/partitioning-tax/): the cost of splitting spatial data into tiles - [Digital elevation model](https://wherobots.com/discover/digital-elevation-model/): surface and terrain models such as Copernicus GLO-30 ## Read more from Wherobots - [Spatial Graph RAG for the physical world](https://wherobots.com/blog/spatial-graph-rag/) - [Introducing developer tools that let AI build with physical world data](https://wherobots.com/blog/spatial-ai-coding-tools/) - [Wherobots for QGIS: eliminating the partitioning tax](https://wherobots.com/blog/spatial-sql-qgis/) - [Raster processing at scale with the out-of-database architecture](https://wherobots.com/blog/raster-processing-at-scale-out-of-database-architecture-wherobotsdb/) Build context records for your own buildings and assets with a Wherobots free trial at [cloud.wherobots.com](https://cloud.wherobots.com). ## Frequently asked questions ### What is a context engine? A context engine is the system that prepares the information an AI model or agent receives at run time. It ingests sources and models them as entities with stable IDs. It indexes them, computes the relations a question needs, and serves the result through an API or tool call. It keeps that result current as sources change. Jason Cui and Jennifer Li of Andreessen Horowitz describe the same need for data agents in [Your Data Agents Need Context](https://a16z.com/your-data-agents-need-context/) (March 2026). ### What does a context engine do for AI agents? It gives an agent a compact, sourced answer to the part of a question that lives in data. That part covers which records match, how they relate, and when each source was last updated. The agent calls the engine as a tool, often over the Model Context Protocol. It receives a small record in place of raw tables or documents. The prompt stays short, and every fact traces to an ID. ### What is the difference between a context engine and context engineering? Context engineering is the practice of choosing which tokens a model receives at each step: instructions, tools, retrieved data, memory, and history. A context engine is the infrastructure that computes, stores, and serves the data part of that context. See [What is context engineering for physical AI?](https://wherobots.com/discover/context-engineering/) ### Is a context engine the same as RAG? No. Retrieval-augmented generation (RAG) searches an index for passages similar to a question and adds them to the prompt. A context engine can include that retrieval step, and it also joins sources, resolves entities, computes derived values such as distances or containment, and keeps the results current. RAG returns stored text. A context engine can return facts that no stored document contains. ### What is the difference between a context engine and a context layer? The terms overlap. A context layer usually names the place in an architecture between data sources and AI agents, and a context engine names the system that does the work in that place. Cui and Li of Andreessen Horowitz describe the context layer for data agents as a superset of the semantic layer. It adds canonical entities, identity resolution, tribal knowledge, and governance guidance to metric definitions. ### How does a context engine use MCP? The Model Context Protocol (MCP), which Anthropic introduced in November 2024, is an open standard that lets AI applications call external tools and read resources over JSON-RPC. A context engine exposes its queries as MCP tools, so any MCP client, such as a coding assistant or an agent framework, can request context with one tool call. The Wherobots MCP server, for example, offers tools to list catalogs and tables, describe a table, and run spatial SQL. ### What are examples of context engines? Code context engines, such as Augment Code’s Context Engine, index a repository and serve relevant code to coding agents over MCP. Streaming and operational context engines, such as Confluent’s Real-Time Context Engine and Materialize, keep derived business objects current as events arrive. Geo-enrichment services such as KnowWhereGraph return linked facts about a region. Wherobots computes spatial relations across vector and raster data for physical world questions. ### Why does physical world data need a context engine? Language models were trained on text, documents, databases, and the internet. Those sources rarely state which building sits inside which flood warning, how far it is from the nearest road, or what the surface height is under it. Those facts are relations between shapes and raster cells. They are computed in a known coordinate system from sources that update on different cycles. A context engine for physical world data computes them at query time and returns them with IDs and sources. ### How does a context engine use satellite imagery and computer vision? Imagery records physical world facts that tables often lack, such as new construction, cleared land, or flood extent. Computer vision inference turns those pixels into features a query can use: building outlines, land cover classes, or detected objects, each with a location. In Wherobots, [RasterFlow](https://docs.wherobots.com/develop/rasterflow) (Public Preview) builds mosaics from imagery, runs computer vision inference on them, and writes the results as vectorized outputs on the same platform. See [AI satellite imagery analysis](https://wherobots.com/discover/satellite-imagery-analysis/) and [land cover classification](https://wherobots.com/discover/land-cover-classification/). ### What is the Context Engine for Physical World Data? It is how Wherobots describes its platform: a context engine whose sources are physical world data, such as buildings, places, roads, boundaries, elevation models, imagery, and weather warnings. WherobotsDB computes the spatial relations in distributed SQL over vector and raster data. RasterFlow turns imagery into mosaics, inference results, and vector outputs. The Havasu catalog stores the tables in Apache Iceberg. The Wherobots MCP server lets AI agents find tables and run queries.