--- title: "What is the Havasu Catalog? A Spatial Data Catalog" url: "https://wherobots.com/discover/havasu-catalog/" description: "The Havasu Catalog is the Wherobots spatial data catalog: open data in Apache Iceberg tables with geometry types, time travel, and spatial pruning." published: "2026-10-04" modified: "2026-10-21" --- # What is the Havasu Catalog? A Spatial Data Catalog The Havasu Catalog is the Wherobots spatial data catalog. It indexes open geospatial datasets, RasterFlow computer vision models, solution notebooks, and demo apps. Each dataset is an Apache Iceberg table that WherobotsDB queries in place with spatial SQL. The name comes from Havasu, the table format Wherobots built to add geometry and raster columns, spatial statistics, and spatial filter pushdown to Iceberg before Iceberg had native geospatial types. This article tests one claim: the table metadata serves as the index. Snapshots list a table's files at each commit. Manifests, the metadata files that list data files, record each file's extent. So a query for one city or one past release is narrowed to its files before any data file opens. ## Key takeaways - A spatial data catalog adds geometry and raster types, a coordinate reference system (CRS) per column, and per-file spatial extents to a data catalog. - Havasu extends the Apache Iceberg table spec with `geometry` and `raster` types and bounding boxes in the manifest files. Iceberg format version 3 now defines native `geometry` and `geography` types. Wherobots upgrades its open data tables to them automatically. - The [Havasu Catalog](https://wherobots.com/havasu/) lists 63 entries: 42 datasets, 6 RasterFlow models, 9 solution notebooks, and 6 demo apps. - Every table carries Iceberg snapshots, so a query can read an earlier Overture Maps release with `VERSION AS OF`. Metadata tables expose snapshots, tags, and data files to SQL. - A Lake Havasu City window overlaps 3 of the 5,141 data files in the Overture buildings table, so a spatial filter skips the rest before reading any data. ## What is a spatial data catalog? A data catalog maps a table name such as `wherobots_open_data.overture_maps_foundation.buildings_building` to its files, their schema, and the history of changes. A spatial data catalog adds spatial types, a coordinate reference system per column, and spatial extents: - **Spatial types.** Geometry columns for points, lines, and polygons, and raster columns for imagery and elevation, stored as typed values. - **A coordinate reference system per column.** The [coordinate reference system](https://wherobots.com/discover/coordinate-reference-system/) maps coordinates to places on Earth, so tables in different systems are never compared by accident. - **Spatial extents.** The bounding box of every data file, recorded in metadata, so a query for one city opens only the files whose boxes touch that city. Asset catalogs such as [STAC](https://docs.ogc.org/cs/25-004/25-004.html), the SpatioTemporal Asset Catalog, describe files, mostly satellite scenes, with JSON records of footprint, time, and band links. Table catalogs such as Iceberg describe rows in analytic tables. The Havasu Catalog is a table catalog that can hold asset catalogs as tables: the [Sentinel-2 L2A scene index](https://wherobots.com/havasu/#l2a_source_items) is a table of STAC items, one row per scene. L2A stands for Level-2A. ## How the Havasu Catalog works Each catalog name points to an Iceberg table whose metadata and Parquet files sit in cloud object storage. ### Catalogs, databases, and tables Each catalog holds databases, each database holds tables, and SQL names a table as `..` ([querying datasets in the Wherobots docs](https://docs.wherobots.com/develop/spatial-catalog/spatial-catalog)). Wherobots Cloud provides `wherobots_open_data` to every organization edition, `wherobots_pro_data` to paid editions, and an organization catalog for a team's own tables. A team can also attach tables through Unity Catalog, AWS Glue, or any Iceberg catalog, so one query can join private and open data. ### Havasu adds geometry and raster columns to Iceberg [Apache Iceberg](https://iceberg.apache.org/) is an open table format for large analytic tables. Its [table spec](https://iceberg.apache.org/spec/) sets goals the Havasu Catalog inherits. Each read sees one committed snapshot. Each write adds and removes files atomically. Columns can be added, dropped, reordered, and renamed. The [Havasu table spec](https://github.com/wherobots/havasu) (version 0.1.0) adds spatial types, raster storage modes, spatial statistics, and spatial transforms to Iceberg format versions 1 and 2. It leaves the rest of Iceberg unchanged: - **Primitive spatial types.** `geometry` follows the [OGC Simple Features standard](https://www.ogc.org/standard/sfa/) from the Open Geospatial Consortium (OGC). It can be stored in Parquet as well-known binary (WKB), extended WKB (EWKB), well-known text (WKT), or GeoJSON, four standard geometry encodings. `raster` stores a grid of cell values, the affine transform that places it on Earth, and its CRS. - **Raster storage modes.** Band values can sit in the table (in-db) or in external files such as Cloud Optimized [GeoTIFFs](https://wherobots.com/discover/geotiff/) (out-db). An out-db row holds only the file's Uniform Resource Identifier (URI), its address, and the geo-referencing. - **Spatial statistics.** Each data file's entry in the manifest records the minimum bounding rectangle of its geometry or raster values. Raster bounds are always stored in WGS 84, the World Geodetic System 1984. - **Spatial transforms.** Functions that partition or cluster rows by location. Havasu supports [ACID transactions, schema evolution, and time travel](https://docs.wherobots.com/reference/havasu-table/introduction) on geometries and rasters, and [every Iceberg feature except merge-on-read tables](https://docs.wherobots.com/reference/havasu/geometry/geometry-overview). ACID stands for atomic, consistent, isolated, and durable. In WherobotsDB, a `geometry(4326)` column is an Iceberg v3 native column with one enforced CRS. Other Iceberg engines with geospatial support read it as geometry. A `geography(4326)` column interprets coordinates on a sphere. Older columns without a declared CRS work the same in WherobotsDB, and other engines read them as binary. Wherobots [upgrades the read-only public tables it manages](https://docs.wherobots.com/reference/havasu/geometry/upgrade-geometry-tables), such as `wherobots_open_data`, automatically. ### Snapshots, time travel, and metadata tables Every commit to an Iceberg table writes a new snapshot, the full list of its data files at that moment. Older snapshots stay readable until they expire, and named references (tags and branches) point at particular snapshots. Wherobots keeps every stable Overture release in the catalog as a tag. `VERSION AS OF '2025-05-21.0'` reads the places table as Overture published it on 21 May 2025. A query with no version reads the latest stable release. Metadata tables, named with a suffix on the table name, expose this history to SQL. `.snapshots` lists commits with a summary of row and file counts, `.refs` lists tags and branches, and `.files` lists every data file with its row count and column statistics. ### Spatial filter pushdown [Spatial filter pushdown](https://docs.wherobots.com/reference/havasu-table/geometry-data/cluster-geometry-table) runs before any data file opens. WherobotsDB compares each file's bounding box in the manifest with the query window and drops files whose box misses it. The Havasu spec calls this test an inclusive projection: `ST_Intersects(geom, Q)` becomes `ST_Intersects(MBR[geom], Q)` per file, where MBR is the minimum bounding rectangle. The test can keep a file with no match and never drops a file with one. Rasters get the same pushdown with `RS_Intersects`, `RS_Contains`, and `RS_Within`. Pruning only helps when nearby rows share files. `CREATE SPATIAL INDEX FOR
USING hilbert(geom, )` rewrites a table in Hilbert curve order, an ordering of grid cells that keeps cells close in the order close on the map. Each file then covers a compact area with a small bounding box. ![Diagram of the five layers behind one Havasu Catalog table. The top two are the catalog name wherobots_open_data.overture_maps_foundation.buildings_building and table metadata with schema ID 7. Below them sit 60 snapshots from 22 April 2025 to 26 September 2026 and manifests with per-file bounding boxes. At the bottom, 5,141 Parquet data files hold 2,533,842,612 rows and 253 GB](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/havasu-catalog-table-layers.webp) *The layers behind the Overture buildings table in the Havasu Catalog, from its snapshot and file metadata: 60 snapshots and 5,141 Parquet files holding 2,533,842,612 rows.* ## Havasu Catalog examples [Vector and raster](https://wherobots.com/discover/raster-vs-vector-data/) datasets are the largest of the catalog's entry types: | Entry type | Count | Examples | | --- | --- | --- | | Datasets | 42 | [Overture buildings](https://wherobots.com/havasu/#buildings_building), [Overture places](https://wherobots.com/havasu/#places_place), [USDA NAIP aerial imagery](https://wherobots.com/havasu/#naip), [Sentinel-2 seasonal mosaics](https://wherobots.com/havasu/#sentinel2), [Copernicus DEM GLO-30](https://wherobots.com/havasu/#glo_30m) | | RasterFlow models | 6 | [Fields of the World](https://wherobots.com/havasu/#fields-of-the-world), Tile2Net, Meta Canopy Height Model (CHM) v1, ChesapeakeRSC, SAM3 (Segment Anything Model 3), bring your own model | | Solution notebooks | 9 | Detecting field boundaries with RasterFlow, [Reading STAC data](https://wherobots.com/havasu/#stac-reader), Spatial Joins | | Demo apps | 6 | [Colorado property risk explorer](https://wherobots.com/havasu/#co-risk-app), California grid wildfire exposure | Dataset sources include the [Overture Maps Foundation](https://overturemaps.org/), the US Census Bureau, [OpenStreetMap](https://www.openstreetmap.org/about), and the European Space Agency (ESA) Copernicus [Sentinel-2](https://wherobots.com/discover/sentinel-2/). Other sources include the U.S. Department of Agriculture (USDA) National Agriculture Imagery Program (NAIP) and Google and Microsoft [building footprints](https://wherobots.com/discover/building-footprint/). Examples by table type: - **Overture buildings and places.** Vector tables with a `geometry` column and a `bbox` struct per row, tagged by Overture release. [Places](https://wherobots.com/discover/poi-data/) carry names, categories, and confidence scores. - **Copernicus DEM GLO-30.** DEM stands for digital elevation model. This is a raster table of elevation tiles, each row a tile with its footprint. GLO-30 is a [digital surface model](https://wherobots.com/discover/digital-elevation-model/), so its heights include roofs and trees. - **Sentinel-2 L2A scene index.** STAC items as rows: footprint, acquisition time, cloud cover, and links to Cloud Optimized GeoTIFF bands that WherobotsDB reads as out-db rasters. - **RasterFlow outputs.** Fields of the World boundaries, Tile2Net sidewalks, and ChesapeakeRSC roads are outputs of the [RasterFlow models](https://docs.wherobots.com/develop/rasterflow/rasterflow-models), stored as vector tables. [RasterFlow](https://wherobots.com/products/ai-for-earth/) runs them over NAIP, Sentinel-2, or a team's own imagery. ## Havasu vs GeoParquet, Iceberg v3, and STAC GeoParquet, Iceberg v3, Havasu, and STAC each describe spatial data in cloud storage, at a different unit: | | GeoParquet 1.1 | Iceberg v3 geo types | Havasu 0.1.0 | STAC 1.1 | | --- | --- | --- | --- | --- | | Unit described | One Parquet file | A table of many files | A table of many files | A catalog of assets (scenes, files) | | Geometry encoding | WKB, or native GeoArrow encodings | WKB, `geometry(C)` and `geography(C, A)` | WKB, EWKB, WKT, or GeoJSON | GeoJSON footprints in JSON records | | Raster | No | No | Yes, in-db and out-db | Links to raster assets | | CRS | PROJJSON in file metadata, default OGC:CRS84 | Column parameter, default OGC:CRS84 | SRID per value | Projection extension | | Spatial statistics | File `bbox` and optional per-row bbox covering column | Per-file bounding box in manifests | Per-file bounding box in manifests | Item and collection extents | | Transactions and snapshots | No | Yes | Yes | No | | Governed by | OGC Standards Working Group | Apache Software Foundation | Wherobots (open spec) | OGC Community Standard | [GeoParquet 1.1](https://geoparquet.org/releases/v1.1.0/), released in June 2024, added a bounding box covering column and GeoArrow-based encodings, which let readers use Parquet row-group statistics. The [Parquet geospatial definitions](https://github.com/apache/parquet-format/blob/master/Geospatial.md) added `GEOMETRY` and `GEOGRAPHY` logical types in Parquet format 2.11 in March 2025, which Iceberg v3 files use. STAC became an [OGC Community Standard](https://www.ogc.org/announcement/ogc-announces-publication-of-the-spatiotemporal-asset-catalog-community-standards/) in 2025, with the core specification at version 1.1.0. The formats stack: an Iceberg v3 table is a set of Parquet files with native geospatial types. ## The Havasu Catalog in Wherobots This walkthrough runs read-only SQL in WherobotsDB for one place: Lake Havasu City, Arizona, inside the box from 114.38° W to 114.28° W and 34.43° N to 34.55° N. Each table is an Iceberg table of Parquet files: | Source | Size (latest snapshot) | What the walkthrough read | | --- | --- | --- | | `wherobots_open_data` catalog | 11 databases | The database list | | Overture `buildings_building` | 2,533,842,612 rows, 5,141 files, 253.2 GB | Snapshot summaries, per-file bounding boxes, and buildings in the Lake Havasu City box | | Overture `places_place` | 81,455,423 rows, 189 files, 9.5 GB | Snapshot summaries, release tags, and places in the box in ten releases | | Overture `transportation_segment` | 352,054,710 rows, 1,393 files, 68.5 GB | Snapshot summaries only | | Census `tiger_county` | 3,235 rows, 2 files, 66.7 MB | Snapshot summaries only | ### List the databases ``` SHOW SCHEMAS IN wherobots_open_data ``` The open data catalog holds 11 databases: `aster_gdem`, `copernicus_dem`, `foursquare`, `meta_canopy_height`, `noaa`, `overture_maps_foundation`, `partner_samples`, `rasterflow_output_samples`, `sentinel2`, `spatial_knowledge_graph`, and `us_census`. ### Count rows and files from snapshot metadata The `.snapshots` metadata table returns each commit with a summary. The latest summary gives row count, file count, and size without reading a data file. ``` -- Latest snapshot summary of four open data tables, read from Iceberg metadata only WITH s AS ( SELECT 'overture buildings_building' AS tbl, committed_at, summary FROM wherobots_open_data.overture_maps_foundation.buildings_building.snapshots UNION ALL SELECT 'overture places_place', committed_at, summary FROM wherobots_open_data.overture_maps_foundation.places_place.snapshots UNION ALL SELECT 'overture transportation_segment', committed_at, summary FROM wherobots_open_data.overture_maps_foundation.transportation_segment.snapshots UNION ALL SELECT 'us_census tiger_county', committed_at, summary FROM wherobots_open_data.us_census.tiger_county.snapshots ) SELECT tbl, COUNT(*) AS snapshots, MIN(committed_at) AS first_commit, MAX(committed_at) AS last_commit, MAX_BY(summary['total-records'], committed_at) AS total_records, MAX_BY(summary['total-data-files'], committed_at) AS data_files, MAX_BY(summary['total-files-size'], committed_at) AS bytes FROM s GROUP BY tbl ORDER BY tbl ``` | Table | Snapshots | Rows | Data files | Size | | --- | --- | --- | --- | --- | | Overture `buildings_building` | 60 | 2,533,842,612 | 5,141 | 253.2 GB | | Overture `places_place` | 60 | 81,455,423 | 189 | 9.5 GB | | Overture `transportation_segment` | 60 | 352,054,710 | 1,393 | 68.5 GB | | Census `tiger_county` | 2 | 3,235 | 2 | 66.7 MB | The query ran in 4.8 seconds. The buildings table's 60 snapshots run from 22 April 2025 to 26 September 2026 (UTC). ### Read earlier Overture releases with time travel `SELECT name, type, snapshot_id FROM wherobots_open_data.overture_maps_foundation.places_place.refs ORDER BY type, name` returns 31 references: the `main` branch and 30 Overture release tags, from `2024-07-22.0` to `2026-09-23.1`. `main` points at the same snapshot as `2026-09-23.1`, the newest release. This query counts places in the Lake Havasu City box in ten releases: ``` -- Places inside the Lake Havasu City box in ten Overture releases, read with time travel SELECT '2024-07-22.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2024-07-22.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2024-10-23.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2024-10-23.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2025-01-22.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2025-01-22.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2025-04-23.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2025-04-23.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2025-07-23.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2025-07-23.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2025-10-22.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2025-10-22.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2026-01-21.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2026-01-21.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2026-04-15.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2026-04-15.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2026-07-22.0' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2026-07-22.0' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) UNION ALL SELECT '2026-09-23.1' AS release, COUNT(*) AS places FROM wherobots_open_data.overture_maps_foundation.places_place VERSION AS OF '2026-09-23.1' WHERE bbox.xmin <= -114.28 AND bbox.xmax >= -114.38 AND bbox.ymin <= 34.55 AND bbox.ymax >= 34.43 AND ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) ORDER BY release ``` ![Bar chart of Overture places inside the Lake Havasu City box in ten releases read with VERSION AS OF. The counts are 2,341 in July 2024, 2,552 in October 2024 and January 2025, 2,664 in April 2025, and 2,600 in July 2025. They are 2,966 in October 2025, 3,015 in January 2026, 3,059 in April 2026, 2,963 in July 2026, and 3,606 in September 2026](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/havasu-catalog-time-travel.webp) *One window, ten Overture releases, one query: places in the Lake Havasu City box grew from 2,341 in July 2024 to 3,606 in September 2026.* All ten versions came back in 8.9 seconds. The count rose from 2,341 in the July 2024 release to 3,606 in September 2026, with dips in July 2025 (2,600) and July 2026 (2,963). ### Join Overture places to building footprints This join counts places inside building footprints in Lake Havasu City by category, on the current release. ``` WITH area AS (SELECT ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55) AS g) SELECT p.basic_category, COUNT(DISTINCT p.id) AS places_in_buildings FROM wherobots_open_data.overture_maps_foundation.places_place p JOIN wherobots_open_data.overture_maps_foundation.buildings_building b ON ST_Contains(b.geometry, p.geometry) CROSS JOIN area WHERE ST_Intersects(p.geometry, area.g) AND ST_Intersects(b.geometry, area.g) AND p.basic_category IS NOT NULL GROUP BY p.basic_category ORDER BY places_in_buildings DESC LIMIT 5 ``` | basic_category | places_in_buildings | | --- | --- | | home_service | 181 | | real_estate_service | 165 | | personal_or_beauty_service | 116 | | restaurant | 92 | | financial_service | 92 | It returned these counts in 10.1 seconds. `COUNT(DISTINCT p.id)` counts a place once where two footprints overlap. ### See spatial pruning in the plan and the file list `EXPLAIN FORMATTED` prints where the spatial filter goes: ``` EXPLAIN FORMATTED SELECT COUNT(*) AS buildings FROM wherobots_open_data.overture_maps_foundation.buildings_building WHERE ST_Intersects(geometry, ST_PolygonFromEnvelope(-114.38, 34.43, -114.28, 34.55)) ``` The scan node of the plan reads: ``` IcebergScan(table=wherobots_open_data.overture_maps_foundation.buildings_building, schemaId=7, snapshotId=461598865520318774, branch=null, filters=st_intersects(geometry, POLYGON ((-114.38 34.43, -114.38 34.55, -114.28 34.55, -114.28 34.43, -114.38 34.43))), ...) ``` The `ST_Intersects` predicate sits inside the Iceberg scan, so it filters files before the scan reads them. An exact `ST_PreparedIntersects` filter then runs on the rows that remain. `schemaId=7` means the table schema has changed several times since creation, and every snapshot still reads. The `.files` metadata table shows how many files the window leaves. Iceberg keeps the minimum and maximum of each Overture table's `bbox` struct column per file: ``` -- Data files of the Overture buildings table whose bbox column statistics overlap the Lake Havasu City box SELECT COUNT(*) AS data_files, SUM(record_count) AS rows_in_table, SUM(CASE WHEN readable_metrics.`bbox.xmin`.lower_bound <= -114.28 AND readable_metrics.`bbox.xmax`.upper_bound >= -114.38 AND readable_metrics.`bbox.ymin`.lower_bound <= 34.55 AND readable_metrics.`bbox.ymax`.upper_bound >= 34.43 THEN 1 ELSE 0 END) AS files_overlapping_box, SUM(CASE WHEN readable_metrics.`bbox.xmin`.lower_bound <= -114.28 AND readable_metrics.`bbox.xmax`.upper_bound >= -114.38 AND readable_metrics.`bbox.ymin`.lower_bound <= 34.55 AND readable_metrics.`bbox.ymax`.upper_bound >= 34.43 THEN record_count ELSE 0 END) AS rows_in_overlapping_files FROM wherobots_open_data.overture_maps_foundation.buildings_building.files ``` Of 5,141 files holding 2,533,842,612 buildings, 3 files overlap the box. They hold 1,576,641 rows. The query read only manifests, in 4.6 seconds. A second query lists the per-file boxes for the US Southwest, 125° W to 102° W and 30° N to 42° N. It returns 84 files: ``` -- Per-file bounding boxes of Overture buildings files that overlap the US Southwest (125 W to 102 W, 30 N to 42 N) SELECT readable_metrics.`bbox.xmin`.lower_bound AS file_xmin, readable_metrics.`bbox.xmax`.upper_bound AS file_xmax, readable_metrics.`bbox.ymin`.lower_bound AS file_ymin, readable_metrics.`bbox.ymax`.upper_bound AS file_ymax, record_count, (readable_metrics.`bbox.xmin`.lower_bound <= -114.28 AND readable_metrics.`bbox.xmax`.upper_bound >= -114.38 AND readable_metrics.`bbox.ymin`.lower_bound <= 34.55 AND readable_metrics.`bbox.ymax`.upper_bound >= 34.43) AS overlaps_lake_havasu_box FROM wherobots_open_data.overture_maps_foundation.buildings_building.files WHERE readable_metrics.`bbox.xmin`.lower_bound <= -102 AND readable_metrics.`bbox.xmax`.upper_bound >= -125 AND readable_metrics.`bbox.ymin`.lower_bound <= 42 AND readable_metrics.`bbox.ymax`.upper_bound >= 30 ``` ![Map of the US Southwest from 125 to 102 degrees west and 30 to 42 degrees north with the bounding boxes of 84 Overture buildings data files drawn as rectangles. Small boxes cluster along the California coast and larger boxes cover the desert. A small red query window at Lake Havasu City overlaps three file boxes, outlined in amber: one covering southern Nevada and western Arizona, and two that extend beyond the map](https://wherobots.com/wp-content/themes/wherobots/assets/img/discover/havasu-catalog-file-pruning.webp) *Per-file bounding boxes of the Overture buildings table over the US Southwest. 84 files overlap the map, and 3 of the table’s 5,141 files overlap the Lake Havasu City window.* File boxes are small where buildings are dense, along the California coast, and large over the desert. One of the three overlapping files spans 117.36° W to 112.43° W, around the lower Colorado River. The other two have continental boxes, so a query almost anywhere in North America opens them. One runs from 179.80° W to 90.10° W and 23.87° N to 79.42° N. ## Research and what changes at scale The Havasu Catalog builds on transactional table formats for object storage and on data skipping, which passes over files that cannot match a query. ### From data lakes to lakehouses Iceberg began at Netflix to replace Hive tables. On Amazon S3, Hive directory listings were slow and inconsistent, and commits were not atomic. Ryan Blue presented the design in [Introducing Iceberg: Tables designed for object stores](https://conferences.oreilly.com/strata/strata-ny-2018/public/schedule/detail/69503.html) at Strata Data New York in September 2018. Iceberg [entered the Apache Incubator](https://incubator.apache.org/projects/iceberg.html) in November 2018 and became a top-level Apache project in May 2020. Tracking every data file in metadata makes snapshots, atomic commits, and file-level statistics possible. Michael Armbrust and colleagues at Databricks described the same move for Delta Lake in [Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores](https://doi.org/10.14778/3415478.3415560) (PVLDB, 2020). A transaction log over Parquet files provides ACID properties, time travel, and fast metadata operations. Armbrust, Ali Ghodsi, Reynold Xin, and Matei Zaharia then named the pattern in [Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics](https://www.cidrdb.org/cidr2021/papers/cidr2021_paper17.pdf) (CIDR 2021). A lakehouse keeps open file formats under a transactional metadata layer and serves SQL analytics and machine learning from one copy of the data. Paras Jain, Peter Kraft, Conor Power, Tathagata Das, Ion Stoica, and Zaharia compared Delta Lake, Hudi, and Iceberg designs in [Analyzing and Comparing Lakehouse Storage Systems](https://www.cidrdb.org/cidr2023/papers/p92-jain.pdf) (CIDR 2023). They also released the LHBench benchmark. ### Spatial data on the lakehouse None of those formats had spatial types. Jia Yu, Jinxuan Wu, and Mohamed Sarwat introduced distributed spatial processing on Apache Spark with [GeoSpark](https://doi.org/10.1145/2820783.2820860) (ACM SIGSPATIAL 2015) at Arizona State University. Yu, Zongsi Zhang, and Sarwat described its spatial partitioning, indexing, and join design in [Spatial data management in Apache Spark: the GeoSpark perspective and beyond](https://doi.org/10.1007/s10707-018-0330-9) (GeoInformatica, 2019). GeoSpark became Apache Sedona, which the [Apache Software Foundation announced as a top-level project](https://news.apache.org/foundation/entry/the-apache-software-foundation-announces-new-top-level-project-apache-sedona) in February 2023. The creators of Sedona founded Wherobots, and [WherobotsDB](https://wherobots.com/products/compute/) is fully code compatible with [Apache Sedona](https://wherobots.com/apache-sedona/) across all spatial functions. [Read the Apache Sedona story](https://wherobots.com/blog/what-is-apache-sedona/). Storage was the next gap. Wherobots began work on Havasu in 2022 and [announced the format](https://wherobots.com/blog/havasu-a-table-format-for-spatial-attributes-in-a-data-lake-architecture/) in December 2023. In parallel, the GeoParquet project, started in 2022, standardized geometry in single Parquet files and reached 1.0.0 in 2023. In early 2024, the spatial and Iceberg communities began adding geospatial types to Iceberg [with the Havasu design as a reference](https://wherobots.com/blog/iceberg-geo-technical-insights-and-implementation-strategies/). Contributors came from Wherobots, CARTO, Planet, Apple, Databricks, Snowflake, and others. In February 2025, Wherobots [announced that Iceberg and Parquet support geo types](https://wherobots.com/blog/apache-iceberg-and-parquet-now-support-geo/). The Iceberg community voted to adopt the v3 spec in May 2025, and the spec now lists versions 1, 2, and 3 as complete and adopted. [Read how Iceberg v3 geospatial types work](https://wherobots.com/blog/iceberg-geospatial-types-v3/). With spatial types native to Iceberg, Wherobots moved the Havasu name to its catalog. The Wherobots Spatial Data Catalog, which held datasets such as [Overture Maps](https://wherobots.com/blog/overture-data-in-wherobots-spatial-data-catalog/), became the Havasu Catalog and widened to RasterFlow models, notebooks, and apps. ### Data skipping and spatial layout File-level bounding boxes descend from two older ideas. Guido Moerkotte's [Small Materialized Aggregates](https://www.vldb.org/conf/1998/p476.pdf) (VLDB 1998) stored the minimum and maximum of each column per block of rows, so a scan could skip blocks that cannot match. Zone maps and Parquet statistics use the same idea. Antonin Guttman's [R-trees: a dynamic index structure for spatial searching](https://doi.org/10.1145/602259.602266) (SIGMOD 1984) organized geometries by nested minimum bounding rectangles. A Havasu manifest stores the same rectangles per file. File layout sets how many files a query can skip. Ibrahim Kamel and Christos Faloutsos sorted rectangles by the Hilbert value of their centers to build better packed R-trees in [Hilbert R-tree: An Improved R-tree using Fractals](https://dl.acm.org/doi/10.5555/645920.673001) (VLDB 1994). `CREATE SPATIAL INDEX` uses that ordering. Ahmed Eldawy, Louai Alarabi, and Mohamed Mokbel compared partitioning methods for distributed range queries and joins in [Spatial partitioning techniques in SpatialHadoop](https://spatialhadoop.cs.umn.edu/publications/p1602-eldawy.pdf) (PVLDB, 2015). The methods included grid, quadtree, Sort-Tile-Recursive (STR), k-d tree, Z-curve, and Hilbert. Liwen Sun, Michael Franklin, Sanjay Krishnan, and Reynold Xin built blocks from the query workload in [Fine-grained partitioning for aggressive data skipping](https://doi.org/10.1145/2588555.2610515) (SIGMOD 2014). Zongheng Yang and colleagues learned layouts with reinforcement learning in [Qd-tree: Learning Data Layouts for Big Data Analytics](https://doi.org/10.1145/3318464.3389770) (SIGMOD 2020). Learned methods now reach spatial data. Varun Pandey and colleagues tested learned indexes for spatial range queries in [The Case for Learned Spatial Indexes](https://arxiv.org/abs/2008.10349) (AIDB 2020). Keizo Hori and colleagues used deep reinforcement learning to pick spatial partitions in [Learned spatial data partitioning](https://doi.org/10.1145/3593078.3593932) (aiDM 2023). On Apache Sedona, they measured run times up to 59.4% lower for distance joins. ### What scale changes At catalog scale, manifest file boxes narrow a city query over 2,533,842,612 buildings to 3 files. The two continental boxes show the limit: file boxes are only as tight as the layout. History adds metadata. The places table's 60 snapshots and 30 release tags keep 30 full versions readable. Every engine that reads the table has to agree on bounds, CRS, and geometry encoding. ### Open problems - **One layout, many queries.** A Hilbert sort serves range queries well. Workloads that mix spatial windows with time, category, or attribute filters need layouts that serve several predicates. - **Geography bounds.** Iceberg v3 lets a `geography` bounding box cross the antimeridian, with xmin greater than xmax. Every engine has to read those boxes the same way, or pruning drops valid rows. - **Metadata growth.** Every snapshot and tag keeps its files alive. A catalog that keeps every Overture release trades storage for reproducibility, and snapshot expiry has to respect tags. - **Rasters in table formats.** Iceberg v3 defines vector types only. Havasu's raster type and out-db bands have no upstream equivalent yet, so raster tables stay Havasu-specific. ## What is a knowledge lake? A knowledge lake is a catalog layer over a data lake that indexes data together with the context needed to use it: schemas, sample queries, models, and applications. Wherobots calls the Havasu Catalog the knowledge lake for the physical world. Language models were trained on text, documents, databases, and the internet. An AI agent asked which buildings sit in a flood zone also needs spatial tables and an index of where they live. The Model Context Protocol (MCP) is a standard way to connect an AI agent to tools and data. The [Wherobots MCP server and AI coding tools](https://docs.wherobots.com/develop/agentic-tools) supply both for [physical AI](https://wherobots.com/discover/physical-ai/) and [geospatial AI](https://wherobots.com/discover/geospatial-ai/) work. ## Conclusions In the Havasu Catalog, metadata narrowed a city query to 3 of 5,141 files, served ten Overture releases by tag, and gave table sizes without a data read. 1. **The catalog skips files by location through its manifests.** A Lake Havasu City window overlaps 3 of 5,141 Overture buildings files. On Overture tables, add the `bbox` predicate to `ST_Intersects` so column statistics prune too. 2. **Pruning in the catalog is only as good as the table layout.** Two of the three overlapping files have continental boxes, so queries across most of North America open them. Where your tables' `.files` bounds are wide, sort with `CREATE SPATIAL INDEX ... USING hilbert`. 3. **Catalog tables keep past Overture releases, so pin one with `VERSION AS OF` for any number you quote.** The same window held 2,341 places in July 2024 and 3,606 in September 2026, with two dips between. An unpinned query can change with each release. 4. **Catalog metadata answers size questions without a data read.** `.snapshots` returned row counts, file counts, and sizes for four tables in 4.8 seconds. 5. **On new catalog tables, declare a native geometry column with the data's actual CRS.** Use `geometry(4326)` for WGS 84 longitude and latitude, which other Iceberg engines with geospatial support read as geometry. Rasters have no Iceberg v3 equivalent, so raster tables stay Havasu-specific for now. ## Related terms - [Geospatial database](https://wherobots.com/discover/geospatial-database/): geometry types, a spatial index and spatial SQL in one system - [Physical AI](https://wherobots.com/discover/physical-ai/): AI that perceives, reasons about, and acts in the physical world - [Raster vs. vector data](https://wherobots.com/discover/raster-vs-vector-data/): the two data models the catalog's tables use - [GeoTIFF](https://wherobots.com/discover/geotiff/): the raster format behind much of the catalog's imagery - [Spatial join](https://wherobots.com/discover/spatial-join/): combining tables by location - [Coordinate reference system](https://wherobots.com/discover/coordinate-reference-system/): the CRS every geometry column declares - [POI data](https://wherobots.com/discover/poi-data/): points of interest, the data in the Overture places table ## Read more from Wherobots - [Havasu: a table format for spatial attributes in a data lake architecture](https://wherobots.com/blog/havasu-a-table-format-for-spatial-attributes-in-a-data-lake-architecture/) - [Apache Iceberg and Parquet now support geo](https://wherobots.com/blog/apache-iceberg-and-parquet-now-support-geo/) - [Iceberg v3 geospatial types](https://wherobots.com/blog/iceberg-geospatial-types-v3/) - [Building a spatial data lakehouse](https://wherobots.com/blog/building-a-spatial-data-lakehouse/) - [Benefits of Apache Iceberg for geospatial data analysis](https://wherobots.com/blog/benefits-of-apache-iceberg-for-geospatial-data-analysis/) - [Introducing developer tools that let AI build with physical world data](https://wherobots.com/blog/spatial-ai-coding-tools/) Start a free trial and query the Havasu Catalog at [login.cloud.wherobots.com](https://login.cloud.wherobots.com/sign-up). ## Frequently asked questions ### What is the Havasu Catalog? The Havasu Catalog is the Wherobots catalog of physical world data, models, and apps. It lists 42 open geospatial datasets, 6 RasterFlow computer vision models, 9 solution notebooks, and 6 demo apps. Each dataset is an Apache Iceberg table with its schema and sample SQL, so a team can query it in Wherobots without a download. [Browse the Havasu Catalog](https://wherobots.com/havasu/). ### Is the Havasu Catalog free to use? The catalog is free to browse at [wherobots.com/havasu](https://wherobots.com/havasu/). Querying its tables in Wherobots is free during the Wherobots free trial, and continued access after the trial requires the Professional tier or above. RasterFlow, the model inference engine behind the catalog’s model entries, is in Public Preview for Professional, Innovation, and Enterprise organizations. ### What is the difference between the Havasu table format and Apache Iceberg? Havasu is the Wherobots spatial table format built on Apache Iceberg. Its specification adds three things to Iceberg format versions 1 and 2: geometry and raster types, per-file bounding boxes in the manifests, and spatial filter pushdown. It leaves the rest of Iceberg unchanged. Iceberg format version 3 now defines native geometry and geography types, designed with Havasu as a reference. In WherobotsDB, a column declared as `geometry(4326)` is an Iceberg v3 native column, and Wherobots upgrades its open data tables automatically. ### What is a knowledge lake? A knowledge lake is a catalog layer over a data lake that indexes data together with the context needed to use it: schemas, sample queries, models, and applications. Wherobots calls the Havasu Catalog the knowledge lake for the physical world because it covers data, models, jobs, and apps about real places. ### Can I add my own data to the Havasu Catalog? Yes. A team’s own spatial tables sit in an organization catalog next to the open datasets. They can also connect through Unity Catalog, AWS Glue, or any Apache Iceberg catalog, so one query can join private and open data. ### Which datasets are in the Havasu Catalog? The catalog holds 42 datasets. Their sources include the Overture Maps Foundation, the US Census Bureau, OpenStreetMap, ESA Copernicus Sentinel-2, and USDA NAIP aerial imagery. Other sources include Copernicus and ASTER (Advanced Spaceborne Thermal Emission and Reflection Radiometer) elevation models and Google and Microsoft building footprints. In SQL, `SHOW SCHEMAS IN wherobots_open_data` lists the 11 databases of the open data catalog. ### Can AI agents use the Havasu Catalog? Yes. The Wherobots MCP server connects AI coding tools such as Claude Code, Cursor, and VS Code to the catalog. An agent can then find a dataset, read its schema, and write and run the spatial SQL from a plain-language request. [Set up the AI coding tools](https://docs.wherobots.com/develop/agentic-tools). ### What is a spatial data catalog? A spatial data catalog maps table names to the files that hold them, like any data catalog. It adds geometry and raster column types, a coordinate reference system per column, and the bounding box of each data file. With those boxes, a query for one area can skip files outside it. ### Does the Havasu Catalog support time travel? Yes. Every table is an Apache Iceberg table with snapshots, and Wherobots keeps each stable Overture Maps release as a tag. `VERSION AS OF '2025-05-21.0'` reads the Overture places table as that release published it, and the `.refs` metadata table lists the available tags. ### How does spatial filter pushdown work? When a table is written, the manifest records the bounding box of the geometries in each data file. A query with a spatial filter such as `ST_Intersects` compares the query window with those boxes first and skips every file whose box does not intersect it. In the Overture buildings table, 3 of 5,141 data files overlap a Lake Havasu City window. ### What is the difference between GeoParquet and Iceberg geometry types? GeoParquet defines how geometry is stored in a single Parquet file. It uses WKB (well-known binary) or GeoArrow encodings and stores the coordinate reference system in file metadata. Iceberg v3 geometry and geography types define spatial columns for a whole table of many Parquet files. Such a table also has per-file bounding boxes in the manifests, snapshots, and transactions. Iceberg v3 data files use the Parquet GEOMETRY and GEOGRAPHY logical types.