---
title: "Detecting sidewalks with RasterFlow"
url: "https://wherobots.com/solution-gallery/rasterflow-tile2net/"
published: "2025-12-02"
modified: "2026-11-10"
---

# Detecting sidewalks with RasterFlow

## Detecting sidewalks with RasterFlow

This notebook will guide you through detecting sidewalks from aerial imagery, using Wherobots RasterFlow and the Tile2Net model. You will gain a hands-on understanding of how to run models like Tile2Net on your selected area of interest, vectorize the model outputs, and work with those vectors in WherobotsDB.

### Tile2Net

The [Tile2Net](https://github.com/VIDA-NYU/tile2net)1 model is an open source segmentation model that can detect sidewalks and other pathways from very high resolution imagery.

This model predicts 4 classes:

- background
- road
- sidewalk
- crosswalk

It was trained on ~19cm aerial imagery from USGS for Cambridge, Manhattan, Brooklyn, and Washington DC.We will demonstrate results using 30cm resolution data from the [National Agriculture Imagery Program (NAIP)](https://www.usgs.gov/centers/eros/science/usgs-eros-archive-aerial-photography-national-agriculture-imagery-program-naip).

## Preview: model inputs and outputs

The interactive map linked below shows the input imagery and model outputs for this notebook’s example AOI. Toggle layers in the sidebar to compare the inputs, the raster model output, and the vectorized geometries side-by-side.

**Layers** (top layer first, as in the map’s layer list):

- *Tile2Net PM Tiles*: Vectorized sidewalk, road and crosswalk polygons in PMTiles format, created from the vector geometries
- *Tile2Net vector geometries*: Vectorized sidewalk / road / crosswalk polygons derived from the raster output
- *Tile2Net model outputs*: Raster output from the model, with bands for sidewalk / road / crosswalk classes
- *Tile2Net input mosaic*: RGB high-resolution aerial imagery the model runs on

View the interactive map [here](https://viz.wherobots.com/?z=14.24&lat=38.99775&lng=-76.93743&l0.id=4b363cf8-20cf-464f-bbaf-820ae32d85de&l0.src=s3%3A%2F%2Fwherobots-examples%2Frasterflow%2Fmosaics%2Fcollegepark_optimized.zarr&l0.name=Tile2Net+input+mosaic&l0.clim=24.672528253600106%2C200.6443167352033&l0.mode=rgb&l0.rgb=r%2Cg%2Cb%2Cband&l0.var=variables&l1.id=11862f01-e388-4788-bcec-0ade21eec794&l1.src=s3%3A%2F%2Fwherobots-examples%2Frasterflow%2Fmodel-outputs%2Ftile2net_optimized.zarr&l1.name=Tile2Net+model+outputs&l1.vis=0&l1.clim=0.00009313203892032774%2C0.8193664079376207&l1.mode=single&l1.rgb=sidewalk%2Croad%2Ccrosswalk%2Cband&l1.var=variables&l2.id=78ecd25a-79c8-49c7-996e-6774a45cc05a&l2.src=s3%3A%2F%2Fwherobots-examples%2Frasterflow%2Fvectors%2Ftile2net_collegepark.parquet%2F&l2.name=Tile2Net+vector+geometries&l2.type=geoparquet&l2.vis=0&l2.fill=%233b82f6&l2.line=%232563eb&l2.lw=2&l2.cc=%233b82f6&l2.cr=5&l3.id=7ba01f8a-40b6-41df-9766-5553491ec152&l3.src=s3%3A%2F%2Fwherobots-examples%2Frasterflow%2Fmodel-outputs%2Ftile2net.pmtiles&l3.name=Tile2Net+PM+Tiles&l3.type=pmtiles&l3.sl.road.vis=0&l3.sl.crosswalk.vis=0&l3.fill=%23ef4444&l3.line=%23dc2626&l3.lw=2&l3.cc=%23ef4444&l3.cr=5&title=Tile2Net+College+Park+MD).

## Selecting an Area of Interest (AOI)

To start, we will choose an Area of Interest (AOI) for our analysis. Tile2Net was trained on select geographies in the Northeastern United States. So we will pick a location in that geographic region that wasn’t in the training data and where there is recent 30cm resolution NAIP data: College Park, Maryland.

```
import os

import shapely
import wkls
from sedona.spark import *

config = SedonaContext.builder().getOrCreate()
sedona = SedonaContext.create(config)

# Generate a WKT geometry for College Park, Maryland using Well-Known Locations (https://github.com/wherobots/wkls)
aoi = wkls.us.md.collegepark.wkt()

min_lon, min_lat, max_lon, max_lat = shapely.from_wkt(aoi).bounds
print(f"Bounds: [{min_lon:.4f}, {min_lat:.4f}, {max_lon:.4f}, {max_lat:.4f}]")
```

## Selecting a time range and verifying NAIP coverage

Because NAIP collects state-level imagery at mixed resolutions, Tile2Net’s required 30cm data may not exist for your selected AOI or timeframe. The USDA’s [NAIP coverage map](https://www.arcgis.com/home/item.html?id=cdaa8c24cf0844abba74fb9d71432fd4#overview) (PDF, 2002-2025) shows what is available where.

```
from datetime import datetime

# Date range for imagery to be used by the model
start_date = datetime(2023, 1, 1)
end_date = datetime(2024, 1, 1)

# The Tile2Net recipe runs on 30cm NAIP imagery from the RGB (visual) product. Wherobots Open Data
# indexes every scene of that product with its footprint, resolution (gsd) and acquisition time.
MODEL_RES = 0.3
sedona.createDataFrame([(aoi,)], ["wkt"]).selectExpr("ST_GeomFromWKT(wkt) AS geom").createOrReplaceTempView("aoi")

# Scenes at the model's resolution touching the AOI (the table and wkls are both EPSG:4326)
sedona.sql(f"""
    SELECT n.geometry, n.year, n.datetime
    FROM wherobots_open_data.imagery.usda_naip_visual_stac n
    JOIN aoi a ON ST_Intersects(n.geometry, a.geom)
    WHERE n.gsd = {MODEL_RES}
""").cache().createOrReplaceTempView("nearby")

# The recipe needs the AOI fully inside the union of the scene footprints in the date range
coverage = sedona.sql(f"""
    SELECT count(*) AS scenes, array_sort(collect_set(n.year)) AS years,
           ST_Contains(ST_Union_Agg(n.geometry), first(a.geom)) AS covered
    FROM nearby n CROSS JOIN aoi a
    WHERE n.datetime >= TIMESTAMP '{start_date.isoformat()}Z' AND n.datetime < TIMESTAMP '{end_date.isoformat()}Z'
""").first()

if not coverage["covered"]:
    # Years that would work, applying the same full-coverage test to each year on its own
    available = [
        row.year
        for row in sedona.sql("""
            SELECT n.year FROM nearby n CROSS JOIN aoi a
            GROUP BY n.year
            HAVING ST_Contains(ST_Union_Agg(n.geometry), first(a.geom))
            ORDER BY n.year
        """).collect()
    ]
    raise ValueError(
        f"{MODEL_RES:g}m NAIP does not fully cover this AOI between "
        f"{start_date:%Y-%m-%d} and {end_date:%Y-%m-%d}. "
        + (
            f"Years with complete {MODEL_RES:g}m coverage over this AOI: {available}."
            if available
            else f"No year has complete {MODEL_RES:g}m NAIP coverage over this AOI."
        )
    )

print(f"NAIP coverage is available. Found {coverage['scenes']} NAIP scenes at {MODEL_RES:g}m covering the AOI")
print(f"Acquisition years: {coverage['years']}")
```

## Using the Wherobots Tasks API

Each RasterFlow workflow runs as a task through the [Wherobots Python SDK](https://pypi.org/project/wherobots-python-sdk/). `run_task` below starts a task run, waits for it to finish, and returns its output value. Recipes return an object with `input_mosaic` and `result`; standalone tasks return a URI. Call `wb.tasks.get("<task name>").input_schema` to see every input a task accepts.

```
import time

from wherobots import TaskRunState, Wherobots

# In a Wherobots notebook the SDK uses your Wherobots credentials, elsewhere it reads WHEROBOTS_API_KEY
wb = Wherobots()

def run_task(task_name, inputs, poll_seconds=30):
    """Run a RasterFlow task, wait for it to finish, and return its output value."""
    run = wb.task_runs.get(wb.tasks.run(task_name, inputs, region="aws-us-west-2").task_run_id)
    print(f"Started {task_name} run {run.task_run_id}")
    while run.is_active:
        time.sleep(poll_seconds)
        run = wb.task_runs.get(run.task_run_id)
    if run.state != TaskRunState.SUCCEEDED:
        raise RuntimeError(f"{task_name} run {run.task_run_id} ended {run.state}: {run.error_message}")
    # A successful run can temporarily have no results if the output read fails.
    for _ in range(3):
        if run.results is not None:
            break
        time.sleep(poll_seconds)
        run = wb.task_runs.get(run.task_run_id)
    if run.results is None:
        raise RuntimeError(
            f"Results for successful run {run.task_run_id} are unavailable. "
            "Fetch this run again with wb.task_runs.get(run_id); do not resubmit the task."
        )
    # The single task output can be a URI or a structured recipe result.
    (output,) = run.results.values()
    return output
```

## Running a model

RasterFlow has pre-defined workflows to simplify orchestration of the processing steps for model inference.These steps include:

- Ingesting imagery for the specified Area of Interest (AOI)
- Generating a seamless image from multiple image tiles (a mosaic)
- Running inference with the selected model

The recipe returns `input_mosaic` for the source imagery and `result` for the predictions. Each contains a `mosaic_index_uri` for its GeoParquet index and a `first_row_mosaic` for its first Zarr store.

Note: This step will take approximately 30 minutes to complete.

```
recipe_output = run_task(
    "predict_mosaic_recipe",
    {
        # Area of Interest as WKT (EPSG:4326)
        "aoi": aoi,

        # Date range for imagery to be used by the model (set in the coverage check above), as RFC 3339 timestamps
        "start": start_date.isoformat() + "Z",
        "end": end_date.isoformat() + "Z",

        # Coordinate Reference System for the output mosaic
        "target_crs": "EPSG:3857",

        # The model recipe to be used for inference (Tile2Net in this case)
        "model_recipe": "tile2net",
    },
)

# The recipe returns both the input imagery and the prediction mosaic.
input_mosaic_index_uri = recipe_output["input_mosaic"]["mosaic_index_uri"]
input_mosaic_store = recipe_output["input_mosaic"]["first_row_mosaic"]
print(f"Input imagery: {input_mosaic_store}")

model_output_index_uri = recipe_output["result"]["mosaic_index_uri"]

# Inspect the prediction index (one row per output mosaic location)
model_output_index_df = sedona.read.format("geoparquet").load(model_output_index_uri)
model_output_index_df.show(truncate=60)
```

```
# This example AOI has one geometry, so we select the first (only) mosaic location.
model_output_store = model_output_index_df.first()["location"]

model_output_store
```

## (Optional) Build an optimized Zarr for visualization

RasterFlow writes its outputs as Zarr stores at native resolution. To explore them interactively on [cloud.wherobots.com/map](https://cloud.wherobots.com/map), you can build an *optimized* multiscale Zarr. `build_zarr_multiscales` adds downsampled overview levels (image pyramids) to the store so the map can stream coarse tiles when zoomed out and full-resolution pixels when zoomed in.

This step is optional and can take a few minutes for large outputs, so the code below is commented out by default — uncomment it to run it.

Pass `aoi` to select a chunk-aligned bounding window from a single Zarr source. It accepts polygon or multipolygon WKT in EPSG:4326, or an S3 GeoJSON or GeoParquet URI with a CRS. Pixels outside the polygon remain within that rectangular window. Omit `aoi` to process the full store.

```
# optimized_store = run_task("build_zarr_multiscales", {"source_store": model_output_store, "aoi": aoi})
# optimized_store
```

## Visualizing outputs

You can visualize the Zarr, GeoParquet, and other geospatial outputs using [cloud.wherobots.com/map](https://cloud.wherobots.com/map).

## Vectorize the raster model outputs

The output for the Tile2Net model is a raster with four classes: background, road, sidewalk, crosswalk.

We can run a separate flow to convert the sidewalks and crosswalks into vector geometries, based on the confidence threshold.Converting these results to geometries allows us to more easily post process the results or join the results with other vector data.

```
import xarray as xr
import s3fs
import zarr
# Determine the classes that are predicted by the model
fs = s3fs.S3FileSystem(profile="default", asynchronous=True)
zstore = zarr.storage.FsspecStore(fs, path=model_output_store[5:])
ds = xr.open_zarr(zstore)
model_features = ds['band'].data.tolist()

# Only vectorize the 'sidewalk' and 'crosswalk' classes 
relevant_features = ['sidewalk', 'crosswalk']   
vector_features = [f for f in model_features if f in relevant_features]
```

```
# Note: this should take about 5 minutes to complete
vectorized_results_uri = run_task(
    "vectorize_mosaic",
    {
        "mosaic_uri": model_output_store,
        "features": vector_features,
        "threshold": 0.05,
        "stats": True,
        "medial_skeletonize": False,
    },
)

print(vectorized_results_uri)
```

## Save the vectorized results to the catalog

We can store these vectorized outputs in the catalog by using WherobotsDB to persist the GeoParquet results.

```
from pyspark.sql.functions import expr

sedona.sql("CREATE DATABASE IF NOT EXISTS examples_temp.tile2net_db")

df = sedona.read.format("geoparquet").load(vectorized_results_uri)
df = df.withColumnRenamed("label", "layer")
df.writeTo("examples_temp.tile2net_db.tile2net_vectorized").createOrReplace()
```

## Visualize the vectorized results

To visualize the vectorized results, we will filter out results with a score lower than 0.2 and keep only geometries larger than 1,000 square metres. The score threshold was determined through observation to strike a balance: it eliminates obvious noise without being overly aggressive, ensuring that we don’t accidentally filter out too many relevant results.

Gaps in prediction for Tile2Net are likely due to it being used with lower resolution imagery (30cm resolution) than what it was trained on (19cm resolution), as well as changes in geographic context, and occlusion from trees over the pathways.

```
df = df.filter("score_mean > 0.2")
df_filtered = df.withColumn(
    "area_m2",
    expr("ST_AreaSpheroid(geometry)")
).filter("area_m2 > 1000")
```

```
from wherobots_gl import Map

# Wherobots-GL Map is URL-based, so write the derived (filtered) vectors to GeoParquet first.
vectors_viz_path = os.getenv("USER_S3_PATH") + "tile2net_vectorized.parquet"
df_filtered.write.format("geoparquet").mode("overwrite").save(vectors_viz_path)

Map(layers=[{"type": "geoparquet", "source": vectors_viz_path, "name": "Vectorized results"}])
```

## Generate PM Tiles for visualization

To improve visualization performance of a large number of geometries, we can use Wherobots built-in high performance PM tile generator.

```
from wherobots import vtiles

full_tiles_path = os.getenv("USER_S3_PATH") + "tile2net_tiles.pmtiles"
vtiles.generate_pmtiles(df_filtered, full_tiles_path)
```

```
vtiles.show_pmtiles(full_tiles_path)
```

### References

1. **Hosseini, M., Sevtsuk, A., Miranda, F., Cesar Jr, R. M., & Silva, C. T. (2023).** Mapping the walk: A scalable computer vision approach for generating sidewalk network datasets from aerial imagery. *Computers, Environment and Urban Systems*, *101*, 101950.