Performance

mapcv’s benchmark suite (benchmarks/) builds the same datasets with mapcv and with other tools, checks that the outputs are the same data, and only then compares time and memory. Everything runs offline against a synthetic tile server, so the numbers measure the tools, not the network.

mapcv against a hand-written script

The baseline, rasterio_script.py, is what a careful user writes without mapcv: mercantile for the tile list, a thread pool fetching tiles with the same 16 connections as mapcv, Pillow to decode and stitch, pyproj to reproject the labels, rasterio.features.rasterize for the masks, numpy windows and Pillow PNGs. Before its time counts, every one of its patches is compared with mapcv’s at the same position: in all scenarios below the images are identical pixel for pixel (except with JPEG output, which is not compared byte for byte) and the masks agree on every pixel.

Scenario	Tool	Wall time	Peak memory	Time vs mapcv	Same data
S: 10 × 10 tiles, 100 patches	mapcv	0.63 s	113 MB	1.00×	—
	rasterio script	0.99 s	127 MB	1.57×	yes
M: 32 × 32 tiles, 1,024 patches	mapcv	1.55 s	217 MB	1.00×	—
	rasterio script	5.89 s	436 MB	3.79×	yes
M-jpg: M written as JPEG	mapcv	1.84 s	216 MB	1.00×	—
	rasterio script	6.94 s	436 MB	3.76×	yes
M-jpgtiles: the server sends JPEG tiles	mapcv	2.73 s	246 MB	1.00×	—
	rasterio script	37.37 s	437 MB	13.69×	yes
M-overlap: stride 128, 3,969 patches	mapcv	2.60 s	431 MB	1.00×	—
	rasterio script	19.87 s	436 MB	7.65×	yes
L: 100 × 100 tiles, 10,000 patches	mapcv	15.46 s	521 MB	1.00×	—
	rasterio script	60.17 s	3,341 MB	3.89×	yes

Medians of 3 runs, release build, on an Intel Core i7-10710U laptop (6 cores, 12 threads, 15 GB RAM, Linux, Python 3.13), mapcv at commit d801d66. The script also writes no manifest, splits or patch summaries, which mapcv’s times include.

Where the difference comes from. mapcv decodes tiles, rasterizes and encodes patches in Rust on all cores and without the GIL; the script does each step once, single-threaded, in Python and Pillow. With JPEG tiles the decode alone takes the script most of its time. The script also holds the whole stitched raster and mask in memory (3.3 GB at L) where mapcv reads strips of tile rows, so its memory grows with the region and mapcv’s does not.

Where mapcv spends its time (scenario L, medians): fetching 4.0 s, writing patches 2.9 s, decoding 1.9 s, rasterizing 1.8 s, planning 1.3 s, reading labels 1.2 s, sampling 0.9 s. Every run is also checked against independent references: images against the tile server’s pixels, masks against rasterio, splits for leakage, and a resumed run against an uninterrupted one, byte for byte.

mapcv against other tools

Four ways to build the same dataset from the same tile server, labels, region, patch size and stride, each in its own script in benchmarks/baseline_scripts/:

• rasterio script: the hand-written pipeline above.
• GDAL command line: gdal_translate reads the tiles through GDAL’s TMS driver (16 connections), ogr2ogr reprojects and numbers the labels, gdal_rasterize burns the mask, and gdal_translate -srcwin cuts each patch, one process per file on every core, as a shell xargs -P loop would.
• TorchGeo 0.10: the tiles stitched into a GeoTIFF (TorchGeo fetches no tiles), then RasterDataset & VectorDataset sampled by GridGeoSampler through a DataLoader with a worker per core.
• leafmap 0.63: map_tiles_to_geotiff downloads the tiles, then rasterio burns the labels and cuts the windows (leafmap has no chipping step of its own).

Scenario	Tool	Wall time	Peak memory	Time vs mapcv	Same data
S: 100 patches	mapcv	0.87 s	110 MB	1.00×	—
	rasterio script	1.34 s	129 MB	1.54×	yes
	GDAL command line	4.73 s	596 MB	5.43×	yes
	TorchGeo	5.53 s	4,788 MB	not comparable	no
	leafmap	7.44 s	464 MB	8.55×	yes
M: 1,024 patches	mapcv	1.56 s	234 MB	1.00×	—
	rasterio script	6.77 s	435 MB	4.33×	yes
	GDAL command line	47.50 s	618 MB	30.37×	yes
	TorchGeo	34.61 s	6,329 MB	not comparable	no
	leafmap	53.86 s	2,263 MB	34.44×	yes

Medians of 3 runs on the same laptop, all tools in one session. Peak memory is the whole process tree: TorchGeo’s includes its DataLoader workers, each of which imports PyTorch.

What the comparison found.

• The rasterio script, the GDAL pipeline and leafmap write the same patches as mapcv: identical images, and masks that agree on every pixel or, for leafmap, on all but 0.02 % (a polygon edge that rounds the other way).
• TorchGeo samples patches on its own grid rather than on the imagery’s pixel grid, so its patches are not the imagery’s own windows and are not compared as the same data; its time is shown for reference only.

	mapcv	rasterio script	GDAL command line	TorchGeo	leafmap
Code for this job	a 24-line YAML config	59 lines	155 lines (Python driving GDAL’s programs; about 10 shell commands)	58 lines + 52 to fetch the tiles	62 lines
Needs GDAL	no	inside the rasterio wheel	yes	inside the rasterio wheel	yes (osgeo bindings)
Install	pip install mapcv	rasterio, mercantile, shapely, pyproj, Pillow	GDAL (conda-forge)	PyTorch, TorchGeo, rasterio, mercantile	conda-forge GDAL, leafmap, rasterio, mercantile

Not compared: raster-vision, whose chip export needs a full pipeline configuration and pins its own PyTorch and GDAL stack.

Reproduce

$ uv sync --all-extras && uv run maturin develop --uv --release
$ uv run python -m benchmarks run --scenarios S M M-jpg M-jpgtiles M-overlap L \
    --baselines rasterio-script --out results.json
$ GDAL_BIN=/path/to/gdal/bin \
  MAPCV_BENCH_TORCHGEO_PYTHON=/path/to/torch-env/bin/python \
  MAPCV_BENCH_LEAFMAP_PYTHON=/path/to/gdal-env/bin/python \
  uv run python -m benchmarks run --scenarios S M \
    --baselines rasterio-script gdal-cli torchgeo leafmap --out others.json
$ uv run python -m benchmarks compare results.json

Always benchmark a release build. The results file records the machine, the library versions, the git commit and every run; see the benchmarks README for the scenarios, the checks and how to add a baseline.