Using pyramids with Dask¶

pyramids reads stay eager NumPy by default. Every lazy entry point is opt-in: you pass chunks= (rasters / NetCDF) or backend='dask' (vectors), and the call returns a Dask object instead of a materialised array. Nothing touches disk until you call .compute().

You have...	Go lazy when...	Entry point
a single raster (Dataset)	it is larger than RAM, or you tile work	read_array(chunks=...), to_zarr
a raster time-series (DatasetCollection)	you reduce / group over many files	from_files(...).data
a vector table (FeatureCollection)	the table does not fit in memory	read_file(..., backend='dask')
a NetCDF cube (NetCDF)	you read one variable across time/files	read_array(chunks=...), open_mfdataset

Install the extras: pip install 'pyramids-gis[lazy]' (rasters, NetCDF, collections) and pip install 'pyramids-gis[parquet]' (vectors).

This page is a tour. Each class has a focused quickstart and an exhaustive cookbook — see the links at the bottom.

Setup¶

In [1]:

%matplotlib inline

from pathlib import Path

import dask.array as da
import numpy as np

DATA = Path('../../../examples/data')
DATA.is_dir()

%matplotlib inline from pathlib import Path import dask.array as da import numpy as np DATA = Path('../../../examples/data') DATA.is_dir()

Out[1]:

True

Dataset — lazy raster reads¶

read_array() returns NumPy; read_array(chunks=...) returns a dask.array. Reductions are nodata-aware only if you mask the sentinel yourself — pyramids hands you the raw band.

In [2]:

from pyramids.base.protocols import is_lazy
from pyramids.dataset import Dataset

ds = Dataset.read_file(DATA / 'acc4000.tif')
lazy = ds.read_array(chunks=(32, 32))
is_lazy(lazy), type(lazy).__name__, lazy.shape, lazy.chunks

from pyramids.base.protocols import is_lazy from pyramids.dataset import Dataset ds = Dataset.read_file(DATA / 'acc4000.tif') lazy = ds.read_array(chunks=(32, 32)) is_lazy(lazy), type(lazy).__name__, lazy.shape, lazy.chunks

2026-07-11 14:38:02 | INFO | pyramids.base.config | Logging is configured.

Out[2]:

(True, 'Array', (13, 14), ((13,), (14,)))

The lazy handle above only describes the graph. The source Dataset is concrete, so we can plot the flow-accumulation band directly to see the data the chunks cover.

In [3]:

ds.plot(band=0, title="acc4000 — flow accumulation")

ds.plot(band=0, title="acc4000 — flow accumulation")

Out[3]:

<cleopatra.array_glyph.ArrayGlyph at 0x7f568b4faba0>

/home/runner/work/pyramids/pyramids/.pixi/envs/docs/lib/python3.14/site-packages/matplotlib/colors.py:816: RuntimeWarning: overflow encountered in multiply
  xa *= self.N

In [4]:

# Mask the nodata sentinel inside the graph, then reduce — no I/O until compute().
nodata = ds.no_data_value[0]
masked = da.where(lazy == nodata, np.nan, lazy)
float(da.nanmin(masked).compute()), float(da.nanmax(masked).compute())

# Mask the nodata sentinel inside the graph, then reduce — no I/O until compute(). nodata = ds.no_data_value[0] masked = da.where(lazy == nodata, np.nan, lazy) float(da.nanmin(masked).compute()), float(da.nanmax(masked).compute())

Out[4]:

(0.0, 88.0)

DatasetCollection — lazy raster time-series¶

from_files stacks co-registered rasters into a 4-D (T, B, R, C) Dask cube exposed as .data. Reductions run over the whole cube and each file opens at most once.

In [5]:

from pyramids.dataset import DatasetCollection

files = sorted((DATA / 'crop_aligned_folder').glob('[0-5].tif'))
cube = DatasetCollection.from_files(files)
cube.data.shape, cube.data.chunks

from pyramids.dataset import DatasetCollection files = sorted((DATA / 'crop_aligned_folder').glob('[0-5].tif')) cube = DatasetCollection.from_files(files) cube.data.shape, cube.data.chunks

Out[5]:

((6, 1, 24, 29), ((1, 1, 1, 1, 1, 1), (1,), (24,), (29,)))

In [6]:

# Time-axis reduction -> a single (B, R, C) NumPy array.
time_mean = cube.mean(skipna=True)
type(time_mean).__name__, time_mean.shape

# Time-axis reduction -> a single (B, R, C) NumPy array. time_mean = cube.mean(skipna=True) type(time_mean).__name__, time_mean.shape

Out[6]:

('ndarray', (1, 24, 29))

FeatureCollection — lazy vector tables¶

backend='dask' returns a LazyFeatureCollection (a partitioned dask-geopandas frame). The bounds reduction is lazy — compute_total_bounds() materialises only the 4-float result, not the rows.

In [7]:

from pyramids.feature import FeatureCollection, is_lazy_fc

lfc = FeatureCollection.read_file(
    DATA / 'coello-gauges.geojson', backend='dask', npartitions=2
)
is_lazy_fc(lfc), lfc.npartitions

from pyramids.feature import FeatureCollection, is_lazy_fc lfc = FeatureCollection.read_file( DATA / 'coello-gauges.geojson', backend='dask', npartitions=2 ) is_lazy_fc(lfc), lfc.npartitions

Out[7]:

(True, 2)

In [8]:

# Lazy reproject; the bounds reduction is computed on demand.
projected = lfc.to_crs(4326)
projected.compute_total_bounds()

# Lazy reproject; the bounds reduction is computed on demand. projected = lfc.to_crs(4326) projected.compute_total_bounds()

Out[8]:

array([-75.5060754 ,   4.32493719, -75.11452282,   4.55188404])

NetCDF — lazy variable reads¶

One variable at a time. chunks=(1, -1, -1) is one chunk per leading-axis step.

In [9]:

from pyramids.netcdf import NetCDF

nc = NetCDF.read_file(DATA / 'netcdf' / 'cf__4v__1d3-3d1__proj__y-desc.nc')
vlazy = nc.read_array('values', chunks=(1, -1, -1))
type(vlazy).__name__, vlazy.shape, vlazy.chunks

from pyramids.netcdf import NetCDF nc = NetCDF.read_file(DATA / 'netcdf' / 'cf__4v__1d3-3d1__proj__y-desc.nc') vlazy = nc.read_array('values', chunks=(1, -1, -1)) type(vlazy).__name__, vlazy.shape, vlazy.chunks

Out[9]:

('Array', (3, 13, 14), ((1, 1, 1), (13,), (14,)))

nc is a concrete NetCDF, so we can plot its values variable to see what the lazy per-step chunks read from.

In [10]:

nc.plot(variable='values', title="cf__4v__1d3-3d1__proj__y-desc — values")

nc.plot(variable='values', title="cf__4v__1d3-3d1__proj__y-desc — values")

Out[10]:

<cleopatra.array_glyph.ArrayGlyph at 0x7f568b146fd0>

/home/runner/work/pyramids/pyramids/.pixi/envs/docs/lib/python3.14/site-packages/matplotlib/colors.py:816: RuntimeWarning: overflow encountered in multiply
  xa *= self.N

In [11]:

# Reduce over the leading axis without materialising the cube.
vlazy.mean(axis=0).compute().shape

# Reduce over the leading axis without materialising the cube. vlazy.mean(axis=0).compute().shape

/home/runner/work/pyramids/pyramids/.pixi/envs/docs/lib/python3.14/site-packages/numpy/_core/_methods.py:49: RuntimeWarning: overflow encountered in reduce
  return umr_sum(a, axis, dtype, out, keepdims, initial, where)

Out[11]:

(13, 14)

What is not lazy¶

• UgridDataset (UGRID meshes) has no native dask.array path: the mesh topology and data variables are eager NumPy. To go lazy, grid a variable to a Dataset with to_dataset(variable, cell_size) and use the raster path above — see the UGRID notebook.
• COG writes are serial through GDAL; COG reads inherit the Dataset lazy path (read_array(chunks=...)) — there is nothing COG-specific to learn.

Where to next¶

Each class has three tiers — pick by depth (quickstart → complete cookbook → live cloud):

• Dataset: quickstart · complete cookbook · cloud — Sentinel-2
• DatasetCollection: quickstart · complete cookbook · cloud: the stack section of the Sentinel-2 notebook
• FeatureCollection: quickstart · complete cookbook · cloud — Overture
• NetCDF: quickstart · complete cookbook · cloud — ERA5

Narrative guides: the Lazy / Dask tutorials (lazy-compute, lazy-raster, lazy-netcdf, lazy-collection, lazy-vector).