Multi-file reads#
Backs NetCDF.open_mfdataset(). Opens many NetCDF files and stacks one named variable into a
single lazy dask.array.Array of shape (n_files, *variable_shape).
| Backend method | open_mfdataset |
| Module | pyramids_lazy._mfdataset |
| Returns | dask.array.Array of shape (n_files, *variable_shape) |
It returns the stacked dask array itself, not a NetCDF wrapping it — there is no cube to call
read_array() on:
from pyramids.netcdf import NetCDF
arr = NetCDF.open_mfdataset("noah_2009*.nc", variable="precipitation")
arr.sum(axis=0).compute()
Deliberately narrow#
One variable at a time, no by_coords inference and no combine strategies. For the common
hydrology and meteorology case — open 365 daily files, stack one gridded variable, reduce along
time — that narrowness is the whole point: no metadata inference means no failure modes when one
file in the set disagrees with the others.
The per-file reads are sequential, and the parallel= argument is deprecated and inert —
passing it emits a DeprecationWarning and changes nothing. It needs to do nothing: every per-file
read is already lazy, so the scheduler parallelises the per-chunk reads at compute time without an
eager dask.delayed fan-out.
API#
pyramids_lazy._mfdataset.open_mfdataset(paths, variable, *, chunks=None, parallel=False, preprocess=None)
#
Open many NetCDFs; stack variable into one lazy dask array.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
paths
|
str | Sequence[str | Path]
|
Glob string ( |
required |
variable
|
str
|
Name of the NetCDF variable to extract from each
file. Must exist in every file with the same spatial
shape + dtype; mismatches produce a :class: |
required |
chunks
|
dict | str | None
|
Chunk specification forwarded to
:meth: |
None
|
parallel
|
bool
|
Deprecated and inert; retained only for backward-compatible
call signatures. Every per-file read returns a lazy dask array, so
the scheduler already parallelises the per-chunk reads at compute
time and there is no separate eager |
False
|
preprocess
|
Callable[[NetCDF], NetCDF] | None
|
Optional callable applied to each
:class: |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
dask.array.Array: A stack of shape |
Any
|
Element |
Any
|
expansion + sort). |
Raises:
| Type | Description |
|---|---|
ImportError
|
When dask is not installed. |
FileNotFoundError
|
When |
Examples:
- Stack a single file into a 1-element dask array:
Source code in src/pyramids_lazy/_mfdataset.py
78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | |