Writing a mesh driver#
pyramids reads and writes UGRID NetCDF itself. Every other unstructured-mesh format — 2DM, Selafin, DFSU, XMDF, PLY and the rest — is read by a separate distribution that registers a driver with pyramids. This page is the contract such a distribution implements.
The dependency runs one way: your package imports pyramids' mesh types and builds them, and pyramids never imports your package. It knows only the entry-point group name.
Registering#
Declare one entry point per driver in the pyramids.mesh_drivers group. The value is a class,
which pyramids instantiates with no arguments:
# your-package/pyproject.toml
[project.entry-points."pyramids.mesh_drivers"]
selafin = "your_package.drivers.telemac:SelafinDriver"
2dm = "your_package.drivers.sms:TwoDmDriver"
A class rather than a function, because can_open matters: several of these formats share
.dat, .mesh and .out, so a suffix alone cannot identify one.
The contract#
from pyramids.netcdf.ugrid import MESH_DRIVER_API_VERSION
class SelafinDriver:
name = "selafin" # unique; the value accepted by `driver=`
long_name = "TELEMAC Selafin/Serafin"
suffixes = (".slf", ".ser", ".res") # lowercase, dot-prefixed
api_version = MESH_DRIVER_API_VERSION
capabilities = frozenset({"read_mesh", "read_data", "lazy_data"})
def can_open(self, path: str) -> bool: ...
def read(self, path, *, mesh_name=None, lazy=True): ...
def read_data(self, path, dataset, *, lazy=True): ... # only for "needs_mesh";
# -> dict[str, MeshVariable]
def mesh_names(self, path: str) -> list[str]: ...
def write(self, dataset, path, **options) -> None: # only for "write_mesh"
...
can_open and mesh_names are required of every driver: pyramids calls both whatever the
capabilities say. can_open must never raise — an empty file, a directory, or a file of
another format all answer False — and must not read the whole file. A raising can_open is
treated as declining the file, with a warning naming your driver.
Capabilities#
can_open and mesh_names are required of every driver, whatever it declares, because
pyramids calls both unconditionally. Beyond those, four tokens gate behaviour, and three of
them also require a method:
| Token | Effect | Method required |
|---|---|---|
read_mesh |
the file carries a topology and read builds it |
read |
needs_mesh |
the file carries values only; read_data supplies them |
read_data |
write_mesh |
to_file(driver=…) will call this driver |
write |
write_data |
write persists variables; without it a write warns they were dropped |
none — it describes what write already does |
The other three are declarative — they describe your driver in info(drivers=True) and in
your documentation, and pyramids does not branch on them:
| Token | Meaning |
|---|---|
read_data |
the driver produces data variables, not just a frame |
lazy_data |
lazy is honoured with a real deferred loader. pyramids forwards lazy to every driver regardless, so declare this only if you act on it |
multi_mesh |
mesh_names can return more than one entry. Every driver is asked, not only these — a single-mesh format returns a one-item list |
A driver declares exactly one of read_mesh or needs_mesh. Declaring neither, or both, is a
registration error.
Results-only formats#
Not every mesh file contains a mesh. An SMS .dat, an XMDF and an XDMF carry values that must be
paired with a topology loaded from somewhere else. Such a driver declares needs_mesh, leaves
read raising, and implements read_data:
read_data returns only the variables it read, as a dict[str, MeshVariable] —
attach_data merges them onto the mesh. Do not return the merged dataset: that division is what
makes a name the mesh already defines an unambiguous collision, and it means your driver cannot
drop the mesh's own variables by rebuilding the mapping.
read_data must also check the file's declared element count against the supplied mesh, and
raise ValueError on a mismatch. That check is the only safeguard there is — a bare results file
carries nothing identifying the mesh it belongs to, so attaching it to the wrong one otherwise
produces a plausible and entirely wrong dataset. If the file declares no count, refuse it rather
than hoping.
What a driver builds#
Construct pyramids' own types and return them:
from pyramids.netcdf.ugrid import (
Connectivity, Mesh2d, MeshVariable, Network1d, UgridDataset,
)
UgridDataset(
mesh=Mesh2d(
node_x, node_y,
face_node_connectivity=Connectivity(
data=faces, # ALWAYS 0-indexed internally
fill_value=-1,
cf_role="face_node_connectivity",
original_start_index=1, # what the FILE used, so a write restores it
),
edge_node_connectivity=..., # needed for edge-located data
crs=wkt_or_epsg,
),
data_variables={"water_level": MeshVariable(...)},
global_attributes={"Conventions": "UGRID-1.0", "source": "TELEMAC Selafin"},
topology_info=None, # see below
crs_wkt=wkt,
file_name=str(path),
)
A format with no NetCDF variables to name — a 2DM or a Selafin — passes topology_info=None,
which UgridDataset documents as the value for a mesh built from arrays. Synthesise
UGRID-conventional names only if you want to_file() to write a tidy NetCDF afterwards, and then
make them match what the writer actually emits.
Return a Network1d for a file whose topology dimension is 1 —
2DM's E2L line elements, or FLO-2D's and 3Di's 1D domains. UgridDataset.read_file refuses one
rather than hand back the wrong type; Network1d.read_file is where it is read:
A driver may serve both: return a Network1d when the file declares topology_dimension == 1 and
a UgridDataset otherwise. Each entry point checks what it received and names the other one if it
is the wrong shape.
Five invariants — each fails silently#
These produce wrong numbers rather than an exception, so each needs its own test in your package:
| # | Invariant | Why |
|---|---|---|
| 1 | Connectivity.data is 0-indexed |
original_start_index only records the file's base |
| 2 | Faces are fill-padded, never ragged | mixed triangles and quads is normal; rows are max_nodes wide |
| 3 | fill_value must not collide with a valid index |
-1 is the safe choice; never 0 on a 0-based array |
| 4 | Face winding is counter-clockwise | area, centroid, polygonize and voronoi all assume it |
| 5 | MeshVariable.shape is correct before any lazy load |
the whole along-time API trusts it without forcing a read |
Lazy reads#
MeshVariable._loader is a zero-argument callable, resolved once by .data and cached. Set
shape from the file header and defer the values:
MeshVariable(
name="water_level", location="face", mesh_name="mesh2d",
shape=(n_time, n_face), # correct BEFORE the loader runs
_loader=lambda: _read_slab(path, variable_index),
)
Close over the path, not an open file handle: the loader may run long after your read
returned, and on Windows a captured handle also keeps the file locked.
How pyramids dispatches#
UgridDataset.read_file resolves in this order:
driver=given — that driver, orDriverNotExistError. Never falls back.driver="ugrid"names pyramids' own reader, which is whyugridis inRESERVED_NAMESand a plugin cannot take it.- A
/vsipath or a NetCDF suffix (.nc,.nc4,.nc3,.cdf,.netcdf) — the built-in reader. No driver discovery runs, so a NetCDF read costs nothing. - Otherwise, the drivers claiming the suffix, each asked
can_open; the first match wins, ties broken by registration order. - Nothing claimed it — the built-in reader anyway, so a NetCDF file under an unusual suffix still opens as it always did.
- That failed too —
UnsupportedMeshFormatError, naming the built-in reader, every installed plugin and its suffixes.
to_file never infers a driver from the suffix: to_file("mesh.slf") writes NetCDF unless you
pass driver="selafin". **options reaches your write uninterpreted.
lazy is forwarded to read and read_data from every entry point that takes it —
read_file, read_all, attach_data and Network1d.read_file. The built-in UGRID reader is
always lazy and raises TypeError on lazy=False rather than ignoring it.
Failure handling#
Every registration problem is a warning, not an exception: a driver that raises on import,
declares a different api_version, collides on a name, or names an unknown capability token is
skipped. One broken third-party package cannot stop a NetCDF file from opening.
Note that resolving the group imports and constructs every registered driver at once — it has
to, since your suffixes are an attribute of the instance and cannot be read without building it.
So your module is imported even when the file being opened is some other format. What is contained
is the failure, not the import.
That is also why a driver whose optional dependency is missing should import it at driver-module
scope, so the failure is contained to that one driver while every other keeps working. A plain
try/except ImportError is enough — your package needs nothing from pyramids' internals:
# your_package/drivers/dhi.py
try:
import mikeio
except ImportError as error: # pragma: no cover
raise ImportError(
"The DFSU driver needs mikeio: pip install 'your-package[dhi]'"
) from error
Import it at this module's top level rather than inside read: the registry loads a driver module
only when its suffix is wanted, so the cost is already deferred, and failing at import is what lets
pyramids skip the driver with a warning instead of raising mid-read.
Compatibility#
MESH_DRIVER_API_VERSION is bumped only on a breaking change to the member names above or to the
constructors a driver builds. A driver declaring a different value is skipped with a warning, so an
old plugin never breaks a pyramids upgrade — it stops being discovered until it is updated.
Checking what is installed#
Everything in the contract is re-exported from the package, so a driver never imports a private module:
from pyramids.netcdf.ugrid import (
CAPABILITIES, # the capability vocabulary
ENTRY_POINT_GROUP, # "pyramids.mesh_drivers"
MESH_DRIVER_API_VERSION,
NETCDF_SUFFIXES, # what the built-in reader claims
RESERVED_NAMES, # names a plugin may not take
MeshDriver,
UnsupportedMeshFormatError,
mesh_drivers,
)
mesh_drivers() # {name: driver}, discovered once and cached
dataset.info(drivers=True) # the same list, in the dataset summary
pyramids.netcdf.ugrid
#
UGRID (Unstructured Grid) NetCDF support for pyramids.
This subpackage provides classes for reading, writing, and operating on UGRID-compliant NetCDF files containing unstructured mesh data.
Beyond the core data model (:class:UgridDataset, :class:Mesh2d, :class:Network1d) it
carries the mesh operation modules: :mod:~pyramids.netcdf.ugrid.graph (topology and graph
operations), :mod:~pyramids.netcdf.ugrid.fill (spatial gap filling),
:mod:~pyramids.netcdf.ugrid.vector_ops (mesh↔vector), :mod:~pyramids.netcdf.ugrid.regrid
(sampling and regridding), :mod:~pyramids.netcdf.ugrid.construct (building a mesh from a
raster or polygons), :mod:~pyramids.netcdf.ugrid.interop_xarray (xarray / zarr hand-off) and
:mod:~pyramids.netcdf.ugrid.network (1-D networks).
Each is reachable through a :class:UgridDataset facade; the modules are exported too for
callers holding a bare :class:Mesh2d.
MeshDriver
#
Bases: Protocol
A third-party reader/writer for one mesh file format.
A driver is a class, not a function, because :meth:can_open matters: several of these
formats share .dat, .mesh and .out, so a suffix alone cannot identify one. The
registry instantiates the class with no arguments.
A driver declares exactly one of read_mesh or needs_mesh. read_mesh means the
file carries a topology and :meth:read builds it. needs_mesh means the file carries
values only — an SMS .dat, an XMDF or an XDMF — and :meth:read_data attaches them to a
mesh the caller already has.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
Short unique identifier, and the value accepted by |
long_name |
str
|
Human-readable format name, shown by :meth: |
suffixes |
tuple[str, ...]
|
Lowercase, dot-prefixed file suffixes this driver claims. |
api_version |
int
|
Must equal :data: |
capabilities |
frozenset[str]
|
A subset of :data: |
Source code in src/pyramids/netcdf/ugrid/_drivers.py
130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 | |
can_open(path)
#
Report whether this driver recognises the file.
Sniff the content; never trust the suffix alone. Must not raise for any input — an empty
file, a directory, or a file of another format all answer False — and must not read
the whole file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The file to inspect. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
True when :meth: |
Source code in src/pyramids/netcdf/ugrid/_drivers.py
mesh_names(path)
#
List the topology names in a file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The file to inspect. |
required |
Returns:
| Type | Description |
|---|---|
list[str]
|
One name per topology; a single-mesh format returns a one-item list. |
Source code in src/pyramids/netcdf/ugrid/_drivers.py
read(path, *, mesh_name=None, lazy=True)
#
Read a file that carries its own topology.
Raises for a needs_mesh format, which has no topology to build a mesh from and is
reached through :meth:read_data instead.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The file to read. |
required |
mesh_name
|
str | None
|
Which topology to read, for a multi-mesh file. |
None
|
lazy
|
bool
|
Defer reading variable values until first access. |
True
|
Returns:
| Type | Description |
|---|---|
UgridDataset | Network1d
|
A dataset, or a :class: |
Source code in src/pyramids/netcdf/ugrid/_drivers.py
read_data(path, dataset, *, lazy=True)
#
Read a results-only file's variables, for a mesh supplied by the caller.
Required only when needs_mesh is in :attr:capabilities.
Return only the variables read from the file; :meth:UgridDataset.attach_data
merges them onto the mesh. That division is what makes a name the dataset already
defines an unambiguous collision, and it means a driver cannot drop the dataset's own
variables by rebuilding the mapping.
dataset is passed in so the driver can check the file's declared element count
against it. That check is the only safeguard there is: nothing in a bare results file
ties it to the right mesh, so an unchecked attach yields a plausible, wrong result. A
mismatch is reported by raising :class:ValueError.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str
|
The results file to read. |
required |
dataset
|
UgridDataset
|
The mesh the values will attach to, for the element-count check. |
required |
lazy
|
bool
|
Defer reading variable values until first access. |
True
|
Returns:
| Type | Description |
|---|---|
dict[str, MeshVariable]
|
The variables read from the file, keyed by name. |
Source code in src/pyramids/netcdf/ugrid/_drivers.py
write(dataset, path, **options)
#
Write a dataset in this driver's format.
Required only when write_mesh is in :attr:capabilities.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dataset
|
UgridDataset
|
The dataset to write. |
required |
path
|
str
|
The destination file. |
required |
**options
|
Any
|
Format-specific settings, passed through by pyramids uninterpreted. |
{}
|
Source code in src/pyramids/netcdf/ugrid/_drivers.py
UnsupportedMeshFormatError
#
Bases: FileFormatNotSupportedError
No installed driver reads this file.
Subclasses the package's existing :class:~pyramids.base._errors.FileFormatNotSupportedError
so except FileFormatNotSupportedError keeps catching an unreadable mesh, as it did when
GDAL was the only reader.
Source code in src/pyramids/netcdf/ugrid/_drivers.py
mesh_drivers()
#
Return the installed mesh drivers, discovered once and cached.
Discovery walks every installed distribution, so it runs on the first call only. Nothing on
the UGRID read path calls it: a .nc read, a /vsi path and a plain
:meth:UgridDataset.info all short-circuit first. An explicit driver=, a non-NetCDF
suffix or info(drivers=True) does reach it.
Returns:
| Type | Description |
|---|---|
dict[str, MeshDriver]
|
A copy of the registry: driver instances keyed by :attr: |
dict[str, MeshDriver]
|
discovery order. Mutating it does not affect the registry, though the driver |
dict[str, MeshDriver]
|
instances themselves are shared. |
Examples:
-
Each key is the driver's own declared name:
- Mutating the result does not reach the registry, so one caller cannot unregister a driver for every other:>>> from pyramids.netcdf.ugrid import _drivers >>> class Stub: ... name, long_name, suffixes = "selafin", "Selafin", (".slf",) ... api_version, capabilities = 1, frozenset({"read_mesh"}) >>> _drivers._CACHE = {"selafin": Stub()} >>> sorted(_drivers.mesh_drivers()) ['selafin']>>> from pyramids.netcdf.ugrid import _drivers >>> class Stub: ... name, long_name, suffixes = "selafin", "Selafin", (".slf",) ... api_version, capabilities = 1, frozenset({"read_mesh"}) >>> _drivers._CACHE = {"selafin": Stub()} >>> _drivers.mesh_drivers().clear() >>> sorted(_drivers.mesh_drivers()) ['selafin'] >>> _drivers.clear_driver_cache()