NetCDF Utilities

Low-level GDAL helpers for attribute reading, no-data extraction, scale/offset handling, time conversion, and dtype utilities.

pyramids.netcdf.utils

resolve_full_name(obj, group_full_name, fallback_name)

Full hierarchical name of a GDAL dimension / array, with a qualified fallback.

Tries obj.GetFullName(); when that raises (some drivers don't expose it on every object), composes the path from the parent group's full name and fallback_name — "<group>/<name>", or "/<name>" when the parent is the root group.

Parameters:

Name	Type	Description	Default
obj	Any	A GDAL dimension or MDArray.	required
group_full_name	str	Full name of the parent group (e.g. "/" or "/grp").	required
fallback_name	str	The object's short name, used to build the fallback path.	required

Returns:

Type	Description
str	The resolved full-name string.

Source code in src/pyramids/netcdf/utils.py

def resolve_full_name(obj: Any, group_full_name: str, fallback_name: str) -> str:
    """Full hierarchical name of a GDAL dimension / array, with a qualified fallback.

    Tries ``obj.GetFullName()``; when that raises (some drivers don't expose it on every
    object), composes the path from the parent group's full name and ``fallback_name`` —
    ``"<group>/<name>"``, or ``"/<name>"`` when the parent is the root group.

    Args:
        obj: A GDAL dimension or MDArray.
        group_full_name: Full name of the parent group (e.g. ``"/"`` or ``"/grp"``).
        fallback_name: The object's short name, used to build the fallback path.

    Returns:
        The resolved full-name string.
    """
    try:
        return cast("str", obj.GetFullName())
    except Exception:
        if group_full_name != "/":
            return f"{group_full_name}/{fallback_name}"
        return f"/{fallback_name}"

create_time_conversion_func(units, out_format='%Y-%m-%d %H:%M:%S', calendar='standard')

Create a converter that maps numeric CF time offsets to date strings.

Parses CF-compliant time unit strings (e.g., "days since 1979-01-01") and returns a callable that converts numeric offsets to formatted date strings.

For standard/proleptic_gregorian calendars, uses Python's datetime + timedelta. For non-standard calendars (360_day, noleap, all_leap, julian), uses cftime.num2date() (optional dependency).

Parameters:

Name	Type	Description	Default
units	str	CF time unit string in the format "<unit> since <origin>". Supported units are days, hours, minutes, seconds, milliseconds, microseconds, and nanoseconds. Sub-second units decode at microsecond resolution (Python datetime's finest), so a nanosecond axis is rounded to the nearest microsecond — exact for date/second output, and visible only in a %f (microsecond) out_format.	required
out_format	str	strftime format for the output strings. Defaults to "%Y-%m-%d %H:%M:%S".	'%Y-%m-%d %H:%M:%S'
calendar	str	CF calendar type. Defaults to "standard". Non-standard calendars are decoded with cftime (a core dependency).	'standard'

Returns:

Name	Type	Description
Callable	Callable	A function that takes a numeric value and returns a formatted date string.

Raises:

Type	Description
ValueError	If the unit string cannot be parsed or uses an unsupported time unit.

Examples:

• Convert day offsets from a 1979 origin:

  >>> from pyramids.netcdf.utils import (
  ...     create_time_conversion_func,
  ... )
  >>> convert = create_time_conversion_func(
  ...     "days since 1979-01-01"
  ... )
  >>> convert(0)
  '1979-01-01 00:00:00'
  >>> convert(365)
  '1980-01-01 00:00:00'
  

• Use hour-based units with a custom format:

  >>> convert = create_time_conversion_func(
  ...     "hours since 2000-01-01",
  ...     out_format="%Y-%m-%d",
  ... )
  >>> convert(24)
  '2000-01-02'
  >>> convert(0)
  '2000-01-01'
  

See Also

_parse_units_origin: Parses the unit string.

Source code in src/pyramids/netcdf/utils.py

def create_time_conversion_func(
    units: str,
    out_format: str = "%Y-%m-%d %H:%M:%S",
    calendar: str = "standard",
) -> Callable:
    """Create a converter that maps numeric CF time offsets to date strings.

    Parses CF-compliant time unit strings (e.g.,
    `"days since 1979-01-01"`) and returns a callable that
    converts numeric offsets to formatted date strings.

    For standard/proleptic_gregorian calendars, uses Python's
    `datetime` + `timedelta`. For non-standard calendars
    (`360_day`, `noleap`, `all_leap`, `julian`), uses
    `cftime.num2date()` (optional dependency).

    Args:
        units: CF time unit string in the format
            `"<unit> since <origin>"`. Supported units are days, hours,
            minutes, seconds, milliseconds, microseconds, and nanoseconds.
            Sub-second units decode at microsecond resolution (Python
            `datetime`'s finest), so a nanosecond axis is rounded to the
            nearest microsecond — exact for date/second output, and visible
            only in a `%f` (microsecond) `out_format`.
        out_format: strftime format for the output strings.
            Defaults to `"%Y-%m-%d %H:%M:%S"`.
        calendar: CF calendar type. Defaults to `"standard"`.
            Non-standard calendars are decoded with `cftime` (a core
            dependency).

    Returns:
        Callable: A function that takes a numeric value and
            returns a formatted date string.

    Raises:
        ValueError: If the unit string cannot be parsed or
            uses an unsupported time unit.

    Examples:
        - Convert day offsets from a 1979 origin:
            ```python
            >>> from pyramids.netcdf.utils import (
            ...     create_time_conversion_func,
            ... )
            >>> convert = create_time_conversion_func(
            ...     "days since 1979-01-01"
            ... )
            >>> convert(0)
            '1979-01-01 00:00:00'
            >>> convert(365)
            '1980-01-01 00:00:00'

            ```

        - Use hour-based units with a custom format:
            ```python
            >>> convert = create_time_conversion_func(
            ...     "hours since 2000-01-01",
            ...     out_format="%Y-%m-%d",
            ... )
            >>> convert(24)
            '2000-01-02'
            >>> convert(0)
            '2000-01-01'

            ```

    See Also:
        _parse_units_origin: Parses the unit string.
    """
    converter = None

    if not _is_standard_calendar(calendar):

        def convert_cftime(value):
            dt = cftime.num2date(value, units, calendar)
            return dt.strftime(out_format)

        converter = convert_cftime
    else:
        unit, origin = _parse_units_origin(units)

        # Microseconds per CF unit. datetime/timedelta is microsecond-resolution, so the
        # offset is resolved as ``origin + timedelta(microseconds=value * factor)``:
        # day/hour/minute/second stay exact over any realistic date range, while the
        # sub-second units (notably the ``nanoseconds`` axis DatasetCollection.to_netcdf
        # writes) round to the nearest microsecond — exact for date/second output, with any
        # residual only in the microsecond digit of a ``%f`` format.
        micros_per_unit = {
            "day": 86_400_000_000.0,
            "hour": 3_600_000_000.0,
            "min": 60_000_000.0,
            "sec": 1_000_000.0,
            "millisecond": 1_000.0,
            "microsecond": 1.0,
            "nanosecond": 0.001,
        }
        factor = next(
            (m for prefix, m in micros_per_unit.items() if unit.startswith(prefix)),
            None,
        )
        if factor is None:
            raise ValueError(f"Unsupported time unit: {unit!r}")

        def convert(value):
            dt = origin + timedelta(microseconds=float(value) * factor)
            return dt.strftime(out_format)

        converter = convert

    return converter

decode_cf_time(values, unit, calendar='standard')

Decode numeric CF time offsets to datetimes.

Standard / gregorian / proleptic_gregorian calendars yield datetime64[ns] when representable; non-standard calendars (360_day / noleap …) yield cftime objects. Arrays whose unit is not a "<interval> since <origin>" string are returned unchanged.

Parameters:

Name	Type	Description	Default
values	ndarray	The numeric values already read for the coordinate.	required
unit	str | None	The coordinate's CF unit string (e.g. "days since 1979-01-01").	required
calendar	str	The CF calendar name. Defaults to "standard".	'standard'

Returns:

Type	Description
NDArray	np.ndarray: Decoded datetimes for a time axis, else values unchanged.

Source code in src/pyramids/netcdf/utils.py

def decode_cf_time(
    values: np.ndarray,
    unit: str | None,
    calendar: str = "standard",
) -> np.typing.NDArray:
    """Decode numeric CF time offsets to datetimes.

    Standard / gregorian / proleptic_gregorian calendars yield ``datetime64[ns]`` when
    representable; non-standard calendars (``360_day`` / ``noleap`` …) yield ``cftime``
    objects. Arrays whose ``unit`` is not a ``"<interval> since <origin>"`` string are
    returned unchanged.

    Args:
        values: The numeric values already read for the coordinate.
        unit: The coordinate's CF unit string (e.g. ``"days since 1979-01-01"``).
        calendar: The CF calendar name. Defaults to ``"standard"``.

    Returns:
        np.ndarray: Decoded datetimes for a time axis, else ``values`` unchanged.
    """
    if not unit or " since " not in unit:
        return values
    standard = _is_standard_calendar(calendar)
    decoded = np.asarray(
        cftime.num2date(values, unit, calendar, only_use_cftime_datetimes=not standard)
    )
    if standard:
        try:
            return decoded.astype("datetime64[ns]")
        except (ValueError, TypeError):
            return decoded
    return decoded

encode_cf_time(value, unit, calendar='standard')

Convert a date string / datetime to a coordinate's numeric CF scale.

The inverse of :func:decode_cf_time for a single value: used to translate a user-supplied selection bound back to the time axis's stored numbers.

Parameters:

Name	Type	Description	Default
value	Any	A date string, :class:datetime, or anything :class:pandas.Timestamp accepts.	required
unit	str	The CF unit string the numeric scale is expressed in.	required
calendar	str	The CF calendar name.	'standard'

Returns:

Name	Type	Description
float	float	value expressed in unit on the given calendar.

Source code in src/pyramids/netcdf/utils.py

def encode_cf_time(value: Any, unit: str, calendar: str = "standard") -> float:
    """Convert a date string / datetime to a coordinate's numeric CF scale.

    The inverse of :func:`decode_cf_time` for a single value: used to translate a
    user-supplied selection bound back to the time axis's stored numbers.

    Args:
        value: A date string, :class:`datetime`, or anything :class:`pandas.Timestamp`
            accepts.
        unit: The CF unit string the numeric scale is expressed in.
        calendar: The CF calendar name.

    Returns:
        float: ``value`` expressed in ``unit`` on the given ``calendar``.
    """
    year, month, day, hour, minute, second, microsecond = _datetime_components(value)
    dt = cftime.datetime(
        year,
        month,
        day,
        hour,
        minute,
        second,
        microsecond,
        calendar=calendar,
    )
    return float(cftime.date2num(dt, unit, calendar))

read_cf_attributes(obj)

Read a GDAL object's attributes the way a CF consumer needs them.

The CF reader for every consumer of units / calendar / standard_name / axis. Use this rather than :func:_read_attributes wherever the attributes are interpreted as CF: attributes alone omit units, because GDAL moves it to the object's unit slot, so a site that reads them raw silently loses it — which is how a time axis came back undecodable, CF axis detection by units never fired, and a UGRID variable's unit vanished on a round trip (#1078).

:func:_read_attributes remains the right call for a verbatim view of what the file declares (serialisation, round-trip writers); this one is for interpreting CF.

The slot is the CF units for the netCDF and HDF5 drivers, which is where this matters. Another multidim driver (Zarr, HDF-EOS) may fill it from a format-specific field, in which case the reported units is that driver's unit rather than a CF attribute the file literally declares — a reason to read attributes raw when the question is "what does this file say", not "what does this axis mean".

Parameters:

Name	Type	Description	Default
obj	Any	Any GDAL object exposing GetAttributes() and GetUnit().	required

Returns:

Type	Description
dict[str, AttributeValue]	dict[str, AttributeValue]: The object's attributes, including units.

Source code in src/pyramids/netcdf/utils.py

def read_cf_attributes(obj: Any) -> dict[str, AttributeValue]:
    """Read a GDAL object's attributes the way a CF consumer needs them.

    The CF reader for every consumer of ``units`` / ``calendar`` / ``standard_name`` /
    ``axis``. Use this rather than :func:`_read_attributes` wherever the attributes are
    interpreted as CF: attributes alone omit ``units``, because GDAL moves it to the object's
    unit slot, so a site that reads them raw silently loses it — which is how a time axis came
    back undecodable, CF axis detection by units never fired, and a UGRID variable's unit
    vanished on a round trip (#1078).

    :func:`_read_attributes` remains the right call for a verbatim view of what the file
    declares (serialisation, round-trip writers); this one is for interpreting CF.

    The slot is the CF ``units`` for the netCDF and HDF5 drivers, which is where this matters.
    Another multidim driver (Zarr, HDF-EOS) may fill it from a format-specific field, in which
    case the reported ``units`` is that driver's unit rather than a CF attribute the file
    literally declares — a reason to read attributes raw when the question is "what does this
    file say", not "what does this axis mean".

    Args:
        obj: Any GDAL object exposing ``GetAttributes()`` and ``GetUnit()``.

    Returns:
        dict[str, AttributeValue]: The object's attributes, including ``units``.
    """
    return _merge_unit(_read_attributes(obj), obj)