Climate indices — catalog & behaviour (offline)¶

Inspect the bundled index catalogue and the backend's design rules without touching the network: which indices ship, their source and ASCII dialect, the two pure parsers, and why this backend ignores spatial arguments and rejects aggregate=. This is the no-network companion to the live quickstart.

In [1]:

import pandas as pd

from earthlens.climate_indices import (
    Catalog,
    ClimateIndices,
    parse_climexp,
    parse_psl,
)

import pandas as pd from earthlens.climate_indices import ( Catalog, ClimateIndices, parse_climexp, parse_psl, )

The shipped indices¶

Catalog() loads the bundled climate_indices_data_catalog.yaml. Each row is a frozen pydantic Index carrying its source, ASCII dialect, file URL, units, and citation.

In [2]:

catalog = Catalog()
pd.DataFrame(
    [
        {
            'id': i,
            'source': catalog.get(i).source,
            'dialect': catalog.get(i).dialect,
            'units': catalog.get(i).units,
            'long_name': catalog.get(i).long_name,
        }
        for i in catalog.available()
    ]
).set_index('id')

catalog = Catalog() pd.DataFrame( [ { 'id': i, 'source': catalog.get(i).source, 'dialect': catalog.get(i).dialect, 'units': catalog.get(i).units, 'long_name': catalog.get(i).long_name, } for i in catalog.available() ] ).set_index('id')

Out[2]:

	source	dialect	units	long_name
id
amo	knmi-climexp	climexp	degC	Atlantic Multidecadal Oscillation (detrended, ...
ao	noaa-psl	psl	std	Arctic Oscillation
censo	noaa-psl	psl	std	Bivariate ENSO index (CENSO)
nao	noaa-psl	psl	std	North Atlantic Oscillation (CPC)
nina34	noaa-psl	psl	degC	Niño 3.4 sea-surface-temperature anomaly
oni	noaa-psl	psl	degC	Oceanic Niño Index (3-month running mean of Ni...
pdo	noaa-psl	psl	std	Pacific Decadal Oscillation
pna	noaa-psl	psl	std	Pacific/North American teleconnection pattern
soi	noaa-psl	psl	std	Southern Oscillation Index
tni	noaa-psl	psl	degC	Trans-Niño Index

The two ASCII dialects¶

Each source ships a different layout. The parsers are pure text → pandas, so we can show them on tiny inline samples — no download needed.

NOAA PSL (psl): a <first_year> <last_year> header, one year jan..dec row per year, then a lone missing-value sentinel line (it varies per file) and a free-text footer.

In [3]:

psl_sample = '''        2000        2001
 2000   0.50   0.10  -0.20   0.30   0.40  -0.10   0.05   0.15  -0.25   0.35   0.45  -0.05
 2001   0.20  -0.30   0.10 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90
  -99.9
  Example index — provenance footer line
'''
parse_psl(psl_sample).head(4)

psl_sample = ''' 2000 2001 2000 0.50 0.10 -0.20 0.30 0.40 -0.10 0.05 0.15 -0.25 0.35 0.45 -0.05 2001 0.20 -0.30 0.10 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.90 -99.9 Example index — provenance footer line ''' parse_psl(psl_sample).head(4)

Out[3]:

	date	value
0	2000-01-01	0.5
1	2000-02-01	0.1
2	2000-03-01	-0.2
3	2000-04-01	0.3

The lone -99.9 line is detected as the sentinel and mapped to NaN; the header and footer are ignored. KNMI climexp (climexp) instead uses #-comment lines and rows of year jan..dec (a trailing annual-mean column is dropped when present); its sentinel is -999.9.

In [4]:

climexp_sample = '''# title :: Example climexp index
# source :: https://climexp.knmi.nl
 2000  0.11  0.22  0.33  0.44  0.55  0.66  0.77  0.88  0.99  1.00  1.11  1.22  0.69
 2001  0.10 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9
'''
parse_climexp(climexp_sample).head(4)

climexp_sample = '''# title :: Example climexp index # source :: https://climexp.knmi.nl 2000 0.11 0.22 0.33 0.44 0.55 0.66 0.77 0.88 0.99 1.00 1.11 1.22 0.69 2001 0.10 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 -999.9 ''' parse_climexp(climexp_sample).head(4)

Out[4]:

	date	value
0	2000-01-01	0.11
1	2000-02-01	0.22
2	2000-03-01	0.33
3	2000-04-01	0.44

Resolving ids — with a did-you-mean hint¶

An unknown id raises a ValueError that suggests the closest match, rather than failing silently.

In [5]:

raises-exception

catalog.get('noo')

catalog.get('noo')

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[5], line 1
----> 1 catalog.get('noo')

File C:\gdrive\algorithms\remote-sensing\earthlens\.claude\worktrees\climate-indices\src\earthlens\climate_indices\catalog.py:260, in Catalog.get(self, index_id)
    243 def get(self, index_id: str) -> Index:
    244     """Resolve an index id to its :class:`Index` row.
    245 
    246     Thin wrapper over the inherited :meth:`get_dataset`, which raises
   (...)    258             adds a did-you-mean hint.
    259     """
--> 260     return self.get_dataset(index_id)

File C:\gdrive\algorithms\remote-sensing\earthlens\.claude\worktrees\climate-indices\src\earthlens\base\abstractdatasource.py:1072, in AbstractCatalog.get_dataset(self, name)
   1070 close = difflib.get_close_matches(name, self.datasets, n=1)
   1071 hint = f" Did you mean {close[0]!r}?" if close else ""
-> 1072 raise ValueError(
   1073     f"{name!r} is not in the {self._catalog_kind}. "
   1074     f"Known {self._entry_noun}: {sorted(self.datasets)}.{hint}"
   1075 ) from None

ValueError: 'noo' is not in the climate-indices catalog. Known indices: ['amo', 'ao', 'censo', 'nao', 'nina34', 'oni', 'pdo', 'pna', 'soi', 'tni']. Did you mean 'nao'?

Global scalars: spatial arguments are ignored¶

Climate indices have no geometry, so the backend accepts a bbox for signature parity but never uses it — the spatial extent is always the whole globe. (Constructing the backend does no network I/O.)

In [6]:

src = ClimateIndices(
    start='2000-01-01',
    end='2001-12-31',
    variables=['oni'],
    lat_lim=[10.0, 20.0],   # accepted ...
    lon_lim=[30.0, 40.0],   # ... but ignored
    path='ci_out',
)
space = src.space
(space.south, space.north, space.west, space.east)

src = ClimateIndices( start='2000-01-01', end='2001-12-31', variables=['oni'], lat_lim=[10.0, 20.0], # accepted ... lon_lim=[30.0, 40.0], # ... but ignored path='ci_out', ) space = src.space (space.south, space.north, space.west, space.east)

Out[6]:

(-90.0, 90.0, -180.0, 180.0)

Nothing to grid-reduce: aggregate= is rejected¶

The values are already monthly scalars, so a non-None aggregate= raises NotImplementedError (the check fires before any network call). Do any rollup on the returned DataFrame instead — see the quickstart's annual-means example.

In [7]:

raises-exception

src.download(aggregate=object())

src.download(aggregate=object())

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[7], line 1
----> 1 src.download(aggregate=object())

File C:\gdrive\algorithms\remote-sensing\earthlens\.claude\worktrees\climate-indices\src\earthlens\climate_indices\backend.py:341, in ClimateIndices.download(self, progress_bar, aggregate)
    318 """Fetch every requested index, write the table, and return it.
    319 
    320 Args:
   (...)    338         (`G8`).
    339 """
    340 if aggregate is not None:
--> 341     raise NotImplementedError(
    342         "ClimateIndices.download(aggregate=...) is not supported: "
    343         "climate indices are tabular monthly scalars, not gridded "
    344         "rasters, so there is no meaningful gridded reduction. Call "
    345         "download() without aggregate= and post-process the returned "
    346         "DataFrame directly."
    347     )
    348 frames = [frame for frame in self._api() if len(frame)]
    349 df = (
    350     pd.concat(frames, ignore_index=True)
    351     if frames
    352     else _helpers.empty_canonical()
    353 )

NotImplementedError: ClimateIndices.download(aggregate=...) is not supported: climate indices are tabular monthly scalars, not gridded rasters, so there is no meaningful gridded reduction. Call download() without aggregate= and post-process the returned DataFrame directly.

An empty selection is an error, not a silent 'fetch everything'¶

In [8]:

raises-exception

ClimateIndices(start='2000-01-01', end='2001-12-31', variables=[])

ClimateIndices(start='2000-01-01', end='2001-12-31', variables=[])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[8], line 1
----> 1 ClimateIndices(start='2000-01-01', end='2001-12-31', variables=[])

File C:\gdrive\algorithms\remote-sensing\earthlens\.claude\worktrees\climate-indices\src\earthlens\base\abstractdatasource.py:560, in AbstractDataSource.__init_subclass__.<locals>.__init__(self, aoi, buffer, cadence, dataset, *args, **kw)
    556 elif buffer is not None:
    557     raise ValueError(
    558         "buffer= only applies to a point aoi=(lon, lat); pass aoi= too"
    559     )
--> 560 orig(self, *args, **kw)
    561 if clip_geometry is not None:
    562     self._attach_clip_geometry(clip_geometry)

File C:\gdrive\algorithms\remote-sensing\earthlens\.claude\worktrees\climate-indices\src\earthlens\climate_indices\backend.py:125, in ClimateIndices.__init__(self, start, end, variables, lat_lim, lon_lim, temporal_resolution, path, fmt, output_format)
    123 index_ids = list(variables) if variables else []
    124 if not index_ids:
--> 125     raise ValueError(
    126         "ClimateIndices needs variables=[index id, ...]; available "
    127         f"indices: {Catalog().available()}."
    128     )
    129 if output_format not in OUTPUT_FORMATS:
    130     raise ValueError(
    131         f"output_format must be one of {list(OUTPUT_FORMATS)}, "
    132         f"got {output_format!r}."
    133     )

ValueError: ClimateIndices needs variables=[index id, ...]; available indices: ['amo', 'ao', 'censo', 'nao', 'nina34', 'oni', 'pdo', 'pna', 'soi', 'tni'].

Takeaway¶

• The catalogue maps each id → source + ASCII dialect + metadata; list it with Catalog().available().
• Two pure parsers (parse_psl / parse_climexp) turn the source text into the canonical date / value frame, sentinel → NaN.
• The backend is tabular and global: spatial args are ignored and aggregate= is rejected by design.