TIGGE and fire fuel — the ECDS and XDS endpoints¶

The earthlens.ecmwf backend reaches five data stores through one facade key and one Personal Access Token. Besides the three Copernicus stores (CDS, ADS, EWDS) there are two ECMWF-hosted ones:

Store	endpoint	Datasets
ECDS — ECMWF Data Store	ecds	TIGGE ensemble forecasts, S2S forecasts + reforecasts
XDS — Cross Data Store	xds	Fire fuel characteristics, burned area (1950–2099)

They run the same CADS software as the Copernicus stores, so the same client, the same token and the same constraint validation all apply — only the URL differs.

Prerequisites¶

1. A Copernicus account and a Personal Access Token in ~/.cdsapirc — the same token that works for CDS/ADS/EWDS also authenticates ECDS and XDS.
2. ECDS additionally needs its portal-scope policy accepted (terms-of-use-ecds) at https://ecds.ecmwf.int. This is separate from every dataset licence, so accepting dataset licences alone is not enough — without it every ECDS retrieve returns 403 … user didn't accept all required site policies.
3. Each dataset's own licence accepted on its dataset page. Licences are versioned: holding revision 4 does not satisfy a dataset that requires 5.

Setup¶

EarthLens is the unified entry point; the ecmwf Catalog lets us inspect what is curated on each store before downloading anything.

In [ ]:

import os
from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
import xarray as xr

from earthlens.core import EarthLens
from earthlens.ecmwf import Catalog

# Downloads land in the shared cache, not in the docs tree and not on the
# system drive. Override with EARTHLENS_CACHE if your cache lives elsewhere.
OUT = Path(os.environ.get('EARTHLENS_CACHE', 'D:/earthlens-cache')) / 'ecds_xds'
OUT.mkdir(parents=True, exist_ok=True)
print('writing to', OUT)

import os from pathlib import Path import matplotlib.pyplot as plt import numpy as np import xarray as xr from earthlens.core import EarthLens from earthlens.ecmwf import Catalog # Downloads land in the shared cache, not in the docs tree and not on the # system drive. Override with EARTHLENS_CACHE if your cache lives elsewhere. OUT = Path(os.environ.get('EARTHLENS_CACHE', 'D:/earthlens-cache')) / 'ecds_xds' OUT.mkdir(parents=True, exist_ok=True) print('writing to', OUT)

What lives on the two stores¶

Catalog.store_for() resolves any dataset id to the store that serves it, and available_datasets is the full universe each store offers. All five are curated today; anything a store added later would show up here as passthrough only and still be downloadable through the raw-request passthrough.

In [ ]:

catalog = Catalog()

for store in ('ecds', 'xds'):
    offered = [d for d in catalog.available_datasets if catalog.store_for(d) == store]
    curated = [d for d in offered if d in catalog.datasets]
    print(f'{store}: {len(offered)} offered, {len(curated)} curated')
    for dataset in offered:
        mark = 'curated' if dataset in catalog.datasets else 'passthrough only'
        print(f'   - {dataset:36s} {mark}')

catalog = Catalog() for store in ('ecds', 'xds'): offered = [d for d in catalog.available_datasets if catalog.store_for(d) == store] curated = [d for d in offered if d in catalog.datasets] print(f'{store}: {len(offered)} offered, {len(curated)} curated') for dataset in offered: mark = 'curated' if dataset in catalog.datasets else 'passthrough only' print(f' - {dataset:36s} {mark}')

TIGGE — a multi-centre ensemble forecast (ECDS)¶

TIGGE collects medium-range ensemble forecasts from 13 NWP centres. The curated row pins origin: ecmwf and a 24-hour lead time; the variable is ECMWF's 2 m temperature.

The request vocabulary is not the MARS idiom — it is ecmwf (not ecmf), single_level (not surface) and 2_m_temperature (with underscores around the m). The curated row already encodes the correct values.

In [ ]:

variable = catalog.get_variable('tigge-forecasts', '2m-temperature')
print('CDS variable :', variable.cds_variable)
print('NetCDF name  :', variable.nc_variable)
print('units        :', variable.units)
print('endpoint     :', variable.endpoint)

variable = catalog.get_variable('tigge-forecasts', '2m-temperature') print('CDS variable :', variable.cds_variable) print('NetCDF name :', variable.nc_variable) print('units :', variable.units) print('endpoint :', variable.endpoint)

In [ ]:

tigge = EarthLens(
    data_source='ecmwf',
    variables={'tigge-forecasts': ['2m-temperature']},
    start='2024-01-01',
    end='2024-01-01',
    temporal_resolution='daily',
    lat_lim=[50.0, 51.0],
    lon_lim=[9.0, 10.0],
    path=str(OUT / 'tigge'),
).download()

tigge

tigge = EarthLens( data_source='ecmwf', variables={'tigge-forecasts': ['2m-temperature']}, start='2024-01-01', end='2024-01-01', temporal_resolution='daily', lat_lim=[50.0, 51.0], lon_lim=[9.0, 10.0], path=str(OUT / 'tigge'), ).download() tigge

TIGGE returns an unstructured grid¶

This is the one place TIGGE differs from every other curated ECMWF dataset: the NetCDF carries a values dimension rather than lat / lon. TIGGE serves its native reduced-Gaussian representation, and the area selector subsets points — it does not regrid. If you need a regular grid, regrid downstream.

In [ ]:

ds = xr.open_dataset(tigge[0])
print('dims :', dict(ds.sizes))
print('vars :', list(ds.data_vars))

t2m = ds['t2m']
print('units:', t2m.attrs.get('units'))
print(f'range: {float(t2m.min()):.1f} .. {float(t2m.max()):.1f} K')

ds = xr.open_dataset(tigge[0]) print('dims :', dict(ds.sizes)) print('vars :', list(ds.data_vars)) t2m = ds['t2m'] print('units:', t2m.attrs.get('units')) print(f'range: {float(t2m.min()):.1f} .. {float(t2m.max()):.1f} K')

In [ ]:

values = np.asarray(ds['t2m'].values).ravel()

fig, ax = plt.subplots(figsize=(7, 3.5))
ax.plot(values - 273.15, marker='o', linestyle='-', color='#c1440e')
ax.set_xlabel('grid point (unstructured `values` index)')
ax.set_ylabel('2 m temperature (°C)')
ax.set_title('TIGGE ECMWF control forecast, +24 h — 2024-01-01')
ax.grid(alpha=0.3)
plt.tight_layout()
plt.show()
ds.close()

values = np.asarray(ds['t2m'].values).ravel() fig, ax = plt.subplots(figsize=(7, 3.5)) ax.plot(values - 273.15, marker='o', linestyle='-', color='#c1440e') ax.set_xlabel('grid point (unstructured `values` index)') ax.set_ylabel('2 m temperature (°C)') ax.set_title('TIGGE ECMWF control forecast, +24 h — 2024-01-01') ax.grid(alpha=0.3) plt.tight_layout() plt.show() ds.close()

Fire fuel moisture — XDS¶

XDS carries modelled fire-fuel characteristics. Unlike TIGGE this comes back on a regular lat/lon grid, wrapped in a ZIP that earthlens unpacks for you.

XDS is not an operational service. Every response carries a notice that it is provided for research use as is, without guaranteed support or maintenance.

In [ ]:

fuel = EarthLens(
    data_source='ecmwf',
    variables={'derived-fire-fuel-biomass': ['live-fuel-moisture-content-group']},
    start='2000-01-01',
    end='2000-01-31',
    temporal_resolution='monthly',
    lat_lim=[50.0, 51.0],
    lon_lim=[9.0, 10.0],
    path=str(OUT / 'fuel'),
).download()

fuel

fuel = EarthLens( data_source='ecmwf', variables={'derived-fire-fuel-biomass': ['live-fuel-moisture-content-group']}, start='2000-01-01', end='2000-01-31', temporal_resolution='monthly', lat_lim=[50.0, 51.0], lon_lim=[9.0, 10.0], path=str(OUT / 'fuel'), ).download() fuel

In [ ]:

fds = xr.open_dataset(fuel[0])
print('dims :', dict(fds.sizes))
print('vars :', list(fds.data_vars))

lfmc = fds['LFMC']
print('units:', lfmc.attrs.get('units'))

fds = xr.open_dataset(fuel[0]) print('dims :', dict(fds.sizes)) print('vars :', list(fds.data_vars)) lfmc = fds['LFMC'] print('units:', lfmc.attrs.get('units'))

In [ ]:

field = lfmc.isel(time=0)

fig, ax = plt.subplots(figsize=(6, 5))
mesh = ax.pcolormesh(fds['lon'], fds['lat'], field, cmap='YlGn', shading='auto')
fig.colorbar(mesh, ax=ax, label='live fuel moisture content (%)')
ax.set_xlabel('longitude')
ax.set_ylabel('latitude')
ax.set_title('LFMC — January 2000')
plt.tight_layout()
plt.show()
fds.close()

field = lfmc.isel(time=0) fig, ax = plt.subplots(figsize=(6, 5)) mesh = ax.pcolormesh(fds['lon'], fds['lat'], field, cmap='YlGn', shading='auto') fig.colorbar(mesh, ax=ax, label='live fuel moisture content (%)') ax.set_xlabel('longitude') ax.set_ylabel('latitude') ax.set_title('LFMC — January 2000') plt.tight_layout() plt.show() fds.close()

S2S reforecasts — two date axes¶

s2s-reforecasts is the subtlest row on either store. It carries two dates: year/month/day select the model cycle (which forecast system version ran), and hyear/hmonth/hday select the reforecast date being re-run. The store only serves a reforecast on the model run's own calendar day, so the two must move together.

request_kind: s2s_reforecast copies the requested month/day into hmonth/hday; only the reforecast year is a per-row choice. (The existing glofas_hindcast kind cannot be reused — it renames year to hyear, which would delete the model-cycle date this dataset also needs.)

In [ ]:

spec = catalog.get_variable(
    's2s-reforecasts', 'maximum-2m-temperature-in-the-last-6-hours'
)
print('request_kind :', spec.request_kind)
print('NetCDF name  :', spec.nc_variable, '|', spec.units)
pinned = {k: v for k, v in spec.extras.items() if k.startswith('h')}
print('pinned h* extras:', pinned, '<- only the year')

spec = catalog.get_variable( 's2s-reforecasts', 'maximum-2m-temperature-in-the-last-6-hours' ) print('request_kind :', spec.request_kind) print('NetCDF name :', spec.nc_variable, '|', spec.units) pinned = {k: v for k, v in spec.extras.items() if k.startswith('h')} print('pinned h* extras:', pinned, '<- only the year')

Reaching an unverified variable¶

Every dataset on both stores is curated, but only the variables verified against a real download carry rows — TIGGE alone exposes 37. Anything else stays reachable through the raw-request passthrough, which takes the store's own request dict verbatim and needs no curated row:

EarthLens(
    data_source='ecmwf',
    dataset='s2s-forecasts',
    request={...},          # the store's own request dict
    endpoint='ecds',        # auto-resolved from the index if omitted
    path='data/s2s',
).download()

Summary¶

• One facade key (ecmwf) and one token reach five stores; only endpoint differs.
• ECDS needs its portal-scope terms-of-use-ecds policy on top of dataset licences.
• TIGGE returns an unstructured values grid — area subsets points, it does not regrid.
• S2S shares TIGGE's vocabulary but returns a regular lat/lon grid.
• S2S reforecasts need both a model-cycle date and a reforecast date.
• XDS returns a regular lat/lon grid inside a ZIP, and is research-grade, not operational.
• Any unverified variable stays reachable through the raw-request passthrough.