CDS quickstart — downloading ERA5 via the EarthLens facade¶

End-to-end example of pulling a small ERA5 retrieval from the Copernicus Climate Data Store using earthlens. The factory-style EarthLens facade keeps the backend choice behind one string key, so the same call shape works for CHIRPS / S3 / GEE — only data_source changes.

What this notebook covers

1. Verifying CDS credentials.
2. Browsing the variable catalog.
3. Submitting a retrieve through the facade.
4. Inspecting the NetCDF that lands on disk.
5. Bundling per-window aggregation into the same download() call.
6. Plotting one of the resulting GeoTIFFs.

Heads-up. The retrieve in Step 4 blocks on the CDS queue — typically 1–10 minutes for a small request. The bbox / date range used below is intentionally tiny so the demo finishes fast.

Prerequisites¶

Three things need to be in place before any CDS retrieve will work:

1. A CDS account — register at https://cds.climate.copernicus.eu.
2. A Personal Access Token in ~/.cdsapirc (Linux/macOS) or C:\Users\<USER>\.cdsapirc (Windows). Format:

   url: https://cds.climate.copernicus.eu/api
   key: <YOUR-TOKEN>
   

3. The dataset's licence accepted on its CDS page. Open https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels and tick the licence at the bottom of the Download tab.

The package itself is installed via pip install earthlens[ecmwf] (the ecmwf extra pulls in cdsapi).

Setup¶

All the imports this notebook needs, gathered up front. earthlens provides the unified EarthLens entry point, the ecmwf Catalog, and the AggregationConfig; pyramids provides NetCDF / Dataset for reading and plotting the files that land on disk.

In [ ]:

from pathlib import Path

import matplotlib.pyplot as plt
import numpy as np
from pyramids.dataset import Dataset
from pyramids.netcdf import NetCDF

from earthlens import AggregationConfig, EarthLens
from earthlens.ecmwf import Catalog

from pathlib import Path import matplotlib.pyplot as plt import numpy as np from pyramids.dataset import Dataset from pyramids.netcdf import NetCDF from earthlens import AggregationConfig, EarthLens from earthlens.ecmwf import Catalog

Step 1 — verify credentials¶

If ~/.cdsapirc is missing, every retrieve raises AuthenticationError with a message pointing at the file location.

In [ ]:

rc = Path.home() / ".cdsapirc"
print(f"~/.cdsapirc found: {rc.is_file()} ({rc})")

rc = Path.home() / ".cdsapirc" print(f"~/.cdsapirc found: {rc.is_file()} ({rc})")

Step 2 — browse the catalog¶

Catalog loads cds_data_catalog.yaml (shipped as package data) and exposes every CDS dataset earthlens knows about. Loading it once gives the full list of datasets available to a retrieve.

In [ ]:

cat = Catalog()
print(f"{len(cat.available_datasets)} CDS datasets in catalog")

cat = Catalog() print(f"{len(cat.available_datasets)} CDS datasets in catalog")

Look up one variable's metadata¶

Lookups are by (dataset_name, variable_code) since the same short code can appear under more than one dataset. The returned spec carries the per-variable metadata each retrieve needs — the CDS code, the NetCDF variable name, units, and whether it is a flux.

In [ ]:

spec = cat.get_variable("reanalysis-era5-single-levels", "2m-temperature")
print(f"cds_variable: {spec.cds_variable}")
print(f"nc_variable:  {spec.nc_variable}")
print(f"units:        {spec.units}")
print(f"is_flux:      {spec.is_flux}  # state -> auto resolves to mean")

spec = cat.get_variable("reanalysis-era5-single-levels", "2m-temperature") print(f"cds_variable: {spec.cds_variable}") print(f"nc_variable: {spec.nc_variable}") print(f"units: {spec.units}") print(f"is_flux: {spec.is_flux} # state -> auto resolves to mean")

Step 3 — pick a small region and date range¶

Three days of 2-metre temperature over a ~1° box around Coello, Colombia. Small enough that the CDS queue normally serves it in a few minutes; the resulting NetCDF is ~10 KB.

In [ ]:

OUT = Path("data/era5")
OUT.mkdir(parents=True, exist_ok=True)

request = dict(
    start="2022-01-01",
    end="2022-01-03",
    temporal_resolution="daily",
    variables={
        "reanalysis-era5-single-levels": ["2m-temperature"],
    },
    lat_lim=[4.0, 5.0],
    lon_lim=[-75.0, -74.0],
    path=str(OUT),
)
request

OUT = Path("data/era5") OUT.mkdir(parents=True, exist_ok=True) request = dict( start="2022-01-01", end="2022-01-03", temporal_resolution="daily", variables={ "reanalysis-era5-single-levels": ["2m-temperature"], }, lat_lim=[4.0, 5.0], lon_lim=[-75.0, -74.0], path=str(OUT), ) request

Step 4 — submit the retrieve via the EarthLens facade¶

EarthLens(data_source="ecmwf", ...) resolves to the ECMWF backend behind the scenes; switching to a different provider is a one-string change. Construct the facade first, then call download() on it as a separate step — the call assembles the cdsapi request, runs the pre-flight RequestValidator, submits to CDS, and blocks on the queue until the NetCDF is written.

In [ ]:

earthlens = EarthLens(data_source="ecmwf", **request)
earthlens.download()  # blocks during CDS queue + retrieve (~1–10 min)

earthlens = EarthLens(data_source="ecmwf", **request) earthlens.download() # blocks during CDS queue + retrieve (~1–10 min)

Step 5 — verify the NetCDF¶

Per-variable NetCDFs land at <path>/<cds_variable>_<cds_dataset>.nc. Open one with pyramids.netcdf.NetCDF and read the requested variable into an array.

In [ ]:

nc_path = OUT / "2m_temperature_reanalysis-era5-single-levels.nc"
nc = NetCDF.read_file(str(nc_path))
print(f"dimensions: {nc.dimension_names}")

nc_path = OUT / "2m_temperature_reanalysis-era5-single-levels.nc" nc = NetCDF.read_file(str(nc_path)) print(f"dimensions: {nc.dimension_names}")

Inspect the values¶

Read the array for spec.nc_variable and print its shape and value range as a quick sanity check on the retrieved field.

In [ ]:

arr = nc.read_array(variable=spec.nc_variable)
print(f"shape:      {arr.shape}  # (time-slots, lat, lon)")
print(f"range:      {np.nanmin(arr):.2f} .. {np.nanmax(arr):.2f} K")
print(f"mean:       {np.nanmean(arr):.2f} K")

arr = nc.read_array(variable=spec.nc_variable) print(f"shape: {arr.shape} # (time-slots, lat, lon)") print(f"range: {np.nanmin(arr):.2f} .. {np.nanmax(arr):.2f} K") print(f"mean: {np.nanmean(arr):.2f} K")

Step 6 — bundle download + aggregation¶

Daily ERA5 NetCDFs carry four 6-hourly slots per day. To get one GeoTIFF per day with the right reduction (mean for state variables like temperature, sum for fluxes like evaporation), pass an AggregationConfig to download(). The backend retrieves the NetCDF and immediately runs aggregate_netcdf against it.

op="auto" reads Variable.is_flux from the catalog — temperature is a state variable, so auto resolves to mean.

In [ ]:

OUT2 = Path("data/era5-aggregated")
OUT2.mkdir(parents=True, exist_ok=True)

OUT2 = Path("data/era5-aggregated") OUT2.mkdir(parents=True, exist_ok=True)

Construct a second facade pointed at the aggregated output directory, then call download() with the aggregate= argument as its own statement.

In [ ]:

earthlens = EarthLens(data_source="ecmwf", **{**request, "path": str(OUT2)})
earthlens.download(
    aggregate=AggregationConfig(freq="1D", op="auto"),
)

earthlens = EarthLens(data_source="ecmwf", **{**request, "path": str(OUT2)}) earthlens.download( aggregate=AggregationConfig(freq="1D", op="auto"), )

Step 7 — list the per-window GeoTIFFs¶

When aggregate.out_dir is left at its None default, the backend lands the GeoTIFFs at <path>/aggregated/. Each filename follows <cds_variable>_<freq>_<window>.tif.

In [ ]:

tifs = sorted((OUT2 / "aggregated").glob("*.tif"))
for t in tifs:
    print(t.name)

tifs = sorted((OUT2 / "aggregated").glob("*.tif")) for t in tifs: print(t.name)

Step 8 — plot one of the daily-mean rasters¶

Read the first daily GeoTIFF with pyramids.dataset.Dataset, then draw it as a quick visual check that the aggregation produced a sensible 2-D array.

In [ ]:

ds = Dataset.read_file(str(tifs[0]))
img = ds.read_array()

fig, ax = plt.subplots(figsize=(7, 5))
im = ax.imshow(img, cmap="coolwarm")
ax.set_title(f"Daily-mean 2m temperature — {tifs[0].stem}")
ax.set_xlabel("lon (pixels)")
ax.set_ylabel("lat (pixels)")
fig.colorbar(im, ax=ax, label=f"K ({spec.units})")
plt.show()

ds = Dataset.read_file(str(tifs[0])) img = ds.read_array() fig, ax = plt.subplots(figsize=(7, 5)) im = ax.imshow(img, cmap="coolwarm") ax.set_title(f"Daily-mean 2m temperature — {tifs[0].stem}") ax.set_xlabel("lon (pixels)") ax.set_ylabel("lat (pixels)") fig.colorbar(im, ax=ax, label=f"K ({spec.units})") plt.show()

Where to go from here¶

• More variables / longer ranges — add entries to the variables dict and widen start / end. Each retrieve still produces one NetCDF per (dataset, variable) pair.
• Monthly means — name the monthly dataset directly in variables, e.g. "reanalysis-era5-single-levels-monthly-means". The catalog auto-synthesizes that entry; the loader picks product_type= ["monthly_averaged_reanalysis"] for you.
• Pressure-level data — use "reanalysis-era5-pressure-levels" and pass level=1000 (or whatever pressure) on AggregationConfig so the reducer runs on a 3-D slice.
• Other reductions — pass op="sum" / "min" / "max" / "std" explicitly to bypass the catalog-driven auto routing.
• Standalone aggregation — call aggregate_netcdf(nc_path, var_info, config) directly against any pyramids-readable NetCDF; no EarthLens instance needed.

Full reference for the aggregation feature lives under the Data Sources → ECMWF → Aggregation tab in the docs.