Regional, ensemble & coastal configurations¶

Beyond CONUS short_range, NWM publishes 55 configurations: regional domains (Alaska / Hawaii / Puerto Rico), GFS-forced ensembles (medium_range 1-6, long_range 1-4), and SCHISM coastal runs. This notebook browses them from the catalog and does one live small regional pull. No credentials.

Setup¶

All the imports up front. earthlens.nwm.Catalog is the offline configuration index; EarthLens is the unified entry point used for the live pull further down. We also instantiate the catalog once and reuse it.

In [ ]:

import datetime as dt
import tempfile
from collections import Counter
from pathlib import Path

from earthlens import EarthLens
from earthlens.nwm import Catalog

cat = Catalog()

import datetime as dt import tempfile from collections import Counter from pathlib import Path from earthlens import EarthLens from earthlens.nwm import Catalog cat = Catalog()

Browse configurations by domain¶

Counting the configurations by domain shows the regional spread, and pulling out the ones with members lists the GFS-forced ensembles.

In [ ]:

by_domain = Counter(c.domain for c in cat.configurations.values())
print('configurations:', len(cat.available_configurations))
print('by domain:', dict(by_domain))
ensembles = {k: c.members for k, c in cat.configurations.items() if c.members}
print('ensembles:', ensembles)

by_domain = Counter(c.domain for c in cat.configurations.values()) print('configurations:', len(cat.available_configurations)) print('by domain:', dict(by_domain)) ensembles = {k: c.members for k, c in cat.configurations.items() if c.members} print('ensembles:', ensembles)

The Hawaii short-range domain is sub-hourly¶

Its steps are 15-minute, written as 5-digit f00015, f00030, … — the catalog pins the step_width and cadence so the backend formats the key correctly.

Reading the short_range_hawaii config back from the catalog surfaces the domain, step_width, cadence and horizon the backend uses to build keys.

In [ ]:

hi = cat.get_config('short_range_hawaii')
print(
    'domain:',
    hi.domain,
    '| step_width:',
    hi.step_width,
    '| cadence(min):',
    hi.step_cadence_h,
    '| horizon:',
    hi.horizon_h,
)

hi = cat.get_config('short_range_hawaii') print( 'domain:', hi.domain, '| step_width:', hi.step_width, '| cadence(min):', hi.step_cadence_h, '| horizon:', hi.horizon_h, )

Live: one Hawaii short-range streamflow file¶

Regional files are small (~0.1 MB), so this is a quick real download.

Build the request¶

We target a recent cycle (3 days back to be safely published), the short_range_hawaii configuration and the chrtout streamflow product, writing into a throwaway temp directory.

In [ ]:

date = (dt.datetime.now() - dt.timedelta(days=3)).strftime('%Y-%m-%d')
nwm = EarthLens(
    data_source="nwm",
    start=date,
    end=date,
    dataset='chrtout',
    variables=['streamflow'],
    aoi=[-180, -90, 180, 90],
    configuration='short_range_hawaii',
    cycles=[0],
    steps=[15],
    path=tempfile.mkdtemp(prefix='nwm_hi_'),
)

date = (dt.datetime.now() - dt.timedelta(days=3)).strftime('%Y-%m-%d') nwm = EarthLens( data_source="nwm", start=date, end=date, dataset='chrtout', variables=['streamflow'], aoi=[-180, -90, 180, 90], configuration='short_range_hawaii', cycles=[0], steps=[15], path=tempfile.mkdtemp(prefix='nwm_hi_'), )

Preview the resolved key¶

_search() enumerates the matching remote files; taking the first one and stripping the bucket prefix shows the exact object key the request resolves to.

In [ ]:

matches = nwm._search()
key = matches[0].href.split('/', 1)[1]
print('key:', key)

matches = nwm._search() key = matches[0].href.split('/', 1)[1] print('key:', key)

Download it¶

Fetch the file to the temp directory and report each written path and its size.

In [ ]:

paths = nwm.download(progress_bar=False)
for p in paths:
    print(p.name, f'{p.stat().st_size/1e6:.2f} MB')
assert paths

paths = nwm.download(progress_bar=False) for p in paths: print(p.name, f'{p.stat().st_size/1e6:.2f} MB') assert paths

A medium-range ensemble member resolves to its member file¶

The member rides on the directory (medium_range_mem2) and the product token (channel_rt_2) — shown here as the enumerated key (no download).

Build the ensemble request¶

Setting member=2 on the medium_range configuration selects the second ensemble member for the ldasout soil-moisture product.

In [ ]:

ens = EarthLens(
    data_source="nwm",
    start=date,
    end=date,
    dataset='ldasout',
    variables=['SOIL_M'],
    aoi=[-180, -90, 180, 90],
    configuration='medium_range',
    member=2,
    cycles=[0],
    steps=[3],
)

ens = EarthLens( data_source="nwm", start=date, end=date, dataset='ldasout', variables=['SOIL_M'], aoi=[-180, -90, 180, 90], configuration='medium_range', member=2, cycles=[0], steps=[3], )

Preview the member key¶

Enumerating the request and taking the first match shows how the member is baked into both the directory and the product token of the resolved key.

In [ ]:

ens_matches = ens._search()
print(ens_matches[0].href.split('/', 1)[1])

ens_matches = ens._search() print(ens_matches[0].href.split('/', 1)[1])