G-Portal search + SFTP download — research scenario¶

Walk the full credentialed path: pick a GCOM-C/SGLI L3 ocean product from the catalog, search the live G-Portal catalogue for a 1-day window, then optionally download the matching products over SFTP into a temp dir. Inspect the resulting HDF5 file's metadata with h5py to confirm it's a real, usable product. This is the workflow an ocean-colour researcher runs to fetch a daily mosaic for a regional study.

Needs the [jaxa] extra. The download step needs $GPORTAL_USERNAME + $GPORTAL_PASSWORD (free account at https://gportal.jaxa.jp/gpr/user/regist1); without those it falls back to the anonymous-search-only path.

Open the catalog and pick a product¶

In [1]:

from earthlens.jaxa import Catalog

cat = Catalog()
row = cat.get('sgli-l3-nwlr')
print('canonical key:', row.key)
print('aliases:      ', row.aliases)
print('protocol:     ', row.protocol)
print('short_name:   ', row.short_name)
print('description:  ', row.description)

from earthlens.jaxa import Catalog cat = Catalog() row = cat.get('sgli-l3-nwlr') print('canonical key:', row.key) print('aliases: ', row.aliases) print('protocol: ', row.protocol) print('short_name: ', row.short_name) print('description: ', row.description)

canonical key: sgli-l3-nwlr
aliases:       ['sgli-l380', 'sgli-ocean-l380', 'gcom-c-nwlr', 'l3-nwlr']
protocol:      gportal
short_name:    10003001
description:   GCOM-C/SGLI — Oceanic sphere/L3-NWLR

Anonymous search¶

gportal.search() does not require credentials. Query the live G-Portal catalogue for daily L3 NWLR products on 2024-01-01.

In [2]:

import gportal

result = gportal.search(
    dataset_ids=[row.short_name],
    start_time='2024-01-01',
    end_time='2024-01-02',
    count=3,
)
matched = result.matched() or 0
print(f'matched products: {matched}')
for p in list(result.products())[:3]:
    print(f'  {p.id}  ({p.data_path})')

import gportal result = gportal.search( dataset_ids=[row.short_name], start_time='2024-01-01', end_time='2024-01-02', count=3, ) matched = result.matched() or 0 print(f'matched products: {matched}') for p in list(result.products())[:3]: print(f' {p.id} ({p.data_path})')

matched products: 6

  GC1SG1_20240101A01D_D0000_3MSG_L380F_3000  (standard/GCOM-C/GCOM-C.SGLI/L3.OCEAN.L380/3/2024/01/GC1SG1_20240101A01D_D0000_3MSG_L380F_3000.h5)
  GC1SG1_20240101D01D_D0000_3MSG_L380F_3000  (standard/GCOM-C/GCOM-C.SGLI/L3.OCEAN.L380/3/2024/01/GC1SG1_20240101D01D_D0000_3MSG_L380F_3000.h5)
  GC1SG1_20240101A08D_D0000_3MSG_L380F_3000  (standard/GCOM-C/GCOM-C.SGLI/L3.OCEAN.L380/3/2024/01/GC1SG1_20240101A08D_D0000_3MSG_L380F_3000.h5)

Credentialed SFTP download (skips when env vars absent)¶

The full backend chain — authenticate, search, SFTP-download — is driven by EarthLens(data_source='jaxa', ...). The cell below executes it when $GPORTAL_USERNAME and $GPORTAL_PASSWORD are set and prints a graceful skip notice otherwise.

In [3]:

import os
import tempfile
from pathlib import Path

if not (os.environ.get('GPORTAL_USERNAME') and os.environ.get('GPORTAL_PASSWORD')):
    print('SKIP: $GPORTAL_USERNAME + $GPORTAL_PASSWORD not set.')
    print('Register at https://gportal.jaxa.jp/gpr/user/regist1 and re-run.')
    written = []
else:
    from earthlens import EarthLens
    out_dir = Path(tempfile.mkdtemp(prefix='earthlens-jaxa-sgli-'))
    lens = EarthLens(
        data_source='jaxa',
        variables=['sgli-l3-nwlr'],
        start='2024-01-01',
        end='2024-01-02',
        lat_lim=[0.0, 30.0],
        lon_lim=[120.0, 150.0],
        path=out_dir,
    )
    written = lens.download()
    print(f'downloaded {len(written)} product(s) to {out_dir}:')
    for p in written:
        print(f'  {p.name}  ({p.stat().st_size // 1024} KB)')

import os import tempfile from pathlib import Path if not (os.environ.get('GPORTAL_USERNAME') and os.environ.get('GPORTAL_PASSWORD')): print('SKIP: $GPORTAL_USERNAME + $GPORTAL_PASSWORD not set.') print('Register at https://gportal.jaxa.jp/gpr/user/regist1 and re-run.') written = [] else: from earthlens import EarthLens out_dir = Path(tempfile.mkdtemp(prefix='earthlens-jaxa-sgli-')) lens = EarthLens( data_source='jaxa', variables=['sgli-l3-nwlr'], start='2024-01-01', end='2024-01-02', lat_lim=[0.0, 30.0], lon_lim=[120.0, 150.0], path=out_dir, ) written = lens.download() print(f'downloaded {len(written)} product(s) to {out_dir}:') for p in written: print(f' {p.name} ({p.stat().st_size // 1024} KB)')

SKIP: $GPORTAL_USERNAME + $GPORTAL_PASSWORD not set.
Register at https://gportal.jaxa.jp/gpr/user/regist1 and re-run.

Inspect one downloaded HDF5 (when present)¶

GCOM-C SGLI products are HDF5; the metadata attributes describe the instrument, observation epoch, geographic coverage, and the data variables inside. We pick the largest downloaded file (typically the global mosaic), open it read-only with h5py, and print the top-level groups + a few global attributes.

In [4]:

if not written:
    print('No file to inspect — re-run the previous cell with creds set.')
else:
    try:
        import h5py
    except ImportError:
        print('h5py not installed — pip install h5py to peek inside the HDF5.')
    else:
        biggest = max(written, key=lambda p: p.stat().st_size)
        print(f'opening {biggest.name} ({biggest.stat().st_size // 1024} KB)')
        with h5py.File(biggest, 'r') as h:
            print('  top-level groups:')
            for k in list(h.keys())[:8]:
                print(f'    /{k}')
            print()
            print('  selected global attributes:')
            for attr in ('Satellite', 'Sensor', 'Observation_start_date',
                         'Observation_end_date', 'Granule_ID'):
                if attr in h.attrs:
                    print(f'    {attr}: {h.attrs[attr]}')

if not written: print('No file to inspect — re-run the previous cell with creds set.') else: try: import h5py except ImportError: print('h5py not installed — pip install h5py to peek inside the HDF5.') else: biggest = max(written, key=lambda p: p.stat().st_size) print(f'opening {biggest.name} ({biggest.stat().st_size // 1024} KB)') with h5py.File(biggest, 'r') as h: print(' top-level groups:') for k in list(h.keys())[:8]: print(f' /{k}') print() print(' selected global attributes:') for attr in ('Satellite', 'Sensor', 'Observation_start_date', 'Observation_end_date', 'Granule_ID'): if attr in h.attrs: print(f' {attr}: {h.attrs[attr]}')

No file to inspect — re-run the previous cell with creds set.