Elevation & terrain — SRTM 30 m¶
NASA SRTM Global 1 arc-second DEM over the Giza plateau — a single static ee.Image, so the pipeline composites the one image and writes it as a GeoTIFF.
Setup¶
First the imports and the per-notebook output directory. pyramids provides Dataset (reading + plotting); earthlens provides the unified EarthLens entry point and the gee Catalog.
import os
from pathlib import Path
from matplotlib.colors import ListedColormap
from pyramids.dataset import Dataset
from pyramids.plot import ColorBar, ColorScaling
from earthlens.core import EarthLens
from earthlens.gee import Catalog, cancel_task
OUT_DIR = Path('out') / 'elevation-terrain'
OUT_DIR.mkdir(parents=True, exist_ok=True)
print(f'output directory: {OUT_DIR.resolve()}')
Credentials¶
The notebook reads the GEE service-account credentials from the GEE_SERVICE_ACCOUNT / GEE_SERVICE_KEY environment variables. Both must be set before running this cell.
SERVICE_ACCOUNT = os.environ['GEE_SERVICE_ACCOUNT']
SERVICE_KEY = os.environ['GEE_SERVICE_KEY']
Inspect the catalog entry¶
Before downloading anything, look at what the bundled catalog knows about the asset — bands, cadence, license, provider.
cat = Catalog()
ds = cat.get_dataset('USGS/SRTMGL1_003')
print(ds)
# The summary clips long text and shows only a band count, so an explorer
# notebook still wants the untruncated title and the fields it omits:
print(f'title (full): {ds.title}')
print(f'ee_type: {ds.ee_type}')
print(f'default_reducer: {ds.default_reducer}')
print(f'license: {ds.license}')
print(f'band ids (first 5): {list(ds.bands)[:5]}')
Download¶
Tiny AOI ([29.9, 30.1] lat, [31.1, 31.3] lon) at 90.0 m, raw cadence — keeps the synchronous download under EE's 32768-px per-axis cap. We build the request first, then authenticate and download as separate steps.
gee = EarthLens(
data_source="gee",
start='2000-02-11',
end='2000-02-12',
dataset='USGS/SRTMGL1_003',
variables=['elevation'],
aoi=[31.1, 29.9, 31.3, 30.1],
cadence='raw',
path=OUT_DIR,
scale=90.0,
reducer='mean',
)
gee.authenticate(service_account=SERVICE_ACCOUNT, service_key=SERVICE_KEY)
download() writes the composited image to disk and returns the list of written GeoTIFF paths.
paths = gee.download(progress_bar=False)
print(f'wrote {len(paths)} GeoTIFF(s):')
for p in paths:
print(f' {p} ({p.stat().st_size / 1024:.1f} KB)')
Quick preview¶
Load the first written GeoTIFF through pyramids and render the single band. (pyramids.dataset.Dataset is the project's GeoTIFF/NetCDF wrapper.) The dataset's nodata value is masked so the colormap isn't pinned to it.
preview = Dataset.read_file(paths[0])
Render the elevation band¶
This window holds two very different landforms: two thirds of it is Nile floodplain inside a 20 m band (p25 = 17 m, p50 = 23 m, p75 = 36 m), and the rest climbs to the Mokattam plateau at 217 m. That combination defeats the obvious approaches — spread the colours evenly over −16 … 217 m and the delta floor, most of the map, collapses into one tone.
Neither reshaping the norm nor clipping fixes it. A power norm at gamma=0.5 gives that 17–36 m band only
9.6% of the colormap and gamma=0.35 makes it worse at 8.7%; clipping the display to the floodplain does
free the delta up, but then a sixth of the scene saturates into a flat block and the plateau stops existing.
The cartographic answer is to classify by quantile — put the breaks at the deciles of this scene, so every class holds about a tenth of the pixels. Each part of the distribution gets its own colour whether it spans 2 m or 130 m, nothing saturates, and the full range still appears on the bar. The uneven spacing you see there is the point: it is the distribution.
The palette is built by hand rather than taken off the shelf. terrain reserves its lowest quarter for
below-sea-level blue, which paints the delta floor as water, and its white top means snow — wrong on a 217 m
desert plateau. This ramp runs delta green → desert sand → plateau brown.
# Deciles of this scene, rounded to map-friendly values: each class holds ~10% of
# the pixels (measured: 5.7% – 12.2%). Recompute these for a different AOI.
ELEVATION_BREAKS = [-20, 14, 16, 18, 20, 23, 27, 33, 48, 86, 220]
# Delta green -> desert sand -> plateau brown. Resampled to 256 entries because
# the boundary norm is built against a 256-colour ramp; a 10-entry colormap would
# clip every class above the first to the last colour.
HYPSOMETRIC = ListedColormap(
[
'#1b5e3a',
'#2f7d4f',
'#57a05f',
'#8cc07a',
'#c2d98d',
'#e8dd94',
'#d9bd7a',
'#bf9760',
'#9c6f47',
'#7a4f33',
]
).resampled(256)
glyph = preview.plot(
cmap=HYPSOMETRIC,
color=ColorScaling.boundary(bounds=ELEVATION_BREAKS),
colorbar=ColorBar(label='elevation (m)'),
title='SRTM GL1 — Cairo and the Nile',
)
glyph.ax.title.set_fontsize(11)
stats = preview.stats(approx_ok=False)
low, high = stats['min'].iloc[0], stats['max'].iloc[0]
print(f'value range: [{low:.4g}, {high:.4g}] m')
Tracking submitted jobs (asynchronous export)¶
The download above uses export_via="url" — a synchronous getDownloadURL round-trip. Nothing was queued, so there's no Earth Engine job to track.
To track an export instead, switch to an asynchronous sink (drive / gcs / asset) and pass wait_for_export=False so .download() returns a TaskInfo at submission time rather than blocking until completion. The cells below submit the same (asset_id, band, AOI, scale) request as an export_via="asset" task into the service account's own asset folder, then walk the four jobs-API calls (list_recent_tasks → wait_for_task_id → ee.data.getAsset → ee.data.deleteAsset) to make the job finish and tidy up. See track-batch-exports.ipynb for a deeper worked example.
Prepare the demo asset folder¶
GEE._export_via_batch writes the image at <asset_id>/<prefix>, so asset_id here is the parent Folder asset (not the final image path). Resolve a folder name under the service account's own project.
import ee
from earthlens.gee import list_recent_tasks, wait_for_task_id
# The asset goes into a `Folder` asset that we own. `GEE._export_via_batch`
# writes the actual image at `<asset_id>/<prefix>`, so `asset_id` here is
# the parent FOLDER (not the final image path). Both must be cleaned up.
_proj = ee.data._get_projects_path().removeprefix('projects/')
PARENT = f'projects/{_proj}/assets'
DEMO_FOLDER = f'{PARENT}/earthlens-demo-elevation-terrain'
print(f'demo folder: {DEMO_FOLDER}')
Listing the parent says whether a previous run left the folder behind, so the cleanup never has to swallow a "not found" from Earth Engine. Then create the parent folder — EE requires it to exist before a child write.
# Listing the parent says whether a previous run left the folder behind, so the
# cleanup below never has to swallow a "not found" from Earth Engine.
listed = ee.data.listAssets({'parent': PARENT})
siblings = [asset['name'] for asset in listed.get('assets', [])]
if DEMO_FOLDER in siblings:
children = ee.data.listAssets({'parent': DEMO_FOLDER})
for child in children.get('assets', []):
ee.data.deleteAsset(child['name'])
print(f'cleared leftover child: {child["name"]}')
ee.data.deleteAsset(DEMO_FOLDER)
print(f'cleared leftover folder: {DEMO_FOLDER}')
# Create the parent folder — EE requires it to exist before a child write.
ee.data.createAsset({'type': 'Folder'}, DEMO_FOLDER)
print(f'created folder: {DEMO_FOLDER}')
Submit¶
Same (asset_id, band, AOI, scale) request as the sync download above, just routed through export_via="asset" + wait_for_export=False. Build the request and authenticate as separate steps.
async_gee = EarthLens(
data_source="gee",
start='2000-02-11',
end='2000-02-12',
dataset='USGS/SRTMGL1_003',
variables=['elevation'],
aoi=[31.1, 29.9, 31.3, 30.1],
cadence='raw',
path=OUT_DIR,
scale=90.0,
reducer='mean',
export_via='asset',
asset_id=DEMO_FOLDER,
wait_for_export=False,
)
async_gee.authenticate(service_account=SERVICE_ACCOUNT, service_key=SERVICE_KEY)
download() returns a TaskInfo per submitted bucket at the moment the task is queued — no blocking.
submitted = async_gee.download(progress_bar=False)
task_info = submitted[0]
print(f'submitted: id={task_info.id} state={task_info.state}')
print(f' description={task_info.description}')
List + wait¶
list_recent_tasks(description_prefix=...) returns every matching task across the current project; wait_for_task_id blocks until the one we care about reaches a terminal state. A real workflow would just poll later from a separate process — the wait here exists so the notebook shows the full success path end-to-end.
recent = list_recent_tasks(
description_prefix=task_info.description,
max_age_min=10,
)
print(f'list_recent_tasks matched {len(recent)} task(s):')
for t in recent:
print(f' {t.id} {t.state:<12} {t.description}')
final = None
try:
final = wait_for_task_id(
task_info.id,
poll_seconds=10,
progress_bar=False,
)
print(f'\nfinal state: {final.state}')
finally:
if final is None:
# The wait raises on FAILED / CANCELLED *and on timeout* — and a
# timeout leaves the export still running. Cancel it so an aborted
# notebook does not leave a live task behind; cancel_task is a no-op
# on an already-terminal task, and the original error still
# propagates out of this finally.
cancel_task(task_info.id)
print(f'cancelled {task_info.id} after the wait failed')
Verify + clean up¶
Confirm the produced asset exists on Earth Engine, then delete it (and the surrounding demo folder) so we don't leak storage between notebook runs. The backend wrote the image at <DEMO_FOLDER>/<task description>.
produced = f'{DEMO_FOLDER}/{task_info.description}'
meta = ee.data.getAsset(produced)
print(f'asset exists: type={meta.get("type")} name={meta.get("name")}')
ee.data.deleteAsset(produced)
print('asset deleted')
# Tear down the parent folder.
ee.data.deleteAsset(DEMO_FOLDER)
print(f'folder deleted: {DEMO_FOLDER}')
What's on disk¶
The GeoTIFF is left under the per-notebook out/ directory for you to inspect. That directory is .gitignored — re-running the notebook overwrites it.
for p in sorted(OUT_DIR.iterdir()) if OUT_DIR.exists() else []:
print(f'{p} ({p.stat().st_size / 1024:.1f} KB)')