Land cover & change — ESA WorldCover v200 (2021)¶
ESA WorldCover 2021 (10 m, 11 classes) — a one-image land-cover map; the default reducer is mosaic because the collection is tiled rather than time-stepped.
Setup¶
First the imports. pyramids provides Dataset (reading + plotting) and the plot-styling types;
earthlens provides the unified EarthLens entry point and the GEE Catalog. ListedColormap is used only
to hand pyramids ESA's published class palette — pyramids still does the rendering.
import os
from pathlib import Path
import numpy as np
from matplotlib.colors import ListedColormap
from pyramids.dataset import Dataset
from pyramids.plot import ColorBar, ColorScaling
from earthlens.core import EarthLens
from earthlens.gee import Catalog, cancel_task
Output directory¶
Each notebook writes its GeoTIFFs into a per-notebook out/ directory (which is .gitignored).
OUT_DIR = Path('out') / 'land-cover-change'
OUT_DIR.mkdir(parents=True, exist_ok=True)
print(f'output directory: {OUT_DIR.resolve()}')
Credentials¶
The notebook reads the GEE service-account credentials from the GEE_SERVICE_ACCOUNT / GEE_SERVICE_KEY environment variables. Both must be set before running this cell.
SERVICE_ACCOUNT = os.environ['GEE_SERVICE_ACCOUNT']
SERVICE_KEY = os.environ['GEE_SERVICE_KEY']
Inspect the catalog entry¶
Before downloading anything, look at what the bundled catalog knows about the asset — bands, cadence, license, provider.
cat = Catalog()
ds = cat.get_dataset('ESA/WorldCover/v200')
print(ds)
# The summary clips long text and shows only a band count, so an explorer
# notebook still wants the untruncated title and the fields it omits:
print(f'title (full): {ds.title}')
print(f'ee_type: {ds.ee_type}')
print(f'default_reducer: {ds.default_reducer}')
print(f'license: {ds.license}')
print(f'band ids (first 5): {list(ds.bands)[:5]}')
Download¶
Tiny AOI ([29.9, 30.1] lat, [31.1, 31.3] lon) at 30.0 m, raw cadence — keeps the synchronous download under EE's 32768-px per-axis cap. We build the request first, then authenticate as a separate step so each is easy to read and re-run.
gee = EarthLens(
data_source="gee",
start='2021-01-01',
end='2021-12-31',
dataset='ESA/WorldCover/v200',
variables=['Map'],
aoi=[31.1, 29.9, 31.3, 30.1],
cadence='raw',
path=OUT_DIR,
scale=30.0,
reducer='mosaic',
)
gee.authenticate(service_account=SERVICE_ACCOUNT, service_key=SERVICE_KEY)
With the request authenticated, download() writes the GeoTIFF(s) to disk and returns their paths.
paths = gee.download(progress_bar=False)
print(f'wrote {len(paths)} GeoTIFF(s):')
for p in paths:
print(f' {p} ({p.stat().st_size / 1024:.1f} KB)')
Quick preview¶
Load the written GeoTIFF through pyramids. (pyramids.dataset.Dataset is the project's GeoTIFF/NetCDF
wrapper.) This band already arrives with its nodata declared, so nothing has to be masked by hand.
preview = Dataset.read_file(paths[0])
Render the land-cover band¶
Map is a categorical band: it holds ESA's class codes, not a measured quantity. A continuous colour ramp
would invent gradients between unrelated classes — "built-up" is not halfway between "cropland" and "bare
ground" — and its colour bar would tick at meaningless intermediate values. So the render uses class breaks
either side of each code, ESA's own published palette, and a colour bar labelled with the class names.
WorldCover v200 also defines 95 (mangroves) and 100 (moss and lichen); neither occurs in this window.
# ESA WorldCover v200 class codes, names and the project's published palette.
# The full legend: 95 (mangroves) and 100 (moss and lichen) do not occur in this
# window, but a different AOI would hit them, and leaving them out would both
# raise a KeyError below and paint them as the neighbouring class on the map.
WORLDCOVER = {
10: ('tree cover', '#006400'),
20: ('shrubland', '#ffbb22'),
30: ('grassland', '#ffff4c'),
40: ('cropland', '#f096ff'),
50: ('built-up', '#fa0000'),
60: ('bare / sparse vegetation', '#b4b4b4'),
70: ('snow and ice', '#f0f0f0'),
80: ('permanent water', '#0064c8'),
90: ('herbaceous wetland', '#0096a0'),
95: ('mangroves', '#00cf75'),
100: ('moss and lichen', '#fae6a0'),
}
# One break either side of each code, so every class gets exactly one block.
# 90/95 are 5 apart, so those two breaks sit at 92.5 and 97.5.
CLASS_BOUNDS = [5, 15, 25, 35, 45, 55, 65, 75, 85, 92.5, 97.5, 105]
# Resampled to 256 entries: the boundary norm is built against a 256-colour
# ramp, so an 11-entry colormap would clip every class above the first to the
# last colour.
palette = ListedColormap([color for _, color in WORLDCOVER.values()]).resampled(256)
glyph = preview.plot(
cmap=palette,
color=ColorScaling.boundary(bounds=CLASS_BOUNDS),
colorbar=ColorBar(label='ESA WorldCover class'),
title='ESA WorldCover — Cairo and the Nile',
)
glyph.ax.title.set_fontsize(11)
glyph.cbar.set_ticks(list(WORLDCOVER))
glyph.cbar.set_ticklabels([f'{code} {name}' for code, (name, _) in WORLDCOVER.items()])
# A class map has no meaningful min/max, so report the mix instead.
classes = preview.read_array(masked=True)
total = classes.count()
print('classes present:')
for code in np.unique(classes.compressed()):
share = (classes == code).sum() / total * 100
print(f' {int(code):3d} {WORLDCOVER[int(code)][0]:<26} {share:5.2f}%')
Tracking submitted jobs (asynchronous export)¶
The download above uses export_via="url" — a synchronous getDownloadURL round-trip. Nothing was queued, so there's no Earth Engine job to track.
To track an export instead, switch to an asynchronous sink (drive / gcs / asset) and pass wait_for_export=False so .download() returns a TaskInfo at submission time rather than blocking until completion. The cells below submit the same (asset_id, band, AOI, scale) request as an export_via="asset" task into the service account's own asset folder, then walk the four jobs-API calls (list_recent_tasks → wait_for_task_id → ee.data.getAsset → ee.data.deleteAsset) to make the job finish and tidy up. See track-batch-exports.ipynb for a deeper worked example.
Imports and the demo asset folder¶
The async helpers live in earthlens.gee. The export writes into a Folder asset that we own — GEE._export_via_batch writes the actual image at <asset_id>/<prefix>, so asset_id here is the parent folder, not the final image path. Both must be cleaned up.
import ee
from earthlens.gee import list_recent_tasks, wait_for_task_id
_proj = ee.data._get_projects_path().removeprefix('projects/')
PARENT = f'projects/{_proj}/assets'
DEMO_FOLDER = f'{PARENT}/earthlens-demo-land-cover-change'
print(f'demo folder: {DEMO_FOLDER}')
Prepare a clean folder¶
Listing the parent says whether a previous run left the folder behind, so the cleanup never has to swallow a "not found" from Earth Engine. Then create the folder Earth Engine requires before a child write.
# Listing the parent says whether a previous run left the folder behind, so the
# cleanup below never has to swallow a "not found" from Earth Engine.
listed = ee.data.listAssets({'parent': PARENT})
siblings = [asset['name'] for asset in listed.get('assets', [])]
if DEMO_FOLDER in siblings:
children = ee.data.listAssets({'parent': DEMO_FOLDER})
for child in children.get('assets', []):
ee.data.deleteAsset(child['name'])
print(f'cleared leftover child: {child["name"]}')
ee.data.deleteAsset(DEMO_FOLDER)
print(f'cleared leftover folder: {DEMO_FOLDER}')
# Create the parent folder — EE requires it to exist before a child write.
ee.data.createAsset({'type': 'Folder'}, DEMO_FOLDER)
print(f'created folder: {DEMO_FOLDER}')
Submit¶
Same (asset_id, band, AOI, scale) request as the sync download above, just routed through export_via="asset" + wait_for_export=False. We build the request and authenticate on separate lines first.
async_gee = EarthLens(
data_source="gee",
start='2021-01-01',
end='2021-12-31',
dataset='ESA/WorldCover/v200',
variables=['Map'],
aoi=[31.1, 29.9, 31.3, 30.1],
cadence='raw',
path=OUT_DIR,
scale=30.0,
reducer='mosaic',
export_via='asset',
asset_id=DEMO_FOLDER,
wait_for_export=False,
)
async_gee.authenticate(service_account=SERVICE_ACCOUNT, service_key=SERVICE_KEY)
download() returns a TaskInfo per submitted bucket at the moment the task is queued — no blocking.
submitted = async_gee.download(progress_bar=False)
task_info = submitted[0]
print(f'submitted: id={task_info.id} state={task_info.state}')
print(f' description={task_info.description}')
List + wait¶
list_recent_tasks(description_prefix=...) returns every matching task across the current project; wait_for_task_id blocks until the one we care about reaches a terminal state. A real workflow would just poll later from a separate process — the wait here exists so the notebook shows the full success path end-to-end.
recent = list_recent_tasks(
description_prefix=task_info.description,
max_age_min=10,
)
print(f'list_recent_tasks matched {len(recent)} task(s):')
for t in recent:
print(f' {t.id} {t.state:<12} {t.description}')
Now block on the task we submitted. If it ends in FAILED / CANCELLED, wait_for_task_id raises — we cancel any still-running task so we don't leak an in-flight job on notebook restart.
final = None
try:
final = wait_for_task_id(
task_info.id,
poll_seconds=10,
progress_bar=False,
)
print(f'final state: {final.state}')
finally:
if final is None:
# The wait raises on FAILED / CANCELLED *and on timeout* — and a
# timeout leaves the export still running. Cancel it so an aborted
# notebook does not leave a live task behind; cancel_task is a no-op
# on an already-terminal task, and the original error still
# propagates out of this finally.
cancel_task(task_info.id)
print(f'cancelled {task_info.id} after the wait failed')
Verify + clean up¶
Confirm the produced asset exists on Earth Engine, then delete it (and the surrounding demo folder) so we don't leak storage between notebook runs. The backend wrote the image at <DEMO_FOLDER>/<task description>.
produced = f'{DEMO_FOLDER}/{task_info.description}'
meta = ee.data.getAsset(produced)
print(f'asset exists: type={meta.get("type")} name={meta.get("name")}')
ee.data.deleteAsset(produced)
print('asset deleted')
# Tear down the parent folder.
ee.data.deleteAsset(DEMO_FOLDER)
print(f'folder deleted: {DEMO_FOLDER}')
What's on disk¶
The GeoTIFF is left under the per-notebook out/ directory for you to inspect. That directory is .gitignored — re-running the notebook overwrites it.
for p in sorted(OUT_DIR.iterdir()) if OUT_DIR.exists() else []:
print(f'{p} ({p.stat().st_size / 1024:.1f} KB)')