Assets: read, metadata, VRT, download, GeoParquet#
The asset-level surface of pyramids.stac: open a single asset, read its
extension metadata without touching the file, mosaic an asset across items into
a lazy VRT, download assets locally, and round-trip Items through GeoParquet.
- Read one asset —
load_assetdispatches by media type (COG/GeoTIFF →Dataset, NetCDF/Zarr →NetCDF, GRIB →open_grib, JPEG2000 →Dataset);which_enginepreviews the reader without opening;resolved_hrefreturns the (optionally signed) href without opening. - Extension metadata —
read_extension_metadataturns a STAC Item'sproj/raster/eofields into a grid + band-metadata dict (CRS, geotransform, shape, nodata/scale/offset, band names) without opening the asset, the way stackstac / odc-stac / rio-tiler do. - VRT mosaic —
build_vrt_from_stacstitches one asset across many items into a lazy GDAL VRT read on demand via/vsicurl/. - Alternate hrefs —
preferred_asset_href/asset_alternate_hrefresolve thealternate-assetsextension, so a caller can prefer ans3://mirror over the canonicalhttps://href and fall back automatically when it is absent.preferred_asset_sourcemakes the same choice but reports the selected location's media type alongside its href, andasset_fieldreads one field off an asset whether it arrives as a raw dict or apystac.Asset. - Verification —
verify_asset(andload_asset(verify=True)) probes an HTTP asset for reachability and a content-type consistent with its declaredtype, before GDAL is asked to open it. Off by default, so no extra request is made. - Physical units & overrides —
load_asset(rescale=True)applies theraster:bandsscale/offset so values come back in physical units instead of raw DN;cfg=suppliesdata_type/nodata/unitthat a thin catalog omits and maps band aliases ("rededge" → "B05"). - Windowed reads from an Item —
read_item_part/read_item_preview/read_item_point/read_item_featureresolve an item + asset and delegate to the COG engine's overview-decimated reads, so a window costs a partial read, not a whole scene. - Download —
download_item,download_item_collectionanddownload_collectioncopy assets to local files (optionalstac-asset, shipped in the[stac]extra). - GeoParquet —
to_geoparquet/from_geoparquetserialize an ItemCollection to a single columnar file and back, in either pyramids' lossless JSON-blob layout or the interoperable stac-geoparquet 1.1 layout (to_geoparquet_spec/from_geoparquet_spec, orspec=True) that DuckDB and pyarrow can query directly (optionalpyarrow, the[parquet]extra).
Reading assets#
pyramids.stac._loader
#
Open a STAC asset as a pyramids Dataset / NetCDF, dispatched by type.
Takes a STAC Item + asset_key (or an Asset directly), resolves the
asset href, and opens it with the right GDAL-backed reader chosen by the asset's
media_type (with the href extension as a fallback):
| media_type / extension | reader |
|---|---|
image/tiff... / .tif .tiff |
:meth:Dataset.read_file |
image/jp2 / .jp2 .jpx |
:meth:Dataset.read_file |
application/x-netcdf / .nc .nc4 .cdf |
:meth:NetCDF.read_file |
application/wmo-grib2 / .grib2 .grb |
:func:pyramids.grib.open_grib |
application/vnd+zarr / .zarr |
:meth:NetCDF.read_file (GDAL Zarr) |
Four opt-ins sit on top of that dispatch, all off by default so an existing read behaves exactly as before:
alternate=prefers analternate-assetshref (ans3://mirror over the public HTTPS copy, say), silently falling back to the canonicalhref.verify=pre-flights the href with one HEAD (:func:verify_asset) and warns when it is unreachable or serves a content type contradicting the declared one. HTTP(S) only, and never requested unless asked for.rescale=returns physical units instead of stored counts, applying the asset'sraster:bandsscale/offsetwith no-data masked first. GDAL (GeoTIFF/COG/JPEG2000) assets only — netCDF, Zarr and GRIB carry their own CF packing, which the readers already unpack.cfg=supplies per-asset metadata a thin catalog omits (data_type,nodata,unit) and resolves band aliases to real asset keys. See :mod:pyramids.stac._configfor the schema. The metadata half is GDAL-only too, but unlikerescalenothing about a CF reader makes it redundant — so acfgoverride resolving to a netCDF / Zarr / GRIB asset is reported with anAssetMetadataWarningrather than silently dropped. Aliases apply on every engine, since they only choose which asset key is read.
The last two cannot be stamped onto the opened handle — an asset is opened
read-only, and a remote /vsicurl COG rejects every metadata setter — so they
materialise a writable in-memory copy instead
(:func:pyramids.stac._config.materialise).
Everything is duck-typed — pyramids does not import or depend on pystac; the
Item / Asset contract is read via getattr + dict lookup (pystac.Asset has
.href / .media_type; raw STAC JSON uses {"href":..., "type":...}). Assets
resolve to pyramids' GDAL-backed wrappers.
which_engine(item_or_asset, asset_key=None, *, alternate=None)
#
Return the reader name :func:load_asset would use, without opening.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item or Asset (pystac object or raw dict). |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
Returns:
| Type | Description |
|---|---|
str
|
One of |
Examples:
- A COG asset dispatches to the GDAL reader:
- A GRIB2 asset (recognised by extension when type is absent):
- An Item + asset key resolves the named asset:
Source code in src/pyramids/stac/_loader.py
resolved_href(item_or_asset, asset_key=None, *, signer=None, alternate=None, verify=False, verify_strict=False)
#
Return an asset's resolved (optionally signed) href without opening it.
The read-free companion to :func:load_asset: it resolves the asset href
and, when a signer is given, applies signer.sign_href — but never opens
the asset. Useful for building a VRT over many assets
(:func:pyramids.stac.build_vrt_from_stac), pre-flighting URLs, or
debugging what load_asset would open.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac object or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
None
|
signer
|
Any
|
Optional signer; when given, its |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
verify
|
bool
|
Pre-flight the chosen href with :func: |
False
|
verify_strict
|
bool
|
With |
False
|
Returns:
| Type | Description |
|---|---|
str
|
The resolved asset href, signed when a |
Raises:
| Type | Description |
|---|---|
StacAssetError
|
The asset is missing or has no href (subclasses
:class: |
StacError
|
|
Examples:
- Resolve a plain asset href:
- Resolve an item's asset and sign it with a simple signer:
- Prefer the asset's
s3mirror, falling back when it has none:
Source code in src/pyramids/stac/_loader.py
612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 | |
load_asset(item_or_asset, asset_key=None, *, signer=None, vsi=None, alternate=None, verify=False, verify_strict=False, rescale=False, cfg=None, collection_id=None)
#
Open a STAC asset as a pyramids Dataset / NetCDF.
Resolves the asset href, optionally rewrites it through a signer
(a :class:~pyramids.stac.signers.Signer), then opens it with the
GDAL-backed reader chosen by media_type / extension. When a signer is
given, both of its hooks are applied: signer.sign_href rewrites the
href, and signer.gdal_env is installed as GDAL config for the duration
of the open (via :class:~pyramids.base.remote.CloudConfig), so the
underlying VSI handle is created with the right credentials / requester-pays
knobs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac.Item or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
None
|
signer
|
Any
|
Optional signer. |
None
|
vsi
|
str | None
|
Optional explicit archive kind forwarded to the reader (e.g. a
GeoTIFF/GRIB inside a |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
verify
|
bool
|
Pre-flight the href with :func: |
False
|
verify_strict
|
bool
|
With |
False
|
rescale
|
bool
|
Return physical units rather than stored counts, applying
|
False
|
cfg
|
Any
|
Optional |
None
|
collection_id
|
str | None
|
The collection |
None
|
Returns:
| Name | Type | Description |
|---|---|---|
A |
Dataset
|
class: |
Dataset
|
class: |
|
Dataset
|
NetCDF / Zarr / GRIB assets. |
Raises:
| Type | Description |
|---|---|
KeyError
|
The asset is missing or has no href. |
ValueError
|
The asset's type/extension matches no supported reader, or
|
StacError
|
|
Examples:
- Open a COG asset from a STAC Item (requires network access):
- Sign the href with an MPC/CDSE-style bearer signer before opening
(the token is installed as a GDAL
Authorizationheader for the open): - Read a Requester-Pays bucket: the signer's
gdal_envopts intoAWS_REQUEST_PAYER=requesterfor the duration of the open: - Read a packed asset in physical units (
raster:bandsscale 0.0001): - Name an asset by alias and supply the nodata the catalog omits:
Source code in src/pyramids/stac/_loader.py
696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 | |
Extension metadata (proj / raster / eo)#
pyramids.stac._extensions
#
Read STAC proj / raster / eo extension metadata (PB-1).
This module builds a cube/grid skeleton — CRS, geotransform, shape, per-band
nodata / scale / offset, band names — directly from the STAC Item JSON, without
opening any asset: pure dict reads (via :func:pyramids.stac._item.asset_field,
no pystac
dependency) that yield a grid/band-metadata dict downstream code can use to build
a VRT (PB-5), a multi-asset cube (PB-2), or a grid match (PC-2) without a
header open.
Scope note: these are readers only. They deliberately do not stamp the
metadata onto a :class:~pyramids.dataset.Dataset returned by
:func:pyramids.stac.load_asset, because that reader opens assets read-only
(remote /vsicurl COGs cannot be opened for write), and mutating a read-only
GDAL handle (SetProjection / SetNoDataValue / SetScale) raises under
gdal.UseExceptions(). Writable consumers (VRT/stack builders) apply the
metadata themselves from the dict this module returns.
parse_number(value, default=None)
#
Coerce a STAC numeric field to a float, honouring nan/inf strings.
The raster extension allows non-finite nodata values to be encoded as
the strings "nan" / "inf" / "-inf".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Any
|
The raw field value (number, numeric string, nan/inf string,
or |
required |
default
|
Any
|
Returned when |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
A float for numeric / nan-inf inputs, otherwise |
Examples:
- A plain number passes through as a float:
- The string
"-inf"becomes negative infinity: - An unparseable value falls back to the default:
Source code in src/pyramids/stac/_extensions.py
affine_to_geotransform(transform)
#
Convert a STAC proj:transform affine to a GDAL geotransform.
proj:transform is the affine ordering [a, b, c, d, e, f]
(mapping (col, row) to (x, y): x = a*col + b*row + c,
y = d*col + e*row + f). GDAL's geotransform is the reordering
(c, a, b, f, d, e) — i.e. (x_origin, x_res, x_rot, y_origin, y_rot,
y_res). A 9-element affine (with a trailing [0, 0, 1] row) is
accepted; only the first six coefficients are used.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
transform
|
Any
|
A 6- or 9-element |
required |
Returns:
| Type | Description |
|---|---|
tuple[float, ...]
|
The 6-tuple GDAL geotransform. |
Raises:
| Type | Description |
|---|---|
ValueError
|
When |
Examples:
- A north-up 30 m grid reorders to the GDAL geotransform:
- The trailing
[0, 0, 1]row of a 9-element affine is ignored:
Source code in src/pyramids/stac/_extensions.py
geotransform_to_affine(geotransform)
#
Convert a GDAL geotransform to a STAC proj:transform affine.
The inverse of :func:affine_to_geotransform. GDAL's geotransform is
(c, a, b, f, d, e) — (x_origin, x_res, x_rot, y_origin, y_rot,
y_res) — and proj:transform is the affine ordering
[a, b, c, d, e, f].
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
geotransform
|
Any
|
A 6-element GDAL geotransform. |
required |
Returns:
| Type | Description |
|---|---|
list[float]
|
The 6-element |
Raises:
| Type | Description |
|---|---|
ValueError
|
When |
Examples:
- A north-up 30 m grid maps back to the affine order:
Source code in src/pyramids/stac/_extensions.py
read_extension_metadata(item, asset_key=None)
#
Read proj / raster / eo extension fields for a STAC asset.
Item-level fields (under properties) are read first and an asset-level
value of the same key overrides them, matching the STAC convention that an
asset narrows item-level metadata. No asset file is opened.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item
|
Any
|
A STAC Item (pystac object or raw dict). When |
required |
asset_key
|
str | None
|
The asset key whose metadata to read, or |
None
|
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A dict with keys: |
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
dict[str, Any]
|
|
Raises:
| Type | Description |
|---|---|
StacAssetError
|
When |
Examples:
- Read a Sentinel-2-style asset's projection metadata from raw JSON:
>>> from pyramids.stac._extensions import read_extension_metadata >>> item = { ... "properties": {"proj:epsg": 32633}, ... "assets": {"B04": { ... "href": "s3://b/B04.tif", ... "proj:shape": [10980, 10980], ... "proj:transform": [10.0, 0.0, 600000.0, 0.0, -10.0, 5300040.0], ... "raster:bands": [{"nodata": 0, "scale": 0.0001}], ... "eo:bands": [{"name": "B04", "common_name": "red"}], ... }}, ... } >>> meta = read_extension_metadata(item, "B04") >>> meta["crs"] 'EPSG:32633' >>> meta["geotransform"] (600000.0, 10.0, 0.0, 5300040.0, 0.0, -10.0) >>> meta["band_names"] ['B04'] - An asset-level
proj:epsgoverrides the item-level value: - A bare asset with no extension fields yields all-empty metadata:
Source code in src/pyramids/stac/_extensions.py
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 | |
VRT mosaic#
pyramids.stac._vrt.build_vrt_from_stac(items, asset, *, signer=None, separate=False, strict=True)
#
Mosaic one STAC asset across items into a lazy VRT-backed Dataset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
Any
|
Iterable of STAC Items (pystac objects, raw JSON dicts, or any
duck-typed equivalent — same contract as
:meth: |
required |
asset
|
str
|
The asset key to mosaic (e.g. |
required |
signer
|
Any
|
Optional signer (e.g. a :class: |
None
|
separate
|
bool
|
When |
False
|
strict
|
bool
|
What to do when GDAL skips a requested source. |
True
|
Returns:
| Name | Type | Description |
|---|---|---|
Dataset |
Dataset
|
A lazy |
Dataset
|
underlying sources on demand ( |
|
Dataset
|
hrefs). |
Raises:
| Type | Description |
|---|---|
ValueError
|
When |
RuntimeError
|
When |
Warns:
| Type | Description |
|---|---|
UserWarning
|
When some sources were skipped and |
Examples:
- Mosaic the
visualasset of several items into one lazy Dataset (requires network for remote hrefs): - Accept a partial mosaic when some hrefs are known to be missing:
See Also
- :func:
pyramids.stac.load_asset: open a single asset instead of mosaicking one asset across items. - :meth:
pyramids.dataset.DatasetCollection.from_stac: stack the same asset across items along a time axis rather than mosaicking it into one image. - :func:
pyramids.dataset.merge.merge_rasters: the eager, file-writing counterpart to this lazy VRT.
Source code in src/pyramids/stac/_vrt.py
551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 | |
Alternate asset hrefs#
pyramids.stac._item
#
Duck-typed accessors for STAC Items and Assets (no pystac dependency).
Both :mod:pyramids.stac._loader (asset → reader dispatch) and
:mod:pyramids.dataset._stac (ItemCollection → :class:DatasetCollection)
need to read the same fields off a STAC Item / Asset that may be a
:class:pystac.Item / :class:pystac.Asset object or a raw STAC JSON
dict. Before this module those two readers each carried their own
getattr-then-dict-fallback resolver, which duplicated the contract and
risked drifting apart.
Centralising the access here gives one duck-typed contract:
- :class:
pystac.Item/ :class:pystac.Assetobjects (attribute access:item.assets,asset.href,asset.media_type,asset.extra_fields). - Raw STAC JSON dicts (
item["assets"],asset["href"],asset["type"], and extension fields as top-level keys on the asset dict).
pyramids does not import or depend on pystac. The extension-field accessor
(:func:asset_field) is the extension point for reading proj / raster /
eo metadata (PB-1).
asset_alternate_href(asset, name)
#
Return asset["alternate"][name]["href"], or None when absent.
Reads the STAC alternate-assets extension, which publishes the same data
at several locations — typically a public https href next to an s3
mirror that is cheaper (or the only readable one) from inside the cloud
region hosting it.
Resolution goes through :func:asset_field, so the block is found both as a
top-level "alternate" key on a raw asset dict and in a pystac.Asset's
extra_fields.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
asset
|
Any
|
A STAC Asset (pystac object or raw dict). |
required |
name
|
str
|
The alternate key to look up (e.g. |
required |
Returns:
| Type | Description |
|---|---|
str | None
|
The alternate href as a string, or |
str | None
|
|
Examples:
- Read an
s3mirror published beside the canonical href: - An absent alternate answers
Nonerather than raising:
See Also
- :func:
preferred_asset_href: picks an alternate with the canonical href as the fallback.
Source code in src/pyramids/stac/_item.py
preferred_asset_href(asset, alternate=None, *, item=None, asset_key=None)
#
Return the first available alternate href, else the canonical href.
alternate is a preference, never a requirement: an asset that publishes
no alternate block, or none under the requested keys, falls back to its
canonical href silently. That keeps a single call site working across a
mixed item collection where only some assets carry the extension.
pyramids accepts one key or an ordered sequence of them; stac-asset's
Config.alternate_assets is always a list. Passing a bare string here is
the single-key spelling of a one-element list, not a different contract.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
asset
|
Any
|
A STAC Asset (pystac object or raw dict). |
required |
alternate
|
str | Sequence[str] | None
|
An alternate key ( |
None
|
item
|
Any
|
The owning Item, used only to enrich the error message. |
None
|
asset_key
|
str | None
|
The asset key, used only to enrich the error message. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The preferred href as a string. |
Raises:
| Type | Description |
|---|---|
StacAssetError
|
No alternate matched and the asset has no canonical
|
Examples:
- A requested alternate wins over the canonical href:
- The first key that resolves wins, and an unmatched preference falls back to the canonical href:
- No preference at all keeps today's behaviour:
See Also
- :func:
preferred_asset_source: the same choice, reported together with the media type of the chosen href — which is what a caller picking a reader needs, since an alternate may declare its owntype.
Source code in src/pyramids/stac/_item.py
preferred_asset_source(asset, alternate=None, *, item=None, asset_key=None)
#
Return the (href, media_type) pair describing the location to read.
The companion of :func:preferred_asset_href for callers that then have to
pick a reader: an alternate may declare its own type, and the
alternate-assets extension does not require it to match the canonical
asset's. An s3:// mirror published as a Zarr store beside an HTTPS COG is
a legitimate pair, so the media type has to travel with the href that was
chosen rather than being read off the asset afterwards.
An alternate that declares no type inherits the asset's, which is the
common case (a mirror of the same bytes) and keeps the href-extension
fallback working for a typeless asset.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
asset
|
Any
|
A STAC Asset (pystac object or raw dict). |
required |
alternate
|
str | Sequence[str] | None
|
An alternate key ( |
None
|
item
|
Any
|
The owning Item, used only to enrich the error message. |
None
|
asset_key
|
str | None
|
The asset key, used only to enrich the error message. |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The chosen href, and the media type describing that href — the |
str | None
|
alternate's own |
tuple[str, str | None]
|
media type, else |
Raises:
| Type | Description |
|---|---|
StacAssetError
|
No alternate matched and the asset has no canonical
|
Examples:
- A typed alternate reports its own media type:
- A typeless alternate inherits the asset's media type:
- No preference keeps the canonical pair:
Source code in src/pyramids/stac/_item.py
411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 | |
asset_field(asset, key, default=None)
#
Return an extension field from an asset (e.g. proj:transform).
Extension fields live as top-level keys on a raw asset dict, and in
asset.extra_fields on a pystac.Asset. This is the building block for
reading proj / raster / eo metadata (PB-1).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
asset
|
Any
|
A STAC Asset (pystac object or raw dict). |
required |
key
|
str
|
The extension field key (e.g. |
required |
default
|
Any
|
Value returned when the field is absent. |
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The field value, or |
Examples:
- Read a projection field from a dict asset:
- Absent fields fall back to the default:
Source code in src/pyramids/stac/_item.py
Reachability & content-type verification#
pyramids.stac._loader.verify_asset(href, media_type=None, *, strict=False, timeout=VERIFY_TIMEOUT, opener=None)
#
Pre-flight an asset href: is it reachable, and is it the type it claims?
The opt-in check behind verify= on :func:load_asset and
:func:resolved_href. It costs one HEAD (or, where HEAD is refused, a
1-byte ranged GET) and catches the two failures that otherwise surface as an
opaque GDAL error several seconds later: an expired or dead URL, and an
asset whose bytes are not what the catalog says they are.
Verification is HTTP-only. An s3:// / gs:// / /vsi* / local href is
skipped and answers None: those are read by GDAL's own VSI layer, whose
credentials this check does not hold, so a probe here would report a
permission failure that the real read does not have.
Hrefs are redacted (:func:pyramids.stac._vrt.redact) before they reach a
message, so a signed URL's token cannot leak into a log handler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
href
|
str
|
The resolved (already signed) asset href. |
required |
media_type
|
str | None
|
The asset's declared media type, compared leniently against
the served |
None
|
strict
|
bool
|
Raise :class: |
False
|
timeout
|
float
|
Per-attempt timeout in seconds. |
VERIFY_TIMEOUT
|
opener
|
Any
|
Optional opener (anything with |
None
|
Returns:
| Type | Description |
|---|---|
str | None
|
The served content type, |
str | None
|
the response carried no |
Raises:
| Type | Description |
|---|---|
StacError
|
|
Examples:
- A matching content type verifies quietly and answers what was served:
>>> import io >>> from pyramids.stac._loader import verify_asset >>> class _Opener: ... def __init__(self, content_type): ... self.headers = {"Content-Type": content_type} ... def open(self, target, timeout=None): ... response = io.BytesIO(b"") ... response.headers = self.headers ... return response >>> verify_asset("https://h/a.tif", "image/tiff", opener=_Opener("image/tiff")) 'image/tiff' - A parameterised declaration still matches the bare served type:
- A contradiction raises under
strict, with the query string redacted:>>> from pyramids.base._errors import StacError >>> try: ... verify_asset( ... "https://h/a.tif?sig=SECRET", ... "image/tiff", ... strict=True, ... opener=_Opener("text/html"), ... ) ... except StacError as exc: ... print(exc) STAC asset 'https://h/a.tif?<redacted>' is declared 'image/tiff' but the server served 'text/html' - A non-HTTP href is skipped rather than probed:
Source code in src/pyramids/stac/_loader.py
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 | |
Metadata overrides and band aliases#
Supply what a thin catalog omits, and name an asset by alias. Passed as cfg= to
load_asset and DatasetCollection.from_stac:
cfg = {
"sentinel-2-l2a": {
"assets": {"*": {"nodata": 0}, "SCL": {"data_type": "uint8"}},
"aliases": {"red": "B04", "rededge": "B05"},
}
}
Most-specific wins: cfg["*"]["assets"]["*"] < cfg["*"]["assets"][key] <
cfg[collection]["assets"]["*"] < cfg[collection]["assets"][key]. nodata and
unit fill gaps only — a value the asset already declares is kept and the skip
warns with AssetMetadataWarning (silence it with "warnings": "ignore").
Item-level windowed reads#
pyramids.stac._windowed
#
Item-level windowed reads: pull a bbox / point / geometry window off an asset.
A caller who wants one window out of one asset of one STAC Item otherwise writes
the same four lines every time — resolve the asset, sign it, open it, crop it.
These helpers are that composition and nothing more: each resolves the asset
through :func:pyramids.stac.load_asset (so signing, the /vsicurl fast-read
preset and the alternate-assets preference all behave identically to a full
read) and then delegates to the overview-decimated reads the COG engine
already owns:
| helper | delegates to |
|---|---|
:func:read_item_part |
:meth:pyramids.dataset.Dataset.read_part |
:func:read_item_preview |
:meth:pyramids.dataset.Dataset.preview |
:func:read_item_point |
:meth:pyramids.dataset.Dataset.point |
:func:read_item_feature |
:meth:pyramids.dataset.Dataset.read_part |
No windowing arithmetic lives here. The decimation, the pixel snapping, the
partial-overlap padding and the CRS transform are the engine's, so an item-level
read and a Dataset-level one answer the same array — and a COG over
/vsicurl/ still fetches only the byte ranges its window needs, because the
window is pushed into GDAL's read rather than applied to an array afterwards.
Scope. These are numpy-returning reads of a single asset. Deliberately
absent, and not planned here: XYZ/TMS tile serving, PNG/JPEG rendering,
colormaps, and cross-asset band-math expressions. That stack is a dynamic tile
server (rio-tiler's territory), not pyramids' mission. The one XYZ primitive
pyramids does own — :meth:pyramids.dataset.Dataset.read_tile, which is a
read_part over Web-Mercator tile bounds — stays at the Dataset level, where
it is a raster read rather than a serving layer.
read_item_part(item_or_asset, asset_key, bbox, *, bbox_crs=STAC_WINDOW_CRS, signer=None, alternate=None, **read_options)
#
Read a bbox window out of one asset of a STAC Item.
Opens the asset through :func:~pyramids.stac.load_asset and delegates to
:meth:~pyramids.dataset.Dataset.read_part, so the read is decimated from
the nearest overview and a COG over /vsicurl/ fetches only the window's
byte ranges.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac object or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
required |
bbox
|
Sequence[float]
|
|
required |
bbox_crs
|
int | str | None
|
CRS of |
STAC_WINDOW_CRS
|
signer
|
Any
|
Optional signer, forwarded to :func: |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
**read_options
|
Any
|
Forwarded verbatim to
:meth: |
{}
|
Returns:
| Type | Description |
|---|---|
NDArray | tuple[NDArray, tuple[float, ...]]
|
numpy.ndarray: |
NDArray | tuple[NDArray, tuple[float, ...]]
|
|
NDArray | tuple[NDArray, tuple[float, ...]]
|
when |
Raises:
| Type | Description |
|---|---|
StacAssetError
|
The asset is missing or has no href. |
UnsupportedAssetError
|
The asset's type matches no reader, or opened as something with no windowed read (a 4-D Zarr cube). |
CRSError
|
A |
OutOfBoundsError
|
The window does not intersect the asset at all. |
Examples:
- Pull a lon/lat window out of a catalog's red band:
Source code in src/pyramids/stac/_windowed.py
read_item_preview(item_or_asset, asset_key=None, *, max_size=1024, signer=None, alternate=None, **read_options)
#
Read a whole-asset thumbnail for one asset of a STAC Item.
Opens the asset through :func:~pyramids.stac.load_asset and delegates to
:meth:~pyramids.dataset.Dataset.preview, which pulls from a coarse
overview where one exists — so previewing a large COG costs a few ranged
requests rather than the whole file.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac object or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
None
|
max_size
|
int
|
Maximum pixels on the longer edge. Defaults to 1024. |
1024
|
signer
|
Any
|
Optional signer, forwarded to :func: |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
**read_options
|
Any
|
Forwarded verbatim to
:meth: |
{}
|
Returns:
| Type | Description |
|---|---|
NDArray
|
numpy.ndarray: The downsampled array, |
NDArray
|
|
Raises:
| Type | Description |
|---|---|
StacAssetError
|
The asset is missing or has no href. |
UnsupportedAssetError
|
The asset's type matches no reader, or opened as something with no windowed read (a 4-D Zarr cube). |
Examples:
- Thumbnail an item's visual asset at 128 px:
Source code in src/pyramids/stac/_windowed.py
read_item_point(item_or_asset, asset_key, point, *, point_crs=STAC_WINDOW_CRS, signer=None, alternate=None, **read_options)
#
Sample one asset of a STAC Item at a single coordinate.
Opens the asset through :func:~pyramids.stac.load_asset and delegates to
:meth:~pyramids.dataset.Dataset.point, which reads the single pixel
covering the coordinate.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac object or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
required |
point
|
Sequence[float]
|
|
required |
point_crs
|
int | str | None
|
CRS of |
STAC_WINDOW_CRS
|
signer
|
Any
|
Optional signer, forwarded to :func: |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
**read_options
|
Any
|
Forwarded verbatim to
:meth: |
{}
|
Returns:
| Type | Description |
|---|---|
NDArray
|
numpy.ndarray: A 0-d array for a single band, or a |
NDArray
|
when no |
Raises:
| Type | Description |
|---|---|
StacAssetError
|
The asset is missing or has no href. |
UnsupportedAssetError
|
The asset's type matches no reader, or opened as something with no windowed read (a 4-D Zarr cube). |
CRSError
|
A |
OutOfBoundsError
|
The coordinate falls outside the asset's extent. |
Examples:
- Sample every band of an item's asset at one lon/lat:
Source code in src/pyramids/stac/_windowed.py
read_item_feature(item_or_asset, asset_key, geometry, *, geometry_crs=STAC_WINDOW_CRS, signer=None, alternate=None, **read_options)
#
Read the window covering a geometry out of one asset of a STAC Item.
The geometry is reduced to its envelope (:func:geometry_bounds) and read
through :func:read_item_part. The result is therefore the geometry's
bounding window, not a masked cut-out: cells inside the window but
outside the geometry keep their values. Masking is a separate operation —
burn the geometry into a raster with
:meth:pyramids.dataset.Dataset.from_features, wrapping it in a
:class:~pyramids.feature.FeatureCollection, or clip the opened asset with
:meth:pyramids.dataset.Dataset.clip, when the shape itself must be
honoured.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_or_asset
|
Any
|
A STAC Item (pystac object or raw dict) or an Asset. |
required |
asset_key
|
str | None
|
Asset name when passing an Item; |
required |
geometry
|
Any
|
A shapely geometry, or a GeoJSON geometry / |
required |
geometry_crs
|
int | str | None
|
CRS of the geometry's coordinates. Defaults to
:data: |
STAC_WINDOW_CRS
|
signer
|
Any
|
Optional signer, forwarded to :func: |
None
|
alternate
|
str | Sequence[str] | None
|
|
None
|
**read_options
|
Any
|
Forwarded verbatim to
:meth: |
{}
|
Returns:
| Type | Description |
|---|---|
NDArray | tuple[NDArray, tuple[float, ...]]
|
numpy.ndarray: The geometry's bounding window, shaped as |
NDArray | tuple[NDArray, tuple[float, ...]]
|
func: |
Raises:
| Type | Description |
|---|---|
ValueError
|
The geometry carries no coordinates. |
StacAssetError
|
The asset is missing or has no href. |
UnsupportedAssetError
|
The asset's type matches no reader, or opened as something with no windowed read (a 4-D Zarr cube). |
OutOfBoundsError
|
The envelope does not intersect the asset at all. |
Examples:
- Read the window covering an AOI polygon:
Source code in src/pyramids/stac/_windowed.py
Download to local files#
pyramids.stac.download
#
Download STAC assets to local files via stac-asset (PC-3).
pyramids reads STAC assets lazily through GDAL /vsicurl/; some workflows want
local copies instead (offline processing, repeated reads, archival). This module
wraps stac_asset's synchronous download behind the optional [stac] extra,
returning the local paths so they can feed
:meth:pyramids.dataset.DatasetCollection.from_files.
Three entry points mirror the three stac_asset.blocking downloaders — one
Item (:func:download_item), a whole ItemCollection
(:func:download_item_collection, e.g. the result of
:func:pyramids.stac.search), or a Collection (:func:download_collection).
All three are blocking: the async stac_asset coroutines cannot run inside a
live event loop, and pyramids exposes no async API.
stac-asset pulls heavy async dependencies (aiohttp, aiobotocore), so it is
not a core dependency — it ships via the [stac] extra (alongside
pystac-client). Install with one of:
- PyPI:
pip install 'pyramids-core[stac]' - conda-forge:
conda install -c conda-forge pyramids-stac(stac-asset is not on conda-forge; install it alone withpip install stac-asset)
The per-protocol client (HTTP / S3 / Planetary Computer / Earthdata) is selected
by stac_asset from each asset href.
download_item(item, directory, *, include=None, exclude=None, alternate_assets=None, s3_requester_pays=False, file_name_strategy=None, error_strategy=None, fail_fast=None, warn=None, max_concurrent=None)
#
Download a STAC Item's assets to a local directory.
A thin, synchronous wrapper over stac_asset.blocking.download_item (the
async download_item cannot run inside a live event loop). The per-protocol
client is chosen by stac_asset from each asset href.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item
|
Any
|
A |
required |
directory
|
str | Path
|
Destination directory for the downloaded assets. |
required |
include
|
list[str] | None
|
Optional asset keys to include (others skipped). |
None
|
exclude
|
list[str] | None
|
Optional asset keys to exclude. |
None
|
alternate_assets
|
list[str] | None
|
Keys of the |
None
|
s3_requester_pays
|
bool
|
Opt into Requester-Pays for |
False
|
file_name_strategy
|
Any | None
|
How downloaded files are named — a
|
None
|
error_strategy
|
Any | None
|
What happens to the already-downloaded files when a
download fails — a |
None
|
fail_fast
|
bool | None
|
Raise on the first asset error instead of gathering them all. |
None
|
warn
|
bool | None
|
Emit a warning per failed asset instead of raising. |
None
|
max_concurrent
|
int | None
|
Cap on concurrent asset downloads (stac-asset's
|
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The downloaded |
Any
|
paths), as returned by |
Raises:
| Type | Description |
|---|---|
OptionalPackageDoesNotExist
|
When |
ValueError
|
A strategy was given as a string naming no enum member,
or |
Examples:
- Download an item's assets, then build a collection from the locals
(requires the
[stac]extra + network):
Source code in src/pyramids/stac/download.py
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 | |
download_item_collection(item_collection, directory, *, include=None, exclude=None, alternate_assets=None, s3_requester_pays=False, file_name_strategy=None, error_strategy=None, fail_fast=None, warn=None, max_concurrent=None)
#
Download every Item's assets from an ItemCollection to a local directory.
The many-item companion of :func:download_item, wrapping
stac_asset.blocking.download_item_collection. Hand it the result of
:func:pyramids.stac.search to take a whole search result offline; the
returned ItemCollection can then feed
:meth:pyramids.dataset.DatasetCollection.from_files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
item_collection
|
Any
|
A |
required |
directory
|
str | Path
|
Destination directory for the downloaded assets. |
required |
include
|
list[str] | None
|
Optional asset keys to include (others skipped). |
None
|
exclude
|
list[str] | None
|
Optional asset keys to exclude. |
None
|
alternate_assets
|
list[str] | None
|
Keys of the |
None
|
s3_requester_pays
|
bool
|
Opt into Requester-Pays for |
False
|
file_name_strategy
|
Any | None
|
How downloaded files are named — a
|
None
|
error_strategy
|
Any | None
|
What happens to the already-downloaded files when a
download fails — a |
None
|
fail_fast
|
bool | None
|
Raise on the first asset error instead of gathering them all. |
None
|
warn
|
bool | None
|
Emit a warning per failed asset instead of raising. |
None
|
max_concurrent
|
int | None
|
Cap on concurrent asset downloads (stac-asset's
|
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The downloaded |
Any
|
its local path, as returned by |
Raises:
| Type | Description |
|---|---|
OptionalPackageDoesNotExist
|
When |
ValueError
|
A strategy was given as a string naming no enum member,
or |
Examples:
- Take a whole search result offline (requires the
[stac]extra + network):
Source code in src/pyramids/stac/download.py
272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 | |
download_collection(collection, directory, *, include=None, exclude=None, alternate_assets=None, s3_requester_pays=False, file_name_strategy=None, error_strategy=None, fail_fast=None, warn=None, max_concurrent=None)
#
Download a STAC Collection's own assets to a local directory.
Wraps stac_asset.blocking.download_collection, which fetches the assets
hanging off the Collection itself (overviews, thumbnails, licences — not the
assets of its Items; for those, search the Collection and call
:func:download_item_collection).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
collection
|
Any
|
A |
required |
directory
|
str | Path
|
Destination directory for the downloaded assets. |
required |
include
|
list[str] | None
|
Optional asset keys to include (others skipped). |
None
|
exclude
|
list[str] | None
|
Optional asset keys to exclude. |
None
|
alternate_assets
|
list[str] | None
|
Keys of the |
None
|
s3_requester_pays
|
bool
|
Opt into Requester-Pays for |
False
|
file_name_strategy
|
Any | None
|
How downloaded files are named — a
|
None
|
error_strategy
|
Any | None
|
What happens to the already-downloaded files when a
download fails — a |
None
|
fail_fast
|
bool | None
|
Raise on the first asset error instead of gathering them all. |
None
|
warn
|
bool | None
|
Emit a warning per failed asset instead of raising. |
None
|
max_concurrent
|
int | None
|
Cap on concurrent asset downloads (stac-asset's
|
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The downloaded |
Any
|
local paths, as returned by |
Raises:
| Type | Description |
|---|---|
OptionalPackageDoesNotExist
|
When |
ValueError
|
A strategy was given as a string naming no enum member,
or |
Examples:
- Fetch a collection's own assets (requires the
[stac]extra + network):
Source code in src/pyramids/stac/download.py
GeoParquet round-trip#
pyramids.stac._geoparquet
#
Serialize STAC Items to/from GeoParquet (PD-3).
stac-geoparquet stores a STAC ItemCollection as one columnar GeoParquet file
(geometry as WKB, WGS84) for bulk transfer + fast spatial filtering, avoiding
thousands of per-item JSON requests. pyramids already has geopandas (core) and a
FeatureCollection (a GeoDataFrame subclass) with GeoParquet I/O, plus the
[parquet] extra (pyarrow) — so the round-trip needs no new dependency.
Two on-disk layouts live here, sharing one pair of entry points:
- The JSON-blob layout (the default,
spec=False): each row carries the item geometry (so the file is a valid, spatially-filterable GeoParquet) plus the full STAC Item as astac_itemJSON column, so :func:from_geoparquetreconstructs the exact item dicts — ready to feed :meth:pyramids.dataset.DatasetCollection.from_stac. Lossless, but only pyramids knows how to read the blob back. - The spec layout (
spec=True, or :func:to_geoparquet_spec/ :func:from_geoparquet_spec): the STAC-GeoParquet 1.1 shape —propertiesflattened into top-level typed columns, WKB geometry in OGC:CRS84, abboxstruct, andassets/linksas Arrow structs. That is what lets DuckDB, pyarrow and the rest of the STAC tooling push spatial and attribute predicates down into the file. Written with pyarrow only — thestac-geoparquetpackage is deliberately not a dependency. It preserves every value, but the round trip is not an identity on the item dict: see the normalisations listed on :func:from_geoparquet_spec.
Requires the [parquet] extra (pyarrow) for the Parquet read/write itself.
to_geoparquet(items, path, *, spec=False)
#
Write a sequence of STAC Items to a GeoParquet file.
With spec=False (the default) each item becomes a row carrying its
geometry (a valid, spatially-filterable GeoParquet geometry in EPSG:4326)
and the full item as a JSON column. With spec=True the whole write is
delegated to :func:to_geoparquet_spec, which produces the columnar
STAC-GeoParquet 1.1 layout instead — different columns, different
metadata, and the extra guards listed under Raises.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
Any
|
Iterable of STAC Items ( |
required |
path
|
str | Path
|
Destination |
required |
spec
|
bool
|
Write the interoperable STAC-GeoParquet 1.1 layout (flattened
property columns, WKB geometry in OGC:CRS84, |
False
|
Raises:
| Type | Description |
|---|---|
ValueError
|
When |
TypeError
|
When an item is neither a dict nor exposes |
OptionalPackageDoesNotExist
|
When pyarrow (the |
Examples:
- Round-trip a couple of item dicts through GeoParquet:
>>> import tempfile, os >>> from pyramids.stac import to_geoparquet, from_geoparquet # doctest: +SKIP >>> items = [{"id": "a", "geometry": {"type": "Point", "coordinates": [1.0, 2.0]}, ... "properties": {"datetime": "2023-01-01T00:00:00Z"}, "assets": {}}] >>> path = os.path.join(tempfile.mkdtemp(), "items.parquet") # doctest: +SKIP >>> to_geoparquet(items, path) # doctest: +SKIP >>> from_geoparquet(path)[0]["id"] # doctest: +SKIP 'a'
See Also
to_geoparquet_spec: The spec layout written directly, without the flag.
Source code in src/pyramids/stac/_geoparquet.py
from_geoparquet(path, *, spec=False)
#
Read STAC Items back from a GeoParquet written by :func:to_geoparquet.
Both layouts are readable through this one entry point. With spec=False
(the default) the file's metadata is probed first: a file carrying the
stac-geoparquet key goes to the spec reader, so a spec file reads
correctly even when the flag was not passed, and everything else takes the
unchanged JSON-blob path. Passing spec=True selects the spec reader
outright.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
Path to a |
required |
spec
|
bool
|
Read the file as the STAC-GeoParquet 1.1 layout by delegating to
:func: |
False
|
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
The list of STAC Item dicts (ready for |
list[dict[str, Any]]
|
meth: |
Raises:
| Type | Description |
|---|---|
OptionalPackageDoesNotExist
|
When pyarrow (the |
KeyError
|
When the file lacks the |
ValueError
|
When |
See Also
from_geoparquet_spec: The spec layout read directly, without the flag.
Source code in src/pyramids/stac/_geoparquet.py
to_geoparquet_spec(items, path)
#
Write STAC Items as a spec-compliant STAC-GeoParquet 1.1 file.
This is the interoperable, columnar layout — the one DuckDB, pyarrow and the
wider STAC tooling can filter on without decoding a JSON blob per row. It is
produced with pyarrow only; the stac-geoparquet package is not a
dependency of pyramids.
On-disk columns, in order:
type,stac_version,stac_extensions,id— the item envelope.geometry— WKB, OGC:CRS84 (written as PROJJSON in the GeoParquetgeofile metadata, not left to the reader's default).bbox—struct<xmin, ymin, xmax, ymax>, widened tostruct<xmin, ymin, zmin, xmax, ymax, zmax>as soon as any item carries a 3D bbox. It is registered as the geometry column'scoveringonly when every row has a bbox: a covering that does not span a row would make a spatial-predicate pushdown skip it.collection— always emitted, null for items without one.datetime—timestamp[us, UTC], always emitted (an all-null column included), unless a value is not an RFC 3339 timestamp at all.- One column per remaining
propertieskey, alphabetically, Arrow-typed. - One column per extra item-root key, alphabetically.
links,assets— Arrow structs (assetskeyed by asset name) when at least one item carries them; a null string column when no item does, since a struct has no fields to infer then.
A column Arrow cannot type — or would type only by widening an int to a
float, or by inferring a zero-field struct from an empty dict such as the
"assets": {} of an asset-less item — is JSON-encoded instead, which keeps
the values intact at the cost of that column's queryability.
Three file-metadata keys are written: geo (GeoParquet 1.1),
stac-geoparquet (the spec version) and pyramids:stac-geoparquet (which
columns are JSON-encoded and which belong at the item root — read back by
:func:from_geoparquet_spec, and ignorable by every other reader).
The file is written through a sibling temporary file and moved into place, so a failed write leaves neither a truncated new file nor a damaged previous one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
items
|
Any
|
Iterable of STAC Items ( |
required |
path
|
str | Path
|
Destination |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
When |
TypeError
|
When an item is neither a dict nor exposes |
OptionalPackageDoesNotExist
|
When pyarrow (the |
Examples:
- Write the columnar layout and read it with plain pyarrow:
>>> import os, tempfile # doctest: +SKIP >>> import pyarrow.parquet as pq # doctest: +SKIP >>> from pyramids.stac import to_geoparquet_spec # doctest: +SKIP >>> items = [{"id": "a", "geometry": {"type": "Point", "coordinates": [1.0, 2.0]}, ... "bbox": [1.0, 2.0, 1.0, 2.0], "assets": {}, ... "properties": {"datetime": "2023-01-01T00:00:00Z"}}] >>> path = os.path.join(tempfile.mkdtemp(), "spec.parquet") # doctest: +SKIP >>> to_geoparquet_spec(items, path) # doctest: +SKIP >>> "datetime" in pq.read_table(path).column_names # doctest: +SKIP True
See Also
to_geoparquet: The pyramids JSON-blob layout (lossless, pyramids-only). from_geoparquet_spec: The matching reader.
Source code in src/pyramids/stac/_geoparquet.py
989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 | |
from_geoparquet_spec(path)
#
Read STAC Items back from a spec-compliant STAC-GeoParquet file.
The inverse of :func:to_geoparquet_spec, and tolerant of files written by
other STAC-GeoParquet writers: the reserved spec columns are recognised by
name, every other column is read back as a properties entry, timestamp
columns are rendered to their RFC 3339 strings and the WKB geometry is
decoded to GeoJSON. Null cells become absent keys, which is what restores
the original sparse items out of Arrow's rectangular structs.
Four normalisations are inherent to the columnar layout, and are the same
ones the stac-geoparquet package applies — so the round trip preserves
every value, but is not an identity on the item dict:
- Timestamps come back in the
Zspelling (...T00:00:00Z), whatever offset spelling they were written with, because the column stores a real instant rather than the original text. - Timestamps are stored with microsecond resolution, so a nanosecond
timestamp loses its last three digits (
...05.123456789Z→...05.123456Z). - A key whose value was explicitly
nullcomes back absent, since the column cannot tell "null" from "not set". That applies to anullgeometryand to a key nested insideassetsjust as much as to a top-level property. - Every item gains a
propertiesdict, empty when the row held no property values, even if the written item carried nopropertieskey at all.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | Path
|
Path to a |
required |
Returns:
| Type | Description |
|---|---|
list[dict[str, Any]]
|
The list of STAC Item dicts (ready for |
list[dict[str, Any]]
|
meth: |
list[dict[str, Any]]
|
carries a |
Raises:
| Type | Description |
|---|---|
OptionalPackageDoesNotExist
|
When pyarrow (the |
ValueError
|
When the file is not in the spec layout — the pyramids
JSON-blob layout (read it with :func: |
See Also
to_geoparquet_spec: The matching writer. from_geoparquet: Reads either layout, auto-detecting the spec one.