Manifest
Every download writes a manifest.json into the request directory. It is collekt’s machine-readable record of what was requested, fetched, reused, and skipped, and it is the contract downstream consumers rely on.
Structure
{
"manifest_schema_version": "0.1",
"request": {
"region": {"west": -6.0, "east": 20.0, "south": 35.0, "north": 45.0},
"start": "2023-06-15T00:00:00Z",
"end": "2023-06-15T23:59:59Z",
"metadata": {}
},
"summary": {"planned": 0, "downloaded": 1, "reused": 0, "skipped": 0, "failed": 0},
"sources": { "cmems_glorys_my": { "kind": "cmems", "variables": ["uo", "vo"], "...": "..." } },
"files": [ { "source": "cmems_glorys_my", "status": "downloaded", "path": "cmems/global/glorys/…nc", "format": "netcdf", "inspection": {"size_bytes": 12345, "data_variables": ["uo", "vo"]} } ],
"planned": [],
"warnings": [],
"failures": []
}- Paths are relative to the request directory, so a collection directory or zip archive is relocatable.
- Downloaded/reused files carry an inspection block: variables/sizes for NetCDF, columns/row count for Parquet, otherwise the file size.
warningsholds skipped sources;failuresholds failed ones. Both are non-blocking unless the run is--strict.
Batch provenance
With batch_days enabled, daily files can share a details.batch record. Collection-level event files instead carry details.batches, a list of the queries used to build that file. Each record contains id, start, end, transport, and the concrete provider request. Native per-day parameters and details.temporal remain attached to daily results.
transport distinguishes range, individual_files, archive_read, and search_then_individual_files. A retrieval group is not necessarily one HTTP request: provider SDKs, pagination, and product downloads can make several. Summary counters continue to count files/results. On reuse, the original batch provenance is retained instead of being replaced by the current batch setting.
A batched download also reports the planner’s provider, method, request, and availability for each file, where the same file downloaded without batch_days carries only temporal. A collection filled in over several runs can therefore hold both shapes. Consumers should read details keys by name rather than assuming a fixed set — which is what the flexible block is for.
This is optional metadata within the existing flexible details block; the manifest version remains 0.1. See Batch downloads.
Draft schema
A permissive draft JSON Schema ships at collekt/resources/schema/manifest.schema.json. During alpha the manifest stays flexible and the schema is provided for reference only — it is not enforced. Core fields are required; provider-specific metadata under details may evolve. manifest_schema_version is bumped when the shape changes materially; a strict, validated schema is a 1.0 goal.