AI4COPSEC AI4COPSEC collekt
  • Home
  • Products
  • CLI
  • Credentials
  • Manifest
  • API reference

On this page

  • Fetch Data
  • Dataset Selection
  • Planning
  • Cache and Strict Mode
  • Catalog Overlays
  • Diagnostics
  • Inspect Configuration

CLI

The collekt command line is a thin wrapper around the same objects used by the Python API: a region and time window, a DatasetConfig YAML selection, and a runtime output directory.

Fetch Data

Use collekt fetch to download source-native files for a request. A dataset selection is required and the output directory is runtime state, not part of the selection file.

collekt fetch \
  --bbox -6 20 35 45 \
  --start 2023-06-15 \
  --end 2023-06-15 \
  --dataset-config westmed.yaml \
  --output-dir data/collections

--bbox uses WEST EAST SOUTH NORTH order. The same request can also use a point radius:

collekt fetch \
  --at-lat 43.3 \
  --at-lon 5.4 \
  --radius 50 \
  --start 2023-06-15 \
  --dataset-config westmed.yaml \
  --output-dir data/collections

or a GeoJSON region:

collekt fetch \
  --region regions/west-med.geojson \
  --start 2023-06-15 \
  --dataset-config westmed.yaml \
  --output-dir data/collections

If --end is omitted, it defaults to the start day. If --start is omitted, the request defaults to the current UTC day.

Dataset Selection

--dataset-config points to a YAML version of DatasetConfig. It names the desired datasets and provider parameters; the source catalog remains bundled with the package, or supplied through --conf-dir.

datasets:
  - provider: cmems
    key: cmems_glorys_my
    variables: [uo, vo]
    depth: [1.0, 1.1]
  - provider: cmems
    key: cmems_duacs_my
    variables: [ugos, vgos]
  - provider: skytruth
    key: skytruth

For CMEMS, select variables from the catalog’s available_variables. Datasets with depth require a depth: [min, max] range. See Products for the bundled source keys and Credentials for provider secrets.

Planning

Use --dry-run to inspect the planned provider requests without downloading files.

collekt fetch \
  --bbox -6 20 35 45 \
  --start 2023-06-15 \
  --dataset-config westmed.yaml \
  --output-dir data/collections \
  --dry-run

The dry-run table reports each source, requested day, availability status, dataset ID, variables, and planned output path.

Cache and Strict Mode

By default, fetch reuses files already present under the output directory. Pass --no-cache to force a new download.

collekt fetch \
  --bbox -6 20 35 45 \
  --start 2023-06-15 \
  --dataset-config westmed.yaml \
  --output-dir data/collections \
  --no-cache

Pass --strict when skipped sources should make the command fail instead of being reported in the result summary.

Catalog Overlays

Use --conf-dir when a project needs to add or override source catalog entries. The directory is merged on top of the bundled catalog.

collekt fetch \
  --bbox -6 20 35 45 \
  --start 2023-06-15 \
  --conf-dir my_brick/conf \
  --dataset-config presets/currents.yaml \
  --output-dir data/collections

The selection file still contains only the requested datasets. The catalog overlay defines what datasets exist.

Diagnostics

collekt doctor checks provider packages, credentials, catalog consistency, and flags any source config key not recognized by its adapter (a likely typo, e.g. pad_dg instead of pad_deg) as a warning.

collekt doctor

Pass --online to validate CMEMS datasets, variables, and declared coverage against the Copernicus Marine catalogue.

collekt doctor --online

Use the same catalog overlay mechanism when checking downstream source entries.

collekt doctor --conf-dir my_brick/conf --online

The command exits with a non-zero status if any check fails.

Inspect Configuration

collekt config show prints the merged configuration as YAML.

collekt config show
collekt config show --conf-dir my_brick/conf

Use it to verify that a catalog overlay is visible before running a fetch.

Part of AI4COPSEC.

Built with Quarto.

GitHub