Fetcher

Fetcher(
    request,
    *,
    config,
    output_dir=None,
    conf_dir=None,
    strict=None,
    progress=null_progress,
)

Fetch source-native data for a request.

Fetcher keeps request, configuration, plan, download, and export state together. It downloads source-native files into a staging collection directory, then exports that directory or a zip archive for downstream use. Analysis-ready outputs are built separately from the returned Result.

Example

import collekt

config = collekt.DatasetConfig(
    collekt.CMEMS("cmems_duacs_my", variables=["ugos", "vgos"]),
)
request = collekt.Request(region=collekt.Region.from_bbox((-6.0, 20.0, 35.0, 45.0)), start="2023-06-15")

fetcher = collekt.Fetcher(request=request, config=config, output_dir="collections")
result = fetcher.download()
fetcher.export_zip("collection.zip")

Parameters

Name	Type	Description	Default
request	Request	Region, time window, and metadata.	required
config	DatasetConfig | Config	Selected datasets and provider parameters.	required
output_dir	str | Path | None	Root directory for staged downloads.	None
conf_dir	str | Path | None	Optional configuration directory that extends the bundled catalog with additional or overriding datasets. Only used when config is a DatasetConfig.	None
strict	bool | None	Override config strict mode. If true, warnings become a final RuntimeError after the manifest is written.	None
progress	ProgressCallback	Optional progress callback.	null_progress

Attributes

Name	Description
output_dir	Root directory used for staged downloads.

Methods

Name	Description
check	Run diagnostics, then return a dry-run plan.
download	Download configured source-native files into the staging directory.
export_directory	Export the downloaded collection as a directory copy.
export_zip	Export the downloaded collection as a zip archive.
plan	Plan provider requests and output paths without downloading files.

check

Fetcher.check(online=False)

Run diagnostics, then return a dry-run plan.

Parameters

Name	Type	Description	Default
online	bool	If true, query provider catalogues where supported.	False

Returns

Name	Type	Description
	Result	Planned collection result. Diagnostics are also stored in
	Result	doctor_checks.

download

Fetcher.download(use_cache=True)

Download configured source-native files into the staging directory.

The staging directory is keyed by region and time range, so cached files are only reused when both match the request.

Parameters

Name	Type	Description	Default
use_cache	bool	If true (default), reuse files already staged for this exact request. If false, delete the request directory and download everything again.	True

Returns

Name	Type	Description
	Result	Collection result with downloaded, reused, skipped, and failed
	Result	source results.

Raises

Name	Type	Description
	RuntimeError	If strict mode is enabled and any source was skipped or failed.

export_directory

Fetcher.export_directory(output_dir, *, overwrite=False)

Export the downloaded collection as a directory copy.

Parameters

Name	Type	Description	Default
output_dir	str | Path	Destination directory.	required
overwrite	bool	If true, replace an existing destination.	False

Returns

Name	Type	Description
	Path	The destination directory.

Raises

Name	Type	Description
	RuntimeError	If download has not been called yet.
	FileExistsError	If the destination exists and overwrite is false.

export_zip

Fetcher.export_zip(output_path, *, overwrite=False)

Export the downloaded collection as a zip archive.

Parameters

Name	Type	Description	Default
output_path	str | Path	Destination zip file.	required
overwrite	bool	If true, replace an existing zip file.	False

Returns

Name	Type	Description
	Path	The destination zip file.

Raises

Name	Type	Description
	RuntimeError	If download has not been called yet.
	FileExistsError	If the destination exists and overwrite is false.

plan

Fetcher.plan()

Plan provider requests and output paths without downloading files.

Returns

Name	Type	Description
	Result	Planned collection result.