Dataset Writing#

When connected to an ODC Database, these methods are available for adding, updating and archiving datasets:

dc = Datacube()
dc.index.datasets.{method}

add(dataset[, with_lineage, archive_less_mature])

Add dataset to the index.

add_location(id_, uri)

Add a location to the dataset if it doesn't already exist.

archive(ids)

Mark datasets as archived

archive_location(id_, uri)

Archive a location of the dataset if it exists and is active.

remove_location(id_, uri)

Remove a location from the dataset if it exists.

restore(ids)

Mark datasets as not archived

restore_location(id_, uri)

Un-archive a location of the dataset if it exists.

update(dataset[, updates_allowed, ...])

Update dataset metadata and location :type dataset: Dataset :param dataset: Dataset model with unpersisted updates :type updates_allowed: Mapping[tuple[str | int, ...], Callable[[tuple[str | int, ...], tuple[str | int, ...], MissingSentinel | str | int | None | Sequence[Any] | Mapping[str, Any], MissingSentinel | str | int | None | Sequence[Any] | Mapping[str, Any]], bool]] | None :param updates_allowed: Allowed updates :type archive_less_mature: int | None :param archive_less_mature: Find and archive less mature datasets with ms delta :rtype: Dataset :return: Persisted dataset model

The following “high level” API helps you get new datasets into a form ready to pass to the above methods in a particular index.

class datacube.index.hl.Doc2Dataset(index, products=None, exclude_products=None, fail_on_missing_lineage=False, verify_lineage=True, skip_lineage=False, eo3='auto', home_index=None)[source]#

Used for constructing Dataset objects from plain metadata documents.

This requires a database connection to perform the automatic matching against available products.

There are options for including and excluding the products to match against, as well as how to deal with source lineage.

Once constructed, call with a dictionary object and location URI, eg:

resolver = Doc2Dataset(index)
dataset = resolver(dataset_dictionary, 'file:///tmp/test-dataset.json')
index.dataset.add(dataset)
Parameters:

  • index (AbstractIndex) – an open Database connection

  • products (Sequence[str] | None) – List of product names against which to match datasets (including lineage datasets). If not supplied we will consider all products.

  • exclude_products (Sequence[str] | None) – List of products to exclude from matching

  • fail_on_missing_lineage (bool) –

    If True fail resolve if any lineage datasets are missing from the DB

    Only False supported if index.supports_external_lineage is True.

  • verify_lineage (bool) –

    If True, check that lineage datasets in the supplied document are identical to DB versions

    Ignored for EO3 documents. Will be dropped in ODCv2 as only eo3 documents will be supported.

  • skip_lineage (bool) – If True, ignore lineage sub-tree in the supplied document and construct dataset without lineage datasets

  • eo3 (bool | str) –

    ‘auto’/True/False by default auto-detect EO3 datasets and pre-process them

    Cannot be ‘False’ if index.supports_legacy is False. Will be dropped in ODCv2 as only eo3 documents will be supported

  • home_index (str | None) – Ignored if index.supports_exernal_home is False. Defaults to None. Optional string labelling the “home index” for lineage datasets. home_index is ignored if an explicit source_tree is passed to the resolver.