Broken paths to accessories after indexing STAC files with relative asset paths

[pjonsson]

Datacube 1.9.20 with the postgis driver.

Doing datacube dataset add /home/ubuntu/test/stac_item.json with a stac_item.json that looks like this:

  "assets": {
    "overview": {
      "href": "overview.jpg",
      "type": "image/jpeg",
      "title": "Overview",
      "description": "Overview",
      "roles": [
        "thumbnail"
      ]
    },
    "B02": {
      "href": "B02.tif",
      "type": "image/tiff; application=geotiff; profile=cloud-optimized",
      "description": "BLUE",
      "raster:bands": [
        { ... }
      ]
    },
  }

results in this database content:

datacube=# select * from odc.dataset where id='<uuid>';
...  "accessories": {"overview": {"path": "overview.jpg"} ... "measurements": {"B02": {"path": "file:///home/ubuntu/test/B02.tif"},

and that gives Explorer faulty paths for the accessories in the information presented for the dataset in https://<explorer-instance>/products/<product-name>/datasets/<uuid>.

I'm not sure where the "path normalization" is done for relative paths of measurements when indexing STAC files, but the same logic should probably apply to accessories with relative paths as well.

[Ariana-B]

It has to do with the asset_absolute_paths param defined in odc-stac (which I believe you yourself added), and it only gets applied to assets that are flagged as measurements, hence the difference in behaviour. For it to be specified with dataset add I'll have to add it as a flag.

[pjonsson]

I remember doing the odc-stac change, but it was odc-tools that I needed that flag for at the time. I don't recall making changes to the code that runs for datacube dataset add except possibly passing the flag through somewhere, but my intention would definitely have been to treat all assets equally, not just doing it for some of them.

[Ariana-B]

A PR to add the flag to datacube dataset add is incoming.

Treating all assets equally would require an update to the logic in odc-stac here: https://github.com/opendatacube/odc-stac/blob/develop/odc/stac/_mdtools.py#L879

[pjonsson]

A PR to add the flag to datacube dataset add is incoming.

Running datacube dataset add on STAC files with relative asset paths already works so that must be passing the flag already, but I think you're right that there needs to be an update in odc-stac at the place you've linked.

[Ariana-B]

Running datacube dataset add on STAC files with relative asset paths already works so that must be passing the flag already, but I think you're right that there needs to be an update in odc-stac at the place you've linked.

When I test this, I see file:/// being prepended to the relative asset link, so there must be something you're doing differently. I'm not sure how the flag could be passed currently since asset_absolute_paths is True by default and dataset add doesn't provide any way to overwrite that value.

[pjonsson]

That is strange, I'm fairly sure we're running a completely stock Datacube 1.9.20. I'll send you a reproducer.