feat(basemaps): Topo raster maps standardisation workflow. BM-1128

[Wentao-Kuang]

Motivation

As a Basemaps user, I want to consume the NZTopo 50 & 250 Maps as a tile service.

Modification

This work defines a new Argo topo-maps-standardising workflow and an update to the existing imagery-import-cogify workflow. We have used this work to generate and publish the NZ Topo Gridless Maps layer to basemaps.

1. topo-maps-standardising (new)

   This work defines a new Argo topo-maps-standardising workflow for coordinating the standardisation of an NZTopo Map Series imagery collection. We have designed the workflow to handle the following collections stored in the AWS S3 TopoReleaseArchive directory:

   s3://topographic-upload/TopoReleaseArchive/NZTopo50_GeoTif_Gridless/
   s3://topographic-upload/TopoReleaseArchive/NZTopo250_GeoTif_Gridless/
   

   To support this workflow, we have introduced new Topo Raster processes to the basemaps\cogify package. This workflow executes these new processes as part of its process flow.

   This workflow has two key steps:

   For an NZTopo Map Series imagery collection:

   • Generate a directory structure of STAC Item and Collection files
   • Standardise the contained GeoTIFF imagery files and save them within the generated directory structure

   From this, we can pass the resulting directories into the imagery-import-cogify workflow to process the standardised imagery into basemaps, either for previewing, or publishing.

2. imagery-import-cogify (update)

   This work also updates the imagery-import-cogify workflow to support the new background parameter for the CLI create command of the basemaps/cogify package. This allows the command to override any transparent pixels with a fixed colour. In this work's context, this allows us to eliminate imagery artifacts that used to appear around the COGs, particularly for NZTopo Map Series imagery.

[tawera-manaena]

@Wentao-Kuang - I've made a minor adjustment to the workflow in Argo. We need to bring those changes here.

Before

        artifacts:
          - name: tiles
            path: /tmp/topo-stac-creation/tiles.json
          - name: brokenTiffs
            path: /tmp/topo-stac-creation/brokenTiffs.json

After

        artifacts:
          - name: tiles
            path: /tmp/topo-stac-creation/tiles.json
          - name: brokenTiffs
            path: /tmp/topo-stac-creation/broken-tiffs.json