Skip to content

Conversation

@Wentao-Kuang
Copy link
Contributor

Motivation

ETL got failed when processing shared layers in different schema with concurrency. The reason is they share a same download file and one of the process might use half downloaded file.

Modifications

  • Add try catch for etl create, to log the error layer id for investigation when failure with async process.
  • Add schema path for the layer download files, so that we don't need to share same downloaded file in the tmp folder.

Verification

Tested locally with multi concurrency. Same test was failure before the fix.

image image

@Wentao-Kuang Wentao-Kuang requested a review from a team as a code owner August 22, 2025 01:21
@Wentao-Kuang Wentao-Kuang added this pull request to the merge queue Aug 24, 2025
Merged via the queue into master with commit 3769175 Aug 24, 2025
21 of 27 checks passed
@Wentao-Kuang Wentao-Kuang deleted the feat/debug-shortbread branch August 24, 2025 23:14
@ccbblin ccbblin mentioned this pull request Aug 27, 2025
This was referenced Aug 27, 2025
@ccbblin ccbblin mentioned this pull request Sep 7, 2025
github-merge-queue bot pushed a commit that referenced this pull request Sep 7, 2025
# [8.8.0](v8.7.0...v8.8.0)
(2025-09-07)


### Bug Fixes

* add required permissions for release please github action
([#3497](#3497))
([a6155c9](a6155c9))
* **cli-vector:** Fix the tmp path of download layers and add try catch.
BM-1352 ([#3494](#3494))
([3769175](3769175))
* intermittent unit tests
([#3503](#3503))
([1e942af](1e942af))
* stop using GITHUB_TOKEN for release-please
([#3495](#3495))
([198cb29](198cb29))


### Features

* **cli-raster:** Fetch chart imagery metadata from backup
location.BM-1345 ([#3492](#3492))
([2b42fa7](2b42fa7))
* **cli-raster:** New cli to standardise charts map. BM-1338
([#3483](#3483))
([35b7854](35b7854))
* **geo:** implement support for all raster topo map series projections
BM-1160 ([#3480](#3480))
([8652578](8652578))
* **geo:** support tile matrixes that are not square
([#3484](#3484))
([7720d02](7720d02))
* **lambda-tiler:** support one band uint16 lerc tiffs
([#3489](#3489))
([906d016](906d016))
* **landing:** quick change cycles through more basemaps BM-1293
([#3493](#3493))
([4bfba36](4bfba36))
@ccbblin ccbblin mentioned this pull request Sep 8, 2025
github-merge-queue bot pushed a commit that referenced this pull request Sep 9, 2025
# [8.9.0](v8.7.0...v8.9.0)
(2025-09-09)


### Bug Fixes

* add required permissions for release please github action
([#3497](#3497))
([a6155c9](a6155c9))
* **cli-vector:** Fix the tmp path of download layers and add try catch.
BM-1352 ([#3494](#3494))
([3769175](3769175))
* **geo:** ensure all supported epsg codes have a projection
([#3508](#3508))
([23a9243](23a9243))
* intermittent unit tests
([#3503](#3503))
([1e942af](1e942af))
* remove release-please from prod deploy
([#3506](#3506))
([05b6f95](05b6f95))
* stop using GITHUB_TOKEN for release-please
([#3495](#3495))
([198cb29](198cb29))


### Features

* **cli-raster:** Fetch chart imagery metadata from backup
location.BM-1345 ([#3492](#3492))
([2b42fa7](2b42fa7))
* **cli-raster:** New cli to standardise charts map. BM-1338
([#3483](#3483))
([35b7854](35b7854))
* **geo:** implement support for all raster topo map series projections
BM-1160 ([#3480](#3480))
([8652578](8652578))
* **geo:** support tile matrixes that are not square
([#3484](#3484))
([7720d02](7720d02))
* **lambda-tiler:** support one band uint16 lerc tiffs
([#3489](#3489))
([906d016](906d016))
* **landing:** quick change cycles through more basemaps BM-1293
([#3493](#3493))
([4bfba36](4bfba36))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants