Skip to content

Commit 6af7d19

Browse files
committed
Don't set Content-Encoding: gzip on the S3 upload
ClickHouse's s3() reader decompresses via the explicit compression arg; a gzip Content-Encoding header on the object makes it mis-read and ingest 0 rows silently (no error). Upload plain gzip bytes so the replicator's compression='gzip' path works.
1 parent 0f6cef7 commit 6af7d19

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

.github/workflows/runner-fleet-metrics.yml

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,9 @@ jobs:
7373
set -euo pipefail
7474
ts="$(date -u +%Y%m%dT%H%M%SZ)"
7575
gzip -c runner_fleet_count.jsonl > runner_fleet_count.json.gz
76+
# NB: do NOT set --content-encoding gzip. ClickHouse's s3() reader
77+
# decompresses via the explicit compression arg in the replicator; a
78+
# gzip Content-Encoding header makes it mis-read the object and ingest
79+
# 0 rows silently (no error). Store plain gzip bytes instead.
7680
aws s3 cp runner_fleet_count.json.gz \
77-
"s3://${S3_BUCKET}/${S3_PREFIX}/${RUNNER_ORG}/${ts}.json.gz" \
78-
--content-type application/json --content-encoding gzip
81+
"s3://${S3_BUCKET}/${S3_PREFIX}/${RUNNER_ORG}/${ts}.json.gz"

0 commit comments

Comments
 (0)