Skip to content

Commit af9170c

Browse files
dtunikovclaude
andauthored
fix(clickhouse): keep IAM-role S3 test TTL within toDateTime range (#4434)
### Problem `connectors/clickhouse/TestIAMRoleCanIssueSelectFromS3` is failing on `main` (e.g. [this run](https://github.com/PeerDB-io/peerdb/actions/runs/27573392839/job/81515384634)). It loads a 3-row Avro fixture into a raw table and asserts `COUNT(*) == 3`, but gets `0` — the rows are silently evicted by the table TTL. ### Root cause #4425 changed the raw-table TTL expression to wrap the timestamp in `toDateTime()`: ```sql TTL toDateTime(fromUnixTimestamp64Nano(_peerdb_timestamp)) + INTERVAL N DAY ``` `toDateTime` is a 32-bit type capped at **2106-02-07**. The test deliberately sets `PEERDB_CLICKHOUSE_RAW_TABLE_TTL_DAYS=36500` (100 years) because the fixture rows carry **2025** timestamps and the default 90-day TTL would otherwise evict them. With the old `DateTime64` expression `2025 + 100y ≈ 2125` was fine; with `toDateTime` it overflows the 2106 ceiling and wraps to a past date, so the TTL evaluates as already-expired and ClickHouse drops the rows → `COUNT(*) = 0`. The production default (90 days) never overflows, so this only affects the test's large override. The backward-compat `toDateTime` fix from #4425 is left intact. ### Fix Lower the test's TTL override from 36500 to 25000 days (~68y, lands around 2093), keeping it within the `toDateTime` range while still preventing eviction of the 2025 fixture rows. Added a comment explaining the 2106 ceiling. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 6641b29 commit af9170c

1 file changed

Lines changed: 4 additions & 2 deletions

File tree

flow/connectors/clickhouse/s3_iam_role_test.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,10 @@ func TestIAMRoleCanIssueSelectFromS3(t *testing.T) {
3535
}
3636
internal.SetupFlowAWSCredentialsFromEnv(t)
3737
t.Setenv("PEERDB_CLICKHOUSE_AWS_S3_BUCKET_NAME", os.Getenv(bucketNameEnvVar))
38-
// Fixture rows have _peerdb_timestamp values from 2025; neuter the raw-table TTL so it doesn't evict them.
39-
t.Setenv("PEERDB_CLICKHOUSE_RAW_TABLE_TTL_DAYS", "36500")
38+
// Fixture rows have _peerdb_timestamp values from 2025; push the raw-table TTL far enough out that it doesn't
39+
// evict them. The TTL expression wraps the timestamp in toDateTime() (a 32-bit type capped at 2106-02-07), so the
40+
// 2025 base + INTERVAL must stay under that ceiling; 25000 days (~68y) lands around 2093, comfortably within range.
41+
t.Setenv("PEERDB_CLICKHOUSE_RAW_TABLE_TTL_DAYS", "25000")
4042
ctx := t.Context()
4143

4244
conn, err := NewClickHouseConnector(ctx, nil,

0 commit comments

Comments
 (0)