Skip to content

Fix Snowflake resync failure due to case mismatch in _resync table suffix#4185

Merged
serprex merged 1 commit into
PeerDB-io:mainfrom
kang8:bugfix/snowflake-resync-case-mismatch
May 21, 2026
Merged

Fix Snowflake resync failure due to case mismatch in _resync table suffix#4185
serprex merged 1 commit into
PeerDB-io:mainfrom
kang8:bugfix/snowflake-resync-case-mismatch

Conversation

@kang8

@kang8 kang8 commented Apr 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #4184

  • The _resync suffix was appended in lowercase, producing mixed-case identifiers like ACCOUNTS_resync
  • getColsFromTable() in qrep.go:187 unconditionally applies strings.ToUpper(), converting it to ACCOUNTS_RESYNC
  • Snowflake treats quoted identifiers as case-sensitive, so the table lookup fails with "does not exist"
  • Changed the suffix to uppercase _RESYNC to ensure consistent identifiers across creation and query paths

Test plan

  • Verified Snowflake resync creates tables with _RESYNC suffix
  • Verified getColsFromTable() can find the resync tables
  • Verified table swap (TrimSuffix) correctly strips _RESYNC after resync completes
  • Confirm no regression for BigQuery, ClickHouse, or PostgreSQL destinations (they don't use getColsFromTable with ToUpper)

@kang8 kang8 had a problem deploying to external-contributor April 16, 2026 08:17 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor April 16, 2026 08:17 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor April 16, 2026 08:17 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor April 16, 2026 08:17 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor April 16, 2026 08:17 — with GitHub Actions Failure
@CLAassistant

CLAassistant commented Apr 16, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@kang8 kang8 force-pushed the bugfix/snowflake-resync-case-mismatch branch from f219dd7 to 0e5edf9 Compare May 16, 2026 14:37
@kang8 kang8 requested a review from a team as a code owner May 16, 2026 14:37
@kang8 kang8 had a problem deploying to external-contributor May 16, 2026 14:37 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor May 16, 2026 14:37 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor May 16, 2026 14:37 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor May 16, 2026 14:37 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor May 16, 2026 14:37 — with GitHub Actions Failure
@kang8

kang8 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

Hi @serprex @Amogh-Bharadwaj, friendly ping on this PR — it's a small 2-line fix for a Snowflake resync bug (#4184). CLA is signed and CI is waiting on a maintainer to approve the workflow run. Could one of you take a look when you have a moment? Thanks!

@serprex

serprex commented May 20, 2026

Copy link
Copy Markdown
Member

cdc_flow.go is shared code between all connectors, this should be fixed in snowflake connector specifically

…ffix (PeerDB-io#4184)

The resync suffix was appended in lowercase ("_resync"), producing
mixed-case identifiers like "ACCOUNTS_resync". However, getColsFromTable()
in qrep.go unconditionally applied strings.ToUpper(), converting it to
"ACCOUNTS_RESYNC". Since Snowflake treats quoted identifiers as
case-sensitive, the table lookup failed with "does not exist".

Fix in Snowflake's getColsFromTable: replace the unconditional
strings.ToUpper with snowflakeSchemaTableNormalize so quoted mixed-case
identifiers are looked up consistently with how they are created. The
shared cdc_flow.go is left untouched since the suffix handling is a
Snowflake-specific concern.
@kang8 kang8 force-pushed the bugfix/snowflake-resync-case-mismatch branch from 0e5edf9 to 95e208f Compare May 20, 2026 06:31
@kang8 kang8 requested a review from a team as a code owner May 20, 2026 06:31
@kang8 kang8 temporarily deployed to external-contributor May 20, 2026 06:31 — with GitHub Actions Inactive
@kang8 kang8 temporarily deployed to external-contributor May 20, 2026 06:31 — with GitHub Actions Inactive
@kang8 kang8 had a problem deploying to external-contributor May 20, 2026 06:31 — with GitHub Actions Failure
@kang8 kang8 had a problem deploying to external-contributor May 20, 2026 06:31 — with GitHub Actions Failure
@kang8 kang8 temporarily deployed to external-contributor May 20, 2026 06:31 — with GitHub Actions Inactive
@kang8

kang8 commented May 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review! I've moved the fix to be Snowflake-specific in getColsFromTable (connectors/snowflake/qrep.go) and left cdc_flow.go untouched. PTAL when you have a moment.

@serprex serprex requested a review from jgao54 May 20, 2026 22:37
@Jeremyyang920 Jeremyyang920 removed the request for review from a team May 20, 2026 22:52
@codecov

codecov Bot commented May 20, 2026

Copy link
Copy Markdown

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
2253 2 2251 203
View the top 2 failed test(s) by shortest run time
github.com/PeerDB-io/peerdb/flow/e2e::TestApiMongo
Stack Traces | 0.01s run time
=== RUN   TestApiMongo
=== PAUSE TestApiMongo
=== CONT  TestApiMongo
--- FAIL: TestApiMongo (0.01s)
github.com/PeerDB-io/peerdb/flow/e2e::TestApiMongo/TestClickHouseMirrorValidation_Pass
Stack Traces | 0.06s run time
=== RUN   TestApiMongo/TestClickHouseMirrorValidation_Pass
=== PAUSE TestApiMongo/TestClickHouseMirrorValidation_Pass
=== CONT  TestApiMongo/TestClickHouseMirrorValidation_Pass
    api_test.go:306: 
        	Error Trace:	.../flow/e2e/api_test.go:306
        	Error:      	Received unexpected error:
        	            	rpc error: code = FailedPrecondition desc = failed to validate source connector mongo: oplog retention must be set to >= 24 hours, but got 23.000000
        	Test:       	TestApiMongo/TestClickHouseMirrorValidation_Pass
    api_test.go:49: begin tearing down postgres schema api_3395sqco
--- FAIL: TestApiMongo/TestClickHouseMirrorValidation_Pass (0.06s)

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@kang8 kang8 temporarily deployed to external-contributor May 20, 2026 23:46 — with GitHub Actions Inactive
@kang8

kang8 commented May 21, 2026

Copy link
Copy Markdown
Contributor Author

The Codecov failure TestApiMongo/TestClickHouseMirrorValidation_Pass (oplog retention must be set to >= 24 hours, but got 23.000000) is unrelated to this PR — it's a pre-existing flaky test.

Root cause: TestApiMongo runs subtests in parallel via RunSuite (e2eshared.go:24). TestMongoDBOplogRetentionValidation (api_test.go:1138) temporarily mutates the shared MongoDB instance's global oplog retention to 23h to exercise the failure path, then restores it to 24h. Any other parallel subtest that performs CDC validation during this window fails.

Prior occurrences of the same flake (different subtests, same error):

This PR only touches flow/connectors/snowflake/qrep.go; it does not touch any MongoDB / ClickHouse / validation code paths. Could we rerun the failed job (or merge as-is)? I'm happy to open a separate issue to track fixing the flake.

@serprex serprex merged commit edeee48 into PeerDB-io:main May 21, 2026
17 of 19 checks passed
@kang8 kang8 deleted the bugfix/snowflake-resync-case-mismatch branch May 21, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Snowflake resync fails due to case mismatch in _resync table suffix

4 participants