[source-postgres] Incremental CDC sync fails with "Saved offset is before replication slot's confirmed lsn" after successful snapshot #70849
Unanswered
melissawessel
asked this question in
Connector Questions
Replies: 1 comment
-
|
I'm having the same issue |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hi all — I’m testing Postgres CDC on our staging environment and running into an issue I haven’t been able to get past. Hoping someone can help me understand what’s going on or confirm whether this is expected behavior.
I’m using source-postgres with CDC enabled. The initial snapshot sync works fine, but the very next sync (the first incremental CDC run) always fails immediately with:
Because of this, I haven’t been able to get a successful incremental CDC sync yet.
What I’m doing
The second sync always fails the same way.
One nuance: if I run an incremental sync without making any table changes, the sync succeeds. As soon as I insert/update rows and advance the slot LSN, the next sync fails with the LSN mismatch.
What I’ve tried
Even with a completely fresh setup, the incremental sync fails immediately.
Log excerpt
Here is the relevant part showing the mismatch:
Possibly related
I found a GitHub discussion where someone ran into a similar LSN mismatch after switching from wal2json to pgoutput, and the solution was to recreate the connection because the saved CDC state and slot state no longer aligned.
(They mentioned that recreating only the slot wasn’t enough.)
In my case, even recreating the slot, publication, source, and connection doesn’t clear the mismatch, so I’m wondering if something similar could be happening with internal state.
What I’m trying to understand
What I’m trying to understand is why the connector’s saved offset ends up behind the slot’s confirmed_flush_lsn immediately after a fresh snapshot, even with almost no DB activity. Is this expected, or is there something specific I should be configuring to prevent that gap? Any guidance would be appreciated.
Beta Was this translation helpful? Give feedback.
All reactions