Skip to content

Conversation

lzshlzsh
Copy link
Contributor

@lzshlzsh lzshlzsh commented Aug 3, 2025

As the design of https://github.com/apache/flink-cdc/pull/2220,CDC only cares about the maximum GTID position and starts from it. For example, if reading from gtid offset 1-7:9-10, it will automatically adjust to read from 1-10, which causes an error in skipping gitd site 8, thus losing data. In particular, when gtid bit 8 is a large transaction, it will cause more serious data loss. We have encountered this problem many times in the production environment.

More details can be found in [FLINK-38183].

@leonardBang
Copy link
Contributor

@ruanhang1993 would you take a look this PR?

@mielientiev
Copy link
Contributor

This is a duplicate PR for the same issue that I opened and proposed fix awhile ago
#3845

cc @leonardBang

@lzshlzsh lzshlzsh marked this pull request as draft August 11, 2025 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants