[FLINK-39207][mysql] Fix mysql cdc could get stuck in backfill binlog reading when reuse snapshot split reader#4311
Merged
lvyanquan merged 4 commits intoapache:masterfrom Mar 20, 2026
Conversation
… reading when reuse snapshot split reader
4efbc47 to
123a860
Compare
lvyanquan
reviewed
Mar 18, 2026
...src/main/java/org/apache/flink/cdc/connectors/mysql/debezium/reader/SnapshotSplitReader.java
Show resolved
Hide resolved
Contributor
|
Please check if the failed MySQL test is related to this change. |
...src/main/java/org/apache/flink/cdc/connectors/mysql/debezium/reader/SnapshotSplitReader.java
Outdated
Show resolved
Hide resolved
Contributor
Author
@lvyanquan Yes, it is. The newly add tests modify the test table, thus the following tests is affected and the assumption of the original records in test table is not satisfied, I'll fix it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This closes FLINK-39207.
What is the purpose of the change
This PR fixes a bug where
MySqlSourceReadercould get stuck in the backfill phase after a failover during the snapshot phase.When
MySqlSourceReaderprocesses multiple snapshot splits sequentially (typically after a failover), it reuses the sameSnapshotSplitReaderinstance. However, thechangeEventSourceContext(which controls the running state of the backfill binlog reading) is not properly reset to the "running" state when submitting the next split. This causes the backfill task for the subsequent split to exit immediately upon checkingcontext.isRunning(), leading to the reader hanging indefinitely.Brief change log
StoppableChangeEventSourceContextto include astartChangeEventSource()method to reset theisRunningflag totrue.SnapshotSplitReader#submitSplitto explicitly invoke the context reset method before starting the task, ensuring the reader is in a valid state for the new split.testMultipleSplitsWithBackfillinSnapshotSplitReaderTestto verify the fix by simulating multiple splits with a forced backfill phase.Verifying this change
This change is verified by the newly added unit test:
SnapshotSplitReaderTest#testMultipleSplitsWithBackfillThis test simulates a scenario where multiple splits are processed sequentially with a backfill phase, ensuring that the second split can be processed correctly without hanging.
Does this pull request potentially affect one of the following parts:
Documentation