KAFKA-19154; Offset Fetch API should return INVALID_OFFSET if requested topic id does not match persisted one #19744
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ed topic id does not match persisted one
dajac
commented
May 16, 2025
| @@ -906,7 +907,7 @@ public OffsetFetchResponseData.OffsetFetchResponseGroup fetchOffsets( | |||
| .setCommittedOffset(INVALID_OFFSET) | |||
| .setCommittedLeaderEpoch(-1) | |||
| .setMetadata("")); | |||
| } else if (offsetAndMetadata == null) { | |||
| } else if (offsetAndMetadata == null || isMismatchedTopicId(offsetAndMetadata.topicId, topic.topicId())) { | |||
dajac
commented
May 16, 2025
Comment on lines
+533
to
+540
| // There are two ways to ensure that committed of recreated topics are not returned. | ||
| // 1) When a topic is deleted, GroupCoordinatorService#onPartitionsDeleted is called to | ||
| // delete all its committed offsets. | ||
| // 2) Since version 10 of the OffsetCommit API, the topic id is stored alongside the | ||
| // committed offset. When it is queried, it is only returned iff the topic id of | ||
| // committed offset matches the requested one. | ||
| // The test tests both conditions but not in a deterministic way as they race | ||
| // against each others. |
There was a problem hiding this comment.
I am not happy with this but I don't have a good idea to make it better. I was thinking about introducing an internal config/parameter to disable 1) in order to test 2) but it feels weird to have this in production code. Thoughts?
I have tested it manually by commenting out 1).
|
@lianetm @squah-confluent @dongnuo123 Could you please review it? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch updates the OffsetFetch API to ensure that a committed offset
is returned iff the requested topic id matches the persisted one; the
invalid offset is returned otherwise.