Skip to content

MINOR: Simplify PartitionStates update logic by removing unnecessary grouping #19442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from

Conversation

zt52875287
Copy link

This patch simplifies the implementation of PartitionStates#update by removing the unnecessary grouping of TopicPartitions by topic. The original logic constructed an intermediate map by topic name, but ultimately only inserted each (TopicPartition, state) pair into the internal LinkedHashMap. This grouping had no effect on the final result.

The updated implementation directly iterates over the input map, improving clarity and reducing overhead.

Additionally, the unit test has been updated to remove reliance on a specific iteration order of partition states. Since both PartitionStates#set and its internal update method accept a Map as input, and standard Java Maps do not guarantee any ordering beyond insertion order (which may not be preserved across implementations), tests should not assume a fixed output order unless explicitly defined by the API.

@github-actions github-actions bot added triage PRs from the community clients small Small PRs labels Apr 11, 2025
map.put(tp, state);
}
}
map.putAll(partitionToState);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PartitionStates#update(Map<TopicPartition, S>) is only used in PartitionStates#set(Map<TopicPartition, S>). Do you think that it's more clear to move the change to set function directly? For example:

    public void set(Map<TopicPartition, S> partitionToState) {
        map.clear();
        map.putAll(partitionToState);
        updateSize();
    }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion. I've updated the code to move the logic directly into the set method as recommended.

@FrankYang0529 FrankYang0529 requested a review from chia7712 April 12, 2025 03:21
@github-actions github-actions bot removed the triage PRs from the community label Apr 13, 2025
Copy link
Collaborator

@m1a2st m1a2st left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this patch, I have a question, Will this change the order in the map? Will it indirectly affect anything?

@zt52875287
Copy link
Author

Thanks for this patch, I have a question, Will this change the order in the map? Will it indirectly affect anything?

I’ve reviewed all call sites of PartitionStates#set() and confirmed that the input Map instances are all HashMap or Java-converted Scala Maps — both of which are unordered by nature.

Specifically:

  • SubscriptionState#assignFromUser and assignFromSubscribed both use HashMap
  • AbstractFetcherThread#updateFetchOffsetAndMaybeMarkTruncationComplete constructs a Scala Map and calls .asJava, which returns a HashMap-backed implementation.

In the original PartitionStates#update method, although entries were grouped by topic before inserting into the internal LinkedHashMap, the final insertion order was still influenced by the iteration order of the original map. In that sense, the order of the internal map was indirectly related to the input map's order.

The new implementation with putAll() similarly inserts entries based on the input map’s iteration order. It remains dependent on the input map's order — which is non-deterministic in all current usages.

Given that the input maps are all unordered, this change does not affect observable behavior.

@m1a2st
Copy link
Collaborator

m1a2st commented Apr 17, 2025

Thanks @zt52875287 for explaination.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants