Skip to content

perf(x/bank): Improve performance of GetAllBalances and GetAccountsBalances. #24660

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

SpicyLemon
Copy link
Contributor

@SpicyLemon SpicyLemon commented May 2, 2025

Description

Closes: #24685

This PR improves the performance of GetAllBalances and GetAccountsBalances. They were using Coins.Add in a loop, which becomes prohibitively expensive as the number of denoms grows. Since the info is coming directly out of state, though, we know there are no duplicated denoms, and they'll be in the correct order already. So we can use append instead of Coins.Add and bypass a whole lot of copying, comparisons and processing.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues. Your PR will not be merged unless you satisfy
all of these items.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change N/A
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification N/A
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Summary by CodeRabbit

  • Documentation

    • Updated the changelog to include a new "Improvements" section under "Unreleased" for better tracking of performance enhancements.
  • Improvements

    • Enhanced the performance of balance retrieval in the banking module, resulting in faster access to account balances. No changes to user-facing functionality.

Copy link

ironbird-prod bot commented May 2, 2025

Ironbird - launch a network To use Ironbird, you can use the following commands:
  • /ironbird start OR /ironbird start --load-test-config= - Launch a testnet with the specified chain and load test configuration.
  • /ironbird chains - List of chain images that ironbird can use to spin-up testnet
  • /ironbird loadtests - List of load test modes that ironbird can run against testnet
Custom Load Test Configuration You can provide a custom load test configuration using the `--load-test-config=` flag:
/ironbird start cosmos --load-test-config={
  "block_gas_limit_target": 0.75,
  "num_of_blocks": 50,
  "msgs": [
    {"weight": 0.3, "type": "MsgSend"},
    {"weight": 0.3, "type": "MsgMultiSend"},
	{"weight": 0.4, "type": "MsgArr", "ContainedType": "MsgSend", "NumMsgs": 3300}
  ]
}

Use /ironbird loadtests to see more examples.

Copy link
Contributor

coderabbitai bot commented May 2, 2025

📝 Walkthrough

Walkthrough

The changes introduce performance improvements to the x/bank module by optimizing how balances are accumulated and grouped in the GetAllBalances and GetAccountsBalances methods. The updated logic removes the use of coin sorting and map-based indexing, instead relying on direct appends and iteration order to group balances. Documentation in the changelog has also been updated to reflect these enhancements.

Changes

File(s) Change Summary
CHANGELOG.md Added an "Improvements" subsection under "Unreleased" noting performance enhancements in x/bank methods.
x/bank/keeper/view.go Refactored GetAllBalances and GetAccountsBalances to avoid sorting and map lookups, simplifying logic.

Sequence Diagram(s)

sequenceDiagram
    participant Caller
    participant Keeper
    participant Store

    Caller->>Keeper: GetAllBalances(ctx)
    Keeper->>Store: Iterate balances
    loop For each balance
        Store-->>Keeper: (address, coin)
        Keeper->>Keeper: Append coin to balances
    end
    Keeper-->>Caller: Return balances
Loading
sequenceDiagram
    participant Caller
    participant Keeper
    participant Store

    Caller->>Keeper: GetAccountsBalances(ctx)
    Keeper->>Store: Iterate balances by address
    loop For each (address, coin)
        Store-->>Keeper: (address, coin)
        alt Last entry is same address
            Keeper->>Keeper: Append coin to last entry's coins
        else
            Keeper->>Keeper: Create new entry for address
        end
    end
    Keeper-->>Caller: Return grouped balances
Loading

📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d1bb429 and c43f8fd.

📒 Files selected for processing (2)
  • CHANGELOG.md (1 hunks)
  • x/bank/keeper/view.go (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (14)
  • GitHub Check: test-system
  • GitHub Check: test-system-legacy
  • GitHub Check: tests (03)
  • GitHub Check: tests (02)
  • GitHub Check: tests (01)
  • GitHub Check: tests (00)
  • GitHub Check: test-sim-nondeterminism
  • GitHub Check: test-e2e
  • GitHub Check: test-integration
  • GitHub Check: build (arm64)
  • GitHub Check: build (amd64)
  • GitHub Check: Analyze
  • GitHub Check: Gosec
  • GitHub Check: Summary
🔇 Additional comments (3)
CHANGELOG.md (1)

41-43: LGTM! Good documentation of the performance improvement.

The changelog entry appropriately documents the performance improvement made to the GetAllBalances and GetAccountsBalances keeper methods in the bank module.

x/bank/keeper/view.go (2)

107-113: Excellent performance optimization for GetAllBalances.

This change improves performance by directly appending coin balances instead of using balances.Add(balance), which would have required additional validation, sorting, and memory allocations on each iteration. Since balances are retrieved directly from the state, there are no duplicate denoms to worry about.

This is a safe optimization because:

  1. The order of iteration is preserved
  2. No duplicate denom checking is needed since balances come directly from state
  3. The final sort operation is unnecessary as coins are already properly ordered in state

120-135: Excellent performance optimization for GetAccountsBalances.

This change significantly improves performance by:

  1. Removing the need for a map to track addresses
  2. Leveraging the fact that balances are iterated in address order
  3. Directly appending coins to the last entry when addresses match
  4. Avoiding unnecessary sorting operations

The implementation now relies on the natural iteration order of IterateAllBalances, which should be consistent by address, making this a safe and effective optimization.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@aljo242
Copy link
Contributor

aljo242 commented May 2, 2025

Awesome! could we add a test that GetAllBalances is deterministic? i.e. that running this multiple times will return the same coins in the same order?

@SpicyLemon
Copy link
Contributor Author

Awesome! could we add a test that GetAllBalances is deterministic? i.e. that running this multiple times will return the same coins in the same order?

I don't think there's a meaningful way to test that. Specifically, I don't know of a way to get it into a state where it might possibly be non-deterministic. That being said, I did add some unit tests on those methods because there weren't any specifically testing their functionality, and have them repeat a bunch to hopefully trigger any nondeterminism problems. But since I don't know of anywhere where they might be nondeterministic, they could still have false positives.

@SpicyLemon
Copy link
Contributor Author

Just a heads up. When I just merged main into this branch, I saw that the changelog now has two Bug Fixes sections in the unreleased section, one above Deprecated, and one below.

@aljo242
Copy link
Contributor

aljo242 commented May 5, 2025

@SpicyLemon thanks for the callout - i'll make sure to clean the changelog up

@aljo242 aljo242 requested a review from technicallyty May 5, 2025 13:47
@aljo242 aljo242 enabled auto-merge May 5, 2025 13:47
@aljo242
Copy link
Contributor

aljo242 commented May 5, 2025

LGTM - good to merge with @technicallyty approval

}

count := 1000
for i := 1; i <= count; i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for i := 1; i <= count; i++ {
for i := range count {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That starts the counting at 0, and I want it stated at one. That way, the failure messages are more natural for us to read without having to add one to it everywhere.

Same applies to the other similar comment.

testFunc := func() {
act = suite.bankKeeper.GetAllBalances(suite.ctx, tc.addr)
}
for i := 1; i <= count; i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
for i := 1; i <= count; i++ {
for i := range count {

@@ -2474,6 +2476,187 @@ func (suite *KeeperTestSuite) TestGetAllSendEnabledEntries() {
})
}

func (suite *KeeperTestSuite) TestGetAllBalances() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in these 2 new tests, can we add a check that the returned balances are sorted? (i.e. copy the return value, sort, compare against original return)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the expected values are created using NewCoins which sorts them. So if an actual result isn't sorted, it won't match the expected and the test will fail. But I added some checks anyway.

auto-merge was automatically disabled May 9, 2025 22:20

Head branch was pushed to by a user without write access

Copy link
Contributor

@technicallyty technicallyty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - pending @aljo242 approval

@aljo242
Copy link
Contributor

aljo242 commented May 13, 2025

@SpicyLemon good to merge if we resolve conflicts

@technicallyty technicallyty enabled auto-merge May 16, 2025 17:51
@technicallyty
Copy link
Contributor

thank you for the contribution @SpicyLemon 🙏🏻

@technicallyty technicallyty added this pull request to the merge queue May 16, 2025
Merged via the queue into cosmos:main with commit 2c43a57 May 16, 2025
40 checks passed
warpbuild-benchmark-bot bot added a commit to WarpBuilds/cosmos-sdk that referenced this pull request May 16, 2025
meetrick pushed a commit to meetrick/cosmos-sdk that referenced this pull request May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve performance of GetAllBalances and GetAccountsBalances
3 participants