-
Notifications
You must be signed in to change notification settings - Fork 19
Thv/benchmark wallet state update #587
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
seems like a good speedup. :-) I don't think the benchmarks.rs belongs in src/api as the code seems quite specific to that benchmark. Also I think the benchmark can leverage the new api stuff to be more concise. It's easier just to demonstrate though, so I've modified the wallet_state bench, and also made a few more things pub in neptune_core. I will push a branch shortly with the suggested changes. |
45e888d
to
e87e212
Compare
here is a branch with suggested changes. At minimum I don't think benchmark.rs belongs in src/api. The rest of the changes demonstrate an alternate way to write the benchmark, based on 45e888d, and a few tweaks to neptune-core code to facilitate. danda/benchmark-wallet-state-update (in this repo) |
e87e212
to
abfb2d4
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since I see more commits that do not yet address my previous comments, I will re-iterate the request to remove benchmarks.rs from src/api with some additional context/rationale, and make this a more official "request changes". rationale:
- I'm not convinced that benchmark related code belongs in src/api at all, even general purpose types. A case could be made for that, but so far has not been advanced.
- the contents seem tailored to the needs of wallet_state.rs benchmark, not general purpose.
- the content and location of benchmarks.rs is inconsistent with other types in src/api, ie not following convention. If we were to support a benchmarks api, it should be in its own sub-directory and probably should define a Benchmark type, and crate::api::Api should have a method for instantiating it.
- it violates the rules of src/api module, as specified in src/api/README-dev.md. In particular rules 3, 4, 6, and 8 which can be summarized as "no panics", "return real error types", "doc-comment all pub items", and "make integration test".
I can suggest a few alternative (and easier) approaches:
- move contents of benchmarks.rs into the wallet_state.rs benchmark. Then make the methods it calls from neptune_cash pub as necessary. Probably the benchmark is just highlighting things that should be pub anyway, eg for 3rd party wallet usage.
- move the contents of benchmarks.rs elsewhere under src.
- use the approach I demonstrated in branch: danda/benchmark-wallet-state-update
abfb2d4
to
f639d7c
Compare
Just rebasing, no new commits. The code from |
f639d7c
to
5e778f6
Compare
There's a lot of code being made public with this PR. I'm not a fan of that since it means that we will not be warned about unused code. |
yeah... there are quite a few things that are presently pub(crate) in the codebase that will likely need to be made pub for the wallet competition. And this benchmark is probably finding some of those. But then it may be exposing others that really needn't be pub, except for the benchmark. So that's a tricky line to walk. A few thoughts:
Personally I would lean towards (2) at this point, but I don't feel strongly about it. @jan-ferdinand and @aszepieniec might have opinions to offer. |
|
||
fn update_wallet_with_block2(bencher: Bencher) { | ||
let rt = tokio::runtime::Runtime::new().unwrap(); | ||
let mut wallet_state = rt.block_on(devops_wallet_state_genesis(Network::Main)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: consider a single block_on() with a single async block.
(except for the bench closure which needs its own)
|
OOooh, I like it! |
Unfortunately, this doesn't work as easily. The
The downside is that compilation will fail with a simple |
5e778f6
to
96151d4
Compare
I managed to better encapsulate the stuff needed for this benchmark. So now only two functions go from Also: This 2x speedup in wallet state update is something we need to roll out ASAP. State updates to the wallet are extremely slow, and by an order of magnitude the bottleneck for state updates, as evidenced by the below log showing a node with ~400 UTXOs applying a block state update.
|
If urgent, then I would suggest backporting this specific feature onto a branch at v0.2.2, and releasing an update of that. I think we are still some weeks away from being able to properly release something from master:
|
When syncing on a node with many UTXOs, the wallet state update is **by far** the slowest part of the state update. So having a benchmark for this critical step is critical. Adds a module `bench_helpers` to encapsulate code needed for benchmarks. Since benchmarks are not part of the main crate, they can only use items that are `pub`. So this `bench_helpers` module prevents us from marking half the repo as `pub`, as all functions needed for benchmarks can be neatly encapsulated here.
This small change gives a 2x speedup on the wallet-updating process, as evidenced by the wallet-state update benchmark which. The wallet state update is *by far* the bottleneck when doing state updates with new blocks. **Before this commit** Timer precision: 20 ns wallet_state fastest │ slowest │ median │ mean │ samples │ iters ╰─ maintain_membership_proofs │ │ │ │ │ ╰─ maintain_100_100 │ │ │ │ │ ╰─ apply_block2 172.2 ms │ 184.5 ms │ 175.6 ms │ 176.2 ms │ 10 │ 10 **After this commit** Timer precision: 20 ns wallet_state fastest │ slowest │ median │ mean │ samples │ iters ╰─ maintain_membership_proofs │ │ │ │ │ ╰─ maintain_100_100 │ │ │ │ │ ╰─ apply_block2 91.05 ms │ 102.8 ms │ 91.97 ms │ 92.91 ms │ 10 │ 10
8bfe31d
to
e8772e7
Compare
Huge performance boost (2x) from the
assert
todebug_assert
change. But I'm mostly interested in feedback on the benchmark, and how to handle state there.