Thv/benchmark wallet state update

[Sword-Smith]

Huge performance boost (2x) from the assert to debug_assert change. But I'm mostly interested in feedback on the benchmark, and how to handle state there.

[dan-da]

seems like a good speedup. :-)

I don't think the benchmarks.rs belongs in src/api as the code seems quite specific to that benchmark.

Also I think the benchmark can leverage the new api stuff to be more concise.

It's easier just to demonstrate though, so I've modified the wallet_state bench, and also made a few more things pub in neptune_core.

I will push a branch shortly with the suggested changes.

[dan-da]

here is a branch with suggested changes. At minimum I don't think benchmark.rs belongs in src/api.

The rest of the changes demonstrate an alternate way to write the benchmark, based on 45e888d, and a few tweaks to neptune-core code to facilitate.

danda/benchmark-wallet-state-update (in this repo)

[dan-da]

Since I see more commits that do not yet address my previous comments, I will re-iterate the request to remove benchmarks.rs from src/api with some additional context/rationale, and make this a more official "request changes". rationale:

1. I'm not convinced that benchmark related code belongs in src/api at all, even general purpose types. A case could be made for that, but so far has not been advanced.
2. the contents seem tailored to the needs of wallet_state.rs benchmark, not general purpose.
3. the content and location of benchmarks.rs is inconsistent with other types in src/api, ie not following convention. If we were to support a benchmarks api, it should be in its own sub-directory and probably should define a Benchmark type, and crate::api::Api should have a method for instantiating it.
4. it violates the rules of src/api module, as specified in src/api/README-dev.md. In particular rules 3, 4, 6, and 8 which can be summarized as "no panics", "return real error types", "doc-comment all pub items", and "make integration test".

I can suggest a few alternative (and easier) approaches:

1. move contents of benchmarks.rs into the wallet_state.rs benchmark. Then make the methods it calls from neptune_cash pub as necessary. Probably the benchmark is just highlighting things that should be pub anyway, eg for 3rd party wallet usage.
2. move the contents of benchmarks.rs elsewhere under src.
3. use the approach I demonstrated in branch: danda/benchmark-wallet-state-update

[Sword-Smith]

Since I see more commits that do not yet address my previous comments ...

Just rebasing, no new commits. The code from api/benchmarks is being moved.

[Sword-Smith]

There's a lot of code being made public with this PR. I'm not a fan of that since it means that we will not be warned about unused code.

[dan-da]

There's a lot of code being made public with this PR. I'm not a fan of that since it means that we will not be warned about unused code.

yeah... there are quite a few things that are presently pub(crate) in the codebase that will likely need to be made pub for the wallet competition. And this benchmark is probably finding some of those. But then it may be exposing others that really needn't be pub, except for the benchmark. So that's a tricky line to walk.

A few thoughts:

1. @jan-ferdinand had suggested a way to mark a pub fn as not visible in docs somehow, so it is not very discoverable and kind of a hint not to use it. edit: it is #[doc(hidden)]

2. consider the other approach I suggested: "move the contents of benchmarks.rs elsewhere under src". Then we need not make anything else pub. My concern was mainly with putting it in src/api, a carefully curated module. But perhaps it could live under src/util_types/benchmarks or something. It could also be marked with the attribute referenced in (1).

3. The branch I created was an attempt to make the benchmark without need to expose many new APIs. It definitely has some warts, but shows it is mostly possible.

Personally I would lean towards (2) at this point, but I don't feel strongly about it.

@jan-ferdinand and @aszepieniec might have opinions to offer.

[dan-da]

style: consider a single block_on() with a single async block.

(except for the bench closure which needs its own)

[jan-ferdinand]

• The annotation to hide an item from the public API is #[doc(hidden]. It is more than a hint: items marked this way are not part of the public API. Tools like cargo-semver-check will correctly pick this up.
• It might be possible to use a crate like visibility to modify an item's visibility based on the compilation profile. I haven't tested it, but the annotation #[cfg_attr(bench, visibility::make(pub))] on a private item might achieve the desired effect.

[aszepieniec]

It might be possible to use a crate like visibility to modify an item's visibility based on the compilation profile. I haven't tested it, but the annotation #[cfg_attr(bench, visibility::make(pub))] on a private item might achieve the desired effect.

OOooh, I like it!

[jan-ferdinand]

I haven't tested it, but the annotation #[cfg_attr(bench, visibility::make(pub))] on a private item might achieve the desired effect.

Unfortunately, this doesn't work as easily. The bench attribute does not exist in this form. What you can do instead (and this time, I have tried it):

• declare a new feature in the Cargo.toml, for example benchmarking = []
• annotate the item in question with #[cfg_attr(feature = "benchmarking", visibility::make(pub))]
• use the annotated item in a benchmark
• run cargo bench --features benchmarking

The downside is that compilation will fail with a simple cargo bench, since the items in question will have the wrong visibility.