Skip to content

chore: move state clones out of benchmark run functions#324

Open
markolazic01 wants to merge 6 commits intoChainSafe:mainfrom
markolazic01:bench/move-clones-out-of-run
Open

chore: move state clones out of benchmark run functions#324
markolazic01 wants to merge 6 commits intoChainSafe:mainfrom
markolazic01:bench/move-clones-out-of-run

Conversation

@markolazic01
Copy link
Copy Markdown

bench: move state clones out of benchmark run functions

Closes #164

Problem

Benchmark run functions were cloning and freeing CachedBeaconState on every iteration. This clone/deinit cost was included in the timed measurement, skewing results for processBlock, processEpoch, and their individual sub-step benchmarks. This is visible through the runImpl inside zbench. This change should also increase readability of the affected files.

Solution

Moved state cloning into zbench's before_each/after_each lifecycle hooks using module-level variables:

  • beforeEach clones the cached state before each timed iteration
  • afterEach frees the clone after the iteration completes
  • run functions now operate directly on the pre-cloned instance

Also removed the now-unused cached_state field from all benchmark structs and cleaned up the corresponding addParam calls.

Applied to process_block.zig and process_epoch.zig.

AI disclosure: Claude was consulted for reviewing the approach and drafting this description. All code changes were authored manually.

@markolazic01 markolazic01 requested a review from a team as a code owner April 23, 2026 20:54
@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request optimizes the benchmark suite by decoupling state management from the timed execution logic. By moving the cloning of CachedBeaconState into lifecycle hooks, the overhead of setup and teardown is no longer included in the benchmark measurements, leading to more accurate performance results for state transition functions.

Highlights

  • Benchmark Optimization: Moved the cloning and deinitialization of CachedBeaconState out of the benchmark run functions to prevent these operations from skewing performance measurements.
  • Lifecycle Management: Utilized zbench's before_each and after_each hooks to handle state cloning and cleanup, ensuring consistent timing for the actual benchmark logic.
  • Code Cleanup: Removed the redundant cached_state field from all benchmark structs and updated addParam calls to reflect the new lifecycle management approach.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@markolazic01 markolazic01 changed the title bench: move state clones out of benchmark run functions chore: move state clones out of benchmark run functions Apr 23, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the state transition benchmarks in process_block.zig and process_epoch.zig to use global state instances managed through beforeEach and afterEach hooks, centralizing state cloning and deinitialization. Feedback focuses on several style guide violations, specifically regarding the requirement for a minimum assertion density of two per function, the need to pass explicit options to library functions, adherence to the 100-column line length limit, and correct 4-space indentation.

Comment on lines +37 to +39
fn beforeEach() void {
g_cached_state_instance = g_cached_state.clone(g_allocator, .{}) catch unreachable;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function violates the repository's style guide regarding assertion density and explicit options.

  1. Assertion Density: The style guide (Line 54) requires an average of at least two assertions per function. Assertions are critical for detecting programmer errors and documenting invariants.
  2. Explicit Options: The style guide (Line 170) requires explicitly passing options to library functions instead of relying on defaults. The clone function should be called with an explicit options struct.
fn beforeEach() void {
    std.debug.assert(g_allocator.ptr != null);
    g_cached_state_instance = g_cached_state.clone(g_allocator, .{ .transfer_cache = true }) catch unreachable;
    std.debug.assert(g_cached_state_instance.state.ptr != null);
}
References
  1. The assertion density of the code must average a minimum of two assertions per function. (link)
  2. Explicitly pass options to library functions at the call site, instead of relying on the defaults. (link)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems not applicable for this particular case, same goes for the other comments mentioning hooks.

Comment on lines +41 to +44
fn afterEach() void {
g_cached_state_instance.deinit();
g_allocator.destroy(g_cached_state_instance);
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function lacks assertions. According to the style guide (Line 54), the assertion density must average at least two assertions per function. Consider asserting that the instance being freed is valid.

fn afterEach() void {
    std.debug.assert(g_cached_state_instance != undefined);
    g_cached_state_instance.deinit();
    g_allocator.destroy(g_cached_state_instance);
}
References
  1. The assertion density of the code must average a minimum of two assertions per function. (link)

allocator.destroy(cloned);
}

const external_data = BlockExternalData{ .execution_payload_status = .valid, .data_availability_status = .available };
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This line exceeds the 100-column limit defined in the style guide (Line 400). Hard limiting line lengths ensures better readability and avoids horizontal scrolling.

            const external_data = BlockExternalData{
                .execution_payload_status = .valid,
                .data_availability_status = .available,
            };
References
  1. Hard limit all line lengths, without exception, to at most 100 columns. (link)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also seems unrelated as the repo isn't enforcing any styling requirements atm, and this line is unaffected by the present changes.

Comment on lines +26 to +28
fn beforeEach() void {
g_cached_state_instance = g_cached_state.clone(g_allocator, .{}) catch unreachable;
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This function violates the style guide requirements for assertion density (Line 54) and explicit options (Line 170).

fn beforeEach() void {
    std.debug.assert(g_allocator.ptr != null);
    g_cached_state_instance = g_cached_state.clone(g_allocator, .{ .transfer_cache = true }) catch unreachable;
    std.debug.assert(g_cached_state_instance.state.ptr != null);
}
References
  1. The assertion density of the code must average a minimum of two assertions per function. (link)
  2. Explicitly pass options to library functions at the call site, instead of relying on the defaults. (link)

Comment thread bench/state_transition/process_epoch.zig Outdated
@markolazic01
Copy link
Copy Markdown
Author

Resolved conflicts in the latest commit.

wemeetagain
wemeetagain previously approved these changes Apr 28, 2026
@markolazic01
Copy link
Copy Markdown
Author

All the checks should pass now

Comment thread bench/state_transition/process_epoch.zig Outdated
Comment thread bench/state_transition/process_epoch.zig Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

move clones out of benchmarking fns

3 participants