fix(benchmark): Filter benchmarks by present on main when testing by teofr · Pull Request #1768 · NomicFoundation/slang

teofr · 2026-05-13T10:39:05Z

#1750 introduced tests to make sure no new benchmarks are added, or expected ones dropped. However, it used all benchmarks in bencher cloud, rather than those we care about (those reported from main).

This PR fixes that by filtering benchmarks for those only present on main

changeset-bot · 2026-05-13T10:39:10Z

⚠️ No Changeset found

Latest commit: 2849cc9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

github-actions · 2026-05-13T10:49:42Z

Bench list diff for `slang_v2` (project `slang-dashboard-cargo-slang-v2`)

New benchmarks (32) — present in code, not yet recorded on bencher. Will start a fresh metric history once a ci:perf run uploads results.

slang_v2::pipeline::compute_contracts_abi::cooldogs
slang_v2::pipeline::compute_contracts_abi::create_x
slang_v2::pipeline::compute_contracts_abi::merkle_proof
slang_v2::pipeline::compute_contracts_abi::multicall3
slang_v2::pipeline::compute_contracts_abi::one_step_leverage_f
slang_v2::pipeline::compute_contracts_abi::pointer_libraries
slang_v2::pipeline::compute_contracts_abi::ui_pool_data_provider_v3
slang_v2::pipeline::compute_contracts_abi::uniswap
slang_v2::pipeline::ir_builder::cooldogs
slang_v2::pipeline::ir_builder::create_x
slang_v2::pipeline::ir_builder::merkle_proof
slang_v2::pipeline::ir_builder::multicall3
slang_v2::pipeline::ir_builder::one_step_leverage_f
slang_v2::pipeline::ir_builder::pointer_libraries
slang_v2::pipeline::ir_builder::ui_pool_data_provider_v3
slang_v2::pipeline::ir_builder::uniswap
slang_v2::pipeline::parser::cooldogs
slang_v2::pipeline::parser::create_x
slang_v2::pipeline::parser::merkle_proof
slang_v2::pipeline::parser::multicall3
slang_v2::pipeline::parser::one_step_leverage_f
slang_v2::pipeline::parser::pointer_libraries
slang_v2::pipeline::parser::ui_pool_data_provider_v3
slang_v2::pipeline::parser::uniswap
slang_v2::pipeline::semantic::cooldogs
slang_v2::pipeline::semantic::create_x
slang_v2::pipeline::semantic::merkle_proof
slang_v2::pipeline::semantic::multicall3
slang_v2::pipeline::semantic::one_step_leverage_f
slang_v2::pipeline::semantic::pointer_libraries
slang_v2::pipeline::semantic::ui_pool_data_provider_v3
slang_v2::pipeline::semantic::uniswap

Orphan benchmarks (32) — recorded on bencher but no longer in code. Either renamed in this PR or candidates for archival.

slang_v2::cooldogs_full_v2::cooldogs_compute_contracts_abi::test
slang_v2::cooldogs_full_v2::cooldogs_ir_builder::test
slang_v2::cooldogs_full_v2::cooldogs_parser::test
slang_v2::cooldogs_full_v2::cooldogs_semantic::test
slang_v2::create_x_full_v2::create_x_compute_contracts_abi::test
slang_v2::create_x_full_v2::create_x_ir_builder::test
slang_v2::create_x_full_v2::create_x_parser::test
slang_v2::create_x_full_v2::create_x_semantic::test
slang_v2::merkle_proof_full_v2::merkle_proof_compute_contracts_abi::test
slang_v2::merkle_proof_full_v2::merkle_proof_ir_builder::test
slang_v2::merkle_proof_full_v2::merkle_proof_parser::test
slang_v2::merkle_proof_full_v2::merkle_proof_semantic::test
slang_v2::multicall3_full_v2::multicall3_compute_contracts_abi::test
slang_v2::multicall3_full_v2::multicall3_ir_builder::test
slang_v2::multicall3_full_v2::multicall3_parser::test
slang_v2::multicall3_full_v2::multicall3_semantic::test
slang_v2::one_step_leverage_f_full_v2::one_step_leverage_f_compute_contracts_abi::test
slang_v2::one_step_leverage_f_full_v2::one_step_leverage_f_ir_builder::test
slang_v2::one_step_leverage_f_full_v2::one_step_leverage_f_parser::test
slang_v2::one_step_leverage_f_full_v2::one_step_leverage_f_semantic::test
slang_v2::pointer_libraries_full_v2::pointer_libraries_compute_contracts_abi::test
slang_v2::pointer_libraries_full_v2::pointer_libraries_ir_builder::test
slang_v2::pointer_libraries_full_v2::pointer_libraries_parser::test
slang_v2::pointer_libraries_full_v2::pointer_libraries_semantic::test
slang_v2::ui_pool_data_provider_v3_full_v2::ui_pool_data_provider_v3_compute_contracts_abi::test
slang_v2::ui_pool_data_provider_v3_full_v2::ui_pool_data_provider_v3_ir_builder::test
slang_v2::ui_pool_data_provider_v3_full_v2::ui_pool_data_provider_v3_parser::test
slang_v2::ui_pool_data_provider_v3_full_v2::ui_pool_data_provider_v3_semantic::test
slang_v2::uniswap_full_v2::uniswap_compute_contracts_abi::test
slang_v2::uniswap_full_v2::uniswap_ir_builder::test
slang_v2::uniswap_full_v2::uniswap_parser::test
slang_v2::uniswap_full_v2::uniswap_semantic::test

github-actions · 2026-05-13T10:49:44Z

Bench list diff for `slang` (project `slang-dashboard-cargo-slang`)

New benchmarks (16) — present in code, not yet recorded on bencher. Will start a fresh metric history once a ci:perf run uploads results.

slang::bindings::build::merkle_proof
slang::bindings::build::weighted_pool
slang::bindings::cleanup::merkle_proof
slang::bindings::cleanup::weighted_pool
slang::bindings::resolve::merkle_proof
slang::bindings::resolve::weighted_pool
slang::cst::cursor::merkle_proof
slang::cst::cursor::weighted_pool
slang::cst::parser::merkle_proof
slang::cst::parser::weighted_pool
slang::cst::query::merkle_proof
slang::cst::query::weighted_pool
slang::v2_binder::cleanup::merkle_proof
slang::v2_binder::cleanup::weighted_pool
slang::v2_binder::run::merkle_proof
slang::v2_binder::run::weighted_pool

Orphan benchmarks (16) — recorded on bencher but no longer in code. Either renamed in this PR or candidates for archival.

slang::merkle_proof_full::merkle_proof_binder_v2_cleanup::test
slang::merkle_proof_full::merkle_proof_binder_v2_run::test
slang::merkle_proof_full::merkle_proof_bindings_build::test
slang::merkle_proof_full::merkle_proof_bindings_resolve::test
slang::merkle_proof_full::merkle_proof_cleanup::test
slang::merkle_proof_full::merkle_proof_cursor::test
slang::merkle_proof_full::merkle_proof_parser::test
slang::merkle_proof_full::merkle_proof_query::test
slang::weighted_pool_full::weighted_pool_binder_v2_cleanup::test
slang::weighted_pool_full::weighted_pool_binder_v2_run::test
slang::weighted_pool_full::weighted_pool_bindings_build::test
slang::weighted_pool_full::weighted_pool_bindings_resolve::test
slang::weighted_pool_full::weighted_pool_cleanup::test
slang::weighted_pool_full::weighted_pool_cursor::test
slang::weighted_pool_full::weighted_pool_parser::test
slang::weighted_pool_full::weighted_pool_query::test

github-actions · 2026-05-13T10:50:25Z

Bench list diff for `comparison` (project `slang-dashboard-cargo-cmp`)

New benchmarks (34) — present in code, not yet recorded on bencher. Will start a fresh metric history once a ci:perf run uploads results.

comparison::parsers::slang::cooldogs
comparison::parsers::slang::create_x
comparison::parsers::slang::merkle_proof
comparison::parsers::slang::mooniswap
comparison::parsers::slang::multicall3
comparison::parsers::slang::one_step_leverage_f
comparison::parsers::slang::pointer_libraries
comparison::parsers::slang::ui_pool_data_provider_v3
comparison::parsers::slang::uniswap
comparison::parsers::slang::weighted_pool
comparison::parsers::slang_v2::cooldogs
comparison::parsers::slang_v2::create_x
comparison::parsers::slang_v2::merkle_proof
comparison::parsers::slang_v2::multicall3
comparison::parsers::slang_v2::one_step_leverage_f
comparison::parsers::slang_v2::pointer_libraries
comparison::parsers::slang_v2::ui_pool_data_provider_v3
comparison::parsers::slang_v2::uniswap
comparison::parsers::solar::cooldogs
comparison::parsers::solar::create_x
comparison::parsers::solar::merkle_proof
comparison::parsers::solar::multicall3
comparison::parsers::solar::one_step_leverage_f
comparison::parsers::solar::pointer_libraries
comparison::parsers::solar::ui_pool_data_provider_v3
comparison::parsers::solar::uniswap
comparison::parsers::solar::weighted_pool
comparison::parsers::tree_sitter::cooldogs
comparison::parsers::tree_sitter::merkle_proof
comparison::parsers::tree_sitter::mooniswap
comparison::parsers::tree_sitter::multicall3
comparison::parsers::tree_sitter::one_step_leverage_f
comparison::parsers::tree_sitter::ui_pool_data_provider_v3
comparison::parsers::tree_sitter::weighted_pool

Orphan benchmarks (34) — recorded on bencher but no longer in code. Either renamed in this PR or candidates for archival.

comparison::cooldogs_group::slang_cooldogs::test
comparison::cooldogs_group::slang_v2_cooldogs::test
comparison::cooldogs_group::solar_cooldogs::test
comparison::cooldogs_group::tree_sitter_cooldogs::test
comparison::create_x_group::slang_create_x::test
comparison::create_x_group::slang_v2_create_x::test
comparison::create_x_group::solar_create_x::test
comparison::merkle_proof_group::slang_merkle_proof::test
comparison::merkle_proof_group::slang_v2_merkle_proof::test
comparison::merkle_proof_group::solar_merkle_proof::test
comparison::merkle_proof_group::tree_sitter_merkle_proof::test
comparison::mooniswap_group::slang_mooniswap::test
comparison::mooniswap_group::tree_sitter_mooniswap::test
comparison::multicall3_group::slang_multicall3::test
comparison::multicall3_group::slang_v2_multicall3::test
comparison::multicall3_group::solar_multicall3::test
comparison::multicall3_group::tree_sitter_multicall3::test
comparison::one_step_leverage_f_group::slang_one_step_leverage_f::test
comparison::one_step_leverage_f_group::slang_v2_one_step_leverage_f::test
comparison::one_step_leverage_f_group::solar_one_step_leverage_f::test
comparison::one_step_leverage_f_group::tree_sitter_one_step_leverage_f::test
comparison::pointer_libraries_group::slang_pointer_libraries::test
comparison::pointer_libraries_group::slang_v2_pointer_libraries::test
comparison::pointer_libraries_group::solar_pointer_libraries::test
comparison::ui_pool_data_provider_v3_group::slang_ui_pool_data_provider_v3::test
comparison::ui_pool_data_provider_v3_group::slang_v2_ui_pool_data_provider_v3::test
comparison::ui_pool_data_provider_v3_group::solar_ui_pool_data_provider_v3::test
comparison::ui_pool_data_provider_v3_group::tree_sitter_ui_pool_data_provider_v3::test
comparison::uniswap_group::slang_uniswap::test
comparison::uniswap_group::slang_v2_uniswap::test
comparison::uniswap_group::solar_uniswap::test
comparison::weighted_pool_group::slang_weighted_pool::test
comparison::weighted_pool_group::solar_weighted_pool::test
comparison::weighted_pool_group::tree_sitter_weighted_pool::test

ggiraldez

Looks good, but would it make sense to get the benchmarks directly from the latest main report?

teofr · 2026-05-13T15:02:23Z

@ggiraldez you're absolutely right, will change

teofr · 2026-05-13T16:09:25Z

@ggiraldez could you give it a quick look? Since it changed following your recommendation.

ggiraldez

Looks good, thank you!

ggiraldez · 2026-05-13T17:50:07Z

+            "--direction",
+            "desc",


I'd add an explicit --sort date_time just in case the default changes in the future.

teofr · 2026-05-19T09:46:17Z

Closing in favor of #1788

teofr · 2026-05-19T11:27:03Z

Actually, reopening, We need a newer version of iai-callgrind to do these tests.

teofr changed the title ~~Filter benchmarks by present on main~~ fix(benchmark): Filter benchmarks by present on main when testing May 13, 2026

teofr marked this pull request as ready for review May 13, 2026 10:55

teofr requested review from a team as code owners May 13, 2026 10:55

ggiraldez approved these changes May 13, 2026

View reviewed changes

teofr force-pushed the teofr/fix-benchmark-name-test branch from 2555a13 to 884568f Compare May 13, 2026 16:03

teofr requested a review from ggiraldez May 13, 2026 16:09

ggiraldez approved these changes May 13, 2026

View reviewed changes

teofr force-pushed the teofr/update-iai-callgrind branch from 8d932d2 to 1faae5f Compare May 14, 2026 11:38

teofr force-pushed the teofr/fix-benchmark-name-test branch 2 times, most recently from 751e8ff to 16d533f Compare May 14, 2026 14:22

teofr force-pushed the teofr/update-iai-callgrind branch 2 times, most recently from cb54152 to 9718ba4 Compare May 14, 2026 19:13

teofr marked this pull request as draft May 15, 2026 09:18

teofr mentioned this pull request May 19, 2026

feat(benchmark): Test names against existing #1788

Closed

teofr closed this May 19, 2026

teofr reopened this May 19, 2026

teofr force-pushed the teofr/fix-benchmark-name-test branch from 16d533f to ce1c39f Compare May 20, 2026 08:14

teofr force-pushed the teofr/update-iai-callgrind branch from 9718ba4 to e0a8a2d Compare May 20, 2026 08:25

teofr force-pushed the teofr/fix-benchmark-name-test branch from ce1c39f to 11ec412 Compare May 20, 2026 08:25

Filter benchmarks by present on main

2849cc9

teofr force-pushed the teofr/update-iai-callgrind branch from e0a8a2d to ac5ea60 Compare May 20, 2026 08:36

teofr force-pushed the teofr/fix-benchmark-name-test branch from 11ec412 to 2849cc9 Compare May 20, 2026 08:36

teofr force-pushed the teofr/update-iai-callgrind branch from 21e96bf to 32bdf49 Compare May 21, 2026 08:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(benchmark): Filter benchmarks by present on main when testing#1768

fix(benchmark): Filter benchmarks by present on main when testing#1768
teofr wants to merge 1 commit into
teofr/update-iai-callgrindfrom
teofr/fix-benchmark-name-test

teofr commented May 13, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 13, 2026 •

edited

Loading

Uh oh!

ggiraldez left a comment •

edited

Loading

Uh oh!

teofr commented May 13, 2026

Uh oh!

teofr commented May 13, 2026

Uh oh!

ggiraldez left a comment

Uh oh!

ggiraldez May 13, 2026

Uh oh!

teofr commented May 19, 2026

Uh oh!

teofr commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

teofr commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

github-actions Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bench list diff for slang_v2 (project slang-dashboard-cargo-slang-v2)

Uh oh!

github-actions Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bench list diff for slang (project slang-dashboard-cargo-slang)

Uh oh!

github-actions Bot commented May 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bench list diff for comparison (project slang-dashboard-cargo-cmp)

Uh oh!

ggiraldez left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

teofr commented May 13, 2026

Uh oh!

teofr commented May 13, 2026

Uh oh!

ggiraldez left a comment

Choose a reason for hiding this comment

Uh oh!

ggiraldez May 13, 2026

Choose a reason for hiding this comment

Uh oh!

teofr commented May 19, 2026

Uh oh!

teofr commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

teofr commented May 13, 2026 •

edited

Loading

changeset-bot Bot commented May 13, 2026 •

edited

Loading

github-actions Bot commented May 13, 2026 •

edited

Loading

Bench list diff for `slang_v2` (project `slang-dashboard-cargo-slang-v2`)

github-actions Bot commented May 13, 2026 •

edited

Loading

Bench list diff for `slang` (project `slang-dashboard-cargo-slang`)

github-actions Bot commented May 13, 2026 •

edited

Loading

Bench list diff for `comparison` (project `slang-dashboard-cargo-cmp`)

ggiraldez left a comment •

edited

Loading