Skip to content

[BENCHMARK TEST] Tune max iterations/hyper parameters to get more consistent runtimes (-2% on benchmark, 10% -> 3% timeout)#639

Merged
seveibar merged 4 commits intomainfrom
opt
Mar 10, 2026
Merged

[BENCHMARK TEST] Tune max iterations/hyper parameters to get more consistent runtimes (-2% on benchmark, 10% -> 3% timeout)#639
seveibar merged 4 commits intomainfrom
opt

Conversation

@seveibar
Copy link
Copy Markdown
Contributor

  • add support for single sample profiling
  • wip
  • Tune max iterations to avoid timing out

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 10, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
capacity-node-autorouter Ready Ready Preview, Comment Mar 10, 2026 0:35am

Request Review

@tscircuitbot
Copy link
Copy Markdown

🏃 Benchmark This PR

Run benchmarks by commenting on this PR:

/benchmark [solver-name|all] [scenario-limit] --concurrency <n> --effort <n>

Examples:

  • /benchmark -> AutoroutingPipelineSolver, all scenarios (default concurrency uses the benchmark runner CPU count)
  • /benchmark AutoroutingPipelineSolver -> one solver, all scenarios
  • /benchmark all 20 -> all solvers, first 20 scenarios
  • /benchmark AutoroutingPipelineSolver 20 --concurrency 8 -> one solver, 20 scenarios, 8 workers
  • /benchmark AutoroutingPipelineSolver 20 --effort 2 -> one solver, 20 scenarios, 2x effort

Any PR whose title contains [BENCHMARK TEST] will automatically run the benchmark workflow on PR updates.

@seveibar seveibar changed the title [BENCHMARK TEST] Tune max iterations/hyper parameters to get more consistent runtimes (-2% on benchmark) [BENCHMARK TEST] Tune max iterations/hyper parameters to get more consistent runtimes (-2% on benchmark, 10% -> 3% timeout) Mar 10, 2026
@seveibar seveibar merged commit ad648b4 into main Mar 10, 2026
14 checks passed
@seveibar seveibar deleted the opt branch March 10, 2026 00:41
@tscircuitbot
Copy link
Copy Markdown


Thank you for your contribution! 🎉

PR Rating: ⭐⭐⭐
Impact: Major

Track your contributions and see the leaderboard at: tscircuit Contribution Tracker


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants