Skip to content

Call closures 2.2x faster: VM fast paths for higher-order and closure calls - #5354

Open
hellovai wants to merge 3 commits into
hellovai/stuck-task-reportfrom
hellovai/vm-closure-call-fast-path
Open

hellovai wants to merge 3 commits into
hellovai/stuck-task-reportfrom
hellovai/vm-closure-call-fast-path

Conversation

@hellovai

@hellovai hellovai commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Calling a closure you were handed is about 2.2x faster. That's the call inside every callback API: a helper taking fn, list.map((x) -> ...), sem.with(fn), baml.sync.atomic(key, fn). No change in behavior.

Numbers

Release build, VM, telemetry off, both binaries built from the same canary and run alternately, seven runs each:

Workload canary this PR
1,000,000 calls of a lambda passed as an argument 290 ns/call 132 ns/call
1,000,000 plain calls (no closure, unaffected) 55 ns/call 52 ns/call

What was slow

Passing a closure to a function. A function-typed parameter makes its function generic over the argument's throws, so apply(f, x) passes a hidden type argument. The call specializer only used its fast instruction for calls with no type arguments, so every higher-order call, and every explicit generic call like g<T>(x), took the general path. That meant leaving the dispatch loop, allocating while copying the type arguments, looking up the call layout and pushing the frame the slow way.

Calling the closure (fn()) always took the general path too.

Changes

  1. Exact calls into generic callees. The fast call instruction now also covers call sites that pass every type argument the callee declares, when those are plain static types. Anything else, such as a type naming a runtime declaration, still takes the general path.
  2. Closure calls stay in the fast dispatch loop. A bytecode function or closure whose parameters are exactly what the call site pushed is entered the way an exact call is, with the closure's captured type arguments and the same telemetry as the general path. These keep the existing path:
    • hooked and pass-through callees, and callees compiled at load time
    • a pending trace or type argument
    • remapped argument layouts
    • bound methods, host closures and generic function values

Risks / what to review

  • Two new call paths in the VM's core dispatch (bex_vm/src/vm.rs). Each must match the general path exactly: type arguments, telemetry entry, frame layout. Anything unusual (hooks, pass-through, load-time compiled callees, remapped layouts, bound methods, host closures, generic function values) falls back to the general path.
  • No language semantics change: only how fast an eligible call is entered.

Testing

  • bex_vm/tests/call_exact_args.rs: the test that pinned the old behavior now checks that baml.json.decode<int>("42") works on the fast path, and fails if the type argument isn't passed through.
  • bex_vm and bex_engine suites and the full runtime corpus pass on top of this stack.

Stack

  1. Task ids (Make VM task ids non-zero and fixed at construction #5323)
  2. baml.sync.atomic (Add baml.sync.atomic for guarding an object's own state #5661)
  3. baml.sync.Semaphore (Replace baml.spawn.Limit with baml.sync.Semaphore #5352)
  4. Stuck-task report (Report where every task is stuck, and name deadlocks #5662)
  5. this PR: VM fast path for closure calls

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Performance
    • Added faster execution for eligible generic, indirect, closure, function, and bound-method calls.
    • Calls with unsupported type arguments or other ineligible conditions continue through the general execution path.
  • Bug Fixes
    • Generic calls now pass their type arguments to callees.
  • Tests
    • Expanded coverage for generic and indirect calls, including closures and bound methods.
  • Documentation
    • Clarified when exact calls use specialized execution and when they fall back.

@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
developer-docs Ready Ready Preview Nov 16, 2026 12:47am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • Resume reviews
  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The VM now specializes exact calls that provide all declared generic type arguments when those arguments are plain static types. It also adds direct entry for eligible indirect calls to bytecode functions and closures. Tests cover generic parsing, indirect calls, and early-yield resumption.

Changes: VM call paths

Layer / File(s) Summary
Exact calls with generic arguments
baml_language/crates/bex_vm/src/call_specialize.rs, baml_language/crates/bex_vm/src/vm.rs, baml_language/crates/bex_vm/tests/call_exact_args.rs, baml_language/crates/bex_vm_types/src/bytecode.rs
Exact-call specialization accepts calls with all declared type arguments. The VM extracts arguments composed of statically declared types and installs them in the callee frame. Other type operands use the general call path. Tests cover generic parsing, and the opcode documentation describes the runtime fallback.
Indirect bytecode-call entry
baml_language/crates/bex_vm/src/vm.rs, baml_language/crates/bex_vm/tests/call_exact_args.rs
Eligible indirect calls enter bytecode functions or closures directly. Other calls continue through existing dispatch. Tests cover closures, functions, bound methods, and early-yield resumption.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant CallIndirect
  participant VM
  participant Callable
  participant CalleeFrame
  CallIndirect->>VM: Attempt direct bytecode-call entry
  VM->>Callable: Check callable and call layout
  VM->>CalleeFrame: Push callee frame
  VM-->>CallIndirect: Switch compact dispatch to callee
Loading

Suggested reviewers: codeshaunted

Merge Risk: Minimal · up to dd6fa

The change speeds up eligible generic and closure calls in the VM and keeps the general call path for anything that does not qualify. The review found no concrete behavioral regression, and the ineligible-call fallbacks leave the operand stack unchanged. The remaining uncertainty is that the tests do not show which route was taken, which is ordinary for a performance change.

Security architecture risk: Minimal · up to dd6fa

The faster call paths retain the existing tracing controls, type-argument handling and resource limits examined. No material security or failure-containment regression was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed execution surface is eligible bytecode call setup within an executing VM. Host closures, native functions, system operations, bound methods and generic-function wrappers are not admitted by the new indirect shortcut and retain general handling. The inspected routes do not establish a new host-operation authority path.

Trust Boundaries and Controls

  • observed — Callable and type-operand selection remain constrained before shortcut entry. Pending trace or type-argument state forces indirect fallback, as do hooks, pass-through behavior, compiled bodies and layouts requiring remapping. Exact specialization excludes written globals and controlled callees. Explicit type operands are drained only after every operand is confirmed to be a plain type without dynamic declaration heads.

Resilience and Maintainability Implications

  • observed — The new entries retain fallible reservations against the shared per-thread frame/value backing-storage budget. Both capacities are reserved before telemetry entry and frame publication. The later locals resize fits the reserved value capacity, resolving the suspected post-publication allocation-failure divergence. Reservation errors use the existing thrown-error and unwind handling.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: faster closure and higher-order calls through VM fast paths. The objectives state the aim is to speed up eligible higher-order and closure calls, and the reported 132 …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: faster closure and higher-order calls through VM fast paths. The objectives state the aim is to speed up eligible higher-order and closure calls, and the reported 132 ns versus 290 ns result matches the 2.2x claim. The changes to vm.rs, call_specialize.rs, bytecode.rs, and the tests all support this focus. The title is specific and accurate, so the check passes.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

A rabbit checks the call path bright,
Static types step into the frame,
Closures join the dispatch flight,
Bound methods answer to their name,
Early yields resume the game,
And tests bring back the number forty-two.

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 819f25c6-78f5-4e06-90de-16bba82bdd31
Commits

Reviewing files that changed from the base of the PR and between 58bda26 and dd6fa19.

Files selected for processing: 2

  • baml_language/crates/bex_vm/src/vm.rs
  • baml_language/crates/bex_vm_types/src/bytecode.rs
Files skipped from review as they are similar to previous changes: 1

  • baml_language/crates/bex_vm_types/src/bytecode.rs

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

@github-actions

Copy link
Copy Markdown

⏭️ Performance benchmarks were skipped

Perf benchmarks (CodSpeed) are opt-in on pull requests — they no longer run on every push. They always run automatically after merge to canary/main.

To run them on this PR, do any of the following, then push a commit (or re-run CI):

  • Add RUN_CODSPEED=1 to the PR description, or
  • Include run-perf or /perf in the PR title or any commit message.

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Binary size checks failed

❌ 2 violations · ✅ 1 passed

⚠️ Please fix the size gate issues or acknowledge them by updating baselines.

Artifact Platform File Gzip Gated on Baseline Delta Status
✅ baml-cli Linux 🔒 62.3 MB 26.5 MB file 73.1 MB -10.8 MB (-14.8%) OK
❌ packed-program Linux 🔒 40.5 MB 16.5 MB file 29.2 MB +11.4 MB (+39.0%) FAIL
❌ bridge_wasm WASM 26.5 MB 🔒 7.8 MB gzip 5.7 MB +2.1 MB (+37.3%) FAIL

🔒 = the size this artifact is GATED on (ceiling + delta). Binaries gate on file size (installed binary); WASM gates on gzip (download size). The other size is shown for information only.

Details & how to fix

Violations:

  • packed-program (Linux) file_bytes: 40.5 MB exceeds limit of 30.1 MB (exceeded by +10.4 MB, policy: max_file_bytes)
  • packed-program (Linux) file_delta_pct: +39.0% exceeds limit of 3.0% (exceeded by +36.0pp, policy: max_delta_pct)
  • bridge_wasm (WASM) gzip_bytes: 7.8 MB exceeds limit of 5.9 MB (exceeded by +1.9 MB, policy: max_gzip_bytes)
  • bridge_wasm (WASM) gzip_delta_pct: +37.3% exceeds limit of 3.0% (exceeded by +34.3pp, policy: max_delta_pct)

Add/update baselines:

.ci/size-gate/wasm32-unknown-unknown.toml:

[artifacts.bridge_wasm]
file_bytes = 26531638
gzip_bytes = 7772879

.ci/size-gate/x86_64-unknown-linux-gnu.toml:

[artifacts.packed-program]
file_bytes = 40516416
gzip_bytes = 16488639

Generated by cargo size-gate · workflow run

@hellovai
hellovai added this pull request to stack #5353 October 21, 2026 18:04
@hellovai
hellovai force-pushed the hellovai/vm-closure-call-fast-path branch from ebc77cd to 30e09f5 Compare October 21, 2026 22:09
@hellovai
hellovai force-pushed the hellovai/vm-closure-call-fast-path branch from 30e09f5 to 73deee6 Compare November 14, 2026 05:44
@hellovai
hellovai removed this pull request from stack #5353 November 14, 2026 05:44
@hellovai hellovai changed the title VM fast paths for higher-order and closure calls Call closures 2.2x faster: VM fast paths for higher-order and closure calls Nov 14, 2026
@hellovai
hellovai changed the base branch from hellovai/baml-sync-semaphore to hellovai/stuck-task-report November 14, 2026 05:45
@hellovai
hellovai added this pull request to stack #5663 November 14, 2026 05:45
@hellovai
hellovai force-pushed the hellovai/vm-closure-call-fast-path branch from 73deee6 to f54a874 Compare November 14, 2026 18:15
@hellovai
hellovai force-pushed the hellovai/vm-closure-call-fast-path branch from f54a874 to f2741a2 Compare November 14, 2026 18:17
hellovai and others added 2 commits November 14, 2026 11:37
…er ones

A function-typed parameter makes its function generic over the argument's
`throws`, so `apply(f, x)` passes one type argument. The load-time call
specializer only used CALL_EXACT_ARGS for calls with no type arguments, so
every higher-order call (and every generic call) took the general path:
leave the dispatch loop, copy the type arguments through three vectors, look
up the call layout and push the frame there.

CALL_EXACT_ARGS now covers sites that pass every type argument the callee
declares. When every type operand is a plain static type, the operands go
straight into the new frame and dispatch stays in run_compact; any other
operand still takes the general path. LOAD_TYPE's static cache now hashes
with FxHash instead of SipHash.

speedtest vm-off, ns per iteration (canary -> this):
  compute/closure-apply-1m  415 -> 230
  compute/pure-call-1m       70.8 -> 71.4

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…compact dispatch

A call through a function value always took the general path: a callable
resolve, layout remap, the general frame push, and a return to
`exec_compact` to re-derive the frame's function, code and pc. A bytecode
function or closure whose parameter slots are exactly what the site pushed
now enters the way `CallExactArgs` does (closure type arguments seeded into
the frame, same telemetry entry as the general path) and stays in
`run_compact`. Hooked and pass-through callees, pending trace or type
lanes, remapped layouts, bound methods, host closures and generic function
values keep the existing path.

The entry is out of line: inlined into `run_compact` it made every ordinary
exact call ~10 instructions slower.

Instructions per iteration (20M loop minus empty run): direct capturing
closure call 1460 -> 1093, `apply(double, i)` 2329 -> 1958, `leaf(i)`
860 -> 858. speedtest vm-off: closure-apply-1m 230.0 -> 193.7 ns/iter,
pure-call-1m 71.5 -> 71.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview – developer-docs — dd6fa19b Deployed Nov 16, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants