Boa and JIT

[EngvallEmill]

Hello! I wonder if boa support JIT? If not, is it possible to integrate Cranelift for JIT support?

[jedel1043]

No, we don't support JIT, but it should definitely be possible to implement. It would just be a very big design task to hook Cranelift into our VM, but it's definitely easier than having to completely reimplement a JIT compiler from scratch.

[zhuzhu81998]

copy-and-patch approach at least on paper looks really cool. From Wikipedia:

In computing, copy-and-patch compilation is a simple compiler technique intended for just-in-time compilation (JIT compilation) that uses pattern matching to match pre-generated templates to parts of an abstract syntax tree (AST) or bytecode stream, and emit corresponding pre-written machine code fragments that are then patched to insert memory addresses, register addresses, constants and other parameters to produce executable code. Code not matched by templates can be either be interpreted in the normal way, or code created to directly call interpreter code.

if i understand correctly, this could be implemented on a step-by-step basis for each opcode, so no need for a gigantic one-step hard-to-reverse upgrade. and it does not require to have a whole compiler embedded in the runtime either.
(Python 3.13 started using it. Lua too. Apparently no patent on this)

Supporting for many different targets might become an issue though (what is boa supposed to support actually?)

[rubys]

Cranelift JIT Prototype — Results and Next Steps

I've been exploring @jedel1043's suggestion of integrating Cranelift for JIT support. There's now a working prototype on a fork branch — about 5,000 lines behind a jit feature flag, with 83 tests passing and all 7 V8 benchmarks running.

What it does

The JIT uses a simple tiering model: interpret a function for 10 calls, then compile it to native code via Cranelift. Currently 109 of Boa's 196 opcodes are supported — the remainder are async/generator, iterator protocol, eval, and class private fields which rarely appear in hot loops. Functions containing unsupported opcodes stay interpreted.

For integer arithmetic (add, subtract, compare, bitwise ops), the JIT inlines NaN-boxing-aware fast paths directly into the native code — checking tag bits, doing native i32 operations with overflow guards, and falling back to runtime helpers for non-integer cases. For property access, it bakes inline cache entries from compile time into the generated code: a shape guard + direct storage load that bypasses the full property lookup when the shape matches.

Results

Comparing origin/main against the JIT branch on the V8 benchmark suite (higher is better):

V8 Benchmark	Boa main	Boa + JIT	QuickJS	JIT vs main
NavierStokes	268	579	1152	2.16x
Crypto	132	212	681	1.61x
Richards	124	180	648	1.45x
DeltaBlue	122	140	624	1.15x
Splay	563	575	1341	1.02x
EarleyBoyer	285	258	1195	0.91x
RayTrace	258	222	834	0.86x

On a tight integer loop (sum(10M) — 10 million iterations of s = (s + i) | 0):

	Boa main	Boa + JIT	QuickJS
sum(10M)	800ms	81ms	128ms

The JIT is 10x faster than the interpreter on tight loops and 1.6x faster than QuickJS. Four of seven V8 benchmarks show clear improvement, with NavierStokes more than doubling.

What we learned

Profiling reveals that the V8 benchmarks are dominated by property access, GC tracing, and function call overhead — not bytecode dispatch. The JIT eliminates dispatch (~13% of interpreter time) and inlines some property access via IC, but can't eliminate the cost of the runtime operations themselves. This is why benchmarks like RayTrace and EarleyBoyer don't improve — their bottleneck is object allocation and GC, not instruction dispatch.

The benchmarks that DO improve (NavierStokes, Crypto, Richards) have hot functions with tight numeric loops or integer-heavy operations where the inlined NaN-boxing fast paths pay off.

Architecture notes

• Cranelift Variable for the register base pointer — handles SSA phi nodes at loop headers automatically
• Stack slot updated by call helpers — when a function call resizes the VM stack, the slot is updated and the Variable reloaded
• Inline IC for GetPropertyByName — bakes the cached shape pointer and storage slot index at compile time, falls back to an IC-aware helper on miss
• GC-safe Move via runtime helper (clone/drop for pointer types)
• --no-jit CLI flag for A/B comparison and debugging
• Feature-gated: cfg(all(feature = "jit", not(feature = "jsvalue-enum"))) — silently excluded when NaN-boxing isn't active

What would improve V8 scores further

1. Fast function calls — the current jit_call helper goes through the full function_call machinery (~200ns per call). A fast path for ordinary functions that skips unnecessary environment cloning and this-binding resolution would help Richards, DeltaBlue, and call-heavy benchmarks.

2. Inline IC for prototype properties — method calls (.splay_(), .isEmpty()) currently fall back to the helper because prototype chain lookups aren't inlined yet. Extending the IC to chase one prototype pointer would cover most method dispatch.

3. Runtime improvements — lighter GC tracing and faster property access would benefit both JIT and interpreter. Profiling shows GC at 10-16% of V8 benchmark time.

Regarding copy-and-patch

@zhuzhu81998 raised the copy-and-patch approach. Having built the Cranelift version, I think both have merit for different tiers. Copy-and-patch would compile much faster (nanoseconds vs microseconds) and could serve as a baseline tier, while Cranelift gives us the optimization path (inlined type checks, IC, potential for speculative optimization). The current Cranelift compilation is fast enough that the 10-call threshold works well in practice.

Try it

git clone https://github.com/rubys/boa.git
cd boa
git checkout cranelift-jit
cargo build --release --bin boa --features jit
echo 'function f(n) { var s=0; for(var i=0;i<n;i++) s=(s+i)|0; return s; } console.log(f(10000000));' | ./target/release/boa

The branch passes format checks, clippy with -D warnings (including --all-features), and all 83 JIT-specific tests plus the existing test suite.

My interest in this comes from Boax, a gem that embeds Boa in Ruby. Performance matters there, and the JIT makes a measurable difference for numeric workloads.

[rubys]

what my current JIT does is swap instruction dispatch overhead with helper calling overhead. Best case that's a wash, but realistically it is a net negative. What it gains is that one by one the most frequently used helpers can be inlined and not only will that overhead be eliminated, but will run at full compiled speed. I shared early progress to see if there was interest in progressing this further.

[jedel1043]

But in this case "full compiled speed" would be "full compiled VM speed", or the equivalent to having a "native" VM. What we want from a JIT compiler is to completely bypass the VM itself, and allow doing things like autovectorization, speculative execution or dynamic PGO, and as far as I'm aware, those can only be enabled if we lower directly from the AST.

EDIT: maybe "only be enabled" is the wrong expression. You might be able to do that using bytecode lowering, but it is much more difficult than doing it while having all the semantic information that the AST provides

[zhuzhu81998]

what my current JIT does is swap instruction dispatch overhead with helper calling overhead

well then your original sentence "This is why benchmarks like RayTrace and EarleyBoyer don't improve — their bottleneck is object allocation and GC, not instruction dispatch." is not accurate

What we want from a JIT compiler is to completely bypass the VM itself

"completely" is maybe a bit too much? you still need to come back to vm at some point?

[zhuzhu81998]

6000 lines of code but still have to use some helpers sounds... off putting. i think cranelift-based jit can wait a bit. there are things that can be done without requiring this much effort.

[jedel1043]

"completely" is maybe a bit too much? you still need to come back to vm at some point?

I mean completely bypass it in the sense that the JIT compiled code just has to interact with the VM at the boundaries of its execution, but otherwise it just uses machine code to execute things.

But it's true that you still need to come back to the VM at some point, so our wins might be bounded by how efficient this boundary glue code is

[rubys]

Can we talk about real examples? For example, nbits from Crypto:

function nbits(x) {
  var r = 1, t;
  if ((t = x >>> 16) != 0) { x = t; r += 16; }
  if ((t = x >> 8) != 0) { x = t; r += 8; }
  if ((t = x >> 4) != 0) { x = t; r += 4; }
  if ((t = x >> 2) != 0) { x = t; r += 2; }
  if ((t = x >> 1) != 0) { x = t; r += 1; }
  return r;
}

Pure numeric, no objects, no property access. The AST approach and bytecode approach would produce identical Cranelift IR for this.

For the case where AST WOULD help, you'd need something like NavierStokes's addFields:

function addFields(x, s, dt) {
  for (var i = 0; i < size; i++) x[i] += dt * s[i];
}

Here the AST knows size is a closure variable and x, s, dt are parameters. But it doesn't know their types or shapes — that requires runtime information.

[jedel1043]

A sufficiently intelligent compiler would analyze the AST to check the shapes of all callers of allFields to see which shape is the input. That's what I'm talking about when I mean lowering from AST would not lose semantic information.

Also, while having an optimized nbits would be nice, the vast majority of JS code works with objects and classes, so I would think trying to JIT from bytecode to only benefit numerical-heavy code is the wrong approach.

[rubys]

Dynamic dispatch, callbacks, module boundaries make statis analysis impractical; V8 and others tried and abandoned that approach. That doesn't mean that it is impossible, but I'm not aware of anybody being able to make that work.

JIT from bytecode doesn't only benefit numerical heavy code. As an an example, property access in loops was made 1.7 times faster that quicksort.

[jedel1043]

I think that's incorrect, or as far as I know, V8 lowers from the AST. It might speculate when to apply JITing from the bytecode, but I think they convert the AST to an intermediate representation, then apply optimizations and finally JIT compile based off of that.

[jedel1043]

Also static analysis is impractical if you don't know the shapes of your arguments. If you do know them after dynamically tracking the shape of your inputs and determining that a function needs to be JIT-ed, you can apply some amount of static analysis based off of the shape of your objects.

[rubys]

See https://v8.dev/blog/ignition-interpreter

Key excerpt:

The Ignition interpreter uses TurboFan’s low-level, architecture-independent macro-assembly instructions to generate bytecode handlers for each opcode. TurboFan compiles these instructions to the target architecture, performing low-level instruction selection and machine register allocation in the process. This results in highly optimized interpreter code which can execute the bytecode instructions and interact with the rest of the V8 virtual machine in a low-overhead manner, with a minimal amount of new machinery added to the codebase.

Also relevant, Rust's has multiple intermediate representations: https://blog.rust-lang.org/2016/04/19/MIR/

[rubys]

It might be helpful if yiou could identify the specific technical debts that have you concerned.

My sense is that we have a nomenclature problem. The current bytecode is an IR. What's needed is a pipeline that applies optimizations properly in passes. Passes may convert a stream of bytes into a structured ir to apply optimizations properly and then lower that to a cranelift ir. As near as I can tell, that's the way every major javascript implementation today works.

The other gap is the feedback vector. Boa's inline cache currently only covers property access. V8's feedback vector covers every operation type (arithmetic, comparisons, call targets). Extending Boa's IC to capture broader type feedback would benefit the interpreter directly (skip type coercion on operations that always see integers) and give the JIT runtime type information for cross-block optimization. That's a separate effort from the IR pipeline, but equally important for closing the performance gap.

What I can work on is splitting the current cranelift IR generation into functions, verify that there are no regressions; then work on a bytecode to IR and IR to cranelift path, verify that there are no regressions and retire the direct path from bytecode. Then apply performance improvement transformations on the IR.

[zhuzhu81998]

the current bytecode of boa is just a Vec<u8> and consists of opcode and its operands (register location etc.).

My sense is that we have a nomenclature problem. The current bytecode is an IR. What's needed is a pipeline that applies optimizations properly in passes. Passes may convert a stream of bytes into a structured ir to apply optimizations properly and then lower that to a cranelift ir. As near as I can tell, that's the way every major javascript implementation today works.

Are you suggesting AST -> current boa bytecode (stream of bytes) -> structured ir -> optimization passes -> cranelift ir? That doesnt sound right.

Boa's IC to capture broader type feedback would benefit the interpreter directly (skip type coercion on operations that always see integers)

So currently the register slots etc. hold JsValue only. If you want to skip type coercion completely, then that probably needs to change. Depending on what you do, the case where the type of a JsValue changes could be somewhat expensive? On the other hand, boa already uses NaN boxes. So type check etc. is not even that slow. So not sure about the equally important part, especially boa doesnt have a JIT to take advantage of any potential runtime info yet.

What I can work on is splitting the current cranelift IR generation into functions, verify that there are no regressions; then work on a bytecode to IR and IR to cranelift path

I mean... the direction doesnt sound right. But Boa can probably use some kind of IR.

[jedel1043]

What I can work on is splitting the current cranelift IR generation into functions, verify that there are no regressions; then work on a bytecode to IR and IR to cranelift path, verify that there are no regressions and retire the direct path from bytecode. Then apply performance improvement transformations on the IR.

Again, I would strongly prefer to repurpose our current bytecompiler to output IR, then create a separate compiler from IR to bytecode. Otherwise you would need to reanalyze the whole bytecode from scratch

[zhuzhu81998]

It seems there are sources that say V8 first lowers AST to bytecode for ignition and then from that bytecode to the IR that Turbofan uses. See e.g. here and here.

There is a stackoverflow answer that says lowering from bytecode in turbofan is just an implementation choice and lowering from AST is also viable and what v8 used to do.

I am guessing maybe bytecode + the feedback vector preserve virtually all useful semantic info anyway?

[jedel1043]

I am guessing maybe bytecode + the feedback vector preserve virtually all useful semantic info anyway?

Kinda but not really. We have a mix of RISC and CISC instructions so some instructions are properly preserved and some others are decomposed into more basic instructions, which loses a lot of the semantic info