Refinements, lexical-scope reflection, and definition-site line numbers - #1064
Merged
sisshiki1969 merged 18 commits intoAug 8, 2026
Merged
Conversation
Ruby 3.4 added three `Binding` methods for the numbered and `it` block parameters, and monoruby had none of them: `implicit_parameters`, `implicit_parameter_defined?` and `implicit_parameter_get`. The names are not stored anywhere separate — a block using `_1`/`_2` or `it` gets them as ordinary leading positional parameters — so the set is recovered from the binding's own iseq: `it` when the parameter list is the `it` form, otherwise the longest `_1`.._9 prefix the parameter names actually spell. Anything else (a real `def`, a block with named parameters) has none, and the two lookup methods raise `NameError` for a name that is not an implicit parameter, matching CRuby. `Binding#local_variable_get` and friends grow the same argument coercion on the way: a non-Symbol, non-String argument now reports CRuby's "… is not a symbol nor a string" rather than the generic conversion error. core/binding: 31 errors → 7 (all remaining ones are refinements, which monoruby does not implement). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Six unrelated small divergences from CRuby, all found by ruby/spec.
- `Proc#arity` on a non-lambda counted an optional positional as
making the arity variable. A proc accepts any argument count
regardless, so CRuby reports only the required ones —
`proc { |a, b=1| }.arity == 1`, where the lambda of the same shape
reports -2. Only an explicit rest goes negative.
- `Proc#to_proc` did not exist. It returns self.
- A String in a non-UTF-8 encoding could not name a constant: the
coercion path only accepted valid UTF-8 and otherwise fell through
to `#to_str`, so `const_get "CS_CONST\u{3bb}".encode("euc-jp")`
raised. Such a name denotes exactly one constant — `::`, being
ASCII, never occurs inside a character of an ASCII-compatible
encoding, so there is nothing to split — and is now interned
bytewise, as `const_set` already stored it. The three copies of the
coercion-and-split code in `const_get` / `const_defined?` /
`const_source_location` became one helper in the process.
- `public :inherited_method` in a subclass froze the ancestor's body
at declaration time, so redefining the ancestor afterwards was not
picked up through the subclass. The entry is a visibility modifier,
not a definition: method lookup now carries its visibility (and its
owner, which stays the subclass) but resolves the body from
whatever the ancestor defines at call time — CRuby's ZSUPER entry.
- `Module#set_temporary_name(nil)` left nested modules reporting
`#<Module:0x..>::Inner`. Erasing a name is not the same as never
having had one: the subtree under an erased module reports `nil`,
and re-naming it brings the names back.
- `Module#using` did not exist, and `main.using` accepted a Class.
Refinements are unimplemented, so `using` cannot do anything, but
it can agree with CRuby on what it accepts (a Module, not a Class)
and where it may appear (not in a method body).
core/proc: 4 failures / 1 error → 0. core/module: 19 failures → 16, and
`Module#using`'s non-refinement examples all pass. The remaining module
failures are refinements, autoload concurrency, definition-site line
numbers and lexical scope — tracked separately.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
`Module.nesting` walked the *runtime* cref stack, which monoruby does
not push per call — so a method called from inside some other class body
reported that body's scope, and inside a method body (where the stack
is empty) it reported `[]`. The nesting captured at `def` time is
already stamped on the iseq; that is the authority, and a block, which
carries none of its own, takes its mother's.
The string forms of `module_eval` / `class_eval` / `instance_eval` had
the mirror-image problem: the eval body's nesting was set to the
receiver *alone*, discarding the scope the call was written in. CRuby
pushes the receiver onto that scope, so `Mod.module_eval "Lookup"`
resolves through the receiver and then through the caller's lexical
scope. The receiver is now pushed onto the outer nesting instead of
replacing it.
That reordering exposed the lookup order `instance_eval "..."` needs —
receiver singleton class, then receiver class, then the caller's scopes
— which used to fall out of the singleton being the only entry. The
walk is now explicitly split around the receiver-class check.
All three eval forms also anchored the body to the *immediate* caller
frame, which is a builtin when they are reached through one (`send`, a
Rust-written helper, mspec). The compiled body already took its outer
*locals* from the nearest Ruby frame, so the two disagreed and the
lexical scope was lost — `send(:module_eval, "Lookup")` could not see
what `module_eval("Lookup")` could. Both now use the nearest Ruby
frame; only the reported `(eval at …)` location still comes from the
immediate caller, whose pc we were handed.
Which in turn uncovered a latent panic: `get_caller_loc` indexed the
sourcemap with that pc without checking it belongs to the frame it
landed on. Through a JIT-inlined `send` it does not, and the process
aborted. It now falls back to the function's own location.
core/module: `Module::Nesting returns the nesting for module/class
declaring the called method` and `Module#module_eval resolves constants
in the caller scope ignoring send` both pass. No regressions across
core/{module,class,proc,binding,basicobject,kernel} and language —
340 files, 6716 examples.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Three places reported the line of the enclosing body where CRuby reports
the line of the definition itself.
`Module#const_source_location` for a constant created by the `class` /
`module` keyword walked the CFP chain for the nearest Ruby frame and
took *its* start line — so every class in a file reported the line of
the module that encloses them. `define_class` runs before the body's
frame exists, so it has nothing better to look at; the body's own iseq
does carry the right line, and `enter_classdef` runs immediately after
on both arches and in both tiers. `define_class` now only marks which
constant is waiting for a location, and `enter_classdef` records it.
`caller_locations` inside `method_added` / `singleton_method_added`
reported the line of the class body rather than the `def`. The hook is
dispatched from Rust through the invoker, which writes no caller pc into
the new frame's cont slot, so the backtrace walk could not recover the
line and fell back to the enclosing function's own location. The
definition site now travels out-of-band on the executor, and the walk
consults it for the first frame whose pc it cannot resolve — which is
exactly that invoker boundary.
Finding that also uncovered why `Module#const_source_location`'s
`#to_str` example failed: it was not about coercion at all, it read the
location of a class-keyword constant.
`const_added` still reports the enclosing line for the `class` / `module`
keyword (assignments and `const_set` are already right). Fixing it needs
the body's FuncId inside `define_class`, which fires the hook — and that
is an ABI change to a runtime call emitted from four places across both
architectures' VM and JIT backends. Left alone deliberately.
core/module: 3 more examples pass. No regressions across
core/{module,class,proc,binding,basicobject,kernel,exception,thread}
and language — 8081 examples.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
`Kernel#binding` captured `cfp.prev()` — the immediate caller. Reached
through a builtin (`send(:binding)`, a Rust-written helper), that is the
builtin's own frame, and a Binding anchored there has no iseq at all:
`Binding#eval` raised "eval with binding requires a Ruby method
context", and `Binding#local_variables` aborted the whole process on an
`unreachable!()` in `as_iseq`. It now walks out to the nearest Ruby
frame, the same rule `Kernel#eval` and the `*_eval` family already use.
core/kernel: `Kernel#binding uses the class as self in a Class.new
block` and `uses the closure's self as self in the binding` pass. No
regressions across core/{binding,kernel,module,proc,basicobject} —
3731 examples.
Two core/binding examples need this too, but they cannot run yet: their
fixture calls `Module#refine`, which monoruby does not implement, so the
file fails to load and takes 59 examples in 7 spec files down with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
`doc/cref.md` noted in one line that `Cref` has no `refinements` field. That undersells it: the missing field is the smallest part of the problem, and the reason the obvious half-measure is unsafe was not written down anywhere. `doc/refinements.md` works through it. Refinements make the resolved method a function of the caller's lexical scope, and every resolution path in the tree is keyed without one: the global method cache (`(name, class_id)` + class version) and the per-class memoized predicates built on the same key, the VM inline cache's two guards, the JIT's compile-time `jit_check_method`, specialized inlining, the inline builtin generators and the process-wide basic-op flag, and `super`'s position-in-the-ancestor-chain model. The one to know about is `update_inline_cache`. On a class-version bump compiled code deopts and that path tries to *repair* rather than recompile, by re-checking each recorded `(recv_class, name) -> comptime_fid`. Those re-checks are cref-free, so a `using` that only bumped the class version would have them confirm the unrefined answer and re-validate machine code that must now dispatch into the refinement — the wrong method, silently, indefinitely. Also recorded: the four semantics that drive all of this, each checked against CRuby 4.0.2 rather than assumed — activation is a runtime event at a lexical position, the set is captured per method body at definition time, basic ops like `Integer#+` are refinable, and `send` / `respond_to?` / `Object#method` see refinements too, so there is no reflection exemption to lean on. Plus the ruby/spec ledger (~96 examples need real refinements; ~58 in core/binding are collateral from a fixture that only needs `refine` to be defined), why the stub that buys those 58 is worse than nothing, and a staged plan whose first stage is the per-frame CREF `doc/cref.md` already describes. Documentation only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
The staged plan opened with "Stage 0 — per-frame cref" without saying what that costs, which reads as a much bigger prerequisite than it is. Two measurements bound it. A block created before a `using` is refined; a method defined before it is not. So the cref is a mutable cell owned by the scope's environment and shared by reference with the blocks that captured it, while `def` snapshots the pointer — a static per-iseq field cannot express both halves. But the mutable half is only needed where `using` is legal, and that excludes method bodies outright (`Module#using` raises there, and `main.using` raises anywhere but toplevel). A method body needs only a definition-time snapshot, which is exactly what `lexical_context` and `nested_definee` already are. The JIT resolves methods at compile time in Rust, so no emitted prologue changes either. That leaves where to put the cell for toplevel / class bodies / eval. Recorded with costs: a new `LFP_CREF` word copying the `LFP_SVAR` template (faithful, known-feasible, but moves `LFP_ARG0` and `RSP_LOCAL_FRAME` through 190 references in 26 files and spends 8 bytes per frame for something only non-method frames use); a side table keyed by LEP like the existing `deferred_unwind` (no layout change, correct per execution, free when unused — recommended); or per-iseq storage next to `lexical_context` (no frame work, but inherits the last-execution-wins staleness that `ISeqInfo` already documents). Documentation only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
The old wording ("a block created before the using is refined, a method
defined before it is not") was right but under-measured, and left it
ambiguous whether a closure snapshots a cref pointer or reads through a
cell.
Two `using` calls in one scope with a proc and a def interleaved settles
it: every proc sees the final state — including the one created before
any `using` ran — while the three methods carry three different sets.
So a block reads the scope's live state at call time; only `def`
snapshots. The cell has to be mutable, owned by the environment, and
read through.
Also noted: runtime CREF mutation is not new to monoruby. Bare `private`
/ `public` / `protected` already write `Cref::visibility` in place,
`module_function` writes its own flag, and class bodies and evals push
and pop entries. What refinements add is that method *resolution* starts
depending on that mutable state, with the visible-to-existing-blocks /
invisible-to-existing-methods split.
Documentation only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
…speed The old §6 was an ordering, not a strategy: it said "extend the method cache key" and "suppress inline generators", which is the framing that makes refinements look like they cost everyone something. The rewrite rests on one representation choice and two observations. Represent an activated set as an interned `RefinementSetId(u32)` rather than a hash of refined classes. That turns the cref's refinement state from something a lookup has to walk into a value a cache entry can hold and compiled code can treat as a constant, which is what makes the rest cheap. Observation one: `using` is illegal in a method body, so a method snapshots its set at `def` and a block's home cell only moves when `using` runs — a load-time event. So an iseq's set is a compile-time constant to speculate on, with `class_version` as the invalidation channel. The bluntness of that bump is acceptable precisely because `using` never happens in a loop. Observation two: the tax should scale with how many *names* are refined, not with whether the program uses refinements. A `refined_names` set lets the global method cache and the per-class memo predicates keep their existing key and simply decline those names, so refining `String#blank?` costs nothing on `Array#each`. From there the pieces get small: the VM inline cache needs no format change (a call site has one set, and the warm path can see it); `inline_cache_map` grows one u32 so `update_inline_cache` re-asks the question the compiler asked, which is the entire fix for the §3.4 silent-wrong-answer path; and the inline-generator problem splits into a per-call-site compile-time gate for the JIT and the global cliff only for the VM's dispatch-table basic ops. The acceptance criterion is stated as identical emitted machine code with the gate off, checkable against an `emit-asm` baseline, rather than as a benchmark that happens not to regress. §3.2 and §3.6 updated to point at the answers instead of leaving the trade-offs open. Documentation only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
`concat` built its result in a local `Array` and then, inside the loop, called `coerce_to_array` on each argument — which dispatches `#to_ary`, i.e. arbitrary Ruby, which can collect. The accumulator was reachable only from a Rust local, so a collection landing in that window freed it and the next `extend_from_slice` read a dead RValue, aborting the process on `Header::ty`'s unwrap. Latent: it needs a collection to fall inside those few allocations. An unrelated change to startup allocation order was enough to make `Array#concat(obj_with_to_ary)` abort in the unit suite. Pushed onto the temp stack for the duration, the way `Array#&` and the other builtins that call back into Ruby already do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
First of the four steps in `doc/refinements.md` §6.9. Everything here is
inert: `refine` and `using` build and record refinement sets, but method
resolution still ignores them, so nothing can change speed or behaviour
for code that does not call `refine`.
**The representation.** A scope's activated refinements are an interned
`RefinementSetId(u32)` (`globals/store/refinement.rs`) rather than a hash
of refined classes: sets are hash-consed, so equal sets get equal ids and
the id can be compared, stored in a cache entry and — from step 3 — baked
into compiled code. `EMPTY` is `0`, the state of every scope in a program
that never refines anything. The table also carries the union of refined
method names and the "any refinement exists" gate that steps 2–4 hang the
fast paths off.
**Where a set lives.** `ISeqInfo::refinements` is an `Option`: `Some` on
the two kinds of body that own one — a method (the snapshot `def` takes,
since `using` is illegal in a method body) and a scope that ran `using` —
and `None` everywhere else, so an ordinary block resolves through its
lexical parent. That is what makes a block created *before* a `using` in
its home scope see it, while a method defined before it does not, which
is the pair of behaviours CRuby has (`doc/refinements.md` §7.1).
The block forms where `using` is legal (`Module.new { using R }`,
`class_eval { using R }`) are scopes in their own right, so a `using`
writes the cell on the body that ran it rather than on its mother; the
activation does not leak out.
**The surface.** `Module#refine` builds (or extends) the module's
refinement for a class and returns it; the refinement is active inside
its own `refine` block, as in CRuby, so a refined method can call its
siblings. `Refinement` is a `Module` subclass with `#target`, its own
`#to_s`, and no `append_features` / `prepend_features` / `extend_object`
— and `include` / `prepend` / `Module#include` / `Object#extend` all
reject it, because a refinement is not a mixin. `Module#refinements`,
`Module.used_refinements` and `Module.used_modules` are real (the last
two report the *caller's* scope); the Ruby-side `used_refinements` mock
is gone. `Refinement#import_methods` copies modules' own Ruby-defined
methods in, validating every argument before importing anything.
core/refinement: 25 errors → 2 failures / 9 errors, the remainder being
the examples that need activation. Over
core/{refinement,module,main,binding,kernel,proc,class,basicobject} and
language, 137 failing examples → 89: the refinement fixtures that used to
abort a whole spec file at load now run, which recovers 58 examples in
core/binding alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Second of the four steps in `doc/refinements.md` §6.9. Refinements now
work end to end through the interpreter; the JIT declines any body that
resolves under one, and everything else keeps the code path it had.
`ClassInfoTable::search_method` grows a variant taking the activated
`(refined class, refinement)` pairs, probing each position's refinements
just before that position's own method table. Probing in chain order
rather than "all refinements first" is what keeps a subclass's real
method ahead of a refinement of its superclass. A class may carry
several activated refinements at once; they are searched
most-recently-activated first, and an earlier one stays reachable behind
a later one rather than being replaced.
`Executor::find_method` reads the caller's set and passes it down. The
`EMPTY` case — every scope in a program that never refines anything, and
every scope outside a `using` in one that does — goes to
`check_method_for_class` exactly as before, global method cache and all.
The whole thing sits behind `RefinementTable::is_active`, false until
the first `refine` call, so a refinement-free program does not even
compute a set.
`using` bumps `class_version`, which is what invalidates the inline
caches and compiled entries that resolved under the old set. It is a
load-time call, never a hot-path one (§6.1).
The JIT refuses to compile a body whose set is non-empty. Step 3 is what
threads the set into `JitContext` and `inline_cache_map`; until then
`update_inline_cache`'s repair would re-confirm the *unrefined*
resolution after a `using` moved the version and re-validate machine
code that must now dispatch into the refinement. Refusing is a
performance choice, a wrong `FuncId` is not.
Also landed here, all of it about where `using` is legal or what it
reaches: `using M` activates the refinements of M's ancestors, not just
its own; all the refinements defined in *one* module are visible from
each other's method bodies, which needs a re-stamp at the end of each
`refine` because a sibling defined later cannot be in the snapshot an
earlier `def` took; `main.using` raises outside the top level and
`refine` without a block raises ArgumentError.
Known limitation, recorded on `ISeqInfo::refinements`: one cell per iseq
means a *block* that runs `using` and executes more than once bases on
the previous execution's set. Re-activating the same module is
idempotent, so it only shows when each execution activates a different
refinement. Class bodies are unaffected — `enter_classdef` re-seeds.
core/refinement 25 errors → 1 failure / 4 errors. Across
core/{refinement,module,main,binding,kernel,proc,class,basicobject} and
language: 89 failing examples → 43, no regressions. The rest are the
reflective entry points (`method`, `respond_to?`, `Symbol#to_proc`),
`super` inside a refinement, and the basic-op fast paths — step 4.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Third of the four steps in `doc/refinements.md` §6.9, and the one that
closes §3.4's silent-wrong-answer path. Step 2's blanket refusal to
compile any body with a non-empty set is lifted.
`inline_cache_map`'s tuples become a named `InlineCacheEntry` carrying
the `RefinementSetId` the compiler resolved under, and
`update_inline_cache`'s re-check passes it back to
`check_method_with_refinements`. That is the whole fix: on a class-version
bump the repair now re-asks the question the compiler asked, instead of
re-asking the *unrefined* one, agreeing with itself, and re-validating
machine code that must dispatch into the refinement.
`JitContext` carries the set of the body being compiled — read once from
the iseq, since it is a compile-time constant of that body (§6.1) — and
`jit_check_method` resolves through it. The inline generators need no
gate for this: they are keyed by `FuncId`, and a refined name now
resolves to the refinement's Ruby-defined body, which is not in the
table. Basic ops still bypass dispatch in both tiers; that is step 4.
The acceptance criterion from §6.4 is met, not assumed: with no
refinement in the program, `--features emit-asm` over a mixed
integer/array/hash/recursion workload emits byte-identical machine code
before and after this change — same six compilations, same byte counts,
same instructions, only the compile timings differ.
Two new tests cover what the JIT has to get right: two classes with
different sets and a third with none, all resolving the same call site
text through compiled code; and reopening the refined class after
compilation, which must leave the refinement alone and be visible
outside it.
No change to the 43 remaining failures across
core/{refinement,module,main,binding,kernel,proc,class,basicobject} and
language — the same examples pass, now with the JIT compiling them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Last of the four steps in `doc/refinements.md` §6.9 — the entry points
that reach a method without going through an ordinary call site.
**Reflection.** `Object#method`, `#public_method`, `#respond_to?` and
`Module#instance_method` resolve through the *caller's* set, which is
what Ruby 4.0 does (§1(d) — checked, not assumed: there is no
"reflection is exempt" shortcut to lean on). `send` and `public_send`
needed nothing; they already went through `find_method`.
**Protocol dispatch.** The `#to_proc` behind a `&obj` argument and the
`#to_s` behind string interpolation are dispatched from Rust, not from a
call site, and interpolation additionally shortcuts non-Object receivers
before dispatching at all. Both now consult the caller's set — the
shortcut only when `to_s` is a refined name anywhere, so nothing changes
for the overwhelming majority of interpolations.
The scope those two must read is the user's, but they are reached from
inside monoruby's own Ruby-level core library (`Array#map` and friends
live in `builtins/*.rb`), whose frames are not the scope anything was
written against. The walk now steps over them — they are already marked
transparent for `$~`, and this is the same question.
**alias.** `alias new old` inside a `refine` block names the *refined
class's* method: the refinement module holds only what the block has
defined so far. `Refinement#instance_methods` answers as the refined
class does, plus whatever the block has added.
**import_methods** imports module by module, so a module whose methods
are not all Ruby-defined raises when the walk reaches it and leaves the
modules before it imported — matching CRuby. The argument type check
stays all-or-nothing.
Nothing needed doing for the JIT inline generators: they are keyed by
`FuncId`, and step 3 already has a refined name resolving to the
refinement's Ruby body, which is not in the table. Basic ops still
bypass dispatch in both tiers — a refinement of `Integer#+` is not seen,
the one piece of §6.7 left undone, and equally undone in the VM.
The §6.4 criterion still holds: with no refinement in the program the
emitted machine code is byte-identical to the pre-refinements baseline.
core/refinement, core/module/refine_spec, using_spec, main/using_spec
and core/binding together: 191 examples, 2 failures. One of the two is
not a defect — monoruby implements Zlib in Ruby, so `import_methods
Zlib` legitimately succeeds where CRuby's C extension cannot be
imported. Across core/{refinement,module,main,binding,kernel,proc,class,
basicobject} and language: 43 failing examples → 30, no regressions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
The document was written as an argument for why refinements were not implemented and what it would take. All four steps have landed, so the framing was stale: §§1–5 stay as the problem statement, §§6–7 are now the implementation rather than a proposal, and the header says so. Records the two known gaps up front — basic ops in both tiers, and the per-iseq cell's re-execution case — plus the ledger the four steps produced (137 failing examples → 30). Documentation only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
`doc/` had grown to 24 markdown files with no entry point, and CLAUDE.md named three of them. Nothing said which documents describe the code as it is, which record why a subsystem is shaped the way it is, and which are proposals that were never built — a distinction that matters, because the three age differently. doc/README.md groups the documents by subsystem and tags each with its language and one of three kinds (reference / design record / plan-history) plus a line on what it answers. A "where to start" section keys the common tasks — adding a builtin, changing dispatch, touching the JIT, anything asynchronous — to the documents that actually cover them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
…eduler-state-diagram-fsi7t4
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1064 +/- ##
==========================================
+ Coverage 91.21% 91.25% +0.04%
==========================================
Files 200 201 +1
Lines 144973 146234 +1261
==========================================
+ Hits 132235 133448 +1213
- Misses 12738 12786 +48 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements
refine/usingend-to-end, plus the lexical-scope anddefinition-site reflection work that had to land first. Across
core/{refinement,module,main,binding,kernel,proc,class,basicobject}andlanguage, failing ruby/spec examples go 137 → 30, with no regressions atany step.
Why refinements were hard here
Refinements make method resolution depend on where the call was written, not
just on the receiver's class. Every cache in the tree — the global method cache
keyed
(name, class_id)+ class version, the VM's per-callsite inline cache,the JIT's
jit_check_method, the class-version repair path, the inlinegenerators, the BOP tables — assumed the callsite's lexical scope was not an
input. Adding refinements naively means either invalidating all of that or,
worse, returning a stale answer silently: the class-version repair path would
have re-resolved a refined callsite without the refinement and cached the
wrong
FuncIdwith no guard to catch it.The design goal was therefore explicit: a program that uses no refinements
must pay nothing. That is checked, not asserted —
--features emit-asmoutputfor a refinement-free workload is byte-identical to the pre-refinements
baseline, verified after both the JIT step and the final step.
How
The activated refinement set is interned to a
RefinementSetId(u32), withEMPTY == 0. That singleu32is what flows through the tree:globals/store/refinement.rs) — a hash-consed pool ofVec<(ClassId, ClassId)>. Activating a set moves an already-active pair tothe front and inserts new pairs at the front, so later
usingwins whileearlier refinements stay reachable; interning sorts stably so equal sets
intern equal. A
refined_names: HashSet<IdentId>and anactive: boolgatethe whole mechanism off for programs that never call
refine.store/class.rs) —search_method_refinedprobes eachancestor position's refinements just before that position's own method table,
which is where CRuby finds them.
store/iseq.rs) —ISeqInfo::refinements, resolved through theouterchain, so a method body carries the set that was active where it waswritten.
InlineCacheEntrygains theRefinementSetId, so the VM inlinecache, the JIT's
JitContext::refinements, and the class-version repair pathall compare scope alongside class. The stale-answer path above is closed by
construction rather than by invalidation.
Refinementitself is a real class (REFINEMENT_CLASS) withrefined_class,refinement_ownerand erased naming, andrefine/using/import_methods/
refinements/used_refinements/used_modules/Module.nestingareimplemented on top of it.
send,respond_to?,methodandsuperseerefinements, matching measured CRuby 4.0.2 behaviour.
Prerequisites that landed in the same series:
Module.nestingand constant resolution now report andresolve from where the code was written, including inside
module_evalandload(path, true).Method#source_locationand friends record the line ofthe
def, not of its first body expression.Binding— the implicit-parameter reflection methods, and anchoring tothe nearest Ruby caller instead of to whichever builtin sits in between.
Fixes found along the way
Array#concatcould abort the process. The accumulator array was notrooted across the
#to_arycall it makes on each argument, so a GC duringcoercion could collect it. Pre-existing; this branch's changed allocation
order made it reproduce. Fixed by holding it on the temp stack
(
d8f12079), and confirmed against the baseline before attributing it here.Known gaps, documented rather than papered over
refine Integer do def +(x) … end) are not seen byeither tier — the BOP fast paths bypass resolution entirely.
usinginside a re-executed block recordsonly the most recent activation.
Both are written up in
doc/refinements.md§6.7 alongside what it would take toclose them.
Documentation
doc/refinements.md(new, 650 lines) — what refinements change, where eachcache layer would have broken, why a partial stub is worse than nothing, the
ruby/spec ledger, the interned-set design, and the frame-layout analysis that
concluded the frame does not have to change.
doc/README.md(new) —doc/had grown to 24 files with no index. Groupsthem by subsystem and tags each with language and kind (reference / design
record / plan-history), since those three age differently.
doc/cref.mdcross-references the above.Testing
cargo test— all 59 test binaries green, after merging currentmaster.regressions measured at each of the four implementation steps.
--features emit-asmbyte-identical to baseline for arefinement-free workload.
mastermoved 16 commits during this work and touches several of the samefiles; it is merged in here and the suite is green on the merge.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Nk3E4n72Q6uyp4hZNN5oUU
Generated by Claude Code