Fix llvmize CI: promote test outputs + use an LLVM release with llvm-project#37#1
Open
julesjacobs wants to merge 13 commits into
Open
Conversation
Add an Xmeasure_frametables GC tweak (OCAMLRUNPARAM=Xmeasure_frametables=1) that walks the frametables registered in each batch and reports descriptor, header and debuginfo sizes together with register/slot statistics, including a per-register usage count.
The runtime can register frametables lazily (e.g. for dynamically linked units), so OCAMLRUNPARAM=Xmeasure_frametables only sees those registered so far. Emit a per-unit pointer into a linker-collected caml_all_frametables section (ELF only) and add an Xmeasure_all_frametables tweak that walks every frametable linked into the executable, registered or not, reading only static data. The hand-written runtime frametable contributes itself to the section. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Frame tables will soon be packed without inter-descriptor alignment, so the runtime must not assume frame descriptors are aligned. Treat a [frame_descr *] as an opaque byte pointer and read each field at an explicit byte offset via new caml_read_unaligned_* helpers, instead of through the frame_descr / frame_descr_long C structs (which are removed). Forming an aligned-typed pointer to unaligned data would be undefined behaviour. No on-disk format change yet: descriptors are still emitted aligned; only the way the runtime reads them changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Now that every runtime read of a frame descriptor is alignment-agnostic, stop aligning the frame tables: the compiler no longer emits inter-descriptor or internal padding (emitaux), the hand-written runtime frame table drops its .align directives (amd64.S), and the runtime frame-table walk no longer rounds up to 32-bit/word boundaries. Backtrace slots are shift-encoded (the descriptor address is stored shifted, with the low bits used as a tag) so they no longer rely on descriptors being aligned. This packs the descriptors: on ocamlopt.opt the average descriptor shrinks from 21.8 to 18.8 bytes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The debuginfo filename and defname strings were emitted inline, once per compilation unit. Split the defname out of the name_info / name_and_loc_info structs into a [defname_offs] field (alongside the existing [filename_offs]), and emit both the filename and defname strings into a mergeable string section (SHF_MERGE|SHF_STRINGS, ".rodata.str1.1") so the linker deduplicates identical strings across all units. The name structs stay in the frame table section, referencing the strings by signed 32-bit offset. Because the strings now live in a shared, deduplicated section, they can no longer be attributed to a single frametable, so the measurement no longer counts them: caml_debuginfo_measure brackets only the name structs and counts records (each two 32-bit words); the frametable measurement reports records*8 + struct-span as the debuginfo size. On ocamlopt.opt this drops the measured debuginfo from 8.50 to 6.84 MiB and, in the linked executable, deduplicates the physical strings across units. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The compact frame-descriptor format will store each return address as a delta from the previous one, which requires the descriptors in a frame table to be ordered by increasing return address. Record them in that order: call frames are recorded inline as they are emitted, and allocation/poll frames are recorded when their out-of-line GC stub is emitted (where the return-address label is about to be defined) rather than at the allocation site. The frame descriptor list is built by prepending, so emit_frames reverses it. Done for both AMD64 and ARM64. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a denser per-descriptor encoding, chosen by the assembler per descriptor:
* A leading ULEB128 byte sequence holds the return-address delta from the
previous descriptor (requiring the increasing-address ordering established
earlier). A leading 0 byte (FRAME_DELTA_ESCAPE) means "escape": the bytes
that follow are a descriptor in the existing normal/long format.
* A short descriptor stores a 6-bit frame size (in 16-byte units) plus flags,
separate num_allocs/num_live bytes, 4-bit alloc-size nibbles, a 1-byte
hot-register bitmap over the eight commonest GC-root registers, and live
stack-slot word offsets. Descriptors that do not fit (large/odd frame size,
cold register, too many live slots, first-in-table, text-section boundary,
...) escape to the normal/long format.
The runtime gains caml_decode_frame_descr to decode either form, and the GC
stack scan, backtrace, signal handling and the frametable measurement all go
through it. The hash table is keyed on {retaddr, fd}.
The format is gated off (every descriptor escapes) where the delta cannot be
emitted: the internal assembler / JIT (which cannot resolve the variable-length
delta directive) and ARM64 (not yet ported), via
Emitaux.disable_short_descriptors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Classify each escaped (non-short) descriptor by why it could not use the short format, mirroring short_encoding in backend/emitaux.ml (same checks, same order). This separates descriptors that escaped only for a positional reason (first-in-frametable, or a function / text-section boundary; their content is short-compatible) from genuine non-fits (cold register, oversized or misaligned frame, too many live slots, an allocation size that overflows a 4-bit nibble, ...). Reported as a per-reason histogram alongside the short/medium/long kind counts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d knows to emit compatible frametables.
…ll frametables" feature on MacOS.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The short-frametable format needs LLVM's OxCamlGCPrinter to emit the escape delta byte and drop descriptor padding (ocaml-flambda/llvm-project#37); the oxcaml-llvmize-16.0.6-minus3 release predates that, so the llvmize tests segfault. Temporarily point the llvmize job at a personal-fork release built from #37's head. Once #37 is merged and an official release exists, drop oxcaml_llvm_repo and bump oxcaml_llvm_tag instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NickBarnes
approved these changes
Jul 6, 2026
NickBarnes
force-pushed
the
frametable-rearrange
branch
7 times, most recently
from
July 14, 2026 08:53
696cb17 to
1992279
Compare
NickBarnes
force-pushed
the
frametable-rearrange
branch
3 times, most recently
from
July 15, 2026 13:58
9b03b5a to
244fb79
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on
frametable-rearrange(oxcaml#6399) — fixes theframe-pointers llvmize-tests (x86_64-linux)CI job, which currently fails in two ways:Stale expected outputs:
id_fn.outputanddls_get.outputwere missing the newoxcaml_short_frametablesmodule flag that llvmize now emits (all the*_ir.outputfiles were already updated). Promoted.SEGVs in all tests that execute LLVM-compiled code: CI downloads the
oxcaml-llvmize-16.0.6-minus3LLVM release, which predates Add oxcaml_short_frametables code for compatibiility with short frametables ocaml-flambda/llvm-project#37 — so LLVM emits classic pointer-aligned frame descriptors while the new runtime walks packed descriptors with delta bytes, and misparses the table.For (2), this points the llvmize job at a temporary release built from ocaml-flambda/llvm-project#37's head (commit 8c50b62e33, tag
oxcaml-llvmize-16.0.6-minus3-pr37on my fork; clang/llc, Release, X86, built on glibc 2.28). Once llvm-project#37 is merged and an official release is minted, drop theoxcaml_llvm_repoline and bumpoxcaml_llvm_tag.Verification
Built LLVM from llvm-project#37 and this branch locally;
make runtest-llvmizewith that clang: the six SEGVs (alloc,exn,switch,csel,extcalls,data_decl) are gone and the whole suite passes after the two promotions. Also checked a linked test binary: the LLVM-emittedcamlAlloc__frametablecontains escape-byte (0x00) descriptors, byte-packed with no padding, matchingcaml_decode_frame_descrexactly.🤖 Generated with Claude Code