Describe the bug
beam.smp segfaults (general protection fault) in sweep_off_heap during a minor GC while another scheduler is concurrently committing a module load (finish_loading_1 → erts_start_staging_code_ix). The GC'd process's off-heap chain contains garbage pointers whose values resemble x86-64 instruction bytes, suggesting overlap between process heap memory and JIT-generated code memory.
We have hit this 5 times over two weeks in CI while running a large Elixir test suite under coverage (mix test --cover). The workload characteristics:
- ~1,300 application modules are cover-compiled (purged and reloaded) at suite start, and ExUnit loads test files during the run, so module-load traffic is constant while other schedulers execute JIT code.
- The crash consistently lands in test files that churn large numbers of refc binaries in tight loops (thousands of generated names/UUIDs inserted into Postgres), i.e. the processes whose minor GCs sweep the longest off-heap chains.
- It only reproduces on freshly provisioned VMs (GitHub Actions
ubuntu-latest runners): about 1 in 20 cold runs crash, while 80 consecutive repeat runs on already-warmed VMs (same commit, same seed available) produced zero crashes.
This looks like a sibling of (or an incomplete fix for) OTP-19599 ("rare bug that could cause an emulator crash after unloading a module or erasing a persistent_term") — the version we run already contains that fix.
Backtrace
Core captured with matching binaries; gdb run on the same machine.
dmesg:
traps: erts_sched_1[3466] general protection fault ip:55b632c35c54 sp:7ff242da1840 error:0 in beam.smp[435c54,55b632c00000+158000]
Crashing scheduler:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 sweep_off_heap (p=p@entry=0x7ff236ab9f78, fullsweep=fullsweep@entry=0) at beam/erl_gc.c:3182
refc_binary = 0x81a48c0fceff413b
prev = 0x7ff1db6427c0
insert_old_here = 0x7ff236aba180
oheap = 0x7ff1db404fb8 ","
oheap_sz = 54176
bin_vheap = 32
shrink_ncandidates = 0
shrink_nactive = 1
shrink_unresolved_end = 0x7ff1db6427b0
br = 0x8948fe4f0f485f5c
#1 0x000055b632c381ae in do_minor (p=p@entry=0x7ff236ab9f78, live_hf_end=live_hf_end@entry=0xfffffffffffffff8, mature=<optimized out>, mature_size=mature_size@entry=3072, new_sz=4185, objv=objv@entry=0x7ff242da1d40, nobj=1) at beam/erl_gc.c:1825
#2 0x000055b632c3b55a in minor_collection (recl=<synthetic pointer>, ygen_usage=2539, nobj=1, objv=0x7ff242da1d40, need=25, live_hf_end=0xfffffffffffffff8, p=0x7ff236ab9f78) at beam/erl_gc.c:1509
#3 garbage_collect (p=p@entry=0x7ff236ab9f78, live_hf_end=live_hf_end@entry=0xfffffffffffffff8, need=need@entry=21, objv=objv@entry=0x7ff242da1d40, nobj=nobj@entry=1, fcalls=fcalls@entry=3526, max_young_gen_usage=0) at beam/erl_gc.c:803
#4 0x000055b632c3c590 in erts_garbage_collect_nobump (p=0x7ff236ab9f78, need=21, objv=0x7ff242da1d40, nobj=1, fcalls=3526) at beam/erl_gc.c:942
#5 0x00007ff23bbffafd in ?? () <- JIT-generated code (anonymous executable mapping)
#6 0x0000000000000000 in ?? ()
Note the values of refc_binary (0x81a48c0fceff413b) and br (0x8948fe4f0f485f5c): both decode as plausible x86-64 instruction byte sequences rather than heap addresses.
Scheduler concurrently committing a module load at the moment of the crash:
Thread 18 (Thread 0x7ff237bfe6c0 (LWP 3467)):
#0 0x000055b632c2b804 in hash_put (h=h@entry=0x55b63324e940 <export_tables+128>, tmpl=0x7ff1ef2a4be8) at beam/hash.c:242
#1 0x000055b632c2bd5d in index_put_entry (t=t@entry=0x55b63324e940 <export_tables+128>, tmpl=<optimized out>) at beam/index.c:76
#2 0x000055b632c2e1ad in export_start_staging () at beam/export.c:520
#3 0x000055b632b06f8e in erts_start_staging_code_ix (num_new=num_new@entry=1) at beam/code_ix.c:104
#4 0x000055b632aeaf8f in finish_loading_1 (A__p=0x55b637a69918, BIF__ARGS=<optimized out>, A__I=<optimized out>) at beam/beam_bif_load.c:418
#5 0x00007ff23bbff63c in ?? ()
#6 0x0000000000000000 in ?? ()
All other threads are idle (poll/cond waits) or in unrelated syscalls.
To Reproduce
No standalone reproducer yet — statistics from our CI:
mix test --partitions 4 --cover --warnings-as-errors over a ~6,600-test Elixir suite; the crashing partition takes ~5 minutes.
- ~5% of runs crash on a freshly provisioned GitHub Actions VM; 0 of 80 repeat runs crashed on warmed VMs (same commit; one run replayed the exact seed of a previous crash without reproducing, so it is a timing race, not input-determined).
- Loaded NIFs (crypto, asn1, a Rustler NIF) were idle at crash time in the captured core; earlier crashes happened in test code with no NIF involvement on the crashing process.
Expected behavior
No emulator crash.
Affected versions
- Erlang/OTP 27.3.4.15 (erts 15.2.7.11), x86_64-unknown-linux-gnu, JIT flavor, from the erlef/setup-beam builds. Elixir 1.19.5.
- Observed on Ubuntu 24.04 (GitHub
ubuntu-latest) runners.
- First observed 2026-08-08; we cannot say whether older versions are affected (the suite/workload is recent).
Additional context
We kept the full 800 MB core file, the exact beam.smp, and the loaded .so files, and can share them privately or run further gdb/etp-macro analysis on the core on request.
Describe the bug
beam.smpsegfaults (general protection fault) insweep_off_heapduring a minor GC while another scheduler is concurrently committing a module load (finish_loading_1→erts_start_staging_code_ix). The GC'd process's off-heap chain contains garbage pointers whose values resemble x86-64 instruction bytes, suggesting overlap between process heap memory and JIT-generated code memory.We have hit this 5 times over two weeks in CI while running a large Elixir test suite under coverage (
mix test --cover). The workload characteristics:ubuntu-latestrunners): about 1 in 20 cold runs crash, while 80 consecutive repeat runs on already-warmed VMs (same commit, same seed available) produced zero crashes.This looks like a sibling of (or an incomplete fix for) OTP-19599 ("rare bug that could cause an emulator crash after unloading a module or erasing a persistent_term") — the version we run already contains that fix.
Backtrace
Core captured with matching binaries; gdb run on the same machine.
dmesg:
Crashing scheduler:
Note the values of
refc_binary(0x81a48c0fceff413b) andbr(0x8948fe4f0f485f5c): both decode as plausible x86-64 instruction byte sequences rather than heap addresses.Scheduler concurrently committing a module load at the moment of the crash:
All other threads are idle (poll/cond waits) or in unrelated syscalls.
To Reproduce
No standalone reproducer yet — statistics from our CI:
mix test --partitions 4 --cover --warnings-as-errorsover a ~6,600-test Elixir suite; the crashing partition takes ~5 minutes.Expected behavior
No emulator crash.
Affected versions
ubuntu-latest) runners.Additional context
We kept the full 800 MB core file, the exact
beam.smp, and the loaded.sofiles, and can share them privately or run further gdb/etp-macro analysis on the core on request.