Namespace-scope variables with dynamic initializers get wrong values when defined in separate transactions (cling 1.2)

[rokabytedev]

Edit (2026-08-17): the mechanism in my first follow-up comment was wrong, and Observation 3 below is inaccurate about const. See the correction comment — the root cause appears to be a __cxx_global_var_init symbol-name collision, and it looks already fixed in 1.3+.

Summary

When two or more namespace-scope variables with dynamic (runtime) initializers are defined in separate inputs, the values read back afterwards are wrong: the first variable holds the value produced by the last initializer, and every subsequent variable reads back as 0.

The initializers themselves all run — a side-effect counter shows exactly the expected number of calls — so the storage that later inputs read does not appear to be the storage the initializers wrote to.

Defining the same variables within a single input is correct, which gives a clean A/B. Upstream clang-repl at the same LLVM version (18.1.8) does not reproduce it.

Reproducer

Enter one line at a time at the cling prompt:

#include <cstdio>
long long n = 0;
long long b(){ return ++n; }
long long P = b();
long long Q = b();
printf("P=%lld Q=%lld n=%lld\n", P, Q, n);

Expected	P=1 Q=2 n=2
Actual	P=2 Q=0 n=2

(Program output goes to stderr, so when scripting this use cling 2>&1 <<'EOF' ... EOF.)

Observations

1. Control — the same definitions in a single input are correct.

long long P = b(); long long Q = b(); printf("P=%lld Q=%lld n=%lld\n", P, Q, n);

→ P=1 Q=2 n=2 ✔

2. It scales with the number of definitions. Three definitions, each its own input:

→ P=3 Q=0 R=0 n=3 (expected 1 2 3)

3. Not specific to auto, to const, or to any library type. Plain int with explicit types, still one definition per input:

#include <cstdio>
int m = 0;
int f(){ return ++m; }
int A = f();
int B = f();
printf("A=%d B=%d m=%d\n", A, B, m);

→ A=2 B=0 m=2 (expected 1 2 2)

4. Does not reproduce with upstream clang-repl at the same LLVM/Clang version cling 1.2 is built on, in the same container with the same libraries and the identical input:

cling 1.2         →  P=2 Q=0 n=2   (wrong)
clang-repl 18.1.8 →  P=1 Q=2 n=2   (correct)

5. Block-scope and function-local declarations are unaffected. In the same session, wrapping the same code in { ... } or in a function gives correct results, so this looks specific to namespace scope across separate transactions.

Practical impact

The most likely way to run into this is the ordinary "take two timestamps and subtract" idiom, which is natural to type as two separate lines:

#include <chrono>
#include <cstdio>
auto t0 = std::chrono::steady_clock::now();
auto t1 = std::chrono::steady_clock::now();
printf("t0=%lld t1=%lld\n", (long long)t0.time_since_epoch().count(), (long long)t1.time_since_epoch().count());

→ t0=1196998825664793 t1=0

so t1 - t0 silently yields a large negative duration. There is no diagnostic, and the failure is easy to misattribute to the timing code itself.

Environment

cling 1.2                       (conda-forge, cling-1.2-h48f18f5_1)
clang / LLVM 18.1.8             (conda-forge, cling_1.2 build)
Debian GNU/Linux 13 (trixie), x86_64

Reproduced identically in two environments that differ only in the C++ runtime (libstdcxx 14.1.0 and 16.1.0), so the behaviour does not appear to depend on the standard library version.

Caveat

Both environments print this at startup:

Warning in cling::IncrementalParser::CheckABICompatibility():
  Possible C++ standard library mismatch, compiled with __GLIBCXX__ '20240521'
  Extraction of runtime standard library version was: '20250605'

I could not get rid of it with the conda-forge packages. I don't think it explains the behaviour, because (a) the reproducer uses no C++ standard library types at all, (b) two environments with different libstdcxx versions produce identical results, and (c) function-scope declarations in the same session are correct. Flagging it in case it is relevant.

Tested by piping to the cling binary; I have not verified this against xeus-cling / Jupyter.

[rokabytedev]

I dug a bit further to narrow down where this goes wrong. Short version: the generated IR looks correct at every level, and the addresses are stable — but each new dynamic initializer writes into the storage of the first dynamically-initialized variable.

Step-by-step trace

Reading all variables back after each definition makes the pattern obvious:

#include <cstdio>
long long n = 0;
long long b(){ return ++n; }
long long P = b();
printf("STEP1 P=%lld n=%lld\n", P, n);
long long Q = b();
printf("STEP2 P=%lld Q=%lld n=%lld\n", P, Q, n);
long long R = b();
printf("STEP3 P=%lld Q=%lld R=%lld n=%lld\n", P, Q, R, n);

STEP1 P=1                n=1      <- fine
STEP2 P=2  Q=0           n=2      <- Q's initializer landed in P
STEP3 P=3  Q=0  R=0      n=3      <- R's initializer landed in P as well

P keeps being overwritten by each subsequent initializer, while Q and R are never written at all. The side-effect counter n is always correct, so every initializer does execute exactly once.

The addresses are distinct and stable

ADDR &P=0x706ca1e51000 &Q=0x706ca1e4b000 &R=0x706ca1e48000 &n=0x706ca1e8c000

Taking the address of a variable from the input that defines it and from any later input yields the same value, and the three variables occupy distinct pages. So this does not look like two symbols being mapped onto one address.

The IR appears correct

With .printDebug 1, each definition gets its own inline namespace and a unique mangled symbol, defined exactly once:

@_ZN11__cling_N501nE = dso_local global i64 0, align 8   ; n
@_ZN11__cling_N521PE = dso_local global i64 0, align 8   ; P
@_ZN11__cling_N531QE = dso_local global i64 0, align 8   ; Q

each module storing to its own symbol:

; module for P
store i64 %1, ptr @_ZN11__cling_N521PE, align 8
define weak dso_local void @_GLOBAL__sub_I_cling_module_4() ...

; module for Q
store i64 %1, ptr @_ZN11__cling_N531QE, align 8
define weak dso_local void @_GLOBAL__sub_I_cling_module_5() ...

and the reading transaction referencing exactly the right externals:

@_ZN11__cling_N521PE = external global i64, align 8
@_ZN11__cling_N531QE = external global i64, align 8
@_ZN11__cling_N501nE = external global i64, align 8

%3 = load i64, ptr @_ZN11__cling_N521PE, align 8
%4 = load i64, ptr @_ZN11__cling_N531QE, align 8
%5 = load i64, ptr @_ZN11__cling_N501nE, align 8

So the AST handling, the namespace shadowing and the code generation all seem to produce what one would expect.

Where this seems to leave the problem

Since the loads resolve correctly and the addresses are stable, but the stores performed from each module's _GLOBAL__sub_I_cling_module_N end up in the first such variable's storage, the discrepancy appears to sit between the emitted module and what the initializer actually writes to at run time, rather than in the AST/CodeGen path.

One consistent observation: variables with a constant initializer (long long n = 0;) are unaffected, and those are exactly the ones with no _GLOBAL__sub_I_cling_module_N entry. Only variables with dynamic initializers show the problem.

I have not built cling from source, so I could not take this further; I hope the above is a useful starting point. Happy to run further experiments on this setup if that would help.

[rokabytedev]

Correction — my earlier mechanism was wrong, and this looks already fixed in 1.3+

I put the previous comment through a proper adversarial re-check and it does not survive. Corrections below, plus what I now believe is the actual root cause. Apologies for the noise.

1. The mechanism I described was wrong

I claimed each new initializer writes into the storage of the first dynamically-initialized variable, and that every initializer runs exactly once. Both are false. My mistake was using a single shared counter n, which cannot distinguish "three different initializers each ran once" from "one initializer ran three times".

Giving each initializer its own counter settles it immediately:

#include <cstdio>
long long np=0, nq=0, nr=0;
long long bp(){ return ++np; }
long long bq(){ return ++nq; }
long long br(){ return ++nr; }
long long P = bp();
long long Q = bq();
long long R = br();
printf("P=%lld Q=%lld R=%lld | np=%lld nq=%lld nr=%lld\n", P,Q,R,np,nq,nr);

P=3 Q=0 R=0 | np=3 nq=0 nr=0

bp runs three times; bq and br never run at all. So nothing is written to the wrong address — the wrong initializer function is executed, repeatedly, and the later variables are simply never initialized. Their storage is fine: manually assigning to Q sticks.

I also wrote "the generated IR appears correct at every level". That was wrong too — the defect is visible in the IR, in the part I omitted.

2. What actually happens

The store is not in _GLOBAL__sub_I_cling_module_N as I wrote; it is in a helper that _GLOBAL__sub_I_cling_module_N calls, and that helper has the same name in every transaction module:

; module 5
define weak dso_local void @__cxx_global_var_init() #0 {
  %1 = call noundef i64 @_ZN11__cling_N512bpEv()
  store i64 %1, ptr @_ZN11__cling_N531PE, align 8
}
define weak dso_local void @_GLOBAL__sub_I_cling_module_5() #0 {
  call void @__cxx_global_var_init()
}

; module 6  -- same name, now discarded
define available_externally dso_local void @__cxx_global_var_init() #0 {
  %1 = call noundef i64 @_ZN11__cling_N522bqEv()
  store i64 %1, ptr @_ZN11__cling_N541QE, align 8
}
define weak dso_local void @_GLOBAL__sub_I_cling_module_6() #0 {
  call void @__cxx_global_var_init()      ; binds to module 5's definition
}

__cxx_global_var_init is internal in a normal TU, but KeepLocalGVPass promotes it to weak so the JIT will not error on duplicate symbols. Every transaction module then exports a weak symbol with the identical name, the first definition wins, and later ones become available_externally and are dropped. So _GLOBAL__sub_I_cling_module_6's call resolves to module 5's initializer — the one for P.

That accounts for every observation, including the ones that made me reach for a memory-corruption story: the variable symbols really are unique and correctly resolved, the addresses really are distinct and stable, constant-initialized variables really are unaffected (no __cxx_global_var_init is emitted for them), and clang-repl at the same LLVM version really is fine.

3. This appears to be already fixed in 1.3+

BackendPasses.cpp in master contains UniqueInitFunctionNamePass, which does exactly the needed renaming:

// make static initialization function names (__cxx_global_var_init) unique
if (F.hasName() && F.getName().starts_with("__cxx_global_var_init")) {
  F.setName(add_module_suffix(F.getName(), ModuleName));

and it is ordered before ReuseExistingWeakSymbols in the pipeline. Grepping a v1.2 source tree for __cxx_global_var_init in BackendPasses.cpp gives 0 hits; master gives 5. This seems to be 1d49255 ("Move static init function renaming to BackendPasses.cpp").

A version bisect on conda-forge packages agrees:

cling 1.0   ->  P=1 Q=2  np=1 nq=1     correct
cling 1.1   ->  P=1 Q=2  np=1 nq=1     correct
cling 1.2   ->  P=2 Q=0  np=2 nq=0     broken

and the IR difference is exactly the renaming:

1.0 / 1.1 :  define weak dso_local void @__cxx_global_var_initcling_module_5_
             define weak dso_local void @__cxx_global_var_initcling_module_6_
1.2       :  define weak dso_local void @__cxx_global_var_init
             define available_externally dso_local void @__cxx_global_var_init

So 1.2 built against stock upstream LLVM appears to be the window where neither the old mechanism nor the new pass was in place. If that reading is right, the remaining questions are whether a regression test is wanted (I could not find one covering two dynamically-initialized namespace-scope variables defined in separate inputs — test/Prompt/initorder.C is the closest but never reads the second variable back from a later transaction), and that conda-forge is still shipping 1.2.

4. Two disclosures I should have made

The conda-forge build enables definition shadowing by default. Upstream defaults to AllowRedefinition(0), but on this package int i = 1; followed by double i = 3.14; is accepted, so it is patched on. That is where the __cling_N5<N> inline namespaces in my IR dumps come from, and I presented that output as stock cling. I believe it is a red herring here, since the collision is on __cxx_global_var_init, whose name does not depend on shadowing — but I have not proven that with AllowRedefinition = 0 and should not have left it implicit.

Observation 3 in the issue body is inaccurate about const. Written as "not specific to auto, to const, or to any library type" it invites the reading that const also reproduces. It does not, in a clean session:

const long long P = b(); const long long Q = b();  ->  P=1 Q=2   correct
const auto      P = b(); const auto      Q = b();  ->  P=1 Q=2   correct
long long       P = b(); long long       Q = b();  ->  P=2 Q=0   broken

What Observation 3 does establish is that neither auto nor <chrono> is required — plain non-const int/long long reproduces. Constness does appear to matter.

5. What stands

The symptom, the single-input control, the version/environment details, the clang-repl A/B, and the <chrono> impact case all reproduce unchanged. The piped-stdin methodology also holds up — I re-ran the reproducer under a real pty via script(1), including with a slow drip-feed, with identical results, and .printDebug 0 vs 1 makes no difference.

Happy to write this up as a lit test in the house style if that would be useful, and to re-run anything against a from-source build if someone can point me at a working recipe.