Skip to content

Commit c9ec45a

Browse files
docs(vrs): fix experiment record shape
The determinism experiment landed in ad0a800 with freeform Question/ Method/Result sections, but the enforced shape (VRS.ENF.experiment-shape) requires Question/Method/Result/Conclusion/VRS Impact. The check failed and was pushed anyway — a `| head` in the verify pipeline masked its non-zero exit. Restructured to the enforced headings; strict check now clean. Content unchanged: a plan's canonical serialization is byte-identical across x86_64-linux, aarch64-linux, and aarch64-darwin. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QzRc44KbA3wfYYKFF9UDMa agent-session-id: cdf2c185-eeb3-4a4d-8ce0-52885f145cac agent-tool: Claude Code agent-tool-version: 2.1.215 agent-model: claude-opus-4-8 agent-runtime-profile: /nix/store/acr8a3l2v366jgmwiq8xdrhgz1py0db5-coding-agent-runtime-profile/share/coding-agents/profile.json agent-skills-manifest: /nix/store/sj1v5j91h8v8d1w9lca4040302lwrd6v-agent-skills-corpus/share/agent-skills/manifest.json tooling-profile: dotfiles@unknown-dirty
1 parent ad0a800 commit c9ec45a

1 file changed

Lines changed: 29 additions & 27 deletions

File tree

context/.experiments/2026-07-25-cross-platform-evaluation-determinism.md

Lines changed: 29 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,14 @@ Validates the property that a plan's identity is the same on every platform —
66
the basis for "same source produces the same version everywhere", which several
77
requirements rest on and which had been an inference rather than a measurement.
88

9-
## Hypothesis
9+
## Question
1010

11-
Evaluating a plan produces byte-identical output on x86_64-linux,
12-
aarch64-linux, and aarch64-darwin, because a plan is pure data construction and
13-
the only platform-variable surface identified in earlier research —
14-
libc-delegated transcendental Math — is not reached by a plan and is removed
15-
from the evaluation global regardless.
11+
Does evaluating a plan produce byte-identical output across x86_64-linux,
12+
aarch64-linux, and aarch64-darwin? A plan is pure data construction, and the only
13+
platform-variable surface earlier research identified — libc-delegated
14+
transcendental Math — is neither reached by a plan nor present in the evaluation
15+
global. If the property holds, "same source produces the same version everywhere"
16+
is measured rather than assumed.
1617

1718
## Method
1819

@@ -43,19 +44,17 @@ The serialized-plan line was hashed on each and compared.
4344
## Result
4445

4546
The canonical plan serialization was **byte-identical on all three platforms**
46-
(SHA-256 `5160e9b3…`). The hypothesis holds: a plan's identity does not depend on
47-
the platform it is evaluated on.
47+
(SHA-256 `5160e9b3…`).
4848

4949
The negative control was more interesting than expected. Every transcendental —
5050
`sin`, `cos`, `exp`, `log`, `pow`, `tanh` — was **also identical across all
5151
three**, including on Apple libm. The predicted glibc-vs-Apple-libm divergence
5252
did not manifest for these inputs on QuickJS-ng 0.14.0.
5353

54-
## Interpretation
54+
## Conclusion
5555

56-
The load-bearing conclusion is unaffected and now measured rather than assumed:
57-
plan identity is platform-independent, because a plan constructs data and touches
58-
none of the platform-variable surface.
56+
Plan identity is platform-independent, now measured rather than assumed: a plan
57+
constructs data and touches none of the platform-variable surface.
5958

6059
The control result is a genuine discrepancy with the earlier research, which read
6160
QuickJS-ng master delegating every transcendental straight to libc and cited
@@ -67,18 +66,21 @@ platforms in general — only that they did here.
6766

6867
The Math lock therefore stands as insurance rather than a demonstrated-necessary
6968
fix: primary sources still show some inputs diverge, a plan has no reason to call
70-
these functions, and removing them costs three lines. What this experiment
71-
establishes is that the lock is not load-bearing for plan identity — identity is
72-
safe because plans do not do arithmetic of this kind at all, not because the lock
73-
catches a divergence that was about to corrupt a hash.
74-
75-
## Threats to validity
76-
77-
- One probe, one set of Math inputs. A wider input sweep could still surface the
78-
predicted transcendental divergence; this experiment did not attempt one,
79-
because transcendentals are outside what a plan evaluates.
80-
- `qjs` is the standalone interpreter, not the embedded `rquickjs` the tool will
81-
use. They share the QuickJS-ng core, so engine-level number and string
82-
behaviour is the same, but the embedding has not itself been run cross-platform.
83-
- Darwin coverage is aarch64 only; no x86_64-darwin was tested, and none is in
84-
the fleet.
69+
these functions, and removing them costs three lines. What this establishes is
70+
that the lock is not load-bearing for plan identity — identity is safe because
71+
plans do not do arithmetic of this kind at all, not because the lock catches a
72+
divergence that was about to corrupt a hash.
73+
74+
Threats to validity: one probe and one set of Math inputs, so a wider sweep could
75+
still surface the predicted transcendental divergence; `qjs` is the standalone
76+
interpreter rather than the embedded `rquickjs` the tool will use, though they
77+
share the QuickJS-ng core; and darwin coverage is aarch64 only, with no
78+
x86_64-darwin in the fleet.
79+
80+
## VRS Impact
81+
82+
Confirms CMP-A03 (evaluation is reproducible), which required testing rather than
83+
assumption; the requirement now cites this record. No change to any decision. The
84+
Math-lock provisions of 0011 are unchanged — reclassified in rationale from
85+
necessary fix to insurance, which is a matter of record here rather than a change
86+
to the decision.

0 commit comments

Comments
 (0)