@@ -5,9 +5,9 @@ and the measurable parts of `NFR-1`. It uses
55` org.jetbrains.kotlinx.benchmark ` 0.4.17 on JVM and the host Kotlin/Native
66target. Reports use average wall time in ` ms/op ` .
77
8- No performance number is claimed in this repository until a report has
9- actually been produced. In particular, the checked- in external-comparison
10- file is explicitly ` unmeasured ` .
8+ No performance number is claimed from a checked-in placeholder. CI creates
9+ the external-comparison report from both runtimes in one benchmark process
10+ and preserves the raw and normalized evidence .
1111
1212## Workloads
1313
@@ -90,34 +90,46 @@ Run the deterministic gate-tool tests without executing benchmarks:
9090
9191No third-party benchmark binary is vendored. Repositories and immutable
9292commits for Sightglass methodology, EEMBC CoreMark, Chasm, and Chicory are in
93- ` upstreams.lock.json ` . A CoreMark asset needs a separate license/provenance
94- review and must be a freestanding module for the current seam. The dedicated
95- external profile runs only CoreMark:
93+ ` upstreams.lock.json ` . EEMBC distributes CoreMark under Apache-2.0 together
94+ with its acceptable-use and result-disclosure terms. The comparison uses the
95+ exact ` coremark.wasm ` fixture shipped by Chasm 1.4.8, records the commit that
96+ introduced it, and rejects any bytes whose SHA-256 is not the locked value.
97+
98+ Prepare the immutable Chasm checkout:
99+
100+ ``` shell
101+ export KWASM_COREMARK_WASM=" $( scripts/prepare-benchmark-upstreams.sh) "
102+ ```
103+
104+ Run CoreMark alone:
96105
97106``` shell
98107KWASM_COREMARK_WASM=/absolute/path/coremark.wasm \
99108KWASM_COREMARK_EXPORT=run \
100109./gradlew :benchmarks:jvmExternalBenchmark
101110```
102111
103- Setting the same environment variables while running ` jvmPerformanceGate `
104- also adds CoreMark to the default report, allowing a complete Chasm geomean
105- comparison to activate.
106-
107- Chasm (JVM and Native) and Chicory interpreter/compiler (JVM) measurements
108- are imported through
109- ` baselines/external-comparisons.schema.json ` . Supply that file to the gate:
112+ For the NFR-1 comparison, ` externalComparison ` runs kwasm and the pinned
113+ Chasm KMP interpreter against the same module bytes and arguments in one
114+ process. It covers fib(35), SHA, JSON, and CoreMark on JVM and every declared
115+ Native benchmark target. The report task verifies the CoreMark checksum and
116+ records both scores, benchmark names, target, machine, command, UTC timestamp,
117+ and Chasm commit:
110118
111119``` shell
120+ ./gradlew :benchmarks:jvmExternalComparisonReport
121+
112122./gradlew :benchmarks:jvmPerformanceGate \
113- -Pkwasm.benchmark.externalComparisons=/path/to/measured -comparisons.json
123+ -Pkwasm.benchmark.externalComparisons=benchmarks/build/performance/external -comparisons-jvm .json
114124```
115125
116- Chicory ratios are informational. Same-target Chasm rows for fib, SHA, JSON,
117- and CoreMark activate the ` NFR-1 ` ≤2.5× geomean gate. A partial set is
118- reported as ` partial ` and is not treated as proof of the requirement. Missing
119- records remain ` unmeasured ` ; they never become a zero, synthetic baseline, or
120- pass.
126+ Replace ` jvm ` with ` macosArm64 ` , ` macosX64 ` , ` linuxArm64 ` , or ` linuxX64 ` for
127+ the per-target Native tasks. Same-target Chasm rows for all four workloads
128+ activate the fatal ` NFR-1 ` ≤2.5× geomean gate. A partial set is reported as
129+ ` partial ` and is not treated as proof. Missing records remain ` unmeasured ` ;
130+ they never become a zero, synthetic baseline, or pass. The schema still
131+ accepts separately collected Chicory interpreter/compiler rows as
132+ informational JVM context.
121133
122134For comparable runs, use an idle dedicated runner, fixed power/performance
123135settings, the same commit and benchmark profile, and preserve the raw report,
@@ -131,3 +143,29 @@ The performance workflow separately builds the real JVM smoke application
131143with GraalVM ` native-image --no-fallback ` and executes it. JMH itself is not
132144used as the native-image proof because its reflective discovery machinery
133145would test JMH configuration rather than kwasm's zero-reflection runtime.
146+
147+ ## iOS incremental footprint
148+
149+ ` NFR-2 ` is enforced with two application-style Kotlin/Native release
150+ frameworks and two final iOS arm64 Mach-O executables:
151+
152+ - the baseline executes a real ` runBlocking ` /` yield ` path, so Kotlin and
153+ ` kotlinx.coroutines ` are present in both outputs;
154+ - the core probe internally decodes, validates, instantiates, and executes a
155+ Wasm function returning ` i32.const 42 ` ;
156+ - ` :core ` is an implementation dependency, not exported through the
157+ Objective-C API;
158+ - both release frameworks use Kotlin/Native ` smallBinary=true ` and
159+ ` latin1Strings=true ` ;
160+ - both final executables use linker dead-code elimination and are stripped
161+ before their byte sizes are compared.
162+
163+ Run the macOS/Xcode-only gate with:
164+
165+ ``` shell
166+ bash scripts/verify-ios-core-footprint.sh
167+ ```
168+
169+ The machine-readable evidence is written to
170+ ` build/ios-footprint/report.json ` . The gate requires the core executable minus
171+ the baseline executable to be at most 1,500,000 bytes.
0 commit comments