Skip to content

Commit afddad6

Browse files
adulbrichadulbrich
committed
fix(wasm): actually enable exception catching, and add CI that proves it
add_compile_options only affects targets created *after* it, and the if(EMSCRIPTEN) block sits below add_library/add_executable -- so -fexceptions was silently doing nothing. LibRaw signals every I/O and format error by throwing and catches in ~78 places, so with emcc's default (catching disabled) each of those catches became abort(): a corrupt or non-RAW file killed the module instead of returning an error the app could show. Switched to target_compile_options/target_link_options on `raw` with PUBLIC, which is order-independent and propagates to every tool linking it. Native and wasm now agree exactly on the error path -- "Cannot open <f>: Input/output error", exit 2 -- for both plain junk and a truncated CR2. Numerically nothing changed: re-ran the reference bracket and every frame is byte-for-byte what it was before the fix. This was found by negative control rather than by reading, which is also why the CI added here leads with it. Built without the flag, the check fails with RuntimeError: Aborted(undefined) ... at ___cxa_throw so it has teeth rather than merely passing. The browser job drives the same error path, which exercises the whole embedding contract -- instantiation, MEMFS staging, INVOKE_RUN being off, callMain's exit code, exception catching -- with no RAW fixture needed. CI deliberately does not claim numerical parity. The one place wasm and native disagree is AHD's cube-root LUT, and reaching it takes a real Bayer mosaic; the reference brackets live outside this repo. Said plainly in the workflow so a green tick is not misread. FORK.md records the first browser run on a real CR2: byte-identical to the NODERAWFS build, 266 MiB peak (6.5% of the wasm32 ceiling), and staging the 26 MB input grew the heap by nothing at all, since MEMFS keeps file bytes outside linear memory. Refs radiantlab/LumiLab#237
1 parent aef029a commit afddad6

6 files changed

Lines changed: 352 additions & 11 deletions

File tree

‎.github/scripts/browser-smoke.html‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
<!doctype html>
2+
<meta charset="utf-8">
3+
<title>LibRaw wasm browser smoke test</title>
4+
<pre id="log"></pre>
5+
<script type="module">
6+
// Instantiates the browser build of dcraw_emu and drives it down the error
7+
// path. That is deliberate rather than a compromise: no RAW fixture is needed
8+
// to exercise the whole embedding contract, which is what this test is about.
9+
//
10+
// - the module instantiates at all
11+
// - FS is reachable from JS, so input can be staged before main() runs
12+
// - INVOKE_RUN is off, so main() has not already run by now
13+
// - callMain returns the process exit code
14+
// - exception catching is linked in, so a bad file returns rather than aborts
15+
//
16+
// Numerical correctness is not in scope here; see FORK.md.
17+
const el = document.getElementById("log");
18+
const out = (s) => { el.textContent += s + "\n"; };
19+
const result = { ok: false };
20+
21+
try {
22+
const mod = (await import("./lib/dcraw_emu.js")).default;
23+
const M = await mod();
24+
25+
M.FS.mkdir("/w");
26+
M.FS.chdir("/w");
27+
28+
// If INVOKE_RUN were on, main() would already have run at instantiation --
29+
// before this file existed -- and the run below would be the second call.
30+
M.FS.writeFile("/w/junk.bin", new TextEncoder().encode("not a raw file"));
31+
out("staged " + M.FS.readFile("/w/junk.bin").length + " bytes into MEMFS");
32+
33+
// No shell in wasm: close fd 2 and reopen, since Emscripten hands out the
34+
// lowest free descriptor. dcraw_emu writes its diagnostic to stderr.
35+
M.FS.close(M.FS.streams[2]);
36+
M.FS.open("/w/err.txt", "w");
37+
38+
const code = M.callMain([
39+
"-T", "-o", "1", "-W", "-j", "-q", "3",
40+
"-g", "2", "0", "-t", "0", "-b", "1.1",
41+
"-Z", "/w/out.tiff", "/w/junk.bin",
42+
]);
43+
out("exit code: " + code);
44+
45+
// FS must still be usable here: EXIT_RUNTIME=1 tears the runtime down when
46+
// main returns, and if it took the filesystem with it there would be no way
47+
// to retrieve output at all.
48+
const stderr = new TextDecoder().decode(M.FS.readFile("/w/err.txt")).trim();
49+
out("stderr: " + stderr);
50+
51+
const peakMiB = (M.HEAPU8.length / 1048576).toFixed(1);
52+
out("heap high-water mark: " + peakMiB + " MiB");
53+
54+
Object.assign(result, {
55+
// A clean rejection is the pass condition. An abort would have thrown out
56+
// of callMain and landed in the catch below instead.
57+
ok: code !== 0 && stderr.includes("Cannot open"),
58+
code, stderr, peakMiB,
59+
});
60+
} catch (e) {
61+
out("ERROR: " + (e && e.stack ? e.stack : e));
62+
result.error = String(e);
63+
}
64+
window.__smoke = result;
65+
</script>

‎.github/scripts/browser-smoke.mjs‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
// Drive browser-smoke.html in headless Chromium and fail the job if the browser
2+
// build does not run. Exists because the browser link configuration has no
3+
// static check: INVOKE_RUN left on, or FS/callMain not exported, produces a
4+
// module that links cleanly and is unusable.
5+
import { chromium } from "playwright";
6+
7+
// dcraw_emu returns 2 for an unreadable input, native and wasm alike.
8+
const EXPECTED_EXIT_CODE = 2;
9+
10+
const browser = await chromium.launch();
11+
const page = await browser.newPage();
12+
page.on("pageerror", (e) => console.error("PAGEERROR:", e.message));
13+
14+
await page.goto("http://127.0.0.1:8731/index.html");
15+
await page
16+
.waitForFunction(() => window.__smoke !== undefined, null, { timeout: 120000 })
17+
.catch(() => {});
18+
19+
console.log(await page.evaluate(() => document.getElementById("log").textContent));
20+
const r = await page.evaluate(() => window.__smoke);
21+
await browser.close();
22+
23+
const fail = (msg) => {
24+
console.error(`::error::${msg}`);
25+
process.exitCode = 1;
26+
};
27+
28+
if (!r || !r.ok) {
29+
fail("dcraw_emu did not run in the browser build");
30+
if (r && r.error) {
31+
fail(r.error);
32+
// An abort escapes callMain as a RuntimeError rather than returning, so
33+
// this is the shape the missing -fexceptions regression takes here.
34+
if (/Aborted|RuntimeError/.test(r.error)) {
35+
fail("aborted rather than returning -- exception handling is not linked in");
36+
}
37+
}
38+
process.exit(1);
39+
}
40+
if (r.code !== EXPECTED_EXIT_CODE) {
41+
fail(`exit code was ${r.code}, expected ${EXPECTED_EXIT_CODE}`);
42+
}
43+
if (process.exitCode) process.exit(1);
44+
45+
console.log(
46+
`browser build OK: rejected bad input with exit ${r.code}, ` +
47+
`heap peaked at ${r.peakMiB} MiB`,
48+
);

‎.github/workflows/wasm.yml‎

Lines changed: 195 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,195 @@
1+
name: WebAssembly
2+
3+
on:
4+
push:
5+
branches: [main]
6+
tags: ['wasm-v*']
7+
pull_request:
8+
branches: [main]
9+
workflow_dispatch:
10+
11+
permissions:
12+
contents: read
13+
14+
env:
15+
# Keep in sync with LIBRAW_TOOLS in CMakeLists.txt.
16+
TOOLS: dcraw_emu
17+
# Pinned deliberately. emcc's floating point code generation decides output
18+
# bytes, so an unpinned toolchain makes byte comparisons non-reproducible.
19+
# This is the version the port was validated against.
20+
EMSDK_IMAGE: emscripten/emsdk:6.0.4
21+
22+
jobs:
23+
build:
24+
name: build ${{ matrix.variant }}
25+
runs-on: ubuntu-latest
26+
strategy:
27+
fail-fast: false
28+
matrix:
29+
include:
30+
# NODERAWFS: real filesystem, same argv and paths as native.
31+
- { variant: node, dir: build-wasm, noderawfs: 'ON' }
32+
# ES module, virtual filesystem, FS/callMain/HEAPU8 exported.
33+
- { variant: browser, dir: build-web, noderawfs: 'OFF' }
34+
35+
steps:
36+
- uses: actions/checkout@v4
37+
38+
- name: Build
39+
run: |
40+
set -euo pipefail
41+
docker run --rm -v "$PWD":/src -w /src "$EMSDK_IMAGE" bash -c "
42+
emcmake cmake -S . -B '${{ matrix.dir }}' \
43+
-DCMAKE_BUILD_TYPE=Release \
44+
-DLIBRAW_WASM_NODERAWFS=${{ matrix.noderawfs }} &&
45+
cmake --build '${{ matrix.dir }}' --target $TOOLS -j \$(nproc)"
46+
47+
- name: Check every tool produced both artifacts
48+
run: |
49+
set -euo pipefail
50+
for t in $TOOLS; do
51+
for ext in js wasm; do
52+
f="${{ matrix.dir }}/$t.$ext"
53+
[ -s "$f" ] || { echo "::error::missing or empty $f"; exit 1; }
54+
done
55+
printf '%-12s %9s bytes of wasm\n' "$t" "$(stat -c%s "${{ matrix.dir }}/$t.wasm")"
56+
done
57+
58+
- uses: actions/upload-artifact@v4
59+
with:
60+
name: libraw-wasm-${{ matrix.variant }}
61+
# one glob per line: upload-artifact does not do brace expansion
62+
path: |
63+
${{ matrix.dir }}/*.js
64+
${{ matrix.dir }}/*.wasm
65+
if-no-files-found: error
66+
67+
# What this catches: build breakage, wrong link flags, a module that aborts on
68+
# startup, and -- specifically -- exception handling being dropped.
69+
#
70+
# What it does NOT catch, and must not be read as covering: numerical parity
71+
# against native. That was measured on the 10-frame reference CR2 bracket (see
72+
# FORK.md) and needs real RAW files, which are ~23 MB each and live outside
73+
# this repository. There is no synthetic substitute: the one place wasm and
74+
# native disagree is AHD's cube-root lookup table, and reaching it takes a
75+
# real Bayer mosaic with enough near-tied interpolation directions.
76+
smoke-node:
77+
name: node smoke test
78+
needs: build
79+
runs-on: ubuntu-latest
80+
steps:
81+
- uses: actions/checkout@v4
82+
83+
- uses: actions/download-artifact@v4
84+
with:
85+
name: libraw-wasm-node
86+
path: wasm-bin
87+
88+
# -ffp-contract=off is what makes native and wasm comparable at all: x86-64
89+
# and arm64 fuse a*b+c with a single rounding step where wasm MVP rounds
90+
# twice. Not load-bearing for the assertions below, but the recipe belongs
91+
# in CI so it stays correct if real imagery is ever added.
92+
- name: Build native with FP contraction disabled
93+
run: |
94+
set -euo pipefail
95+
cmake -S . -B build-native \
96+
-DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_FLAGS="-ffp-contract=off"
97+
cmake --build build-native --target $TOOLS -j "$(nproc)"
98+
99+
# The regression this exists for. LibRaw signals every I/O and format
100+
# error by throwing, and catches in ~78 places; emcc disables exception
101+
# catching by default, so without -fexceptions each of those catches
102+
# becomes `abort()`. The module then dies on a corrupt or non-RAW file
103+
# instead of returning an error, and the app gets an unrecoverable
104+
# RuntimeError rather than a message it can show.
105+
#
106+
# This has a verified negative control: built without -fexceptions, the
107+
# first case below fails with
108+
# RuntimeError: Aborted(undefined) ... at ___cxa_throw
109+
# so the check has teeth rather than merely passing.
110+
#
111+
# A truncated CR2 is used as well as plain junk because it reaches a
112+
# different throw: a real RAW header that then runs out of data.
113+
- name: Non-RAW and truncated input must fail cleanly, not abort
114+
run: |
115+
set -uo pipefail
116+
printf 'not a raw file at all\n' > junk.bin
117+
head -c 4096 /dev/urandom > trunc.CR2
118+
printf 'II\x2a\x00\x08\x00\x00\x00' >> trunc.CR2
119+
120+
fail=0
121+
for f in junk.bin trunc.CR2; do
122+
for impl in "native build-native/dcraw_emu" "wasm node wasm-bin/dcraw_emu.js"; do
123+
set -- $impl
124+
label=$1; shift
125+
out=$("$@" -T -o 1 -W -j -q 3 -g 2 0 -t 0 -b 1.1 -Z /dev/null "$f" 2>&1)
126+
rc=$?
127+
printf '%-7s %-10s rc=%s %s\n' "$label" "$f" "$rc" "$out"
128+
case "$out" in
129+
*Aborted*|*RuntimeError*)
130+
echo "::error::$label aborted on $f -- exception handling is not linked in"
131+
fail=1 ;;
132+
esac
133+
if [ "$rc" -eq 0 ]; then
134+
echo "::error::$label reported success on $f"
135+
fail=1
136+
fi
137+
case "$out" in
138+
*"Cannot open"*) ;;
139+
*) echo "::error::$label printed no diagnostic for $f"; fail=1 ;;
140+
esac
141+
done
142+
done
143+
[ "$fail" -eq 0 ] || exit 1
144+
echo "both builds reject bad input cleanly"
145+
146+
- name: Usage banner runs and exits
147+
run: |
148+
set -uo pipefail
149+
node wasm-bin/dcraw_emu.js > usage.txt 2>&1
150+
grep -q "dcraw emulator" usage.txt || {
151+
echo "::error::the module did not reach main()"; cat usage.txt; exit 1; }
152+
echo "module instantiates and reaches main()"
153+
154+
# The browser build has failure modes no static check catches: INVOKE_RUN left
155+
# on would run main() before JS can stage input, and an unexported FS or
156+
# callMain leaves a filesystem JS cannot reach. Neither shows up until the
157+
# module is actually instantiated in a browser.
158+
#
159+
# No RAW fixture is needed: driving the error path exercises instantiation,
160+
# MEMFS writes, callMain, exit-code propagation and exception catching, which
161+
# is the whole of the embedding contract.
162+
smoke-browser:
163+
name: browser smoke test
164+
needs: build
165+
runs-on: ubuntu-latest
166+
steps:
167+
- uses: actions/checkout@v4
168+
169+
- uses: actions/download-artifact@v4
170+
with:
171+
name: libraw-wasm-browser
172+
path: web/lib
173+
174+
- uses: actions/setup-node@v4
175+
with: { node-version: '22' }
176+
177+
# `npx playwright install` only downloads browsers. The driver script does
178+
# `import { chromium } from "playwright"`, so the package itself has to be
179+
# installed where node can resolve it.
180+
- name: Install Playwright and Chromium
181+
run: |
182+
set -euo pipefail
183+
npm install --no-save --no-audit --no-fund playwright@1.62.0
184+
npx playwright install --with-deps chromium
185+
186+
- name: Instantiate dcraw_emu in a browser
187+
run: |
188+
set -euo pipefail
189+
cp .github/scripts/browser-smoke.html web/index.html
190+
python3 -m http.server 8731 --bind 127.0.0.1 --directory web &
191+
for _ in $(seq 1 30); do
192+
curl -sf -o /dev/null http://127.0.0.1:8731/index.html && break
193+
sleep 1
194+
done
195+
node .github/scripts/browser-smoke.mjs

‎CMakeLists.txt‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -85,11 +85,18 @@ if(EMSCRIPTEN)
8585
# LibRaw reports every I/O and format error by throwing
8686
# (LIBRAW_EXCEPTION_IO_EOF and friends) and catches in ~78 places. emcc
8787
# disables exception catching by default, which turns each of those catches
88-
# into an abort -- so a truncated or unrecognised file would take the whole
88+
# into an abort -- so a truncated or unrecognised file takes the whole
8989
# module down instead of returning an error code. Needed at compile *and*
9090
# link time.
91-
add_compile_options(-fexceptions)
92-
add_link_options(-fexceptions)
91+
#
92+
# target_compile_options rather than add_compile_options, deliberately:
93+
# add_compile_options only affects targets created *after* it, and this
94+
# block sits below add_library/add_executable, so the directory-scoped form
95+
# silently does nothing. That failure is invisible until a malformed file
96+
# reaches the module, which is exactly when it matters least to discover it.
97+
# PUBLIC propagates the flag from the library to every tool linking it.
98+
target_compile_options(raw PUBLIC -fexceptions)
99+
target_link_options(raw PUBLIC -fexceptions)
93100

94101
foreach(_tool ${LIBRAW_TOOLS})
95102
target_link_options(${_tool} PRIVATE

‎FORK.md‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,7 @@ native speed (~2.0 s vs ~1.0 s per frame).
132132
| wasm vs **stock native** (arm64, FMA on) | frames 1-5 identical; 6-10 differ by 16 to 704 bytes |
133133
| wasm vs native **`-ffp-contract=off`** | frames 1-5 identical; 6-10 differ by 1 to 28 bytes |
134134
| wasm run 1 vs wasm run 2 | **identical** |
135+
| wasm under node vs wasm in a browser | **identical** |
135136

136137
So most of the gap is arm64 FMA contraction, exactly as found for the Radiance
137138
tools: arm64 fuses `a*b+c` into one instruction with a single rounding step and
@@ -197,13 +198,38 @@ No pixel changes exponent, and the mean luminance -- which is what becomes
197198
`COMPUTED_VERTICAL_ILLUMINANCE` downstream -- agrees to nine significant
198199
figures.
199200

201+
### In a browser, on a real CR2
202+
203+
`build-web` was driven in headless Chromium against `capt10.CR2`, staged into
204+
MEMFS by JS and run through `callMain` with the pipeline's argument list:
205+
206+
```
207+
fetched CR2: 26,010,200 bytes
208+
heap after instantiate: 16.0 MiB
209+
heap after staging input: 16.0 MiB
210+
exit code: 0 (1.9 s)
211+
heap high-water mark: 266.1 MiB (6.5% of the wasm32 ceiling)
212+
output TIFF: 67,293,432 bytes
213+
sha256: 8137c98ac37c930c3fdaf5be0fc1d17fe40985dc5faa7411dc06d1f6a78af2ab
214+
```
215+
216+
Three things worth keeping:
217+
218+
- The browser output is **byte-identical to the NODERAWFS build**, same digest.
219+
The virtual filesystem changes nothing about the numbers.
220+
- **266 MiB peak, a twelfth of the wasm32 budget.** Memory was the open risk in
221+
[#237](https://github.com/radiantlab/HDRICalibrationTool/issues/237); it is not
222+
a constraint. Frames are converted one at a time, and each stage gets a fresh
223+
module instance, so this is the whole per-frame cost.
224+
- Staging the 26 MB input **did not grow the heap at all**, 16.0 MiB before and
225+
after. MEMFS keeps file bytes outside wasm linear memory, confirming what
226+
[#234](https://github.com/radiantlab/HDRICalibrationTool/issues/234) found for
227+
the Radiance tools. Input size does not compete with the working set.
228+
200229
### Not yet done
201230

202-
Browser-tab execution. `build-web` links, and its embedding mechanism is the
203-
same one verified for the Radiance tools, but `dcraw_emu.wasm` has never been
204-
instantiated in a browser. Memory has not been measured against the wasm32
205-
ceiling either; a 5796x3870 CR2 at `-q 3` should be a few hundred MB, but that
206-
is an estimate rather than a measurement.
231+
Nothing in this repository. Wiring `dcraw_emu` into HDRICalibrationTool's
232+
TypeScript pipeline so RAW inputs actually reach it is tracked there.
207233

208234
## Licensing
209235

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
> **Note — this fork adds a WebAssembly build of `dcraw_emu`.**
1+
> **Note: this fork adds a WebAssembly build of `dcraw_emu`.**
22
>
33
> Upstream's sources need no patching at all: every optional dependency
44
> self-guards behind a preprocessor macro, so not defining them is enough, and
55
> an `emcmake` configure succeeds on the first attempt with the tree unmodified.
6-
> What upstream does not ship is a CMake build — `README.cmake` redirects to
6+
> What upstream does not ship is a CMake build. `README.cmake` redirects to
77
> [LibRaw/LibRaw-cmake](https://github.com/LibRaw/LibRaw-cmake), unsupported
8-
> since 2014 — so this fork adds one file, `CMakeLists.txt`, and changes nothing
8+
> since 2014, so this fork adds one file, `CMakeLists.txt`, and changes nothing
99
> else.
1010
>
1111
> It replaces LibRaw-cmake rather than vendoring it: ~700 of its 737 lines are

0 commit comments

Comments
 (0)