Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/deploy_branches.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,9 @@ jobs:
eval $(opam env)
export OPAMYES=1
opam clean --all-switches --download-cache --logs --repo-cache --unused-repositories
- name: Setup zarith native BigInt runtime
run: opam exec -- make setup-zarith
working-directory: ./source
- name: Build Release
run: |
export DUNE_CACHE=enabled
Expand Down
22 changes: 15 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,23 @@ TEST_DIR="$(shell pwd)/_build/default/test"
HTML_DIR="$(shell pwd)/_build/default/src/web/www"
SERVER="http://0.0.0.0:8000/"

.PHONY: all deps change-deps setup-instructor setup-student dev dev-helper dev-student fmt watch watch-release release release-student grade echo-html-dir serve serve2 repl test clean
.PHONY: all deps change-deps setup-instructor setup-student dev dev-helper dev-student fmt watch watch-release release release-student grade echo-html-dir serve serve2 repl test clean setup-zarith

all: dev

# Install native BigInt runtime for zarith_stubs_js to fix WebWorker postMessage serialization.
# The vendored runtime.js uses native JS BigInt (from zarith_stubs_js v0.17.0) which survives
# structured clone, unlike the BigInteger.js library used in older versions.
setup-zarith:
@echo "Installing native BigInt zarith runtime..."
@cp vendor/zarith_native_bigint_runtime.js "$$(opam var lib)/zarith_stubs_js/runtime.js"

deps:
opam repo add archive git+https://github.com/ocaml/opam-repository-archive
opam update
opam install ./hazel.opam.locked --deps-only --with-test --with-doc
npm install
$(MAKE) setup-zarith

change-deps:
opam update
Expand All @@ -25,7 +33,7 @@ setup-instructor:
setup-student:
cp src/web/exercises/settings/ExerciseSettings_student.re src/web/exercises/settings/ExerciseSettings.re

dev-helper:
dev-helper: setup-zarith
dune fmt --auto-promote || true
dune build @ocaml-index @src/fmt --auto-promote src --profile dev

Expand All @@ -36,16 +44,16 @@ dev-student: setup-student dev-helper
fmt:
dune fmt --auto-promote

watch: setup-instructor
watch: setup-instructor setup-zarith
dune build @ocaml-index @src/fmt --auto-promote src --profile dev --watch

watch-release: setup-instructor
watch-release: setup-instructor setup-zarith
dune build @src/fmt --auto-promote src --profile release --watch

release: setup-instructor
release: setup-instructor setup-zarith
dune build @src/fmt --auto-promote src --profile release

release-student: setup-student
release-student: setup-student setup-zarith
dune build @src/fmt --auto-promote src --profile dev # Uses dev profile for performance reasons. It may be worth it to retest since the ocaml upgrade

grade:
Expand Down Expand Up @@ -86,7 +94,7 @@ coverage:
dune runtest --instrument-with bisect_ppx --force
bisect-ppx-report summary

ci:
ci: setup-zarith
dune build --profile dev
dune runtest --instrument-with bisect_ppx --force

Expand Down
86 changes: 86 additions & 0 deletions docs/bigint-webworker-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# BigInt WebWorker Serialization Fix

This document explains the vendored zarith runtime and why it's needed.

**Vendored file:** `vendor/zarith_native_bigint_runtime.js`
**Source:** `zarith_stubs_js` v0.17.0
**Purpose:** Fixes BigInt serialization through WebWorker `postMessage`

### The Problem

Hazel uses a WebWorker for evaluation to avoid blocking the UI. Data (including
the AST with BigInt values) is sent to/from the worker via `postMessage`, which
uses the browser's [structured clone algorithm](https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Structured_clone_algorithm).

The `zarith_stubs_js` package (v0.16.x) that provides BigInt support for
js_of_ocaml uses the [BigInteger.js](https://github.com/peterolson/BigInteger.js)
library internally. BigInteger.js creates custom JavaScript objects with
prototype methods like `.lt()`, `.add()`, etc. When these objects pass through
structured clone, **the prototype chain is lost** - the data survives but the
methods don't. This caused crashes when displaying evaluation results containing
large integers.

### The Solution

Starting in v0.17.0, `zarith_stubs_js` switched to using **native JavaScript
`BigInt`** instead of BigInteger.js. Native `BigInt` is a primitive type that
fully survives structured clone.

However, we cannot simply upgrade to zarith_stubs_js v0.17.0 because it requires
upgrading the Jane Street packages. At the time of this fix (January 2026),
bonsai v0.17.0 on opam constrains `js_of_ocaml < 5.7.0`, and versions of
`js_of_ocaml-compiler` before 5.7.0 require `ocaml < 5.2`. Note that bonsai's
master branch on GitHub has updated to require `js_of_ocaml >= 6.0.0`, so a
future bonsai release should resolve this constraint issue.

### Our Approach

We vendor the `runtime.js` from zarith_stubs_js v0.17.0 and copy it into the
opam switch at build time, replacing the v0.16.x version. This gives us native
BigInt support without upgrading the package.

The `Makefile` has a `setup-zarith` target that performs this copy:

```makefile
setup-zarith:
cp vendor/zarith_native_bigint_runtime.js "$$(opam var lib)/zarith_stubs_js/runtime.js"
```

This target is automatically run before builds (`make dev`, `make release`, etc.)
and in CI.

### Alternatives Considered

1. **Full package upgrade** - Blocked by OCaml version / js_of_ocaml constraints
as described above.

2. **O(n) serialization shim** - Our previous approach: recursively walk the
entire message payload before/after `postMessage`, converting BigInts to
tagged strings (`{__hazel_bigint__: "123"}`) and back. This worked but added
overhead proportional to message size (4 traversals per worker round-trip).

### Future Considerations

- **When a new bonsai release is published to opam** (with the js_of_ocaml >= 6.0.0
constraint from master), we should be able to remove this vendored file and do
a proper upgrade. At that point:
1. Upgrade zarith_stubs_js to v0.17.0+ via opam
2. Remove `vendor/zarith_native_bigint_runtime.js`
3. Remove the `setup-zarith` target from the Makefile
4. Remove the `setup-zarith` dependencies from other Makefile targets

- **If zarith_stubs_js changes its internal representation again**, we may need
to update the vendored file. Check the [zarith_stubs_js releases](https://github.com/janestreet/zarith_stubs_js)
for changes.

- **The vendored runtime.js API is stable** - it implements the same `ml_z_*`
functions as the original, just with native BigInt instead of BigInteger.js.
There should be no compatibility issues.

### Related Files

- `vendor/zarith_native_bigint_runtime.js` - The vendored runtime file
- `src/web/util/WorkerClient.re` - Client-side worker communication
- `src/web/util/WorkerServer.re` - Worker-side message handling
- `Makefile` - `setup-zarith` target
- `.github/workflows/deploy_branches.yml` - CI setup-zarith step
3 changes: 2 additions & 1 deletion src/web/dune
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,13 @@
(library
(name workerServer)
(modules WorkerServer)
(wrapped false)
(instrumentation
(backend bisect_ppx))
(libraries language)
(js_of_ocaml)
(preprocess
(pps ppx_yojson_conv ppx_sexp_conv ppx_deriving.show)))
(pps js_of_ocaml-ppx ppx_yojson_conv ppx_sexp_conv ppx_deriving.show)))

(library
(name web)
Expand Down
8 changes: 4 additions & 4 deletions src/web/util/WorkerServer.re
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ let work = (req_value: Request.value): Response.value => {
};
};

let on_request = (req: Request.t): unit =>
req
|> List.map(((k, v)) => (k, work(v)))
|> Js_of_ocaml.Worker.post_message;
let on_request = (req: Request.t): unit => {
let resp: Response.t = req |> List.map(((k, v)) => (k, work(v)));
Js_of_ocaml.Worker.post_message(resp);
};

let start = () => Js_of_ocaml.Worker.set_onmessage(on_request);
Loading
Loading