Skip to content

feat(wasm): JSON text export of deep values - #1087

Open
zxch3n wants to merge 1 commit into
mainfrom
feat/wasm-json-text-export
Open

feat(wasm): JSON text export of deep values#1087
zxch3n wants to merge 1 commit into
mainfrom
feat/wasm-json-text-export

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 3, 2026

Copy link
Copy Markdown
Member

Stack: 4/6 — merge order: #1093#1085#1086#1087#1090#1091

Summary

Stacked on #1086 (which stacks on #1085); merge in order, GitHub retargets automatically.

Adds two bulk-read APIs on LoroDoc and every container class (LoroMap/LoroList/LoroMovableList/LoroTree/LoroText/LoroCounter):

  • getDeepValueJson(): string — serde_json serialization of the deep value in ONE wasm call; same content as JSON.stringify(x.toJSON()).
  • getDeepValueJsonWithIds(): { json: string, cids: ContainerID[] }json is the deep value without ids; cids lists container ids in pre-order DFS of the serialized tree, so a consumer (e.g. loro-mirror) can re-attach ids in a single JS walk to reconstruct the getDeepValueWithID() shape.

The (json, cids) pair is produced by converting the with-id deep value to a serde_json::Value and stripping { cid, value } nodes in one pass, so cids order always matches the key/item order after JSON.parse, regardless of serde_json's preserve_order feature.

Benchmark (generated doc: 70,051 containers — Map 15,632 / List 9,956 / Text 44,463; 4.1 MB JSON; pnpm bench-deep-value-json)

Release build, median of 5 rounds:

case time
toJSON() 75.6 ms
getDeepValueWithID() 137.0 ms
getDeepValueJson() 46.0 ms
getDeepValueJson() + JSON.parse 56.3 ms
getDeepValueJsonWithIds() + parse + re-attach walk 180.2 ms

The ≥5x target is not reachable: measured 2.4x end-to-end (getDeepValueJson()+JSON.parse vs getDeepValueWithID()). CPU profiling shows the bottleneck is the Rust-side deep-value walk + serialization (getDeepValueJson alone is 46 ms), not the boundary crossing — JSON.parse of 4.1 MB costs only ~5 ms. The JsValue structured-clone conversion accounts for ~75–80 ms of getDeepValueWithID()'s 137 ms. Dev (unoptimized wasm) build numbers: 370 ms → 280 ms (1.3x), dominated by dlmalloc/debug checks. Further wins would require reducing Rust-side materialization cost, not JS-side changes.

Caveats (documented in the changeset, doc comments, and context/wasm-bulk-read.md):

  • Tree node meta maps serialize as plain deep values (matching getDeepValueWithID()), so meta container ids do not appear in cids.
  • A plain object with exactly the keys cid + value where cid is a valid container id string is indistinguishable from a container node (inherent to the format).

Tests

  • Rust: crates/loro-internal/tests/deep_value_json.rs (5 tests: content equality with plain deep value, exact pre-order cids, per-container variants, empty doc, detached errors).
  • JS: tests/deep_value.test.ts extended (17 tests): JSON.parse(getDeepValueJson()) deep-equals toJSON() for doc + all 6 container types; re-attach walk reconstructs getDeepValueWithID() exactly.
  • Full wasm suite: 27 files / 366 tests pass; tsc --noEmit clean; cargo test -p loro-internal / -p loro pass.

Wasm binary size

Dev build with debug info: +524 KB (+0.52%) vs the parent branch.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3139.98 KB
  • Gzipped size: 1040.15 KB
  • Brotli size: 729.47 KB

@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 1430e55 to 80ac32f Compare September 4, 2026 12:17
@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 80ac32f to 55093ca Compare September 4, 2026 16:38
Base automatically changed from feat/wasm-container-bulk-read to main September 5, 2026 05:14
Add getDeepValueJson(): string on LoroDoc and every container class —
serde_json serialization of the deep value in one WASM call, same content
as JSON.stringify(x.toJSON()) — and getDeepValueJsonWithIds():
{ json, cids } where cids lists container ids in pre-order DFS of the
serialized tree so a consumer can re-attach ids in a single JS walk.

The (json, cids) pair is produced by converting the with-id deep value to
a serde_json::Value and stripping { cid, value } nodes in one pass, so the
cids order always matches the key/item order a JS consumer sees after
JSON.parse, regardless of serde_json's preserve_order feature.

Benchmark on a ~70k-container doc (Map 15,632 / List 9,956 / Text 44,463,
4.1 MB JSON), release build: getDeepValueWithID 137.0 ms vs
getDeepValueJson()+JSON.parse 56.3 ms (2.4x). The 5x target is not
reachable from the JS side: profiling shows the Rust-side deep-value walk
and serialization dominate (getDeepValueJson alone is 46 ms; JSON.parse
of 4.1 MB is ~5 ms), not the boundary crossing.

Wasm size (dev build, with debug info): +524 KB (+0.52%).
@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 55093ca to 7e99105 Compare September 5, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant