Problem
Nemotron's encoder is materialized as a full JavaScript Uint8Array before ORT session creation. During session build, this likely doubles the largest model allocation and makes mobile/lower-memory browsers much more likely to kill the tab.
Evidence
nemotron-engine.js:180-190 fetches encoder/decoder/tokenizer bytes, then passes them into WasmNemotron.create(enc, dec, tok).
nemotron-engine.js:383-400 streams the encoder for progress, but accumulates the full file into new Uint8Array(len).
crates/nemotron-asr/src/backend_web.rs:159-204 constructs ORT sessions from in-memory bytes.
Acceptance criteria
- Investigate whether ORT web/session creation can avoid a second full encoder copy.
- If possible, stream or transfer bytes into the WASM/worker side to lower peak memory.
- If not possible with current ORT APIs, document the hard blocker and add a memory guard that routes low-memory devices away from Nemotron before loading.
- Measure peak memory before/after or provide a reproducible browser memory trace.
Problem
Nemotron's encoder is materialized as a full JavaScript
Uint8Arraybefore ORT session creation. During session build, this likely doubles the largest model allocation and makes mobile/lower-memory browsers much more likely to kill the tab.Evidence
nemotron-engine.js:180-190fetches encoder/decoder/tokenizer bytes, then passes them intoWasmNemotron.create(enc, dec, tok).nemotron-engine.js:383-400streams the encoder for progress, but accumulates the full file intonew Uint8Array(len).crates/nemotron-asr/src/backend_web.rs:159-204constructs ORT sessions from in-memory bytes.Acceptance criteria