Merge pull request #5 from ReactVision/develop

Release 1.0.1

Author: doranteseduardo

CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 1.0.1
+
+Needs the virocore web build from the same change: the new renderer calls are
+absent from 1.0.0's binary, and each falls back to the old path when missing.
+
+### Fixed
+
+- **A runtime abort is reported.** `ViroWebRendererOptions.onAbort`, `renderer.addAbortListener()` and `renderer.abortedWith`. An abort — out of memory past the heap's ceiling, most often — is terminal, and it used to surface only as a rejection a model load caught, so nothing reached a host's error tracking. Model loads in flight settle as failed.
+- **Model files are removed from the virtual FS once loaded.** MEMFS holds every file in JS memory until unlinked, so each model stayed resident for the life of the page.
+- **The camera feed is drawn from the stream, not the tracker's buffer.** The stream is requested at `feedWidth`/`feedHeight` (default 1280x960) and uploaded to a source texture straight from the `<video>`; the tracker gets a downscaled copy at `captureWidth` along the stream's aspect. No readback, no per-frame texture. Older renderers keep the RGBA path.
+- **Relocalising frames keep their rotation.** tinyvio's `poseConfidence` now decides the renderer's state: Full or PositionNoScale is Normal, RotationOnly is Limited (rotation applied, position held), None holds the last pose. An engine without it is drawn only while Running.
+- **The pose is smoothed** with a One Euro filter before `arSetPose` (`poseSmoothing`, or `false` for the raw solve). tinyvio does not filter, and the raw pose shook content visibly while tracking was healthy.
+- **`onStatus` is debounced** out of Normal (15 frames) and back at once, and carries the frame's `confidence` and `reason`. `reason === NoGravity` that persists means no motion events are arriving.
+- **The default asset paths keep their trailing slash under Vite.** Vite rewrites `new URL("../slam/", import.meta.url)` at dev time to the directory without its `/`, so the tracker asked for `.../slamtinyvio-slam.wasm` and AR aborted ("both async and sync fetching of the wasm failed"). The renderer's `../wasm/` base had the same pattern. Both bases now end in `/`.
+- **An engine loaded through `loadSlam` finds its own `.wasm`.** The session pointed every engine at this package's `slam/` copy, including one a host loaded from its own script, which then fetched a file that was not there. With a custom `loadSlam` and no `slamBaseUrl`, the engine resolves the `.wasm` next to its script.
+- Playback reports untracked frames as Unavailable, so they hold the last tracked pose as the live path does.
+
+### Added
+
+- `PoseConfidence`, `TrackingReason`, `PoseFilter`, `ViroRendererAbortError`; `ViroSceneApi.createSourceTexture` and `updateTextureFromSource`.
+- `test/session.test.mjs` in `npm test`, and `npm run test:browser`: taps, the heap, the abort, source textures and dropout holding, in a real Chrome. Against 1.0.0's binary nine of its checks fail.
+
 ## 1.0.0
 
 First published release.

RELEASE_NOTES.md

@@ -1,5 +1,24 @@
 # Release Notes
 
+## 1.0.1
+
+Fixes found running the web player against real scenes. The compiled renderer in this release is virocore 3.0.2; the new calls fall back to the old path on an older binary, but the heap, tap and dropout fixes live in the binary and need this one.
+
+### Fixed
+
+- **Large models no longer kill the renderer.** The heap was fixed at 256 MB, and a 30 to 40 MB generated GLB crossed it once decoded; `abort("OOM")` is permanent, so every later call failed and the canvas stayed black. The heap now grows to 2 GB, and glTF loads no longer copy the model.
+- **An abort is reported.** `onAbort` in the options, `renderer.addAbortListener()` and `renderer.abortedWith`. Model loads in flight settle as failed.
+- **Model files leave the virtual FS once loaded,** instead of staying in JS memory for the life of the page.
+- **Taps land where they are drawn.** They were mirrored about the horizontal axis.
+- **Web AR holds its pose through a dropout** instead of snapping to a default orientation. tinyvio's `poseConfidence` decides the state: RotationOnly applies rotation and holds position, None holds the last pose.
+- **The pose is smoothed** with a One Euro filter (`poseSmoothing`, or `false` for the raw solve), and `onStatus` is debounced out of Normal and carries `confidence` and `reason`.
+- **The camera feed draws from the stream** at `feedWidth`/`feedHeight` (default 1280x960) through a source texture, with no per-frame readback; the tracker gets a downscaled copy.
+- **Web AR starts under Vite.** The tracker's wasm path lost its trailing slash in dev, and an engine loaded through `loadSlam` was pointed at the package's copy instead of its own.
+
+### Added
+
+- `PoseConfidence`, `TrackingReason`, `PoseFilter`, `ViroRendererAbortError`; `ViroSceneApi.createSourceTexture` and `updateTextureFromSource`.
+
 ## 1.0.0
 
 First published release. The Viro renderer — the same C++ engine ViroReact runs natively on iOS, Android, Apple Vision Pro and Meta Quest — compiled to WebAssembly and drawing through WebGL2.

package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "@reactvision/viro-web-renderer",
-  "version": "1.0.0",
+  "version": "1.0.1",
   "description": "WebAssembly/WebGL2 build of the Viro renderer (virocore) for the web platform",
   "license": "MIT",
   "author": "ReactVision",
@@ -48,7 +48,8 @@
   "scripts": {
     "copy-wasm": "bash scripts/copy-wasm.sh",
     "build": "tsc",
-    "test": "node test/hitTest.test.mjs && node test/package.test.mjs",
+    "test": "node test/hitTest.test.mjs && node test/session.test.mjs && node test/package.test.mjs",
+    "test:browser": "node test/browser/run.mjs",
     "clean": "rm -rf dist *.tgz",
     "prepare": "npm run build",
     "prepublishOnly": "npm run copy-wasm && npm run copy-slam && npm run build && VIRO_REQUIRE_WASM=1 npm test",

package.json

@@ -7,6 +7,7 @@ import {
   type ViroNodeEventHandlers,
   type ViroAnimationHandlers,
 } from "./sceneApi.js";
+import { ViroRendererAbortError } from "./types.js";
 import type { ViroWebModule, ViroWebRendererOptions } from "./types.js";
 
 const MODEL_EXT: Record<ViroModelFormat, string> = {
@@ -79,6 +80,8 @@ export class ViroWebRenderer {
   private readonly eventHandlers = new Map<ViroHandle, ViroNodeEventHandlers>();
   private readonly modelLoadResolvers = new Map<ViroHandle, (success: boolean) => void>();
   private readonly animationHandlers = new Map<ViroHandle, ViroAnimationHandlers>();
+  private readonly abortListeners = new Set<(error: ViroRendererAbortError) => void>();
+  private abortError: ViroRendererAbortError | null = null;
 
   private constructor(
     private readonly module: ViroWebModule,
@@ -163,6 +166,7 @@ export class ViroWebRenderer {
     }
     const path = `${dir}/viro_model_${nodeHandle}.${MODEL_EXT[format]}`;
     this.module.FS.writeFile(path, bytes);
+    const written = [path, ...resources.map((res) => `${dir}/${res.name}`)];
     return new Promise<boolean>((resolve) => {
       // One resolver per node, and a second load replaces the first. Settle the
       // one being replaced rather than dropping it: the native callback carries
@@ -171,7 +175,25 @@ export class ViroWebRenderer {
       this.modelLoadResolvers.get(nodeHandle)?.(false);
       this.modelLoadResolvers.set(nodeHandle, resolve);
       this.module.viroLoadModel(nodeHandle, path, format);
-    });
+    }).finally(() => this.unlinkAll(written));
+  }
+
+  /**
+   * The loaders read the virtual FS synchronously and are done with it by the
+   * time the load settles. MEMFS keeps every file in JS memory until unlinked,
+   * so without this each model stayed resident for the life of the page, once
+   * per load.
+   */
+  private unlinkAll(paths: string[]): void {
+    const unlink = this.module.FS.unlink;
+    if (!unlink) return;
+    for (const p of paths) {
+      try {
+        unlink.call(this.module.FS, p);
+      } catch {
+        // Already gone, or a second load into the node replaced it.
+      }
+    }
   }
 
   /**
@@ -183,7 +205,12 @@ export class ViroWebRenderer {
     const bytes = new Uint8Array(await (await fetch(url)).arrayBuffer());
     const path = `/viro_env_${lightingEnvCounter++}.hdr`;
     this.module.FS.writeFile(path, bytes);
-    const handle = this._scene.loadRadianceHDRTexture(path);
+    let handle: ViroHandle;
+    try {
+      handle = this._scene.loadRadianceHDRTexture(path);
+    } finally {
+      this.unlinkAll([path]);
+    }
     if (handle) this._scene.setLightingEnvironment(handle);
     return handle;
   }
@@ -237,14 +264,19 @@ export class ViroWebRenderer {
     canvas.width = width;
     canvas.height = height;
 
+    // The runtime can abort before there is a renderer to tell (during init);
+    // create() then rejects on its own and there is no one else to notify.
+    let renderer: ViroWebRenderer | undefined;
     const module = await loadViroWebModule(canvas, {
       locateFile: options.locateFile,
       baseUrl: options.assetBaseUrl,
       importGlue: options.importGlue,
+      onAbort: (what) => renderer?.handleAbort(what),
     });
     module.initViroScene(selector, width, height);
 
-    const renderer = new ViroWebRenderer(module, canvas);
+    renderer = new ViroWebRenderer(module, canvas);
+    if (options.onAbort) renderer.addAbortListener(options.onAbort);
     // Once, at startup: what a bug report needs pasted back and cannot find
     // anywhere else, since the binary reaches an app through two hand copies.
     const buildId = renderer.scene.getBuildId();
@@ -326,6 +358,49 @@ export class ViroWebRenderer {
     this.disposed = true;
   }
 
+  /**
+   * Set once the WASM runtime has aborted (see ViroWebRendererOptions.onAbort).
+   * A renderer in that state cannot recover; a new one has to be created.
+   */
+  get abortedWith(): ViroRendererAbortError | null {
+    return this.abortError;
+  }
+
+  /**
+   * Be told when the runtime aborts. Called at most once per renderer, and
+   * straight away when it already has. Returns an unsubscribe.
+   */
+  addAbortListener(listener: (error: ViroRendererAbortError) => void): () => void {
+    if (this.abortError) {
+      listener(this.abortError);
+      return () => {};
+    }
+    this.abortListeners.add(listener);
+    return () => {
+      this.abortListeners.delete(listener);
+    };
+  }
+
+  private handleAbort(what: unknown): void {
+    if (this.abortError) return;
+    // Emscripten passes "OOM" for a heap that could not grow and nothing at all
+    // for most other aborts, a failed allocation past MAXIMUM_MEMORY included.
+    const reason = (what instanceof Error ? what.message : String(what ?? "")) || "unknown";
+    this.abortError = new ViroRendererAbortError(reason);
+    // Nothing in flight can finish: the loader that would call back is gone.
+    for (const resolve of this.modelLoadResolvers.values()) resolve(false);
+    this.modelLoadResolvers.clear();
+    const listeners = [...this.abortListeners];
+    this.abortListeners.clear();
+    for (const listener of listeners) {
+      try {
+        listener(this.abortError);
+      } catch (err) {
+        console.error("[Viro web] onAbort listener threw:", err);
+      }
+    }
+  }
+
   private assertLive(): void {
     if (this.disposed) {
       throw new Error("ViroWebRenderer: instance has been disposed");