web: model discovery, live download progress, VLM deadlock fix, RAG/vision download UI#510
Conversation
… refresh), live download progress via async http_download slot, VLM pthread-pool deadlock, RAG + vision download UI, prebuilt ONNX/sherpa WASM vendoring
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Docs buttons miss hydrate refresh
- Subscribed the Documents tab to shared model state changes and refreshes the download buttons on activation so hydration updates are reflected.
Or push these changes by commenting:
@cursor push a41097fbdf
Preview (a41097fbdf)
diff --git a/examples/web/RunAnywhereAI/src/views/documents.ts b/examples/web/RunAnywhereAI/src/views/documents.ts
--- a/examples/web/RunAnywhereAI/src/views/documents.ts
+++ b/examples/web/RunAnywhereAI/src/views/documents.ts
@@ -23,6 +23,7 @@
type RAGDocumentSummary,
type RAGSearchResult,
} from '@runanywhere/web';
+import { onModelStateChange } from '../components/model-selection';
import { escapeHtml } from '../services/escape-html';
import { formatError } from '../services/format-error';
@@ -37,6 +38,7 @@
let selectedLlmModelId = '';
/** Model-id pair the current native pipeline was created with. */
let createdPipelineKey: string | null = null;
+let unsubscribeState: (() => void) | null = null;
// ---------------------------------------------------------------------------
// Init
@@ -114,8 +116,14 @@
void downloadSelectedModel(selectedLlmModelId, 'LLM');
});
refreshModelButtons();
+ unsubscribeState?.();
+ unsubscribeState = onModelStateChange(() => refreshModelButtons());
- return {};
+ return {
+ onActivate: () => {
+ refreshModelButtons();
+ },
+ };
}
// ---------------------------------------------------------------------------You can send follow-ups to the cloud agent here.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 2bbba92. Configure here.
| container.querySelector('#docs-llm-download-btn')!.addEventListener('click', () => { | ||
| void downloadSelectedModel(selectedLlmModelId, 'LLM'); | ||
| }); | ||
| refreshModelButtons(); |
There was a problem hiding this comment.
Docs buttons miss hydrate refresh
Medium Severity
The Documents tab’s new embedding/LLM download controls call refreshModelButtons() only at init. They don’t listen for models.hydrated or onModelStateChange, unlike Vision/Chat. After async OPFS hydration marks models downloaded, the buttons can stay enabled as “Download” until the user changes a picker.
Reviewed by Cursor Bugbot for commit 2bbba92. Configure here.
1 participant
Web SDK testing-pass fixes found while exercising the example app end-to-end.
Model discovery (was: every downloaded model showed "Download" after reload)
_rac_inference_framework_from_proto+_rac_framework_raw_valuetoRAC_EXPORTED_FUNCTIONS_BASE. Without themframeworkOPFSDir()returned null andhydrateModelRegistry()skipped every model, so OPFS-backed models never re-detected on cold start.models.hydratedevent, so the picker/Storage/Docs views never re-queried after the async cold-start hydrate.model-selection.tsnow subscribes once and refreshes consumers.Download progress (was: UI froze at 0% until 100%)
http_downloadplatform-adapter slot inPlatformAdapter.ts(streamingfetch+ReadableStream→ MEMFS, per-chunk progress, resume viaRange, cancel viaAbortController, 416-on-resume treated as complete).download_orchestrator.cppthat uses the slot when present (native path unchanged:#ifdef __EMSCRIPTEN__+ slot-null fallback).VLM inference (was: hung forever in the image encode)
PTHREAD_POOL_SIZE=4, so on-demand worker spawn deadlocked the blocked main thread. BumpedPTHREAD_POOL_SIZEto 8 (covers the engine's capped max). VLM now completes.RAG + Vision UI
%, reflects downloaded state).ONNX/sherpa WASM vendoring
sherpa-onnx-racrelease) instead of the slow from-source-only path.Known follow-ups (not in this PR)
wasmTableexport under emcc 5.0.0) — under investigation; blocks RAG embeddings + speech end-to-end.OffscreenRuntimeBridge) but not wired.Note
Medium Risk
Touches core download orchestration and platform I/O on Emscripten (behavior change for all Web model downloads); WASM link/export and pthread sizing affect VLM builds. Example-only UI changes are low risk.
Overview
End-to-end fixes for the Web SDK and RunAnywhereAI example after full-app testing.
Model discovery after reload exports
_rac_inference_framework_from_protoand_rac_framework_raw_valueso JS can resolve framework OPFS folders forhydrateModelRegistry(). The example subscribes once tomodels.hydratedinmodel-selection.tsso the picker and toolbar refresh when async cold-start hydration marks models as on disk.Download UX wires the platform
http_downloadslot inPlatformAdapter.ts(streaming fetch → MEMFS, live progress, Range resume, abort/cancel, HTTP 416 treated as complete). On Emscripten,download_orchestrator.cppuses a new event-driven driver when that slot exists; the blocking sync worker remains the fallback. Web streaming skips per-byte SHA-256 but keeps size validation and HTTPS.VLM raises
PTHREAD_POOL_SIZEfrom 4 to 8 so ggml compute threads do not deadlock the main thread during synchronous encode.Example UI: Documents gets per-picker Download buttons with
%status; Vision adds Load image… (RGB decode + analyze without the camera). ONNX/sherpa vendor scripts prefer prebuilt WASM tarballs fromsherpa-onnx-rac, with from-source fallback.Reviewed by Cursor Bugbot for commit 2bbba92. Configure here.