Skip to content

Commit a3fb2bc

Browse files
artizclaude
andcommitted
wasm scan demo: load the layout model same-origin (release assets aren't CORS-fetchable)
GitHub Release assets are served from release-assets.githubusercontent.com with no Access-Control-Allow-Origin header, so a browser fetch of the layout model failed with a CORS error the moment the URL stopped 404ing. The recognition models avoid this only because Hugging Face sends CORS headers; the official docling-layout-heron-onnx on HF can't substitute — it exports a different contract (images + orig_target_sizes in, labels/boxes/scores out) than our decode_layout expects (raw logits/pred_boxes). So load the layout model from the page's own origin: prefer ./layout_heron_int8.onnx (~68 MB), fall back to ./layout_heron.onnx (~172 MB), and if neither is present open a setup note explaining how to fetch them (download_dependencies.sh) and drop one next to the page. These are the exact conformance-validated exports the native pipeline uses; *.onnx is already gitignored so the copied model isn't committed. Refs #157 Signed-off-by: artiz <artem.kustikov@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT
1 parent a38baa0 commit a3fb2bc

1 file changed

Lines changed: 55 additions & 17 deletions

File tree

crates/docling-wasm/www/scan.html

Lines changed: 55 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,31 @@
4141
<h1>docling.rs → wasm: scanned PDF / image, full lite profile</h1>
4242
<p>
4343
Stage 2 of <a href="https://github.com/docling-project/docling.rs/issues/157">#157</a>:
44-
RT-DETR layout detection (~172&nbsp;MB — cached after the first load)
45-
+ PP-OCRv3 recognition via ONNX Runtime Web; pdf.js rasterizes PDF pages
46-
at the native pipeline's 2&nbsp;px/point; region refinement, OCR and
47-
reading-order assembly run in Rust/wasm — the same code as the native
48-
pipeline. Tables come out via the geometric reconstruction
49-
(TableFormer is stage 3). Nothing leaves this page.
44+
RT-DETR layout detection + PP-OCRv3 recognition via ONNX Runtime Web;
45+
pdf.js rasterizes PDF pages at the native pipeline's 2&nbsp;px/point;
46+
region refinement, OCR and reading-order assembly run in Rust/wasm — the
47+
same code as the native pipeline. Tables come out via the geometric
48+
reconstruction (TableFormer is stage 3). Nothing leaves this page.
5049
</p>
50+
<details id="setup">
51+
<summary>The layout model is served from this page's own origin — fetch it once (~68&nbsp;MB int8)</summary>
52+
<p>
53+
Unlike the recognition model (fetched from Hugging Face, which sends CORS
54+
headers), the RT-DETR layout export lives on this repo's GitHub Release,
55+
and <strong>GitHub Release assets can't be fetched cross-origin from a
56+
browser</strong> (no <code>Access-Control-Allow-Origin</code>). So this
57+
page loads the layout model <em>same-origin</em> — put it next to this
58+
file and serve the directory:
59+
</p>
60+
<pre>curl -fsSL https://raw.githubusercontent.com/docling-project/docling.rs/master/scripts/install/download_dependencies.sh | sh
61+
cp models/layout_heron_int8.onnx crates/docling-wasm/www/ # ~68 MB int8 (or layout_heron.onnx, ~172 MB fp32)</pre>
62+
<p>
63+
These are the exact models the native pipeline uses. The page prefers the
64+
int8 export and falls back to fp32; both decode identically here (same
65+
<code>pixel_values</code>&nbsp;→&nbsp;<code>logits</code>,&nbsp;<code>pred_boxes</code>
66+
contract).
67+
</p>
68+
</details>
5169
<div id="drop">drop a scanned PDF or image here, or click to pick</div>
5270
<input type="file" id="pick" accept=".pdf,.png,.jpg,.jpeg,.tif,.tiff,.webp,.bmp" hidden>
5371
<p>
@@ -70,11 +88,13 @@ <h1>docling.rs → wasm: scanned PDF / image, full lite profile</h1>
7088
pdfjs.GlobalWorkerOptions.workerSrc =
7189
"https://cdn.jsdelivr.net/npm/pdfjs-dist/build/pdf.worker.min.mjs";
7290

73-
// The models-v1 release hosts only the fp32 layout export (the int8
74-
// variant is produced locally by scripts/install/quantize_models.py and
75-
// was never uploaded) — fetch what is actually there.
76-
const LAYOUT_URL =
77-
"https://github.com/docling-project/docling.rs/releases/download/models-v1/layout_heron.onnx";
91+
// The layout model is loaded SAME-ORIGIN (see the setup note above): GitHub
92+
// Release assets carry no CORS header, so a browser fetch of the release URL
93+
// fails — the model has to sit next to this page instead. Prefer the
94+
// conv-only static-INT8 export (~68 MB, a big first-load win over the
95+
// ~172 MB fp32) and fall back to fp32; both share the same interop
96+
// contract, so the wrapper below is unchanged.
97+
const LAYOUT_PATHS = ["./layout_heron_int8.onnx", "./layout_heron.onnx"];
7898
const SCALE = 2.0; // px per PDF point — the native pipeline's RENDER_SCALE
7999

80100
const meta = document.getElementById("meta");
@@ -171,14 +191,32 @@ <h1>docling.rs → wasm: scanned PDF / image, full lite profile</h1>
171191
return recCache[lang];
172192
}
173193

194+
// Load the first layout model that resolves same-origin; if none is
195+
// present, open the setup note and stop (the page is otherwise inert).
196+
async function loadLayout() {
197+
for (const path of LAYOUT_PATHS) {
198+
try {
199+
const buf = await fetchProgress(path, "layout model (first load only)");
200+
status("starting layout session …", true);
201+
return await ort.InferenceSession.create(buf, {
202+
executionProviders: ["wasm"],
203+
logSeverityLevel: 3,
204+
});
205+
} catch (e) {
206+
// 404 / decode failure → try the next candidate.
207+
}
208+
}
209+
return null;
210+
}
211+
174212
busy("loading wasm module …");
175213
await init();
176-
const layoutModel = await fetchProgress(LAYOUT_URL, "layout model (first visit only)");
177-
status("starting layout session …", true);
178-
const layoutSession = await ort.InferenceSession.create(layoutModel, {
179-
executionProviders: ["wasm"],
180-
logSeverityLevel: 3,
181-
});
214+
const layoutSession = await loadLayout();
215+
if (!layoutSession) {
216+
document.getElementById("setup").open = true;
217+
ready("layout model not found next to this page — see the note above");
218+
throw new Error("no layout model available");
219+
}
182220
await recFor(lang.value);
183221
ready("ready — drop a scanned PDF or image");
184222

0 commit comments

Comments
 (0)