wasm: one demo page with the whole pipeline (#157)

The browser demo had grown into three pages: index.html (declarative only,
no PDF/image, no output-format parity) plus ocr.html and scan.html, which
were really OCR test harnesses. Fold everything into index.html and delete
the other two.

The single page now covers what the module can actually do: any supported
document, PDF via its text layer, and — falling back automatically when a
PDF has no text layer — scanned pages and images through the ONNX pipeline.
Output picks between Markdown, docling JSON and DocLang XML; a Markdown
preview renders the result. The OCR half (ORT, pdf.js, models) loads lazily,
so a visitor converting a DOCX downloads none of it.

Supporting changes:
- convert() takes `images` ("placeholder" | "embedded"), mirroring
  docling-serve's option, so pictures can ride along as data URIs — there is
  no `referenced` mode because a page cannot write files.
- ScannedConverter::finish and convert_scanned_image accept "doclang" too,
  so both paths offer the same three output grammars; the format is threaded
  through worker.js and pipeline.js.
- A dead module worker (blocked CDN, offline, strict extension) used to
  leave every pending RPC unsettled, so the page sat on "loading …"
  forever. onerror/onmessageerror now reject the waiters with a message that
  names the cause. Verified: the page reports the failure in ~30 s instead of
  hanging.
- Drop the temporary layout-region diagnostic probe and its JS plumbing, and
  the per-page console timing logs.
- Inline favicon so a static host never 404s on /favicon.ico.

Verified in headless Chromium against the real wasm module: DOCX → md/json/
doclang, images=placeholder vs embedded (data URI present), a corpus
text-layer PDF, the Markdown preview, and cross-origin isolation via coi.js
(crossOriginIsolated = true, so ORT gets threads).

Refs #157

Signed-off-by: artiz <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT

Author: artiz

crates/docling-wasm/README.md

@@ -26,12 +26,59 @@ converters and the PDF text parser are pure Rust.
 ## API
 
 ```ts
-convert(bytes: Uint8Array, filename: string, to?: "md" | "json" | "doclang"): string
+convert(
+  bytes: Uint8Array,
+  filename: string,
+  to?: "md" | "json" | "doclang",          // default "md"
+  images?: "placeholder" | "embedded",     // default "placeholder", Markdown only
+): string
 supported_extensions(): string   // JSON array, e.g. for <input accept=…>
 version(): string
 ```
 
-The filename's extension drives format detection, same as the CLI.
+The filename's extension drives format detection, same as the CLI. `images`
+mirrors docling-serve's option: `placeholder` emits docling's `<!-- image -->`,
+`embedded` inlines the picture as a `data:` URI so the Markdown is
+self-contained (a page has no filesystem, so there is no `referenced` mode).
+
+### Convert a file the user picked
+
+```js
+import init, { convert } from "./pkg/docling_wasm.js";
+await init();
+
+const file = input.files[0];
+const bytes = new Uint8Array(await file.arrayBuffer());
+const markdown = convert(bytes, file.name, "md");
+const json     = convert(bytes, file.name, "json");
+const withPics = convert(bytes, file.name, "md", "embedded");
+```
+
+### Scanned pages (OCR)
+
+Scanned PDFs and images need the ML models; everything else runs with no
+network at all. The full wiring — model resolution, Web Worker, pdf.js
+rasterization — is [`www/index.html`](./www/index.html); the short version:
+
+```js
+import { createOcr } from "./pipeline.js";
+
+const ocr = createOcr({ onStatus: (msg) => console.log(msg) });
+await ocr.boot();                        // wasm + layout model
+
+// A standalone image is its own page.
+const md = await ocr.convertImage(await file.arrayBuffer(), file.name, "en", "md");
+
+// A scanned PDF: feed rasterized pages in order (2 px/point = the native
+// pipeline's RENDER_SCALE), then finish.
+await ocr.startDoc("en", /* TableFormer */ false);
+for (const page of pages) await ocr.addPage(page.rgba, page.w, page.h, 2.0);
+const doc = ocr.finishDoc(file.name, "md");
+```
+
+Models resolve **device file → local `./models/` → Hugging Face**, so a page can
+ship with no models and still work: `ocr.setProvidedModels({ "layout_heron_int8.onnx": buf })`
+takes files the user picked, and anything not provided is fetched.
 
 ## Build
 
@@ -49,15 +96,20 @@ wasm-bindgen --target web --out-dir crates/docling-wasm/www/pkg \
 
 ## Demo
 
-[`www/index.html`](./www/index.html) is a drop-a-file demo page over the
-module (output selector, conversion timing, automated-test hook). After the
-`wasm-bindgen` step above:
+[`www/index.html`](./www/index.html) is the whole thing on one page: drop a
+file, pick the output (Markdown / JSON / DocLang), pick how images render, and
+optionally turn on OCR for scanned pages. After the `wasm-bindgen` step above:
 
 ```bash
 python3 -m http.server -d crates/docling-wasm/www 8901
 # open http://127.0.0.1:8901/
 ```
 
+It is a plain static page — copy `www/` behind any web server (or into a mobile
+app's webview) and it works as-is. The OCR half loads lazily, so a visitor who
+only converts a DOCX never downloads ONNX Runtime, pdf.js or any model. PDFs
+try their text layer first and fall back to OCR only when there isn't one.
+
 Verified end-to-end in headless Chromium: Markdown/DOCX→md, DOCX→JSON, a
 corpus PDF→md through the text-layer path, and the scanned-PDF error path all
 exercised through the real wasm module.
@@ -113,10 +165,9 @@ Stage 3 is the same but `conv.addPageTf(rgba, w, h, 2.0, layout, rec, tf)`,
 where `tf` is a stateful session over the three TableFormer graphs (the heavy
 cross-attention K/V and the growing decoder KV-cache stay on the JS side, so
 each decode step marshals only the last tag and gets back logits+hidden — see
-[`www/pipeline.js`](./www/pipeline.js)'s `JsTfSession`). The demos wire all of
-this up: [`www/ocr.html`](./www/ocr.html) (stage 1),
-[`www/scan.html`](./www/scan.html) (stages 2–3, pdf.js + a TableFormer toggle +
-a device model-file picker).
+[`www/pipeline.js`](./www/pipeline.js)'s `JsTfSession`).
+[`www/index.html`](./www/index.html) wires all three stages up behind the OCR
+language selector, the TableFormer toggle and the model picker.
 
 ## Run it on your phone
 
@@ -136,8 +187,8 @@ same-origin, from a CORS host, **or straight from files on your device**.
    Pages or githack.) Build the wasm `pkg/` first (see **Build** above).
 
 3. **Provide the models to the page**, either:
-   - **from your device** — tap the *"Models from device"* picker in
-     `scan.html` and select `layout_heron_int8.onnx` and, for TableFormer,
+   - **from your device** — tap the model picker under *"Scanned PDFs and
+     images"* and select `layout_heron_int8.onnx` and, for TableFormer,
      `encoder.onnx`, `decoder_kv.onnx` (+`.data`), `bbox.onnx` (+`.data`). Each
      file is read to an `ArrayBuffer` on the client and used directly — no
      download, and a single allocation per file (a 225 MB encoder over

crates/docling-wasm/src/lib.rs

@@ -16,7 +16,7 @@
 //! const md = convert(new Uint8Array(await file.arrayBuffer()), file.name, "md");
 //! ```
 
-use docling::{DocumentConverter, InputFormat, SourceDocument};
+use docling::{DocumentConverter, ImageMode, InputFormat, SourceDocument};
 use wasm_bindgen::prelude::*;
 
 #[cfg(feature = "ocr")]
@@ -39,16 +39,28 @@ fn start() {
 
 /// The whole conversion body, host-testable (`JsError` can only be
 /// constructed on the wasm target, so the JS boundary stays a thin shim).
-fn convert_impl(bytes: &[u8], filename: &str, to: Option<&str>) -> Result<String, String> {
+fn convert_impl(
+    bytes: &[u8],
+    filename: &str,
+    to: Option<&str>,
+    images: Option<&str>,
+) -> Result<String, String> {
     let ext = filename.rsplit('.').next().unwrap_or_default();
     let format = InputFormat::from_extension(ext)
         .ok_or_else(|| format!("unknown or unsupported extension: {filename:?}"))?;
     let source = SourceDocument::from_bytes(filename.to_string(), format, bytes.to_vec());
     let result = DocumentConverter::new()
         .convert(source)
         .map_err(|e| e.to_string())?;
+    let image_mode = image_mode(images)?;
     match to.unwrap_or("md") {
-        "md" | "markdown" => Ok(result.document.export_to_markdown()),
+        // `Referenced` is deliberately unreachable here: it hands the caller
+        // loose image files to write next to the Markdown, which a page with no
+        // filesystem cannot do — the browser equivalent is `embedded`.
+        "md" | "markdown" => Ok(result
+            .document
+            .export_to_markdown_with_images(image_mode, "artifacts")
+            .0),
         "json" => Ok(result.document.export_to_json()),
         "doclang" => Ok(result.document.export_to_doclang()),
         other => Err(format!(
@@ -57,13 +69,36 @@ fn convert_impl(bytes: &[u8], filename: &str, to: Option<&str>) -> Result<String
     }
 }
 
+/// Picture rendering for Markdown output, mirroring docling-serve's `images`
+/// option: `placeholder` (docling's default `<!-- image -->`) or `embedded`
+/// (`![Image](data:…;base64,…)`, self-contained — the only way to carry pixels
+/// out of a page that cannot write files).
+fn image_mode(images: Option<&str>) -> Result<ImageMode, String> {
+    match images.unwrap_or("placeholder") {
+        "placeholder" => Ok(ImageMode::Placeholder),
+        "embedded" => Ok(ImageMode::Embedded),
+        other => Err(format!(
+            "unknown images={other:?} (expected \"placeholder\" or \"embedded\")"
+        )),
+    }
+}
+
 /// Convert a document (as bytes + filename, the extension drives format
 /// detection) to `to`: `"md"` (Markdown, default), `"json"` (docling-core's
 /// `DoclingDocument` wire format, schema 1.10.0) or `"doclang"` (docling's
 /// DocLang XML serialization).
+///
+/// `images` controls how pictures render in Markdown — `"placeholder"`
+/// (default) or `"embedded"` (base64 data URIs), the same option
+/// docling-serve exposes.
 #[wasm_bindgen]
-pub fn convert(bytes: &[u8], filename: &str, to: Option<String>) -> Result<String, JsError> {
-    convert_impl(bytes, filename, to.as_deref()).map_err(|e| JsError::new(&e))
+pub fn convert(
+    bytes: &[u8],
+    filename: &str,
+    to: Option<String>,
+    images: Option<String>,
+) -> Result<String, JsError> {
+    convert_impl(bytes, filename, to.as_deref(), images.as_deref()).map_err(|e| JsError::new(&e))
 }
 
 /// The file extensions this build can convert, as a JSON string array —
@@ -98,16 +133,16 @@ mod tests {
     #[test]
     fn markdown_roundtrip() {
         let md = b"# Title\n\nHello *world*\n";
-        let out = convert_impl(md, "note.md", None).unwrap();
+        let out = convert_impl(md, "note.md", None, None).unwrap();
         assert!(out.contains("# Title"));
-        let json = convert_impl(md, "note.md", Some("json")).unwrap();
+        let json = convert_impl(md, "note.md", Some("json"), None).unwrap();
         assert!(json.contains("\"schema_name\""));
     }
 
     #[test]
     fn ml_formats_rejected() {
         // Images still need the full ML pipeline.
-        let err = convert_impl(&[0x89, b'P', b'N', b'G'], "scan.png", None).unwrap_err();
+        let err = convert_impl(&[0x89, b'P', b'N', b'G'], "scan.png", None, None).unwrap_err();
         assert!(
             err.contains("unknown or unsupported") || err.contains("pdf"),
             "should reject the ML-only format: {err}"
@@ -129,7 +164,7 @@ mod tests {
             "/../../tests/data/pdf/sources/code_and_formula.pdf"
         ))
         .expect("corpus pdf");
-        let out = convert_impl(&bytes, "code_and_formula.pdf", None).unwrap();
+        let out = convert_impl(&bytes, "code_and_formula.pdf", None, None).unwrap();
         assert!(!out.trim().is_empty(), "text layer should extract");
     }
 
@@ -141,7 +176,7 @@ mod tests {
         if docling::PDF_ML_COMPILED {
             return;
         }
-        let err = convert_impl(b"%PDF-1.4\n%%EOF", "scan.pdf", None).unwrap_err();
+        let err = convert_impl(b"%PDF-1.4\n%%EOF", "scan.pdf", None, None).unwrap_err();
         assert!(
             err.contains("text layer") || err.contains("OCR"),
             "should point at the missing text layer: {err}"
@@ -156,10 +191,29 @@ mod tests {
             "/../docling/tests/data/docx/sources/docx_lists.docx"
         ))
         .expect("corpus docx");
-        let out = convert_impl(&bytes, "docx_lists.docx", None).unwrap();
+        let out = convert_impl(&bytes, "docx_lists.docx", None, None).unwrap();
         assert!(!out.trim().is_empty());
     }
 
+    /// `images=embedded` inlines picture bytes as data URIs (docling-serve's
+    /// option, and the only way a page with no filesystem can carry pixels
+    /// out); the default stays docling's `<!-- image -->` placeholder.
+    #[test]
+    fn embedded_images_inline_as_data_uris() {
+        let bytes = std::fs::read(concat!(
+            env!("CARGO_MANIFEST_DIR"),
+            "/../docling/tests/data/docx/sources/word_image_anchors.docx"
+        ))
+        .expect("corpus docx with images");
+        let placeholder = convert_impl(&bytes, "word_image_anchors.docx", None, None).unwrap();
+        assert!(placeholder.contains("<!-- image -->"), "{placeholder}");
+        let embedded =
+            convert_impl(&bytes, "word_image_anchors.docx", None, Some("embedded")).unwrap();
+        assert!(embedded.contains("](data:image/"), "expected a data URI");
+        let err = convert_impl(&bytes, "word_image_anchors.docx", None, Some("nope")).unwrap_err();
+        assert!(err.contains("unknown images="), "{err}");
+    }
+
     #[test]
     fn extensions_json_parses() {
         let v: Vec<String> = serde_json::from_str(&supported_extensions()).unwrap();

crates/docling-wasm/src/ocr.rs

@@ -27,7 +27,7 @@
 //!   },
 //! });
 //! ```
-//! (see `www/ocr.html` for the complete wiring, including output-name
+//! (see `www/index.html` for the complete wiring, including output-name
 //! discovery and model/dict caching.)
 
 use docling_core::{DoclingDocument, Node};

crates/docling-wasm/src/scanned.rs

@@ -18,7 +18,7 @@
 //! }
 //! const markdown = conv.finish("scan.pdf", "md");
 //! ```
-//! (`www/scan.html` is the complete wiring.)
+//! (`www/index.html` is the complete wiring.)
 
 use docling_pdf::layout::{decode_layout, layout_input, SIDE};
 use docling_pdf::ocr_prep::{
@@ -33,14 +33,6 @@ use wasm_bindgen::prelude::*;
 use crate::ocr::{tensor_parts, RecSession};
 use crate::tableformer::TfSession;
 
-#[wasm_bindgen]
-extern "C" {
-    // On-page diagnostic sink (defined by the host — worker.js forwards it to
-    // the main thread, which shows it; console isn't reachable on a phone).
-    #[wasm_bindgen(js_name = __docling_diag)]
-    fn diag(s: &str);
-}
-
 #[wasm_bindgen]
 extern "C" {
     /// The JS-side layout session: a wrapper around an `ort.InferenceSession`
@@ -188,19 +180,6 @@ impl ScannedConverter {
         let regions = decode_layout(&logits, &boxes, q, c, page_w, page_h);
         let regions = refine_regions(regions, &[], page_w, page_h);
 
-        // Diagnostic: the region-label histogram, so it's visible (browser
-        // console) whether the layout model flagged any `table` region on this
-        // page — tables only render when it does (geometric or TableFormer).
-        {
-            let mut hist: std::collections::BTreeMap<&str, usize> =
-                std::collections::BTreeMap::new();
-            for r in &regions {
-                *hist.entry(r.label).or_default() += 1;
-            }
-            let summary: Vec<String> = hist.iter().map(|(l, n)| format!("{l}×{n}")).collect();
-            diag(&format!("layout regions: {}", summary.join(", ")));
-        }
-
         // OCR the text regions (same gather/batch/decode as native ocr_page).
         let (bboxes, lines) = prep_region_lines(&img, &regions, scale);
         let texts = self.ocr_lines(rec, &lines).await?;
@@ -268,16 +247,26 @@ impl ScannedConverter {
     }
 
     /// Assemble the accumulated pages into the final document and render it
-    /// as `"md"` (default) or `"json"`. Resets the converter.
+    /// as `"md"` (default), `"json"` or `"doclang"` — the same three the
+    /// declarative [`crate::convert`] entry point offers. Resets the converter.
     pub fn finish(&mut self, name: &str, to: Option<String>) -> Result<String, JsError> {
         let doc = finish_document(name, std::mem::take(&mut self.pages));
-        match to.as_deref().unwrap_or("md") {
-            "md" | "markdown" => Ok(doc.export_to_markdown()),
-            "json" => Ok(doc.export_to_json()),
-            other => Err(JsError::new(&format!(
-                "unknown output format {other:?} (expected \"md\" or \"json\")"
-            ))),
-        }
+        render(&doc, to.as_deref())
+    }
+}
+
+/// Render an assembled document in one of the three output grammars. The OCR
+/// pipeline recovers no picture *pixels* (regions are recognized, not cropped
+/// out), so Markdown always uses docling's `<!-- image -->` placeholder — there
+/// is nothing to embed, unlike the declarative path's `images` option.
+fn render(doc: &docling_core::DoclingDocument, to: Option<&str>) -> Result<String, JsError> {
+    match to.unwrap_or("md") {
+        "md" | "markdown" => Ok(doc.export_to_markdown()),
+        "json" => Ok(doc.export_to_json()),
+        "doclang" => Ok(doc.export_to_doclang()),
+        other => Err(JsError::new(&format!(
+            "unknown output format {other:?} (expected \"md\", \"json\" or \"doclang\")"
+        ))),
     }
 }
 

crates/docling-wasm/src/tableformer.rs

@@ -9,7 +9,7 @@
 //! encoder's constant cross-attention K/V and `enc_out`, and the growing
 //! decoder KV-cache, entirely on the JS side. Each decode step sends only the
 //! last tag (one int) and gets back `logits` + `hidden` (525 floats). See
-//! `www/scan.html` for the JS wiring (the `decoder_kv` graph: N_LAYERS=6,
+//! `www/index.html` for the JS wiring (the `decoder_kv` graph: N_LAYERS=6,
 //! KV_HEADS=8, head_dim=64, cross length 784).
 
 use docling_pdf::pdfium_backend::TextCell;