wasm: lite-profile scanned pipeline in the browser (#157 stage 2)

Scanned PDFs and images now convert fully client-side through the real
pipeline shape: RT-DETR layout detection + PP-OCRv3 recognition via
ONNX Runtime Web, everything else in Rust — the native pipeline's own
functions, no second implementation.

docling-pdf (all under the existing ocr-prep feature, wasm-safe):
- layout.rs: input packing (640x640 CHW) and RT-DETR decoding (sigmoid
  top-q, center->corners, per-label thresholds downstream) extracted as
  layout_input/decode_layout; the native batch path now decodes through
  the same function per page.
- ocr_prep: prep_region_lines — the exact region-crop/line-split/prep
  gathering native ocr_page does — moved out and shared; ocr.rs
  consumes it.
- new scanned module: the Worker::process refinement chain
  (label thresholds -> resolve -> orphans -> false-picture drops ->
  contained drops), page assembly with the geometric table fallback
  (the lite profile: TableFormer is stage 3), and cross-page
  continuation merging, re-exposed without the ml feature.
Native path byte-identical after the extraction — the scanned-tiff
conversion diffs empty against the pre-refactor output (rebuilt and
re-verified after a disk-full broke the first attempt), full
docling-pdf/docling suites green.

docling-wasm:
- ScannedConverter: feed pdf.js-rendered RGBA pages ({scale: 2} — the
  native RENDER_SCALE) one by one, finish() assembles the document with
  the same continuation merging as native; page_count() for progress.
- convert_scanned_image: one-shot image path (its own page at scale 1,
  like native).
- LayoutSession interop mirrors RecSession: run(data) ->
  {logits: {data, dims}, boxes: {data, dims}}.
- www/scan.html: complete demo — pdf.js from CDN, layout int8
  (~165 MB, models-v1 release, browser-cached) + rec model + dict,
  per-page progress, timing.

Stage 3 (TableFormer + enrichment in the browser) remains: the
autoregressive decoder loop is deeply ort-typed and needs its own
careful extraction pass; table regions meanwhile take the geometric
reconstruction, matching --no-table-former.

Refs #157

Signed-off-by: artiz <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT

Author: artiz

crates/docling-pdf/src/layout.rs

@@ -47,13 +47,12 @@ pub struct Region {
     pub b: f32,
 }
 
-#[cfg(feature = "ml")]
 /// Base confidence threshold (docling-ibm-models `base_threshold`): the raw
 /// RT-DETR floor before docling's `LayoutPostprocessor` applies its stricter
 /// per-label thresholds ([`label_threshold`]).
 const THRESHOLD: f32 = 0.3;
-#[cfg(feature = "ml")]
-const SIDE: u32 = 640;
+/// RT-DETR's fixed square input side.
+pub const SIDE: u32 = 640;
 
 /// Per-label confidence threshold, ported from docling's
 /// `LayoutPostprocessor.CONFIDENCE_THRESHOLDS`. The raw predictor keeps every
@@ -222,46 +221,81 @@ impl LayoutModel {
             let logits =
                 &logits[p * num_queries * num_classes..(p + 1) * num_queries * num_classes];
             let boxes = &boxes[p * num_queries * 4..(p + 1) * num_queries * 4];
-
-            // sigmoid over every (query, class); take the top `num_queries` scores.
-            let mut scored: Vec<(f32, usize)> = (0..num_queries * num_classes)
-                .map(|idx| (sigmoid(logits[idx]), idx))
-                .collect();
-            scored.sort_unstable_by(|a, b| b.0.total_cmp(&a.0));
-            scored.truncate(num_queries);
-
-            let mut regions = Vec::new();
-            for (score, idx) in scored {
-                if score <= THRESHOLD {
-                    continue;
-                }
-                let label_id = idx % num_classes;
-                let q = idx / num_classes;
-                let cx = boxes[q * 4];
-                let cy = boxes[q * 4 + 1];
-                let w = boxes[q * 4 + 2];
-                let h = boxes[q * 4 + 3];
-                // center_to_corners, then scale normalized coords to page points.
-                let l = (cx - w / 2.0) * page_w;
-                let t = (cy - h / 2.0) * page_h;
-                let r = (cx + w / 2.0) * page_w;
-                let b = (cy + h / 2.0) * page_h;
-                regions.push(Region {
-                    label: LABELS.get(label_id).copied().unwrap_or("text"),
-                    score,
-                    l,
-                    t,
-                    r,
-                    b,
-                });
-            }
-            all.push(regions);
+            all.push(decode_layout(
+                logits,
+                boxes,
+                num_queries,
+                num_classes,
+                *page_w,
+                *page_h,
+            ));
         }
         Ok(all)
     }
 }
 
-#[cfg(feature = "ml")]
 fn sigmoid(x: f32) -> f32 {
     1.0 / (1.0 + (-x).exp())
 }
+
+/// Pack one page image into the model's `(1, 3, SIDE, SIDE)` input: resize
+/// (aspect ignored, RT-DETR convention), rescale to `[0,1]`, CHW. Shared
+/// with the browser build (#157), which delegates only the session call.
+#[cfg(feature = "ocr-prep")]
+pub fn layout_input(img: &image::RgbImage) -> Vec<f32> {
+    let n = (SIDE * SIDE) as usize;
+    let mut data = vec![0f32; 3 * n];
+    let resized = image::imageops::resize(img, SIDE, SIDE, image::imageops::FilterType::Triangle);
+    for (i, px) in resized.pixels().enumerate() {
+        data[i] = px[0] as f32 / 255.0;
+        data[n + i] = px[1] as f32 / 255.0;
+        data[2 * n + i] = px[2] as f32 / 255.0;
+    }
+    data
+}
+
+/// Decode one page's raw RT-DETR outputs into scored [`Region`]s in page
+/// points — sigmoid over every (query, class), top-`num_queries` kept, boxes
+/// converted center→corners and scaled. Shared with the browser build; the
+/// native batch path calls it per page, so both decode identically.
+pub fn decode_layout(
+    logits: &[f32],
+    boxes: &[f32],
+    num_queries: usize,
+    num_classes: usize,
+    page_w: f32,
+    page_h: f32,
+) -> Vec<Region> {
+    let mut scored: Vec<(f32, usize)> = (0..num_queries * num_classes)
+        .map(|idx| (sigmoid(logits[idx]), idx))
+        .collect();
+    scored.sort_unstable_by(|a, b| b.0.total_cmp(&a.0));
+    scored.truncate(num_queries);
+
+    let mut regions = Vec::new();
+    for (score, idx) in scored {
+        if score <= THRESHOLD {
+            continue;
+        }
+        let label_id = idx % num_classes;
+        let q = idx / num_classes;
+        let cx = boxes[q * 4];
+        let cy = boxes[q * 4 + 1];
+        let w = boxes[q * 4 + 2];
+        let h = boxes[q * 4 + 3];
+        // center_to_corners, then scale normalized coords to page points.
+        let l = (cx - w / 2.0) * page_w;
+        let t = (cy - h / 2.0) * page_h;
+        let r = (cx + w / 2.0) * page_w;
+        let b = (cy + h / 2.0) * page_h;
+        regions.push(Region {
+            label: LABELS.get(label_id).copied().unwrap_or("text"),
+            score,
+            l,
+            t,
+            r,
+            b,
+        });
+    }
+    regions
+}

crates/docling-pdf/src/lib.rs

@@ -33,6 +33,8 @@ pub mod pdfium_backend;
 mod reading_order;
 #[cfg(feature = "ml")]
 pub mod resample;
+#[cfg(feature = "ocr-prep")]
+pub mod scanned;
 #[cfg(feature = "ml")]
 pub mod tableformer;
 pub mod textparse;

crates/docling-pdf/src/ocr.rs

@@ -5,16 +5,15 @@
 //! is recognised and decoded with CTC — producing [`TextCell`]s the normal
 //! layout assembly then consumes. This avoids a separate text-detection model.
 
-use image::{imageops, RgbImage};
+use image::RgbImage;
 use ort::session::Session;
 use ort::value::Tensor;
 
 use crate::layout::Region;
 // The ONNX-free half (line prep, batching, CTC decode) lives in `ocr_prep`
 // so the wasm build shares it verbatim (#79 phase 2).
 use crate::ocr_prep::{
-    batch_input, decode_row, dict_chars, prep_line, segment_lines, width_batches, PrepLine,
-    REC_HEIGHT,
+    batch_input, decode_row, dict_chars, prep_region_lines, width_batches, PrepLine, REC_HEIGHT,
 };
 use crate::pdfium_backend::TextCell;
 
@@ -24,21 +23,6 @@ pub struct OcrModel {
     chars: Vec<String>,
 }
 
-/// Layout labels whose content is recognised as running text.
-fn is_text_label(label: &str) -> bool {
-    matches!(
-        label,
-        "text"
-            | "title"
-            | "section_header"
-            | "list_item"
-            | "caption"
-            | "footnote"
-            | "code"
-            | "formula"
-    )
-}
-
 /// OCR recognition language: which PP-OCRv3 model + dictionary pair runs.
 ///
 /// The default is **English** (`models/ocr_rec_en.onnx` + `models/en_dict.txt`):
@@ -183,37 +167,10 @@ impl OcrModel {
         regions: &[Region],
         scale: f32,
     ) -> Result<Vec<TextCell>, String> {
-        let (iw, ih) = img.dimensions();
-        // Gather every line crop on the page first, so equal-width lines can
-        // share a recognition run regardless of which region they came from.
-        let mut bboxes: Vec<(f32, f32, f32, f32)> = Vec::new();
-        let mut lines: Vec<PrepLine> = Vec::new();
-        for region in regions {
-            if !is_text_label(region.label) {
-                continue;
-            }
-            let l = (region.l * scale).max(0.0) as u32;
-            let t = (region.t * scale).max(0.0) as u32;
-            let r = ((region.r * scale).max(0.0) as u32).min(iw);
-            let b = ((region.b * scale).max(0.0) as u32).min(ih);
-            if r <= l || b <= t {
-                continue;
-            }
-            let crop = imageops::crop_imm(img, l, t, r - l, b - t).to_image();
-            for (lx, ly, rx, ry) in segment_lines(&crop) {
-                let line = imageops::crop_imm(&crop, lx, ly, rx - lx, ry - ly).to_image();
-                let Some(pl) = prep_line(&line) else {
-                    continue;
-                };
-                bboxes.push((
-                    (l + lx) as f32 / scale,
-                    (t + ly) as f32 / scale,
-                    (l + rx) as f32 / scale,
-                    (t + ry) as f32 / scale,
-                ));
-                lines.push(pl);
-            }
-        }
+        // Gather every line crop on the page first (shared with the browser
+        // path), so equal-width lines can share a recognition run regardless
+        // of which region they came from.
+        let (bboxes, lines) = prep_region_lines(img, regions, scale);
 
         // Deterministic width-batching (shared with the wasm path).
         let mut texts = vec![String::new(); lines.len()];

crates/docling-pdf/src/ocr_prep.rs

@@ -152,6 +152,66 @@ pub fn segment_lines(crop: &RgbImage) -> Vec<(u32, u32, u32, u32)> {
         .collect()
 }
 
+/// Layout labels whose content is recognised as running text.
+pub fn is_text_label(label: &str) -> bool {
+    matches!(
+        label,
+        "text"
+            | "title"
+            | "section_header"
+            | "list_item"
+            | "caption"
+            | "footnote"
+            | "code"
+            | "formula"
+    )
+}
+
+/// A line's page-point bounding box, `(l, t, r, b)`.
+pub type LineBox = (f32, f32, f32, f32);
+
+/// Gather every text-region line crop on a page, in page order: crop each
+/// text region (page points × `scale` → image px), split it into lines, prep
+/// each line, and keep the line's page-point bbox. The exact gathering the
+/// native `ocr_page` does — shared so the browser path produces the same
+/// cells given the same probabilities.
+pub fn prep_region_lines(
+    img: &RgbImage,
+    regions: &[crate::layout::Region],
+    scale: f32,
+) -> (Vec<LineBox>, Vec<PrepLine>) {
+    let (iw, ih) = img.dimensions();
+    let mut bboxes = Vec::new();
+    let mut lines = Vec::new();
+    for region in regions {
+        if !is_text_label(region.label) {
+            continue;
+        }
+        let l = (region.l * scale).max(0.0) as u32;
+        let t = (region.t * scale).max(0.0) as u32;
+        let r = ((region.r * scale).max(0.0) as u32).min(iw);
+        let b = ((region.b * scale).max(0.0) as u32).min(ih);
+        if r <= l || b <= t {
+            continue;
+        }
+        let crop = imageops::crop_imm(img, l, t, r - l, b - t).to_image();
+        for (lx, ly, rx, ry) in segment_lines(&crop) {
+            let line = imageops::crop_imm(&crop, lx, ly, rx - lx, ry - ly).to_image();
+            let Some(pl) = prep_line(&line) else {
+                continue;
+            };
+            bboxes.push((
+                (l + lx) as f32 / scale,
+                (t + ly) as f32 / scale,
+                (l + rx) as f32 / scale,
+                (t + ry) as f32 / scale,
+            ));
+            lines.push(pl);
+        }
+    }
+    (bboxes, lines)
+}
+
 /// Whole-image line preparation for the browser OCR path (no layout model:
 /// the page itself is the single text region). Returns page-order prepared
 /// lines; callers that need geometry use [`segment_lines`] directly.

crates/docling-pdf/src/scanned.rs

@@ -0,0 +1,54 @@
+//! Scanned-page assembly facade for the browser build (#157 stage 2).
+//!
+//! The native `Worker::process` chain, re-exposed without the `ml` feature so
+//! `docling-wasm` can run it around JS-delegated inference: refine the raw
+//! layout detections exactly like the native pipeline, then assemble the page
+//! with the geometric table fallback (the lite profile — TableFormer is
+//! stage 3) and no enrichments. One shared implementation, one behavior.
+
+use docling_core::{DoclingDocument, Node};
+
+/// One assembled page: its nodes and `(anchor, href)` hyperlink pairs.
+pub type AssembledPage = (Vec<Node>, Vec<(String, String)>);
+
+use crate::layout::{label_threshold, Region};
+use crate::pdfium_backend::{PdfPage, TextCell};
+
+/// The native pipeline's region-refinement chain, in its exact order:
+/// per-label score thresholds → overlap resolution → orphan text regions for
+/// detector-missed cells → false-picture drops → contained-regular drops.
+/// `cells` is the page's (possibly empty, pre-OCR) text-cell set.
+pub fn refine_regions(
+    regions: Vec<Region>,
+    cells: &[TextCell],
+    page_w: f32,
+    page_h: f32,
+) -> Vec<Region> {
+    let mut regions = regions;
+    regions.retain(|r| r.score >= label_threshold(r.label));
+    let mut regions = crate::assemble::resolve(regions);
+    crate::assemble::add_orphan_regions(&mut regions, cells);
+    crate::assemble::drop_false_pictures(&mut regions, cells, page_w, page_h);
+    crate::assemble::drop_contained_regulars(&mut regions);
+    regions
+}
+
+/// Assemble one refined page — geometric tables (no TableFormer), no
+/// enrichments — into its nodes and hyperlink pairs.
+pub fn assemble_page(page: &PdfPage, regions: Vec<Region>) -> AssembledPage {
+    let table_rows = vec![None; regions.len()];
+    let enrich = vec![None; regions.len()];
+    crate::assemble::assemble_page(page, regions, &table_rows, &enrich)
+}
+
+/// Stitch per-page results into a document: cross-page paragraph
+/// continuations merge exactly like the native pipeline's final pass.
+pub fn finish_document(name: &str, pages: Vec<AssembledPage>) -> DoclingDocument {
+    let mut doc = DoclingDocument::new(name);
+    for (nodes, links) in pages {
+        doc.nodes.extend(nodes);
+        doc.links.extend(links);
+    }
+    crate::assemble::merge_continuations(&mut doc.nodes);
+    doc
+}

crates/docling-wasm/README.md

@@ -83,10 +83,32 @@ const rec = {
 const markdown = await ocr_image(imageBytes, dictText, rec, "md");
 ```
 
-[`www/ocr.html`](./www/ocr.html) is the complete demo (model/dict fetched
-from their public hosting and browser-cached). Whole-image projection
-segmentation only — best on single-column scans; layout detection is
-stage 2, scanned PDFs (pdf.js rasterization) arrive with it.
+[`www/ocr.html`](./www/ocr.html) is the recognition-only demo (model/dict
+fetched from their public hosting and browser-cached).
+
+**Stage 2 — the lite profile** (`ScannedConverter` / `convert_scanned_image`):
+RT-DETR layout detection (int8, ~165 MB from the `models-v1` release) +
+region-cropped OCR + reading-order assembly, with tables via the geometric
+reconstruction (the native `--no-table-former` path — TableFormer is
+stage 3). Scanned **PDFs** convert too: the host page rasterizes pages with
+pdf.js at `{scale: 2}` (2 px/point — the native pipeline's `RENDER_SCALE`)
+and feeds the bitmaps page by page:
+
+```js
+const conv = new ScannedConverter(dictText);
+for (let p = 1; p <= pdf.numPages; p++) {
+  const viewport = page.getViewport({ scale: 2 });
+  // … render to canvas, then:
+  await conv.add_page(rgba, canvas.width, canvas.height, 2.0, layout, rec);
+}
+const markdown = conv.finish(file.name, "md");
+```
+
+[`www/scan.html`](./www/scan.html) is the complete demo (pdf.js + both
+models). Region refinement, OCR gathering/batching/decoding and page
+assembly are the native pipeline's own functions
+(`docling_pdf::{layout, ocr_prep, scanned}`) — the wasm path adds no second
+implementation.
 
 ## Host-side tests
 

crates/docling-wasm/src/lib.rs

@@ -22,7 +22,11 @@ use wasm_bindgen::prelude::*;
 #[cfg(feature = "ocr")]
 mod ocr;
 #[cfg(feature = "ocr")]
+mod scanned;
+#[cfg(feature = "ocr")]
 pub use ocr::ocr_image;
+#[cfg(feature = "ocr")]
+pub use scanned::{convert_scanned_image, ScannedConverter};
 
 #[wasm_bindgen(start)]
 fn start() {