Skip to content

Commit b1f3af1

Browse files
authored
Merge pull request #22 from artiz/claude/focused-curie-zkkv2p
feat(html): key-value form regions (docling field_region/field_item)
2 parents cb7ed00 + 1246547 commit b1f3af1

28 files changed

Lines changed: 1954 additions & 1272 deletions

MIGRATION.md

Lines changed: 22 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ PyPI; run via `scripts/conformance.sh <fmt>`), not the committed groundtruth
6666
|---|---|---|
6767
| Markdown | `markdown.rs` (pulldown-cmark) | **10/10 exact** |
6868
| CSV | `csv.rs` (`csv` crate) | **9/9 exact** |
69-
| HTML | `html.rs` (scraper/html5ever) | **28/32 exact** (rest need a headless browser — §5) |
69+
| HTML | `html.rs` (scraper/html5ever) | **30/32 exact** (rest need a headless browser — §5) |
7070
| AsciiDoc | `asciidoc.rs` (regex) | **4/4 exact** |
7171
| DeepSeek-OCR Markdown | `deepseek.rs` | **3/3 exact** (auto-detected VLM-token variant) |
7272
| XLSX | `xlsx.rs` (calamine) | **9/9 exact** |
@@ -187,9 +187,11 @@ These are deliberate or unavoidable divergences, not bugs.
187187
the converter routes by content markers (`us-patent` → USPTO, `us-gaap`/`dei`
188188
→ XBRL, else JATS) rather than the extension alone.
189189

190-
8. **No headless-browser pass.** A few HTML behaviours depend on rendering the
191-
page (nav/visibility suppression, form key-value regions, rendered bounding
192-
boxes) — see §5.
190+
8. **Headless-browser pass is opt-in.** Form key-value regions and inline
191+
visibility are handled statically by default. Stylesheet-driven (CSS-cascade)
192+
nav/visibility suppression needs a rendered page, available behind the
193+
optional `web-browser` feature / `--use-web-browser` flag (Rust-driven
194+
Chromium); rendered-bounding-box nested-table padding is still out — see §5.
193195

194196
---
195197

@@ -203,9 +205,6 @@ Explicitly **not done**, with the reason:
203205
classification, formula understanding, code understanding). Model-bound; out of
204206
scope for the discriminative port.
205207

206-
(**TableFormer is now done** — ported to ONNX and run on every table region; see
207-
§5 and `PDF_CONFORMANCE.md`. Geometric reconstruction remains only as the fallback
208-
when the TableFormer graphs aren't present.)
209208
- **XML DocLang / DocTags** input backend — no `.dclg` sources in the corpus to
210209
verify against, and not in the requested scope.
211210
- **Older patent schemas.** USPTO covers the modern `v4x` XML only; the
@@ -228,9 +227,22 @@ when the TableFormer graphs aren't present.)
228227
inline-group spacing and stay attached to their list item; `\operatorname`
229228
functions, limit-label space escaping and the two-space symbol padding match
230229
pylatexenc byte-for-byte).
231-
- **HTML browser-render subsystem** — nav/visibility suppression (`wiki_duck`),
232-
form key-value-pair regions (`kvp_data_example`), deep nested-table cell padding
233-
from rendered bounding boxes. ~4 HTML fixtures + KVP.
230+
- **HTML browser-render subsystem** — the browser-free parts are **done**: form
231+
key-value regions (`kvp_data_example`, detected statically from docling's
232+
`keyN` / `keyN_valueM` / `keyN_marker` `id`-convention), docling-faithful
233+
inline-image handling (inline images emit nothing; only block / `<a>`-wrapped
234+
/ `<figure>` images become pictures), and inline visibility suppression
235+
(`hidden` / inline `display:none` / `visibility:hidden`).
236+
For stylesheet-driven (CSS-cascade) visibility — `wiki_duck`'s collapsed
237+
menus, kept distinct from the still-visible table of contents — there is now
238+
an **optional headless-browser pre-render** behind the `web-browser` Cargo
239+
feature / `--use-web-browser` flag: it drives the system Chromium from Rust
240+
(via `headless_chrome`, no Node/Playwright), strips computed-`display:none`
241+
subtrees, and hands the cleaned HTML back to the Rust backend. It resolves the
242+
cascade only when the page's CSS is reachable (inline `<style>`, or external
243+
stylesheets fetchable with a base host), so a saved page whose stylesheets are
244+
external + offline still needs the real network. Deep nested-table cell
245+
padding from rendered bounding boxes remains the last rendered-geometry gap.
234246

235247

236248
---

README.md

Lines changed: 34 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,11 @@ with [`mail-parser`](https://crates.io/crates/mail-parser) (which conforms to
3636
routed through the HTML backend, with embedded images resolved from the
3737
archive by `Content-Location`/`cid:`. The discriminative PDF/image pipeline
3838
lives in `fleischwolf-pdf`: a pure-Rust PDF text parser, pdfium for page
39-
rasterization, and an ONNX layout/TableFormer/OCR stack. Audio/ASR is the main
40-
format still on the roadmap (see `MIGRATION.md`).
39+
rasterization, and an ONNX layout/TableFormer/OCR stack. TableFormer is ported
40+
to ONNX and run on every detected table region to recover its structure;
41+
geometric reconstruction from cell positions remains only as the fallback when
42+
the TableFormer graphs aren't present (see `PDF_CONFORMANCE.md`). Audio/ASR is
43+
the main format still on the roadmap (see `MIGRATION.md`).
4144

4245
Output is checked against upstream Python docling — declarative formats
4346
byte-for-byte against live docling, the ML pipeline against a deterministic
@@ -170,6 +173,35 @@ by a wide margin, but a scanned/image-only PDF (no embedded text layer) comes
170173
back empty rather than erroring, so a caller can detect that and re-convert
171174
without the flag.
172175

176+
### Headless-browser HTML pre-render (optional)
177+
178+
Almost everything in the HTML backend is pure Rust, but one thing a static
179+
parse can't do is resolve the **CSS cascade** — whether a stylesheet- or
180+
class-driven rule makes an element `display:none` (e.g. a collapsed nav menu).
181+
The optional `--use-web-browser` flag renders the page in the system Chromium
182+
first, drops every element the browser computes as hidden, then feeds the
183+
cleaned HTML through the normal Rust backend (so all structure/table/KVP/
184+
formatting logic still runs in Rust — the browser only decides visibility). It
185+
applies to every HTML-routing input: direct HTML, plus MHTML and EPUB (which
186+
assemble HTML from their archives). It's driven straight from Rust over the
187+
DevTools protocol via
188+
[`headless_chrome`](https://crates.io/crates/headless_chrome) — no Node,
189+
Playwright, or other runtime.
190+
191+
It's gated behind the off-by-default `web-browser` Cargo feature, so the standard
192+
build stays browser-dependency-free:
193+
194+
```bash
195+
cargo run -p fleischwolf-cli --features web-browser -- --use-web-browser page.html
196+
```
197+
198+
Chromium is located via `$FLEISCHWOLF_CHROME`/`$CHROME`, then
199+
`$PLAYWRIGHT_BROWSERS_PATH/chromium`, else autodetected. The page's CSS must be
200+
reachable for the cascade to resolve — inline `<style>` works offline, but a
201+
saved page that links external stylesheets needs those fetchable (with a base
202+
host). Without the feature, `--use-web-browser` is a clear error rather than a
203+
silent no-op.
204+
173205
## Node.js / Bun bindings
174206

175207
Fleischwolf ships as an npm package, [**`fleischwolf`**](https://www.npmjs.com/package/fleischwolf)

crates/fleischwolf-cli/Cargo.toml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,5 +15,11 @@ rust-version.workspace = true
1515
name = "fleischwolf"
1616
path = "src/main.rs"
1717

18+
[features]
19+
# Pass through to the library's optional headless-browser HTML pre-render, so
20+
# `cargo run -p fleischwolf-cli --features web-browser -- --use-web-browser …`
21+
# activates it. Off by default (keeps the default binary browser-dependency-free).
22+
web-browser = ["fleischwolf/web-browser"]
23+
1824
[dependencies]
1925
fleischwolf = { path = "../fleischwolf", version = "0.10.0" }

crates/fleischwolf-cli/src/main.rs

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
//! A stand-in for `docling.cli.main`; the full Typer-style CLI (batch mode,
44
//! pipeline options) is a later phase.
55
//!
6-
//! Usage: fleischwolf [--strict] [--to md|json] [--images MODE] [--fetch-images] [--no-stream] [--no-table-former] [--no-ocr] <input-file>
6+
//! Usage: fleischwolf [--strict] [--to md|json] [--images MODE] [--fetch-images] [--no-stream] [--no-table-former] [--no-ocr] [--use-web-browser] <input-file>
77
//! --to md|json output format (default: md). `json` emits docling-core's
88
//! native DoclingDocument JSON (export_to_dict).
99
//! --images MODE picture handling for Markdown (mirrors docling's
@@ -31,6 +31,10 @@
3131
//! fastest option, but a scanned/image-only PDF (no
3232
//! embedded text layer) yields no text — convert those
3333
//! without this flag.
34+
//! --use-web-browser pre-render HTML/MHTML/EPUB in the system Chromium (driven
35+
//! from Rust) so stylesheet-driven `display:none` elements
36+
//! (e.g. a collapsed nav menu) are dropped before parsing.
37+
//! Requires building with `--features web-browser`.
3438
3539
use std::io::{self, Write};
3640
use std::path::Path;
@@ -46,6 +50,7 @@ fn main() -> ExitCode {
4650
let mut no_stream = false;
4751
let mut no_table_former = false;
4852
let mut no_ocr = false;
53+
let mut use_web_browser = false;
4954
let mut bench_warm: Option<usize> = None;
5055
let mut path: Option<String> = None;
5156
let mut args = std::env::args().skip(1);
@@ -56,6 +61,7 @@ fn main() -> ExitCode {
5661
"--no-stream" => no_stream = true,
5762
"--no-table-former" => no_table_former = true,
5863
"--no-ocr" => no_ocr = true,
64+
"--use-web-browser" => use_web_browser = true,
5965
"--to" => to = args.next().unwrap_or_default(),
6066
"--images" => images = args.next().unwrap_or_default(),
6167
// Hidden benchmarking aid: load the PDF/image pipeline once, then time
@@ -90,7 +96,7 @@ fn main() -> ExitCode {
9096
};
9197

9298
let Some(path) = path else {
93-
eprintln!("usage: fleischwolf [--strict] [--to md|json] [--images MODE] [--fetch-images] [--no-stream] [--no-table-former] [--no-ocr] <input-file>");
99+
eprintln!("usage: fleischwolf [--strict] [--to md|json] [--images MODE] [--fetch-images] [--no-stream] [--no-table-former] [--no-ocr] [--use-web-browser] <input-file>");
94100
return ExitCode::from(2);
95101
};
96102

@@ -124,7 +130,8 @@ fn main() -> ExitCode {
124130
.strict(strict)
125131
.fetch_images(fetch_images)
126132
.no_table_former(no_table_former)
127-
.no_ocr(no_ocr);
133+
.no_ocr(no_ocr)
134+
.use_web_browser(use_web_browser);
128135

129136
// Stream Markdown by default: print each chunk as the converter produces it
130137
// (page by page for PDF). JSON needs the whole tree, and the referenced image

crates/fleischwolf-core/src/document.rs

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,22 @@ pub enum Node {
6565
},
6666
/// A logical grouping of child nodes (e.g. a list, a section).
6767
Group { label: String, children: Vec<Node> },
68+
/// A form key-value region (docling's `field_region`): a set of form fields,
69+
/// each pairing an optional marker, key, and value. Backends detect these
70+
/// from form structure (e.g. HTML's `keyN` / `keyN_valueM` / `keyN_marker`
71+
/// `id`-convention); the serializers render each item's parts as separate
72+
/// labelled texts (`marker` / `field_key` / `field_value`).
73+
FieldRegion { items: Vec<FieldItem> },
74+
}
75+
76+
/// One entry of a [`Node::FieldRegion`]: a marker/key/value triple, any of which
77+
/// may be absent. Mirrors docling's `field_item` with its `marker` / `field_key`
78+
/// / `field_value` child texts.
79+
#[derive(Debug, Clone, PartialEq, Default)]
80+
pub struct FieldItem {
81+
pub marker: Option<String>,
82+
pub key: Option<String>,
83+
pub value: Option<String>,
6884
}
6985

7086
/// An extracted picture's raw encoded bytes plus its mimetype and pixel size —

crates/fleischwolf-core/src/json.rs

Lines changed: 69 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ pub fn to_json(doc: &DoclingDocument) -> Value {
9797
let mut b = Builder::default();
9898
let body = b.walk_into(&doc.nodes, "#/body");
9999

100-
json!({
100+
let mut out = json!({
101101
"schema_name": "DoclingDocument",
102102
"version": SCHEMA_VERSION,
103103
"name": doc.name,
@@ -127,7 +127,23 @@ pub fn to_json(doc: &DoclingDocument) -> Value {
127127
"key_value_items": [],
128128
"form_items": [],
129129
"pages": {},
130-
})
130+
});
131+
132+
// docling only emits `field_regions` / `field_items` when a document has
133+
// form fields, and places them just before `pages`. Insert them in that slot
134+
// (re-appending `pages` afterwards, since `preserve_order` keeps insertion
135+
// order) so non-KVP documents' JSON is byte-identical to before.
136+
if !b.field_regions.is_empty() {
137+
if let Some(obj) = out.as_object_mut() {
138+
let pages = obj.remove("pages");
139+
obj.insert("field_regions".into(), Value::Array(b.field_regions));
140+
obj.insert("field_items".into(), Value::Array(b.field_items));
141+
if let Some(pages) = pages {
142+
obj.insert("pages".into(), pages);
143+
}
144+
}
145+
}
146+
out
131147
}
132148

133149
#[derive(Default)]
@@ -136,6 +152,8 @@ struct Builder {
136152
groups: Vec<Value>,
137153
tables: Vec<Value>,
138154
pictures: Vec<Value>,
155+
field_regions: Vec<Value>,
156+
field_items: Vec<Value>,
139157
}
140158

141159
impl Builder {
@@ -165,11 +183,60 @@ impl Builder {
165183
Some(self.add_picture(caption.as_deref(), image.as_ref(), parent))
166184
}
167185
Node::Group { label, children } => Some(self.add_group(label, children, parent)),
186+
Node::FieldRegion { items } => Some(self.add_field_region(items, parent)),
168187
// Handled by `add_list` in `walk`.
169188
Node::ListItem { .. } => None,
170189
}
171190
}
172191

192+
/// A form key-value region: `field_regions/N` holds the region, each field is
193+
/// a `field_items/M` whose children are its `marker` / `field_key` /
194+
/// `field_value` texts (absent parts are simply omitted).
195+
fn add_field_region(&mut self, items: &[crate::FieldItem], parent: &str) -> String {
196+
let self_ref = format!("#/field_regions/{}", self.field_regions.len());
197+
self.field_regions.push(Value::Null);
198+
let region_index = self.field_regions.len() - 1;
199+
let mut item_refs = Vec::new();
200+
for item in items {
201+
item_refs.push(json!({ "$ref": self.add_field_item(item, &self_ref) }));
202+
}
203+
self.field_regions[region_index] = json!({
204+
"self_ref": self_ref,
205+
"parent": { "$ref": parent },
206+
"children": item_refs,
207+
"content_layer": "body",
208+
"label": "field_region",
209+
"prov": [],
210+
});
211+
self_ref
212+
}
213+
214+
fn add_field_item(&mut self, item: &crate::FieldItem, parent: &str) -> String {
215+
let self_ref = format!("#/field_items/{}", self.field_items.len());
216+
self.field_items.push(Value::Null);
217+
let item_index = self.field_items.len() - 1;
218+
let mut child_refs = Vec::new();
219+
for (label, text) in [
220+
("marker", &item.marker),
221+
("field_key", &item.key),
222+
("field_value", &item.value),
223+
] {
224+
if let Some(text) = text {
225+
child_refs
226+
.push(json!({ "$ref": self.add_text(label, text, &self_ref, json!({})) }));
227+
}
228+
}
229+
self.field_items[item_index] = json!({
230+
"self_ref": self_ref,
231+
"parent": { "$ref": parent },
232+
"children": child_refs,
233+
"content_layer": "body",
234+
"label": "field_item",
235+
"prov": [],
236+
});
237+
self_ref
238+
}
239+
173240
fn add_text(&mut self, label: &str, text: &str, parent: &str, extra: Value) -> String {
174241
let self_ref = format!("#/texts/{}", self.texts.len());
175242
let raw = unescape_text(text);

crates/fleischwolf-core/src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,6 @@ mod json;
1616
mod labels;
1717
mod markdown;
1818

19-
pub use document::{DoclingDocument, Node, PictureImage, Table};
19+
pub use document::{DoclingDocument, FieldItem, Node, PictureImage, Table};
2020
pub use labels::DocItemLabel;
2121
pub use markdown::{ImageMode, MarkdownStreamer};

crates/fleischwolf-core/src/markdown.rs

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -346,11 +346,28 @@ fn render_one(node: &Node, blocks: &mut Vec<String>, ctx: &mut Ctx) {
346346
blocks.push(picture_marker(image.as_ref(), ctx));
347347
}
348348
Node::Group { children, .. } => render(children, blocks, ctx),
349+
Node::FieldRegion { items } => {
350+
// docling renders the region container (which carries no text of its
351+
// own) as a `<!-- missing-text -->` marker, then each field item the
352+
// same way, followed by that item's marker/key/value as separate
353+
// paragraphs.
354+
blocks.push(MISSING_TEXT.to_string());
355+
for item in items {
356+
blocks.push(MISSING_TEXT.to_string());
357+
for part in [&item.marker, &item.key, &item.value].into_iter().flatten() {
358+
blocks.push(strict_text(part, ctx.strict));
359+
}
360+
}
361+
}
349362
// Handled by the run-merging branch in `render`.
350363
Node::ListItem { .. } => unreachable!("list items are rendered in runs"),
351364
}
352365
}
353366

367+
/// docling's placeholder for a structural node (a field region / item) that has
368+
/// no text of its own.
369+
const MISSING_TEXT: &str = "<!-- missing-text -->";
370+
354371
/// The Markdown for a picture under the active [`ImageMode`]; Referenced mode also
355372
/// records the bytes in `ctx.artifacts` for the caller to write.
356373
fn picture_marker(image: Option<&crate::PictureImage>, ctx: &mut Ctx) -> String {

crates/fleischwolf/Cargo.toml

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,18 @@ rust-version.workspace = true
1515
# corpus by path, so they're dev-only too.
1616
exclude = ["tests", "benches"]
1717

18+
[features]
19+
# Optional headless-browser HTML pre-render (the `--use-web-browser` flag /
20+
# `DocumentConverter::use_web_browser`). Off by default so the standard build
21+
# stays pure-Rust with no browser/runtime dependency; enable with
22+
# `--features web-browser`. Drives the system Chromium over the DevTools
23+
# protocol purely from Rust (no Node/Playwright) via `headless_chrome`.
24+
web-browser = ["dep:headless_chrome"]
25+
1826
[dependencies]
1927
calamine = { version = "0.26", features = ["dates"] }
2028
csv = "1.4.0"
29+
headless_chrome = { version = "1.0", optional = true }
2130
fleischwolf-core = { path = "../fleischwolf-core", version = "0.10.0" }
2231
fleischwolf-pdf = { path = "../fleischwolf-pdf", version = "0.10.0" }
2332
image = { version = "0.25", default-features = false, features = ["png", "jpeg", "gif", "bmp", "tiff", "webp"] }

0 commit comments

Comments
 (0)