Skip to content

Commit 183aba7

Browse files
authored
Merge pull request #11 from artiz/claude/pdf-drop-pdfium-text-path
feat(pdf): drop pdfium's word path — docling-parse word grouping in the Rust parser
2 parents 9d3cf08 + ec9c3c3 commit 183aba7

17 files changed

Lines changed: 645 additions & 209 deletions

File tree

PDF_CONFORMANCE.md

Lines changed: 26 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ The groundtruth is regenerated from **live published docling**, so it agrees wit
1313
1414
## Current state
1515

16-
**6 / 14 strict** · **7 / 14 whitespace-normalized.**
16+
**6 / 16 strict** · **7 / 16 whitespace-normalized.**
1717

1818
| PDF | diff | dominant remaining blocker |
1919
|---|---:|---|
@@ -24,20 +24,28 @@ The groundtruth is regenerated from **live published docling**, so it agrees wit
2424
| right_to_left_01 | **exact** | — (RTL period attachment) |
2525
| right_to_left_02 | **exact** | — (kashida dedup + page-number layout) |
2626
| amt_handbook_sample | 2 *(ws-ok)* | docling's spurious fraction double space — ours is more faithful |
27+
| 2305.03393v1 | 32 | title-page reading order + author-ID run spacing |
28+
| skipped_1page | 40 | image/diagram page (Korean); layout picture-detection |
29+
| skipped_2pages | 44 | image/diagram pages (Korean); layout picture-detection |
2730
| normal_4pages | 54 | reading order (heading numbering, footnote order) |
2831
| right_to_left_03 | 66 | RTL bidi |
29-
| 2305.03393v1 | 93 | title-page reading order + author-ID run spacing |
3032
| table_mislabeled_as_picture | 108 | layout over-detects tables (survey rendered as tables) |
31-
| 2206.01062 | 198 | TableFormer multi-row headers + title-page reading order |
32-
| 2203.01017v2 | 209 | TableFormer structure + reading order |
33-
| redp5110_sampled | 226 | TOC mis-classified as a picture; cover-page ordering |
33+
| 2206.01062 | 164 | TableFormer multi-row headers + title-page reading order |
34+
| redp5110_sampled | 173 | TOC mis-classified as a picture; cover-page ordering |
35+
| 2203.01017v2 | 183 | TableFormer structure + reading order |
3436

3537
`amt` is the 7th under the whitespace-normalized metric: its only diff is
3638
docling's spurious double space before the `1⁄4` fraction, where our single-spaced
3739
output is the more faithful rendering. The remaining non-exact PDFs are heavy
3840
multi-column / table docs whose gaps are model-level (TableFormer structure,
3941
layout classification, title-page reading order), not text-layer.
4042

43+
The heavy table docs improved with the docling-parse **word-cell** grouping now
44+
feeding TableFormer (2305.03393v1 93→32, 2203.01017v2 209→183, 2206.01062
45+
198→164, redp5110 226→173): the parser's per-word cells reproduce docling-parse's
46+
`word_cells` byte-for-byte, so cell-to-grid matching tracks docling more closely.
47+
See "Text reconstruction" below and `PDF_PARSER_NOTES.md` item 6.
48+
4149
## How the pipeline works
4250

4351
pdfium extracts the glyph layer and renders each page to a bitmap; an ONNX stack
@@ -79,8 +87,9 @@ on `lopdf`) that reconstructs each glyph's box from the *font's own advance
7987
widths* and the PDF text/graphics matrices — the same information docling-parse
8088
uses. It is the **default** text layer; set `DOCLING_PDFIUM_TEXT=1` to fall back
8189
to pdfium. Pages without a parseable text layer fall back to pdfium
82-
automatically, so scanned/OCR pages are unaffected. (pdfium still provides page
83-
rasters and word/code cells for TableFormer.)
90+
automatically, so scanned/OCR pages are unaffected. The parser also supplies the
91+
**word cells** TableFormer matches against (`DOCLING_PDFIUM_WORDS` keeps pdfium's);
92+
pdfium still provides page rasters, code cells, and link annotations.
8493

8594
The parser handles Type0/CID + Identity-H and simple Type1/TrueType fonts,
8695
ToUnicode CMaps (`bfchar`/`bfrange`), WinAnsi/MacRoman + `/Differences`
@@ -99,6 +108,16 @@ ligature recomposition, and loose-box geometry. On the clean parser boxes it use
99108
the Euclidean corner gap (matching docling); on pdfium's loose boxes it keeps the
100109
signed horizontal gap.
101110

111+
The same contraction also produces **word cells** (`dp_lines::word_cells`): a word
112+
is a maximal run of glyphs the contraction merges *without* inserting a separator
113+
space, so the per-word segments split at exactly the `delta < gap` points — which
114+
reproduces docling-parse's `word_cells` byte-for-byte (377/377 on 2305-pg9). These
115+
are the per-word tokens TableFormer matches against table-grid cells, replacing
116+
pdfium's word cells (roadmap item 6). Code cells stay on pdfium for now — the
117+
parser's space-glyph-only code grouping drops monospace spacing
118+
(`function add``functionadd`); opt into the parser code path with
119+
`DOCLING_PARSER_CODE` once that's fixed.
120+
102121
Other text/serializer/layout fixes matching docling: markdown escaping (`_``\_`,
103122
then HTML-escape `&`/`<`/`>`), typographic-punctuation normalization
104123
(```'`, ``/```-`, `“”``"`, or `'` for Hangul fonts), `@`-glue

PDF_PARSER_NOTES.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,33 @@ numbering and footnote reading order.)
163163
4. ~~Korean quote normalization~~ — DONE (normal_4pages 74→54).
164164
5. **Now: 6/14 strict, 7/14 whitespace-normalized.** Blocker B (amt) needs a
165165
font-metrics layer for strict 7/14.
166-
6. Long term: drop pdfium's text path (keep it for rasterisation).
166+
6. **Word cells off pdfium — DONE (parser is now the default word source).** The
167+
parser is already the sole *prose* source (item 2); **word** cells now come
168+
from it too. The insight: docling-parse's `word_cells` are exactly the line
169+
contraction's runs, split at the points where `merge_with` inserts a separator
170+
space (`delta < gap`). So instead of the legacy gap-heuristic
171+
(`words_from_glyphs`, which blob-joined TJ-spaced runs like
172+
`highlydiversesetoftables…`), `dp_lines::word_cells` tracks per-word segments
173+
*through the same proven contraction* that already reproduces `textline_cells`.
174+
On `2305` pg9 it now emits **377/377 words byte-identical to the docling-parse
175+
`word_cells` oracle** (x-coords matching to 0.01 pt; the only residue is the
176+
~3 pt-taller vertical box from the blocker-B font-metrics gap, which the
177+
TableFormer matcher tolerates).
178+
179+
Result vs the docling **groundtruth**: strictly *better* on the heavy
180+
multi-column docs (`redp5110` +22, `2206.01062` +16, `2203.01017v2` +7
181+
groundtruth-matching lines), neutral on the rest, **no regression** anywhere —
182+
so it's the default (opt out with `DOCLING_PDFIUM_WORDS`, or `DOCLING_PDFIUM_TEXT`
183+
for full pdfium text). The 7 affected committed snapshots were regenerated to
184+
the now docling-faithful output (e.g. `bold ,` / `x 2`, which docling-parse's
185+
own `word_cells` confirm — the old `bold,` / `x2` were pdfium punct-gluing).
186+
187+
**Code cells still pdfium.** The parser's space-glyph-only code grouping drops
188+
the inter-token spaces pdfium recovers in monospace listings
189+
(`function add``functionadd`, a regression vs groundtruth), so code cells
190+
stay on pdfium, gated behind `DOCLING_PARSER_CODE` for a future fix (the parser
191+
needs faithful monospace spacing before the *code* path can retire too). pdfium
192+
thus remains for code cells + page rasterisation + link annotations.
167193

168194
## Tooling (under `scripts/`)
169195

README.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -188,6 +188,25 @@ The comparison scripts install the latest published Python `docling` from PyPI
188188
into `.venv-compare` automatically on first run. See
189189
[`COMPARING.md`](./COMPARING.md).
190190

191+
## Deploy in a container
192+
193+
For a real-world service, bake the binary, native libs, and models into one image
194+
so the runtime needs no Python. [`examples/Dockerfile`](./examples/Dockerfile) is a
195+
3-stage build that does exactly this — a `models` stage exports the layout +
196+
**TableFormer** (KV-cached decoder) ONNX with torch and fetches the OCR model +
197+
pdfium, a `builder` stage compiles the CLI, and a slim `runtime` stage carries just
198+
the binary, `libonnxruntime`, pdfium, and the models, with the `DOCLING_*` env vars
199+
preset:
200+
201+
```bash
202+
docker build -f examples/Dockerfile -t fleischwolf .
203+
docker run --rm -v "$PWD:/data" fleischwolf /data/input.pdf # Markdown to stdout
204+
docker run --rm -v "$PWD:/data" fleischwolf /data/input.pdf --to json
205+
```
206+
207+
The image converts PDFs/images fully offline; the model export (torch +
208+
`docling-ibm-models`) happens only at build time, never at runtime.
209+
191210
## Performance
192211

193212
`scripts/performance.sh` runs the **largest fixture of each supported type** through
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
//! Dump the Rust parser's word cells for a page: `<pdf> <page_index_0based>`.
2+
//! TSV: l<tab>t<tab>r<tab>b<tab>text (top-left page-point coords).
3+
fn main() {
4+
let path = std::env::args().nth(1).expect("pdf");
5+
let pi: usize = std::env::args()
6+
.nth(2)
7+
.and_then(|s| s.parse().ok())
8+
.unwrap_or(0);
9+
let bytes = std::fs::read(&path).unwrap();
10+
let pages = fleischwolf_pdf::textparse::pdf_words(&bytes);
11+
if let Some((_, _, cells)) = pages.get(pi) {
12+
for c in cells {
13+
let t = c.text.replace(['\t', '\n'], " ");
14+
println!("{:.2}\t{:.2}\t{:.2}\t{:.2}\t{}", c.l, c.t, c.r, c.b, t);
15+
}
16+
}
17+
}

crates/fleischwolf-pdf/src/dp_lines.rs

Lines changed: 112 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,62 @@ struct Cell {
3333
active: bool,
3434
lig_carry: bool, // last_merged_cell_was_ligature
3535
font: u64, // hash of the PDF font name+flags (for enforce_same_font)
36+
/// Sub-word segments accumulated during contraction, in final logical order.
37+
/// A word boundary is recorded wherever `merge_with` inserts a separator space
38+
/// (`delta < gap`); within a boundary the glyphs share one segment. Flattening
39+
/// these across all cells yields docling-parse's `word_cells` (item 6). The
40+
/// line path ignores this; only [`word_cells`] reads it.
41+
words: Vec<WordSeg>,
42+
}
43+
44+
/// One word's accumulated text and native-coordinate bounding box (y up).
45+
#[derive(Clone)]
46+
struct WordSeg {
47+
text: String,
48+
l: f64,
49+
b: f64,
50+
r: f64,
51+
t: f64,
52+
}
53+
54+
impl WordSeg {
55+
fn from_glyph(text: String, l: f64, b: f64, r: f64, t: f64) -> Self {
56+
WordSeg { text, l, b, r, t }
57+
}
58+
/// Absorb `o` into this segment (same word): union the box, append text.
59+
fn absorb(&mut self, o: &WordSeg) {
60+
self.text.push_str(&o.text);
61+
self.l = self.l.min(o.l);
62+
self.b = self.b.min(o.b);
63+
self.r = self.r.max(o.r);
64+
self.t = self.t.max(o.t);
65+
}
66+
/// Extend the box to cover a single glyph (ligature recompose into one word).
67+
fn extend(&mut self, l: f64, b: f64, r: f64, t: f64) {
68+
self.l = self.l.min(l);
69+
self.b = self.b.min(b);
70+
self.r = self.r.max(r);
71+
self.t = self.t.max(t);
72+
}
73+
}
74+
75+
/// Concatenate two word runs (in final logical order). With a separator space
76+
/// the runs stay distinct (a word boundary); without one, `left`'s last word and
77+
/// `right`'s first word are the same word and merge. Mirrors `merge_with`'s
78+
/// space decision so word grouping tracks the line contraction exactly.
79+
fn merge_word_runs(mut left: Vec<WordSeg>, mut right: Vec<WordSeg>, space: bool) -> Vec<WordSeg> {
80+
if left.is_empty() {
81+
return right;
82+
}
83+
if right.is_empty() {
84+
return left;
85+
}
86+
if !space {
87+
let first = right.remove(0);
88+
left.last_mut().unwrap().absorb(&first);
89+
}
90+
left.extend(right);
91+
left
3692
}
3793

3894
impl Cell {
@@ -86,18 +142,25 @@ impl Cell {
86142
} else {
87143
other.rx0 - self.rx1
88144
};
145+
let space = delta < gap;
89146
if !self.ltr || !other.ltr {
90-
if delta < gap {
147+
if space {
91148
self.text.insert(0, ' ');
92149
}
93150
self.text = format!("{}{}", other.text, self.text);
94151
self.ltr = false;
152+
// RTL: `other` is logically first, so its words precede self's. The
153+
// junction is between other's last word and self's first.
154+
self.words =
155+
merge_word_runs(other.words.clone(), std::mem::take(&mut self.words), space);
95156
} else {
96-
if delta < gap {
157+
if space {
97158
self.text.push(' ');
98159
}
99160
self.text.push_str(&other.text);
100161
self.ltr = true;
162+
self.words =
163+
merge_word_runs(std::mem::take(&mut self.words), other.words.clone(), space);
101164
}
102165
// Extend the right edge to `other`.
103166
self.rx1 = other.rx1;
@@ -206,8 +269,9 @@ fn contract(cells: &mut Vec<Cell>, euclidean: bool) {
206269
cells.retain(|c| c.active);
207270
}
208271

209-
/// Build line cells from a page's glyph stream via the docling-parse contraction.
210-
pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<TextCell> {
272+
/// Build per-glyph char cells from a page's glyph stream (shared by the line and
273+
/// word paths): drop degenerate spaces, recompose ligatures, init word segments.
274+
fn build_cells(glyphs: &[Glyph], euclidean: bool) -> Vec<Cell> {
211275
let mut cells: Vec<Cell> = Vec::new();
212276
for g in glyphs {
213277
// Use the loose box (uniform font ascent/descent + advance) so adjacent
@@ -241,12 +305,23 @@ pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<
241305
}
242306
last.text.push(g.ch);
243307
last.ltr = !is_right_to_left(&last.text);
308+
if let Some(w) = last.words.last_mut() {
309+
w.text.push(g.ch);
310+
w.extend(g.ll as f64, g.lb as f64, g.lr as f64, g.lt as f64);
311+
}
244312
continue;
245313
}
246314
}
247315
let text = g.ch.to_string();
248316
let ltr = !is_right_to_left(&text);
249317
cells.push(Cell {
318+
words: vec![WordSeg::from_glyph(
319+
text.clone(),
320+
g.ll as f64,
321+
g.lb as f64,
322+
g.lr as f64,
323+
g.lt as f64,
324+
)],
250325
text,
251326
rx0: g.ll as f64,
252327
ry0: g.lb as f64,
@@ -262,6 +337,12 @@ pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<
262337
font: g.font,
263338
});
264339
}
340+
cells
341+
}
342+
343+
/// Build line cells from a page's glyph stream via the docling-parse contraction.
344+
pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<TextCell> {
345+
let mut cells = build_cells(glyphs, euclidean);
265346
contract(&mut cells, euclidean);
266347
cells
267348
.into_iter()
@@ -281,6 +362,33 @@ pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<
281362
.collect()
282363
}
283364

365+
/// Build **word** cells from a page's glyph stream via the same contraction as
366+
/// [`line_cells`], then split each line into its constituent words at exactly the
367+
/// points where the contraction inserted a separator space. This reproduces
368+
/// docling-parse's `word_cells` (the per-word tokens TableFormer matches against
369+
/// table-grid cells), letting the pipeline drop pdfium's text path entirely
370+
/// (roadmap item 6). Empty words (overprint-cleared) are skipped.
371+
pub(crate) fn word_cells(glyphs: &[Glyph], page_h: f32, euclidean: bool) -> Vec<TextCell> {
372+
let mut cells = build_cells(glyphs, euclidean);
373+
contract(&mut cells, euclidean);
374+
let mut out = Vec::new();
375+
for c in cells {
376+
for w in c.words {
377+
if w.text.trim().is_empty() {
378+
continue;
379+
}
380+
out.push(TextCell {
381+
text: w.text,
382+
l: w.l as f32,
383+
t: page_h - w.t as f32,
384+
r: w.r as f32,
385+
b: page_h - w.b as f32,
386+
});
387+
}
388+
}
389+
out
390+
}
391+
284392
fn is_rtl_char(c: char) -> bool {
285393
let ch = c as u32;
286394
(0x0600..=0x06FF).contains(&ch)

0 commit comments

Comments
 (0)