Skip to content

Commit dd95c83

Browse files
artizclaude
andcommitted
feat(pdf): docling-parse migration phase 3.1 — space-box glyph fidelity
Two pdfium glyph quirks broke the sanitizer's merging: - Generated space glyphs carry a default font that blocked every word↔space merge under enforce_same_font. Spaces are now font-neutral (0), and the font check treats 0 as a wildcard. - pdfium gives generated spaces a degenerate *zero-width* loose box at the wrong baseline, so corner-distance adjacency failed and lines fragmented into words. Drop only zero-width spaces (the inter-word gap then drives merge_with's space insertion); spaces with real width are kept so justified double-spaces survive. Net effect with DOCLING_DP_LINES on: the 2305-pg9 word-interleaving is gone, and general text improves — multi_page 54→22, normal_4pages 108→82, redp5110 300→256, amt_handbook 16→14. Default path (flag off) unchanged (EXACT 3 intact). Blocker before default-on: pdfium decomposes fi/ffi ligatures into separate glyphs whose gaps trip the inserted-space rule (`conf iguration`, `di f f i cult`), regressing 2305-pg9 to 4. docling-parse keeps the ligature as one cell — next fix is to recompose them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 0964992 commit dd95c83

2 files changed

Lines changed: 24 additions & 3 deletions

File tree

crates/fleischwolf-pdf/src/dp_lines.rs

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,13 @@ fn applicable(a: &Cell, b: &Cell) -> bool {
101101
if !a.active || !b.active {
102102
return false;
103103
}
104-
if a.font != b.font && !is_ligature(&a.text) && !is_ligature(&b.text) {
104+
// font 0 = unknown/space (font-neutral); ligatures bridge fonts too.
105+
if a.font != 0
106+
&& b.font != 0
107+
&& a.font != b.font
108+
&& !is_ligature(&a.text)
109+
&& !is_ligature(&b.text)
110+
{
105111
return false;
106112
}
107113
a.same_orientation(b)
@@ -186,7 +192,15 @@ pub(crate) fn line_cells(glyphs: &[Glyph], page_h: f32) -> Vec<TextCell> {
186192
// Use the loose box (uniform font ascent/descent + advance) so adjacent
187193
// glyphs share a top edge, matching docling-parse's `compute_rect`.
188194
if !g.ll.is_finite() {
189-
return None; // a space pdfium gave no box for; can't place it
195+
return None;
196+
}
197+
// Drop *degenerate* space glyphs (zero-width loose box): pdfium's
198+
// generated spaces get a zero-width box at the wrong baseline that
199+
// breaks corner-distance adjacency. Without them the inter-word gap
200+
// drives `merge_with`'s space insertion. Spaces with a real width are
201+
// kept (they carry justified double-space information).
202+
if g.ch == ' ' && (g.lr - g.ll).abs() < 0.5 {
203+
return None;
190204
}
191205
let text = g.ch.to_string();
192206
let ltr = !is_right_to_left(&text);

crates/fleischwolf-pdf/src/pdfium_backend.rs

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -317,7 +317,14 @@ fn glyphs(b: &dyn PdfiumLibraryBindings, tp: FPDF_TEXTPAGE, fetch_font: bool) ->
317317
if ch == '\r' || ch == '\n' {
318318
continue;
319319
}
320-
let font = if fetch_font { font_hash(b, tp, i) } else { 0 };
320+
// Spaces are font-neutral (0): pdfium's generated spaces carry a default
321+
// font that would otherwise block every word↔space merge under
322+
// enforce_same_font; docling-parse's spaces inherit the run's font.
323+
let font = if fetch_font && !ch.is_whitespace() {
324+
font_hash(b, tp, i)
325+
} else {
326+
0
327+
};
321328
let (mut l, mut r, mut bot, mut top) = (0f64, 0f64, 0f64, 0f64);
322329
let has_box = b.FPDFText_GetCharBox(tp, i, &mut l, &mut r, &mut bot, &mut top) != 0;
323330
// Loose box: font ascent/descent + glyph advance, uniform per font/size.

0 commit comments

Comments
 (0)