Commit dd95c83
feat(pdf): docling-parse migration phase 3.1 — space-box glyph fidelity
Two pdfium glyph quirks broke the sanitizer's merging:
- Generated space glyphs carry a default font that blocked every word↔space
merge under enforce_same_font. Spaces are now font-neutral (0), and the font
check treats 0 as a wildcard.
- pdfium gives generated spaces a degenerate *zero-width* loose box at the wrong
baseline, so corner-distance adjacency failed and lines fragmented into words.
Drop only zero-width spaces (the inter-word gap then drives merge_with's space
insertion); spaces with real width are kept so justified double-spaces survive.
Net effect with DOCLING_DP_LINES on: the 2305-pg9 word-interleaving is gone, and
general text improves — multi_page 54→22, normal_4pages 108→82, redp5110
300→256, amt_handbook 16→14. Default path (flag off) unchanged (EXACT 3 intact).
Blocker before default-on: pdfium decomposes fi/ffi ligatures into separate
glyphs whose gaps trip the inserted-space rule (`conf iguration`, `di f f i cult`),
regressing 2305-pg9 to 4. docling-parse keeps the ligature as one cell — next fix
is to recompose them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 0964992 commit dd95c83
2 files changed
Lines changed: 24 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
104 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
105 | 111 | | |
106 | 112 | | |
107 | 113 | | |
| |||
186 | 192 | | |
187 | 193 | | |
188 | 194 | | |
189 | | - | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
190 | 204 | | |
191 | 205 | | |
192 | 206 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
317 | 317 | | |
318 | 318 | | |
319 | 319 | | |
320 | | - | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
321 | 328 | | |
322 | 329 | | |
323 | 330 | | |
| |||
0 commit comments