Commit ea78a2e
feat(pdf): normalize U+2212 minus + merge hyphen-ending paragraph fragments
- Map U+2212 (minus sign) to ASCII `-` in clean_text alongside the en/em dashes
(author-ID `[0000 −0002]` → `[0000 -0002]`).
- merge_continuations now also joins a fragment ending in a wrap hyphen/dash with
a lowercase continuation (`vocab-` + `ulary` → `vocab- ulary`), matching docling.
2305v1 44→40; the 4 exact PDFs unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent e262439 commit ea78a2e
1 file changed
Lines changed: 6 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
157 | 157 | | |
158 | 158 | | |
159 | 159 | | |
160 | | - | |
| 160 | + | |
161 | 161 | | |
162 | 162 | | |
163 | 163 | | |
| |||
613 | 613 | | |
614 | 614 | | |
615 | 615 | | |
616 | | - | |
617 | | - | |
618 | | - | |
619 | | - | |
620 | | - | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
621 | 621 | | |
622 | 622 | | |
623 | 623 | | |
| |||
0 commit comments