PDF: hyphenated line wraps inside list items split into multiple items

[bfuerea]

Bug

When a list item wraps across a line break at a hyphenated word (e.g. "algo-rithms"), docling splits it into two separate list items. The continuation fragment gets a new list-item number, corrupting the entire list numbering.

Example from a real PDF:

1. Digital Competence for Deep Learning - this step moves beyond mere computer literacy, demanding that students master data-interpretation skills, generate and critique artificial intelligence (AI) outputs, and understand the algo-
2. rithms embedded in clinical imaging devices;
3. Evidence-Based Practice with Quality Digital Resources - ...
4. Learning Environments with Applicable Digital Technology - ...
5. Synergy Between Human Teachers and Trustworthy AI - ...

Item "2. rithms" is a continuation of item 1 — "algorithms" was split across lines. This corrupts the numbering (should be 4 items, not 5).

Steps to reproduce

from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("fuerea_digital_transformation_2026.pdf")
md = result.document.export_to_markdown()

# The orphaned fragment appears in the output
assert "2. rithms" in md  # Bug: continuation treated as new item

Expected behavior

The hyphenated wrap should be detected and merged back into the parent list item. The word "algorithms" should appear complete, and the list should have exactly 4 items.

Source location

readingorder_model.py:363-412: list items are emitted one-per-cluster. The merge machinery at :414 _merge_elements and :457 predict_merges exists, but only merges TEXT labels — LIST_ITEM is explicitly excluded at line 193. The wrap fragment gets a distinct cluster ID from the layout stage and never enters el_merges_mapping.

Suggestion

Extend predict_merges to handle hyphenated wraps inside LIST_ITEM clusters. The heuristic: if a list item ends with - (or soft hyphen \u00ad) and the next list item starts with a lowercase letter, merge them. GROBID uses this exact rule.

Docling version

2.115.0

Regression test PDF

• fuerea_digital_transformation_2026.pdf (1.5 MB) — 2-column bilingual review paper, list items at lines 53-56 of the markdown output demonstrate the bug

[dosubot]

Hi @bfuerea! I'm Dosu and I'm helping the IBM Docling team.

Your analysis is accurate. The root cause is that predict_merges in docling-ibm-models only considers elements with label == DocItemLabel.TEXT as merge candidates [1]. Since LIST_ITEM is not TEXT, list items never enter the merge mapping and are never eligible for cross-element merging.

Docling actually has two separate dehyphenation mechanisms, and neither covers this case:

1. Within-cluster (sanitize_text in page_assemble_model.py) — handles hard hyphens (-) when joining lines inside a single cluster [2]. This works fine when the hyphenated word stays within one cluster's cells, but here the continuation fragment gets assigned a distinct cluster ID by the layout model, so this path never sees it.

2. Cross-element (_merge_elements in readingorder_model.py) — handles soft hyphens (\u00ad) when merging consecutive elements [3]. This would handle the dehyphenation, but it only runs for elements that predict_merges included in el_merges_mapping — which excludes LIST_ITEM.

So the fix site is as you identified: extending the label filter in predict_merges (in docling-ibm-models, reading_order_rb.py around line 192-193) to also accept DocItemLabel.LIST_ITEM under a hyphenation heuristic. The _merge_elements function in readingorder_model.py already handles the actual text joining and soft-hyphen stripping [3], so that part should work once list items are allowed through.

Your suggested heuristic (item ends with -/\u00ad + next item starts with lowercase) is reasonable and matches the existing within-cluster logic in sanitize_text. One thing to watch: the cross-element merge currently checks for soft hyphens (\u00ad) while the within-cluster check uses hard hyphens (-). A fix would likely need to handle both, since the PDF parser may emit either depending on how the hyphen was encoded in the original PDF.

No existing issue or PR was found tracking this specific problem.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[Success6666]

I traced the regression through both layers and split the fix into two focused PRs:

• fix(reading-order): merge hyphenated list continuations docling-ibm-models#171 makes a LIST_ITEM eligible for a cross-column merge only when it is a hyphenated lowercase continuation.
• fix(reading-order): dehyphenate hard continuations #3888 removes the hard hyphen during the final merge, while preserving hard hyphens before uppercase continuations.

Together they cover the reported algo- + rithms case. Both PRs include positive and negative regression tests.