fix: generate page_break for skipped pages in export functions

[jhchoi1182]

Description

Currently, pages that fail during PDF parsing are silently skipped. Taking the test.pdf attached to the issue (composed of pages 78, 79, 83, and 84) as an example, pages 79 and 83 failed to parse, causing page 78 to be followed immediately by page 84. Previously, this resulted in the generation of only a single <page_break> for page 84. This made it impossible for users to detect that pages 79 and 83 were missing, leading to potential issues during post-processing.

This PR modifies the _iterate_items() function to generate individual <page_break> markers for each skipped page when parsing failures result in non-consecutive page transitions. For instance, when jumping from page 78 to 84, the function now generates page breaks for the missing pages (79, 83) in addition to page 84, ensuring all skipped pages are explicitly marked.

Changes

common.py:

• Added _PageBreakNode class to represent page break nodes with prev_page and next_page fields
• Added _yield_page_breaks() helper function that generates page break nodes for each page in a non-consecutive range
• Updated _iterate_items() to use the new helper, ensuring all skipped pages get their own <page_break> marker

html.py:

• Updated serialize_doc() to handle skipped pages in SPLIT_PAGE mode
• Skipped pages are now rendered as empty rows in the split page view, maintaining page numbering consistency
• Added logic to track all physical pages from page breaks and render the full range including missing pages

Affected Export Functions

Function	Behavior
export_to_doctags()	Generates <page_break> tokens for skipped pages
export_to_markdown(page_break_placeholder=str)	Generates placeholder for skipped pages
export_to_html(split_page_view=True)	Renders skipped pages as empty rows in split view

Design Decision

When page breaks are enabled, breaks are now generated for all page transitions, including pages that failed to parse. This is the default behavior without additional parameters, as it provides the most intuitive result: users can detect missing pages in the output. If maintainers prefer to make this configurable, a skip_missing_page_breaks parameter could be added to CommonParams in a follow-up PR.

Example

export_to_doctags()

Before (page 78, 79, 83, 84):

<doctag>
... content ...      78page
<page_break>
... content ...      84page
</doctag>

After (page 78, 79, 83, 84):

<doctag>
... content ...      78page
<page_break>
<page_break>  (79, 83page)
<page_break>
... content ...      84page
</doctag>

export_to_markdown(page_break_placeholder="---PAGE BREAK---")

Before (page 78, 79, 83, 84):

... content ...      78page

---PAGE BREAK---

... content ...      84page

After (page 78, 79, 83, 84):

... content ...      78page

---PAGE BREAK---

---PAGE BREAK---

---PAGE BREAK---

... content ...      84page

export_to_html(split_page_view=True)

Before (page 78, 79, 83, 84):

<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
... content ...      78page

</div>
</td>
</tr>
<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
... content ...      84page

</div>
</td>
</tr>

After (page 78, 79, 83, 84):

<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
... content ...      78page

</div>
</td>
</tr>
<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
                     (79page)

</div>
</td>
</tr>
<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
                     (83page)

</div>
</td>
</tr>
<tr>
<td>
<figure>no page-image found</figure>
</td>
<td>
<div class='page'>
... content ...      84page

</div>
</td>
</tr>

Testing

Unit tests for this feature are available in test/test_page_break_skipped_pages.py. Run the tests with:

pytest test/test_page_break_skipped_pages.py -v

Issue resolved by this Pull Request:

Resolves #472

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• New and existing unit tests pass locally with my changes

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Require two reviewer for test updates

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[dosubot]

Related Documentation

1 document(s) may need updating based on files changed in this PR:

Docling

• How can I use granite-docling to process all PDFs in a directory and output doctags?

^{How did I do? Any feedback?}  

[github-actions]

✅ DCO Check Passed

Thanks @jhchoi1182, all your commits are properly signed off. 🎉

[Copilot]

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jhchoi1182]

After merging the main branch, I applied ruff checks to my changes.
Note that running ruff check --fix and ruff format also modifies 14 other files, but I only applied it to my changes as I think it would be better to handle those separately.

[jhchoi1182]

Hi @dolfim-ibm, checking in on this PR. Is there anything else needed from my side to get this merged? I'm happy to address any feedback or conflicts.

[ezphyki]

@vagenas

Please process this matter quickly.

[jhchoi1182]

I found that my previous PR code conflicted with the .filter() method.

To resolve this, I've submitted a separate PR to the docling repository that adds code to track failed pages by adding them to DoclingDocument.pages. This companion PR in docling-core modifies _yield_page_breaks() to only generate page breaks for pages present in the pages dict.

How it works:

• docling: _add_failed_pages_to_document() method adds failed/skipped pages to DoclingDocument.pages with their size information (empty content)
• docling-core: _yield_page_breaks() now takes page_numbers parameter (extracted from doc.pages.keys()) and only generates page breaks for pages in that set

This approach ensures:

• ✅ Failed pages get proper page break markers (pages are in pages dict)
• ✅ filter() method continues to work correctly (excluded pages are removed from pages dict, so no spurious page breaks)
• ✅ Filter + failed page scenario works (e.g., filtering {2,3,5} with page 3 failing still generates correct breaks)

Related PR: docling-project/docling#2939

[jhchoi1182]

@cau-git

I've rebased on main to clean up the messy commit history.

With the companion docling PR (docling-project/docling#2939) now merged into main, failed pages are guaranteed to be present in doc.pages. This allowed me to simplify the split_page_view logic in html.py by removing defensive code that handled missing pages.

I'd appreciate feedback on whether this overall approach is the right direction. Happy to adjust if needed.