Requested feature
For multi-page PDF processing, sometimes we wanna split the file into several chunks to do parallel processing, and then merge the docling document at the end. If the file includes multiple tables, and the function add_document() is used, sometimes I got the error
Document hierarchy is inconsistent. #/tables/14 has cell #/groups/0 with parent #/tables/3 [type=value_error, input_value=DoclingDocument(schema_na...age=None, page_no=170)}), input_type=DoclingDocument]
To solve this issue, the function concatenate(docs) is used. However, it has one limitation that cannot keep the original page number. If the processing is just part of the file based on the page range, for example, page 5 - 20, and the chunks are page (5-10), (11-20), after calling concatenate(docs) to merge the Docling docs, the page no will be reset from 1, not keeping the starting page from 5. I have to calculate the page offset and update the page no.
The code I added to update the page no to support page range:
def update_docling_doc_page_no(self, merged_doc: DoclingDocument, page_offset: int):
if page_offset < 1:
return
# 1. Update pages dict keys and PageItem.page_no
new_pages = {}
for old_no, page_item in merged_doc.pages.items():
new_no = old_no + page_offset
page_item.page_no = new_no
new_pages[new_no] = page_item
merged_doc.pages = new_pages
# 2. Update ProvenanceItem.page_no on all content items
for item, _level in merged_doc.iterate_items(with_groups=False):
if hasattr(item, "prov"):
for prov in item.prov:
prov.page_no += page_offset
# 3. Update GraphCell.prov.page_no in KeyValueItem / FormItem
if isinstance(item, (KeyValueItem, FormItem)):
for cell in item.graph.cells:
if cell.prov is not None:
cell.prov.page_no += page_offset
Samples of Docling docs that got the "Document hierarchy is inconsistent." error
Requested feature
For multi-page PDF processing, sometimes we wanna split the file into several chunks to do parallel processing, and then merge the docling document at the end. If the file includes multiple tables, and the function
add_document()is used, sometimes I got the errorTo solve this issue, the function
concatenate(docs)is used. However, it has one limitation that cannot keep the original page number. If the processing is just part of the file based on the page range, for example, page 5 - 20, and the chunks are page (5-10), (11-20), after calling concatenate(docs) to merge the Docling docs, the page no will be reset from 1, not keeping the starting page from 5. I have to calculate the page offset and update the page no.The code I added to update the page no to support page range:
Samples of Docling docs that got the "Document hierarchy is inconsistent." error
0704.3575_Averages_of_b-hadron_properties_at_the_end_of_2006_11_pages.pdf
segment_0.json
segment_1.json