VLM doclang fallback - #159
Merged
Merged
Conversation
…ocling-project#153) The VLM pipeline routes non-DocTags model output through the DocLang XML reader, which is a strict parser: a model that leaves a <heading>/<text> unclosed produces markup it rejects with `expected 'heading' tag, not 'doclang'`, and that error propagated all the way out — dropping the entire normal_4pages fixture from the docling-project#153 corpus run ("rust side failed"). The DocTags path never hard-fails on hostile model output (docling-project#152: best-effort document out); the DocLang fallback now matches it. parse_doclang_fragments tries the strict reader first (well-formed DocLang keeps its heading levels and table structure) and, on a parse error, salvages the fragments through the tolerant DocTags parser, which degrades broken/unknown tags to text and never errors. Two regression tests cover both branches. Refs docling-project#153 Signed-off-by: artiz <artem.kustikov@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT
…-project#153) picture_classification scored 1.3% similarity in the live VLM corpus run. Root-causing it deterministically: our DocTags parser reproduces docling's groundtruth Markdown for that fixture byte-for-byte (pictures with captions before the image placeholder, multi-page <page_break>, reading order), so the divergence is live-model / render variance — which the harness itself flags as expected ("triage as render, not parser") — not a defect on our side. Add an end-to-end regression test through convert_vlm that feeds the model's exact DocTags for the document and asserts the Markdown matches the groundtruth, so our side of the comparison stays provably correct. Image leg, so it needs no pdfium and runs in CI everywhere. Refs docling-project#153 Signed-off-by: artiz <artem.kustikov@gmail.com> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.