docling-pdf: probe the xref/trailer when a PDF will not load

A real invoice fails at the very first stage — lopdf reports "invalid file
trailer" while pdfium reads the same file fine — so the browser spends ~10 s
on OCR for a document whose text layer is right there. The failure says
nothing about which part of the cross-reference machinery is wrong, and the
file is personal, so the diagnosis has to come from the reporter's own
machine.

On a load failure the example now dumps what the parser tripped over: which
markers exist (trailer / startxref / xref / /XRef / %%EOF), how many bytes
trail the final %%EOF, the last startxref block, and the bytes at the offset
it points to. A classic xref table, an xref stream, an offset past the end of
the file and plain garbage each read differently there and call for different
repairs.

Exercised on two deliberately broken copies of a corpus PDF: appended junk
after %%EOF still loads (lopdf tolerates it), while a corrupted startxref
offset reports "startxref points to 999999999, past the end of the file".

Refs #157

Signed-off-by: artiz <artem.kustikov@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EY5KAiquN4YpVf2PXEQkVT

Author: artiz

crates/docling-pdf/examples/text_layer.rs

@@ -35,6 +35,7 @@ fn main() {
         Err(e) => {
             eprintln!("1. lopdf load: FAILED — {e}");
             eprintln!("   (the browser reports this as \"no embedded text layer\")");
+            probe_tail(&bytes);
         }
         Ok(doc) => {
             eprintln!(
@@ -80,3 +81,70 @@ fn main() {
         }
     }
 }
+
+/// When the document will not load, show the cross-reference machinery the
+/// parser choked on: the last `startxref`, where it points, and what actually
+/// sits there. A classic `xref` table, an xref *stream* (`/Type /XRef`), bytes
+/// past the end of the file or plain garbage each point at a different repair,
+/// and this is enough to tell them apart without handing the PDF around.
+fn probe_tail(bytes: &[u8]) {
+    let find_last = |needle: &[u8]| {
+        bytes
+            .windows(needle.len())
+            .rposition(|w| w == needle)
+            .map(|i| (i, needle.len()))
+    };
+    let printable = |b: &[u8]| -> String {
+        b.iter()
+            .map(|&c| {
+                if (0x20..0x7f).contains(&c) {
+                    c as char
+                } else if c == b'\n' || c == b'\r' {
+                    '\u{23ce}'
+                } else {
+                    '.'
+                }
+            })
+            .collect()
+    };
+
+    eprintln!("   file: {} bytes", bytes.len());
+    eprintln!(
+        "   markers: trailer={} startxref={} xref={} /XRef={} %%EOF={}",
+        find_last(b"trailer").is_some(),
+        find_last(b"startxref").is_some(),
+        find_last(b"xref").is_some(),
+        find_last(b"/XRef").is_some(),
+        find_last(b"%%EOF").is_some(),
+    );
+
+    // Trailing bytes after the final %%EOF confuse a strict tail scan.
+    if let Some((i, len)) = find_last(b"%%EOF") {
+        let after = bytes.len() - (i + len);
+        eprintln!("   bytes after the last %%EOF: {after}");
+    }
+
+    let Some((sx, _)) = find_last(b"startxref") else {
+        eprintln!("   no startxref at all — the xref would have to be rebuilt by scanning objects");
+        return;
+    };
+    let tail = &bytes[sx..bytes.len().min(sx + 60)];
+    eprintln!("   last startxref block: {:?}", printable(tail));
+
+    // The number after `startxref` is the byte offset of the xref section.
+    let digits: String = tail
+        .iter()
+        .skip(b"startxref".len())
+        .skip_while(|c| c.is_ascii_whitespace())
+        .take_while(|c| c.is_ascii_digit())
+        .map(|&c| c as char)
+        .collect();
+    match digits.parse::<usize>() {
+        Ok(off) if off < bytes.len() => {
+            let end = bytes.len().min(off + 120);
+            eprintln!("   at offset {off} -> {:?}", printable(&bytes[off..end]));
+        }
+        Ok(off) => eprintln!("   startxref points to {off}, past the end of the file"),
+        Err(_) => eprintln!("   startxref carries no parseable offset"),
+    }
+}