fix(chunker): respect contextualized table token limits

[pablopupo]

• Apply the HybridChunker metadata-adjusted budget when splitting Markdown table rows.
• Count exact combined fragments at token boundaries and preserve table content when an oversized prefix is split.
• Add regressions for contextualized overflow, boundary tokenization, and prefix-to-body reconstruction.

Resolves docling-project/docling#3428

Verification

• pytest --cov=docling_core --cov-report=xml test: 566 passed, 6 skipped
• ruff format --check docling_core test, ruff check docling_core test, and mypy docling_core test
• Supplied PDF: maximum contextualized chunk 8,189 / 8,192; both tables reconstructed byte-for-byte

[github-actions]

✅ DCO Check Passed

Thanks @pablopupo, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🔴 1 of 2 protections blocking · waiting on 👀 reviews

	Protection	Waiting on
🔴	Require two reviewer for test updates	👀 reviews
🟢	Enforce conventional commit	—

🔴 Require two reviewer for test updates

Waiting for

• #approved-reviews-by >= 2

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

---

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[codecov]

Codecov Report

❌ Patch coverage is 82.35294% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling_core/transforms/chunker/line_chunker.py	81.25%	3 Missing ⚠️

📢 Thoughts on this report? Let us know!