fix: add a pre-scan step to detect the true last non-empty row/column and limit the scan range accordingly

[HuangruiChu]

Fix #2307, Follow the instruction of #2307 (comment).
Add a pre-scan step to detect the true last non-empty row and column (including merged cells), then use those bounds to limit all subsequent scans. This will avoid iterating over millions of empty cells and dramatically improve performance on large, sparse sheets.

Issue resolved by this Pull Request:
Resolves #2307

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[github-actions]

✅ DCO Check Passed

Thanks @HuangruiChu, all your commits are properly signed off. 🎉

[dosubot]

Related Documentation

Checked 2 published document(s). No updates required.

^{You have 5 draft document(s). Publish docs to keep them always up-to-date}

^{How did I do? Any feedback?}  

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Require two reviewer for test updates

This rule is failing.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[HuangruiChu]

Need to fix the "Ruff formatter...........................................................Failed".

[codecov]

Codecov Report

❌ Patch coverage is 4.54545% with 21 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/msexcel_backend.py	4.54%	21 Missing ⚠️

📢 Thoughts on this report? Let us know!

[cau-git]

@HuangruiChu thanks for your contribution. Can you please restore the test_backend_msexcel test unit, which apparently was deleted in this PR?

[ceberam]

Thanks @HuangruiChu for your contribution. Here are some comments:

• Did you test and have any evidence that this PR solves the intended issue #2307 ? I tested the PR against large spreadsheets with empty cells and the processing time was more than double compared to the current implementation.
• The new function _find_true_data_bounds is computationally expensive but it limits the scan range in _find_data_tables and overall it may pay off using it. However, it should also be leveraged in _find_table_bottom and _find_table_right, which still have some unbounded scans. This could explain the increase of processing time in this PR.
• Restore the test_backend_msexcel.py to ensure regression tests on this backend pass, as pointed out by @cau-git .
• Remove some unnecessary comments like # Example integration in MsExcelDocumentBackend._find_data_tables

[HuangruiChu]

Thank you for your reply. Sorry I mistakenly deleted the content of the "test_backend_msexcel.py". I was planning to add a test for "_find_true_data_bounds" under "test_backend_msexcel.py".

1. "it should also be leveraged in _find_table_bottom and _find_table_right," Thank you for remind me I am working on update this.
2. "Remove some unnecessary comments" Sure, I will remove that.
3. "Did you test and have any evidence that this PR solves the intended issue Large spreadsheets - very slow processing #2307 ?" I create a version where the max_row is close to 1M by copy "test-01.xlsx". The processing time for the whole datasheet1 is around 2.0 seconds
   .