Skip to content

test: add percentage-based tolerance for image size verification#3660

Open
ceberam wants to merge 3 commits into
mainfrom
test/add-fuziness-verify-picture
Open

test: add percentage-based tolerance for image size verification#3660
ceberam wants to merge 3 commits into
mainfrom
test/add-fuziness-verify-picture

Conversation

@ceberam

@ceberam ceberam commented Jun 19, 2026

Copy link
Copy Markdown
Member

Summary

Implements percentage-based tolerance for image size verification in verify_picture_image_v2() to handle cross-platform rendering differences, following the same pattern as bounding box verification.

Problem

Image verification tests were failing when run on different platforms (Linux vs macOS) due to minor pixel differences in image dimensions caused by platform-specific rendering. These failures occurred even when the code was working correctly, blocking legitimate test runs.

Example failure:

  • Ground truth: 254x267 pixels (generated on Linux)
  • Test output: 251x267 pixels (generated on macOS)
  • Difference: 3 pixels (~1.2%) - within normal cross-platform variance

Solution

Added percentage-based tolerance thresholds similar to existing bbox verification:

Constants added:

  • STRICT_IMAGE_SIZE_TOL_RATIO = 0.015 (~1.5% tolerance for cross-platform variance)
  • FUZZY_IMAGE_SIZE_TOL_RATIO = 0.05 (~5% tolerance for OCR/image processing variance)

Key features:

  1. True percentage-based comparison: Compares actual percentage differences against tolerance ratios
  2. Precise for all image sizes: No minimum pixel tolerance that could allow excessive percentage differences
  3. Independent width/height checks: Each dimension validated separately
  4. Clear error messages: Shows actual values, pixel differences, and percentage differences vs tolerance
  5. Consistent with existing patterns: Follows the same approach as _assert_bbox_close()
  6. Demonstrates that DOCX images now pass verification in strict mode with the new percentage-based tolerance

Testing

✅ All existing tests pass (15 tests in test_verify_utils.py)
✅ New parameterized test covers 8 scenarios (strict/fuzzy modes, width/height, edge cases)
✅ Original failing case (254x267 vs 251x267) now passes in strict mode
✅ DOCX tests now run with fuzzy=False, demonstrating proper cross-platform handling
✅ Cross-platform compatibility verified on macOS and Fedora Linux

Related Issues

Fixes #3659

ceberam added 3 commits June 19, 2026 11:09
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
The images in docx should pass the verification with small tolerance due to cross-platform

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>
@ceberam ceberam self-assigned this Jun 19, 2026
@ceberam ceberam added the tests issue related to changes needed in the tests label Jun 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@codecov

codecov Bot commented Jun 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

tests issue related to changes needed in the tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add percentage-based tolerance for image size verification

2 participants