feat(degradations): model paper that is not flat and light that is not even - #15
Open
hBouanane wants to merge 1 commit into
Open
feat(degradations): model paper that is not flat and light that is not even#15hBouanane wants to merge 1 commit into
hBouanane wants to merge 1 commit into
Conversation
…t even Rendering-based synthesis produces a page lying perfectly flat under perfectly uniform light. Real captures never do, and the literature puts the gap at several points on documents photographed in the wild. - Wrinkles: local smooth displacement plus the shading its ridges catch. - PageCurl: a bound page bending away from the sensor, compressing text toward one edge and darkening into the gutter - an artefact flat perspective warping cannot produce. - IlluminationField: smooth arbitrary lighting, distinct from Shadow (a linear ramp) and Vignette (radial, centred). Real lighting is neither. The deformations map the annotation through the same displacement field, sampled at each box corner rather than inverted analytically: the field is smooth so corner sampling tracks the ink, and it works for any field without a closed form. Verified by measuring where the ink actually landed and requiring the boxes to be there. Fields are generated small and upsampled, because paper deformation and lighting are smooth at a scale far larger than a glyph - per-pixel would be both wrong and slow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #14. Targets the sim-to-real gap that reviewers always attack — realistic scene synthesis reports +8.21 on wild documents from modelling deformation and illumination.
What was missing
OCRSmith modelled flat perspective only. A real capture has a sheet that bends and light that falls unevenly.
WrinklesPageCurlIlluminationFieldPageCurlproduces the single most recognisable artefact of a photographed book page, and flat perspective warping cannot make it.IlluminationFieldis deliberately distinct from the existingShadow(a linear ramp) andVignette(radial, centred) — real lighting is neither.The part that matters
These displace pixels, so they carry the annotation through the same field. Rather than inverting each field analytically, the displacement is sampled at every box corner — the fields are smooth, so corner sampling tracks the ink, and it generalises to any field without needing a closed form.
Verified by measuring the actual ink extent in the degraded image and requiring the boxes to be there: IoU > 0.85 for both deformations.
Implementation note
Fields are generated at 1/28 scale and upsampled. Paper deformation and lighting are smooth at a scale far larger than a glyph, so a per-pixel field would be both physically wrong and slow. Resampling is nearest-neighbour deliberately: bilinear would soften strokes in a way indistinguishable from a blur degradation, which would confound any ablation.
Verified
522 tests (22 new). The
photoandarchivepresets now use these.