kittest: Add UPDATE_SNAPSHOTS=force
#7508
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new mode,
UPDATE_SNAPSHOTS=force
, which will lower the threshold to zero, overwriting every image that is not exactly the same.Most comparisons has a threshold because different GPUs render slightly differently. However, setting that threshold accurately can be hard.
Sometimes a test will pass locally, but fail on CI. In those cases you want to force an update of the failing test. You can use
UPDATE_SNAPSHOTS=force
for that.And sometimes a small change should update all images, but the change is so tiny that it falls under the threshold. Still, you want to make a point of showing that these images have changes. You can use
UPDATE_SNAPSHOTS=force
for that.