Description
Describe the Bug
Hi, first of all huge thanks for your work on this great plugin.
I've noticed [...].actual.png
files don't have FRSOURCE_CPVRD_V
set, which causes problems when they're used to update the baseline images.
In our workflow we mostly use Cypress on the command line (cypress run
). We "accept" a new baseline image by overwriting the old baseline with the new [...].actual.png
file (see "Additional context" for why we do this).
This mostly works fine, but this .actual.png
file doesn't have the FRSOURCE_CPVRD_V
field set. This means that when we run the same tests later, FRSOURCE_CPVRD_V
gets set by the plugin, and therefore there is a diff according to git
, but not visually. This can show up later in the diff for an unrelated change, which is difficult for a reviewer to understand, and will happen repeatedly as baselines are updated over time.
Steps to reproduce
Developer A:
- Create a test and run it, generating an initial baseline
- Modify the component under test such that the test will fail
- Run the test, which fails, producing
.actual.png
and.diff.png
files - Rename
.actual.png
to overwrite the baseline - Commit, push etc.
Developer B:
- Run the test, which passes, but which automatically updates the baseline to include
FRSOURCE_CPVRD_V
- Now any commits/PR etc. this developer makes may include a confusing rewrite of that image
- Any subsequent baseline updates (performed like in step 4) will unset
FRSOURCE_CPVRD_V
and cause this cycle to happen again!
Expected behavior
I would expect to see no diff after step 6. The implication of step 8 is that seemingly random results are introduced into diffs, including PRs. I think setting FRSOURCE_CPVRD_V
when generating the .actual.png
files would solve the issue.
Additional context
I'm aware that updateImages
does not have this problem, but it doesn't work for our GHA workflow: we run the tests and would like indicate which are failing (not possible with updateImages
) and then automatically open a PR with new baselines. It is these PRs which contain many irrelevant diffs.
Despite our (potentially unusual) use case, I imagine all users who rename .actual.png
to update their baselines are running into similar problems.
Let me know if I'm missing anything, I could look into opening a PR otherwise.
Package version
3.3.10
Device
Any
OS and version
Any
Browser and version
Any
Node version
Any
I can work on it by myself
- Yes
- No
Code of Conduct
- I agree to follow this project's Code of Conduct