Skip to content

Include a README.md as part of vrt:generate-backstop-config #79

@dalin-

Description

@dalin-

We need to give people more guidance. In the same way that we include a starter backstop.json, we can include a starter README.md and link to it from the root README.md

Example to append to the root README.md


## Visual Regression Testing (VRT)
Run VRT with `fire vrt:run`
For details see the BackStop [README.md](./tests/backstop/README.md)

Example README.md to add to tests/backstop

# What Is It?

BackstopJS uses a headless browser to create screenshots of webpages on different
environments (or at different moments in time) and then creates a diff of the
two images; the changed areas are highlighted.

# Pre-requisites

* Ddev
  * Orbstack is recommended for _much_ faster test runs.
* FIRE

# Terminology
It's important to understand the Backstop terminology:
* **reference** — the environment that we're comparing against.  For us this is typically the `live` environment, or maybe `test`.
* **test** — the environment where we've made changes, and we want to ensure that only the things that we expect appear different.  For us this is typically a `pr-*` environment, or maybe `dev`.

# Run the tests

`fire vrt:run`

# Review the results

At the end the report will open in your browser.

If you see no differences, you're done.

If the differences are:
* Intentionally caused by your code changes (move on to the next step about updating the reference)
* Unintentionally caused by a bug in your code changes. Either:
  * fix the code (You can run a single scenario with `--filter=<scenarioLabelRegex>`)
  * agree with the team to live with it (then continue)
* Obviously because of a content difference between the two environments (then continue)
* Possibly because of a content difference, but you're not sure (consult with someone who knows the site better, or who knows VRT better)

# Known Issues

* False positives
    * Any time content fades in, moves, or animates it will happen at slightly different speeds each time.  If you find one like this, add some `postInteractionWait` to delay the screenshot until after this is complete.
    * Sometimes if you are testing a local site (not a Pantheon environment) images will show as half loaded.  This is
      because Stage File Proxy is still downloading the image. Re-run the test
      to see better results.
    * Styleguide tests may fail because images/videos pull from a random image generator.
    * Search results may fail due to minor sorting differences ¯\_(ツ)_/¯ .
    * Lazy-loaded images are problematic.  See https://github.com/garris/BackstopJS/pull/1601
* Things we don't currently test:
    * Admin screens, or anything else logged in.  See `get-logged-in-cookie.example.sh` in combination with `cookiePath`
    * User interaction (e.g. clicking menus, submitting forms, scrolling),
      To add more, see the [BackstopJS Advanced Scenarios](https://github.com/garris/BackstopJS/#advanced-scenarios).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions