Skip to content

feat: Visual regression tests #736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

feat: Visual regression tests #736

wants to merge 4 commits into from

Conversation

tuliomir
Copy link
Collaborator

@tuliomir tuliomir commented Feb 14, 2025

Motivation

Having end-to-end tests with Cypress ensure the overall flow of the application is working as expected, however the visual aspects are not considered. See more about why to do visual tests on this external article.

The first implementation for the visual regression was made on #703, using only the local resources of the CI environment. This, however, led to inconsistent renderings of the screens and false positives on the diffs.

Rationale and Alternatives

The best solution to this is to export the HTML to a third party to have it processed in a consistent environment and the diffs are generated there. Most of the commercially available third parties are more focused in managing interaction between designer and development teams.

Happo.io has a workflow closer to the one at Hathor Labs, in which the renderings are generated in each PR and compared to the latest rendering in master. There is also the need to review and approve any visual change that the PR may cause, and this reviewing process integrates well with the GitHub PR interface.

PR Objectives and Reviewing Process

The proposal of this PR sits somewhere between a Proof-of-Concept and a pilot implementation for a larger feature. Since it requires the installation of new GitHub plugins to work, the first contact of reviewers with it will naturally see a failing CI.

So, it is expected that the PR will have to undergo some steps:

  1. Initial review, with a failing CI
    This review aims to get to know the proposed feature and see the code that will be implemented. Something close to the review of a design. A working version of this PR can be seen at a fork of this repository on my personal account ( link ).

  2. Proper configuration of the plugins
    Upon approval of the general concept and code, an administrator of this repository ( in this case, @pedroferreira1 ) will have to install and configure the Happo plugin. All the steps necessary to perform this are described below, in the "Pending Activities" section.

  3. Second review, with a passing CI
    After all the configurations are done, The CI will pass and the reviewers will be able to interact with the Happo platform to validate all features of this PR.

Only after this second review and posterior approval will this PR be merged and become a part of the workflow for the Hathor Desktop Wallet.

Acceptance Criteria

Pending activities

  • Create a Happo.io account for Hathor Labs
  • Install the Happo.io application on this repository ( docs )
  • Add its API keys to the repository secrets
  • Second review to ensure all configurations are properly working and the workflow is functional

Future implementations

This PR only contains an initial implementation of this feature, upon which a full test suite for the whole application can be built. This will be done gradually in many PRs.

Security Checklist

  • Make sure you do not include new dependencies in the project unless strictly necessary and do not include dev-dependencies as production ones. More dependencies increase the possibility of one of them being hijacked and affecting us.

@tuliomir tuliomir added enhancement New feature or request tests labels Feb 14, 2025
@tuliomir tuliomir requested a review from r4mmer February 14, 2025 15:04
@tuliomir tuliomir self-assigned this Feb 14, 2025
@tuliomir tuliomir mentioned this pull request Feb 14, 2025
1 task
@tuliomir tuliomir marked this pull request as ready for review February 14, 2025 16:42
@@ -15,6 +15,7 @@ on:
jobs:
cypress-run:
runs-on: ubuntu-22.04
environment: Happo CI Env
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line enforces a specific environment for retrieving the API secrets and must be adjusted for this repository.

@tuliomir tuliomir mentioned this pull request Mar 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request tests
Projects
Status: In Progress (Done)
Development

Successfully merging this pull request may close these issues.

2 participants