Skip to content

chore(tests): visual regression testing POC #5214

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a74328b
feat(setup): initial set for the playwright
alionazherdetska Apr 3, 2025
de5f31d
feat(testing): added the screenshot for the button
alionazherdetska Apr 3, 2025
76605d8
feat(feat): implemented the initial setup for the playwright visual t…
alionazherdetska Apr 20, 2025
c105222
feat(setup): added the scripts for the visual tests
alionazherdetska Apr 20, 2025
de14127
Merge branch 'visual-testing-playwright' into 4923-using-playwright-f…
alionazherdetska Apr 20, 2025
6e3b1da
chore(workflow): added missing permissions
alionazherdetska Apr 20, 2025
7437476
chore(tests): added the screenshots for browsers
alionazherdetska Apr 21, 2025
4f2fa22
removed screenshots
alionazherdetska Apr 21, 2025
893cda6
remove redundant files
alionazherdetska Apr 21, 2025
d079f96
chore: added screenshots
alionazherdetska Apr 21, 2025
8ed36d6
test
alionazherdetska Apr 21, 2025
7e918fe
removed the screenshots
alionazherdetska Apr 21, 2025
ff92071
added creenshots
alionazherdetska Apr 21, 2025
0154a92
remove screeshots
alionazherdetska Apr 21, 2025
62292d7
added new screenshots
alionazherdetska Apr 21, 2025
43648cf
chore(test): removed redundant code
alionazherdetska Apr 22, 2025
cb126d2
chore: removed redundant comments
alionazherdetska Apr 24, 2025
39fa628
chore(workflow): refactore upload screenshot on failure job
alionazherdetska Apr 24, 2025
fc962f2
test
alionazherdetska Apr 24, 2025
22c8d37
Merge branch 'main' into 4923-using-playwright-for-visual-regression-…
alionazherdetska Apr 24, 2025
a86f5ca
fixed broken lock.file
alionazherdetska Apr 24, 2025
1a2b6de
fixed the button.html
alionazherdetska Apr 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions .github/workflows/visual-tests.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
name: Visual Regression Tests
on:
pull_request:
paths:
- 'packages/styles/**'
workflow_dispatch:
inputs:
update-snapshots:
description: 'Update visual snapshots'
type: boolean
default: false

permissions:
contents: read
actions: read
pull-requests: read

jobs:
visual-tests:
name: Visual Tests
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
with:
fetch-depth: 0

- name: Setup
uses: ./.github/actions/setup-pnpm

- name: Install dependencies and build packages
run: pnpm bootstrap

- name: Install Playwright browsers
working-directory: packages/styles
run: npx playwright install --with-deps

- name: Get snapshot cache
id: snapshot-cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684
with:
path: packages/styles/tests/visual/__screenshots__
key: visual-snapshots-${{ github.ref }}
restore-keys: |
visual-snapshots-refs/heads/main

- name: Update Snapshots
if: github.event.inputs.update-snapshots == 'true' || steps.snapshot-cache.outputs.cache-hit != 'true'
working-directory: packages/styles
run: pnpm test:visual:update

- name: Run visual tests
working-directory: packages/styles
run: pnpm test:visual

- name: Upload test results
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: playwright-report
path: packages/styles/test-results
retention-days: 14

# Upload diff screenshots on failure
- name: Upload diff screenshots
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
with:
name: playwright-screenshots
path: |
packages/styles/test-results/**/*-diff.png
packages/styles/test-results/**/*-actual.png
packages/styles/test-results/**/*-expected.png
retention-days: 14
6 changes: 6 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,12 @@
"styles:unit": "pnpm --filter design-system-styles unit",
"styles:lint": "pnpm --filter design-system-styles lint",
"styles:lint:fix": "pnpm --filter design-system-styles lint:fix",
"styles:visual": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual",
"styles:visual:update": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual:update",
"styles:visual:ui": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual:ui",
"styles:visual:chrome": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual:chrome",
"styles:visual:firefox": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual:firefox",
"styles:visual:safari": "pnpm --filter design-system-styles build && pnpm --filter design-system-styles test:visual:safari",
"components": "pnpm components:start",
"components:play": "pnpm --filter design-system-components play",
"components:start": "pnpm --filter design-system-components start",
Expand Down
15 changes: 15 additions & 0 deletions packages/styles/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,18 @@ src/utilities/_post-icon-version.scss

# Temporary files
src/tokens/temp/

# Visual testing ignores
test-results/
test-results-*
playwright-report/

# Screenshot artifacts
**/tests/visual/__screenshots__/**/*-actual.png
**/tests/visual/__screenshots__/**/*-diff.png
**/tests/visual/__screenshots__/**/*-expected.png

# Playwright
/playwright/.cache/
playwright/.auth/
*.tsbuildinfo
12 changes: 10 additions & 2 deletions packages/styles/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,13 @@
"unit": "gulp sass:tests",
"lint": "stylelint src/**/*.scss !src/tokens/temp/**",
"lint:fix": "stylelint src/**/*.scss !src/tokens/temp/** --fix",
"format": "prettier src/**/*.scss --write"
"format": "prettier src/**/*.scss --write",
"test:visual": "playwright test",
"test:visual:update": "playwright test --update-snapshots",
"test:visual:ui": "playwright test --ui",
"test:visual:chrome": "playwright test --project=chromium",
"test:visual:firefox": "playwright test --project=firefox",
"test:visual:safari": "playwright test --project=webkit"
},
"peerDependencies": {
"@angular/core": "^19.0.0",
Expand All @@ -50,6 +56,7 @@
"gulp-sourcemaps": "3.0.0"
},
"devDependencies": {
"@playwright/test": "1.52.0",
"@swisspost/design-system-icons": "workspace:9.0.0-next.34",
"@swisspost/design-system-tokens": "workspace:9.0.0-next.34",
"@types/node": "22.10.5",
Expand All @@ -60,6 +67,7 @@
"gulp-newer": "^1.4.0",
"gulp-postcss": "10.0.0",
"gulp-sass": "6.0.0",
"http-server": "14.1.1",
"jest": "29.7.0",
"postcss": "8.4.49",
"postcss-scss": "4.0.9",
Expand Down Expand Up @@ -90,4 +98,4 @@
"ui",
"frontend"
]
}
}
51 changes: 51 additions & 0 deletions packages/styles/playwright.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
testDir: './tests/visual',

testMatch: '**/*.spec.ts',

timeout: 30000,

reporter: [
['html', { outputFolder: 'test-results/html-report' }],
['list']
],

snapshotPathTemplate: '{testDir}/__screenshots__/{projectName}/{testFilePath}/{arg}{ext}',

outputDir: 'test-results/artifacts',

use: {
baseURL: 'http://localhost:9000',
},

// Projects for different browsers
projects: [
// Desktop Chrome
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] },
},

// Desktop Firefox
{
name: 'firefox',
use: { ...devices['Desktop Firefox'] },
},

// Desktop Safari
{
name: 'webkit',
use: { ...devices['Desktop Safari'] },
}
],

// Start a web server for the tests
webServer: {
command: 'npx http-server . -p 9000',
url: 'http://localhost:9000',
reuseExistingServer: true,
timeout: 60000,
},
});
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 18 additions & 0 deletions packages/styles/tests/visual/button.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { test, expect } from '@playwright/test';

test.describe('Button Component Visual Test', () => {
test('primary button appearance', async ({ page }) => {
await page.goto('/tests/visual/components/button.html');

await page.waitForTimeout(2000);

const button = page.locator('.btn-primary');
await expect(button).toHaveScreenshot('button-primary.png');

await button.hover();
await expect(button).toHaveScreenshot('button-primary-hover.png');

await button.focus();
await expect(button).toHaveScreenshot('button-primary-focus.png');
});
});
13 changes: 13 additions & 0 deletions packages/styles/tests/visual/components/button.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<link rel="stylesheet" id="styles" href="/dist/post-default.css" />
<title>Button Test</title>
</head>
<body>

<button class="btn btn-primary">Button</button>
</body>
</html>
18 changes: 18 additions & 0 deletions packages/styles/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"compilerOptions": {
"target": "ES2019",
"module": "commonjs",
"moduleResolution": "node",
"esModuleInterop": true,
"sourceMap": true,
"outDir": "./dist",
"baseUrl": ".",
"skipLibCheck": true,
"strict": true,
"resolveJsonModule": true,
"forceConsistentCasingInFileNames": true,
"types": ["node", "@playwright/test"]
},
"include": ["tests/**/*.ts", "playwright.config.ts"],
"exclude": ["node_modules"]
}
Loading