Skip to content

Commit 3ee3d1e

Browse files
florkbrflorkbr
authored
feat(playwright-konflux): setup pipelines for running playwright tests in Konflux (#850)
* feat(playwright-konflux): setup pipelines for running playwright tests in Konflux * fix(konflux-playwright): test timing and selector fixes
1 parent f13d0ad commit 3ee3d1e

10 files changed

+441
-56
lines changed

‎.tekton/landing-page-frontend-pull-request.yaml‎

Lines changed: 70 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ metadata:
99
pipelinesascode.tekton.dev/max-keep-runs: "3"
1010
pipelinesascode.tekton.dev/on-cel-expression: event == "pull_request" && target_branch
1111
== "master"
12-
pipelinesascode.tekton.dev/pipeline: https://github.com/RedHatInsights/konflux-pipelines/blob/main/pipelines/docker-build.yaml
12+
pipelinesascode.tekton.dev/pipeline: https://github.com/RedHatInsights/konflux-pipelines/raw/main/pipelines/platform-ui/docker-build-run-all-tests.yaml
1313
creationTimestamp: null
1414
labels:
1515
appstudio.openshift.io/application: landing-page-frontend
@@ -29,10 +29,77 @@ spec:
2929
value: 5d
3030
- name: dockerfile
3131
value: build-tools/Dockerfile
32+
- name: unit-tests-script
33+
value: |
34+
#!/bin/bash
35+
set -ex
36+
37+
npm run lint
38+
npm test -- --runInBand --no-cache
39+
- name: test-app-name
40+
value: "landing-page-frontend"
41+
- name: test-app-port
42+
value: "8000"
43+
- name: run-app-script
44+
value: |
45+
#!/bin/bash
46+
set -ex
47+
# Start Caddy to serve app assets on port 8000
48+
caddy run --config /etc/caddy/Caddyfile --adapter caddyfile
49+
- name: frontend-proxy-routes-configmap
50+
value: "landing-page-frontend-dev-proxy-caddyfile"
51+
- name: e2e-playwright-image
52+
value: "mcr.microsoft.com/playwright:v1.58.2-noble"
53+
- name: e2e-tests-script
54+
value: |
55+
#!/bin/bash
56+
set -ex
57+
58+
# Set CI environment variable for Playwright config
59+
export CI=true
60+
61+
# Wait for the application to be ready
62+
echo "Waiting for application to be ready..."
63+
for i in {1..120}; do
64+
# Check that both local app and proxied resources work
65+
if curl -k -s https://stage.foo.redhat.com:1337/apps/landing/fed-mods.json > /dev/null 2>&1 && \
66+
curl -k -s https://stage.foo.redhat.com:1337/apps/chrome/fed-mods.json > /dev/null 2>&1; then
67+
echo "Application is ready!"
68+
break
69+
fi
70+
echo "Attempt $i/120: Application not ready yet..."
71+
sleep 1
72+
done
73+
74+
# Run Playwright tests
75+
npm run test:playwright
76+
- name: e2e-credentials-secret
77+
value: "landing-page-frontend-credentials-secret"
78+
- name: workspace-setup-script
79+
value: |
80+
#!/bin/bash
81+
set -ex
82+
83+
npm install
3284
pipelineRef:
33-
name: docker-build
85+
resolver: git
86+
params:
87+
- name: url
88+
value: https://github.com/RedHatInsights/konflux-pipelines
89+
- name: revision
90+
value: main
91+
- name: pathInRepo
92+
value: pipelines/platform-ui/docker-build-run-all-tests.yaml
3493
taskRunTemplate:
3594
serviceAccountName: build-pipeline-landing-page-frontend
95+
podTemplate:
96+
hostAliases:
97+
- ip: "127.0.0.1"
98+
hostnames:
99+
- "stage.foo.redhat.com"
100+
- ip: "::1"
101+
hostnames:
102+
- "stage.foo.redhat.com"
36103
workspaces:
37104
- name: workspace
38105
volumeClaimTemplate:
@@ -43,7 +110,7 @@ spec:
43110
- ReadWriteOnce
44111
resources:
45112
requests:
46-
storage: 1Gi
113+
storage: 2Gi
47114
status: {}
48115
- name: git-auth
49116
secret:

‎E2E_TEST_COVERAGE.md‎

Lines changed: 245 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,245 @@
1+
# E2E Test Coverage for landing-page-frontend
2+
3+
This document outlines all Playwright E2E tests for the landing-page-frontend application, their purpose, and execution strategy.
4+
5+
## Overview
6+
7+
All UI-related tests have been migrated to the Playwright framework. The test suite consists of **9 test spec files** covering approximately **820 lines** of test code.
8+
9+
## Test Execution Strategy
10+
11+
### In-Pipeline Tests (Target: < 5 minutes)
12+
13+
Tests that will run on every pull request in the Konflux pipeline:
14+
15+
#### 1. Authentication Tests
16+
**File**: `playwright/e2e/login.spec.ts`
17+
- **Purpose**: Verify SSO authentication flow
18+
- **Test Count**: 1 test
19+
- **Coverage**:
20+
- User can login via RH SSO
21+
- Authenticated user reaches the app
22+
- User avatar is visible after login
23+
- No redirect to sso.redhat.com after authentication
24+
- **Estimated Duration**: 5-10 seconds
25+
- **In-Pipeline**: YES
26+
27+
#### 2. Session Initialization Tests
28+
**File**: `playwright/e2e/session-init.spec.ts`
29+
- **Purpose**: Verify user session is properly initialized
30+
- **Test Count**: 1 test
31+
- **Coverage**:
32+
- User session initialized via storageState
33+
- User avatar visible after session init
34+
- **Estimated Duration**: 5 seconds
35+
- **In-Pipeline**: YES
36+
37+
#### 3. Landing Page Smoke Tests
38+
**File**: `playwright/e2e/landing-smoke.spec.ts`
39+
- **Purpose**: Critical smoke tests for core landing page functionality
40+
- **Test Count**: 2 tests
41+
- **Coverage**:
42+
- Landing page loads successfully
43+
- "My favorite services" section is visible
44+
- Default widget containers are present:
45+
- landing-rhel-widget
46+
- landing-openshift-widget
47+
- landing-ansible-widget
48+
- landing-exploreCapabilities-widget
49+
- landing-recentlyVisited-widget
50+
- chrome-favoriteServices-widget
51+
- landing-openshiftAi-widget
52+
- landing-imageBuilder-widget
53+
- landing-acs-widget
54+
- **Estimated Duration**: 15-20 seconds
55+
- **In-Pipeline**: YES
56+
57+
#### 4. Widget Basic Tests
58+
**File**: `playwright/e2e/widgets-basic.spec.ts`
59+
- **Purpose**: Verify basic widget presence, links, and removal functionality
60+
- **Test Count**: 11 tests
61+
- **Coverage**:
62+
- RHEL widget: existence, link validation, removal
63+
- Ansible widget: presence, link to `/ansible/ansible-dashboard`, removal
64+
- OpenShift widget: existence, link to `/openshift`, removal
65+
- OpenShift AI widget: presence, external link validation, removal
66+
- ACS widget: descriptive copy validation
67+
- **Estimated Duration**: 45-60 seconds
68+
- **In-Pipeline**: YES
69+
70+
#### 5. Chrome Tooltips Tests
71+
**File**: `playwright/e2e/chrome-tooltips.spec.ts`
72+
- **Purpose**: Verify chrome-level UI elements (Settings/Help tooltips)
73+
- **Test Count**: 1 test
74+
- **Coverage**:
75+
- Settings button tooltip appears on hover
76+
- Help button tooltip appears on hover with correct content
77+
- **Estimated Duration**: 10 seconds
78+
- **In-Pipeline**: YES
79+
- **Note**: Tests chrome-level elements, included in migration plan
80+
81+
### Nightly Scheduled Tests (Longer-Running)
82+
83+
Tests that should run nightly against the stage environment via app-interface:
84+
85+
#### 6. Widget Layout Operations
86+
**File**: `playwright/e2e/widgets-layout.spec.ts`
87+
- **Purpose**: Comprehensive widget layout manipulation tests
88+
- **Test Count**: 15+ tests (includes complex drag-and-drop operations)
89+
- **Coverage**:
90+
- Widget drag and drop reordering
91+
- Widget locking/unlocking
92+
- Widget removal and restoration
93+
- Layout persistence
94+
- Responsive layout behavior
95+
- Chrome overlay handling
96+
- **Estimated Duration**: 2-3 minutes
97+
- **In-Pipeline**: NO (scheduled nightly)
98+
- **Reason**: Complex interactions, longer execution time
99+
100+
#### 7. Favorite Services Widget
101+
**File**: `playwright/e2e/favorite-services-widget.spec.ts`
102+
- **Purpose**: Test favoriting/unfavoriting services in the chrome widget
103+
- **Test Count**: 8+ tests
104+
- **Coverage**:
105+
- Opening services menu
106+
- Adding services to favorites
107+
- Removing services from favorites
108+
- Category navigation (Automation, etc.)
109+
- Widget state persistence
110+
- **Estimated Duration**: 1-2 minutes
111+
- **In-Pipeline**: NO (scheduled nightly)
112+
- **Reason**: Complex chrome interactions, multiple async operations
113+
114+
#### 8. Integrations Widget
115+
**File**: `playwright/e2e/integrations-widget.spec.ts`
116+
- **Purpose**: Test integrations widget functionality
117+
- **Test Count**: 4 tests
118+
- **Coverage**:
119+
- Widget not in default layout
120+
- Adding/removing widget from layout
121+
- Expanding integration categories
122+
- Creating new integration (wizard flow)
123+
- **Estimated Duration**: 45-60 seconds
124+
- **In-Pipeline**: NO (scheduled nightly)
125+
- **Reason**: Depends on external integrations service
126+
127+
#### 9. Explore Capabilities Widget
128+
**File**: `playwright/e2e/explore-capabilities.spec.ts`
129+
- **Purpose**: Validate Explore Capabilities widget content and CTAs
130+
- **Test Count**: 1 test (validates 5 tiles)
131+
- **Coverage**:
132+
- "Get started with a tour" tile and link
133+
- "Try OpenShift on AWS" (ROSA) tile and link
134+
- "Developer Sandbox" tile and external link
135+
- "Analyze RHEL environments" tile and link
136+
- "Convert from CentOS to RHEL" tile and link
137+
- **Estimated Duration**: 30-45 seconds
138+
- **In-Pipeline**: MAYBE (borderline - monitor execution time)
139+
- **Timeout**: 90 seconds configured
140+
141+
## Test Categories Summary
142+
143+
### By Category
144+
- **Authentication**: 2 tests
145+
- **Smoke Tests**: 2 tests
146+
- **Widget Functionality**: 11 tests
147+
- **Widget Layout**: 15+ tests
148+
- **Chrome Integration**: 8+ tests
149+
- **External Integrations**: 4 tests
150+
- **Navigation/CTAs**: 1 test (5 validations)
151+
152+
### By Execution Location
153+
- **In-Pipeline (PR)**: ~18 tests (estimated < 2.5 minutes total)
154+
- **Nightly Scheduled**: ~28 tests (estimated 4-6 minutes total)
155+
156+
## Test Infrastructure
157+
158+
### Authentication
159+
- **Method**: Global setup with `storageState`
160+
- **File**: `playwright/global-setup.ts`
161+
- **Flow**: Login performed once, session saved to `playwright/.auth/user.json`
162+
- **Credentials**: Sourced from Vault via `landing-page-frontend-credentials-secret`
163+
164+
### Page Objects
165+
- **LandingPage**: `playwright/pages/LandingPage.ts`
166+
- Provides helper methods for common operations
167+
- Widget selectors and interactions
168+
- Layout reset functionality
169+
- Navigation and waiting utilities
170+
171+
### Test Configuration
172+
- **Config File**: `playwright.config.ts`
173+
- **Base URL**: `https://stage.foo.redhat.com:1337`
174+
- **Browser**: Chromium (primary), Firefox/WebKit (commented out)
175+
- **Parallelization**: Disabled on CI (workers: 1)
176+
- **Retries**: 2 on CI, 0 locally
177+
- **Reporter**: HTML
178+
179+
## Nightly Test Configuration (app-interface)
180+
181+
The nightly scheduled tests should be configured in app-interface with the following parameters:
182+
183+
### Test Selection
184+
Run only the nightly test suite:
185+
```bash
186+
npx playwright test \
187+
playwright/e2e/widgets-layout.spec.ts \
188+
playwright/e2e/favorite-services-widget.spec.ts \
189+
playwright/e2e/integrations-widget.spec.ts \
190+
playwright/e2e/explore-capabilities.spec.ts
191+
```
192+
193+
### Environment
194+
- **Target**: Stage environment (`https://stage.foo.redhat.com:1337`)
195+
- **Frequency**: Nightly
196+
- **Reporting**: Results sent to Ibutsu
197+
198+
### Credentials
199+
- Use the same Vault credentials as in-pipeline tests
200+
- Ensure E2E_USER and E2E_PASSWORD environment variables are set
201+
202+
## Test Maintenance
203+
204+
### Adding New Tests
205+
1. Create spec file in `playwright/e2e/`
206+
2. Import Page Objects as needed
207+
3. Follow existing patterns for authentication and setup
208+
4. Document in this file
209+
5. Categorize as in-pipeline or nightly based on execution time
210+
211+
### Monitoring Execution Time
212+
- In-pipeline tests MUST complete in < 5 minutes total
213+
- If in-pipeline suite exceeds 5 minutes, move longest tests to nightly
214+
- Monitor Konflux pipeline runs for timing data
215+
216+
### Test Failure Investigation
217+
1. Check Playwright HTML report
218+
2. Review traces (enabled on first retry)
219+
3. Check Konflux pipeline logs for environment issues
220+
4. Verify credentials are valid
221+
5. Check for stage environment outages
222+
223+
## Migration Notes
224+
225+
All tests have been migrated from the previous Cypress framework to Playwright:
226+
- Cypress component tests → Playwright E2E tests
227+
- Cypress E2E tests → Playwright E2E tests
228+
- Authentication flow updated to use Playwright global setup
229+
- Page Object pattern maintained for consistency
230+
231+
## Success Criteria
232+
233+
- All in-pipeline tests pass on every PR
234+
- Total in-pipeline execution time < 5 minutes
235+
- Nightly tests run successfully against stage
236+
- Test failures reported to Ibutsu
237+
- No test flakiness (retries should rarely be needed)
238+
239+
## Links
240+
241+
- Playwright Documentation: https://playwright.dev/
242+
- Playwright Config: `/playwright.config.ts`
243+
- Global Setup: `/playwright/global-setup.ts`
244+
- Page Objects: `/playwright/pages/`
245+
- Test Files: `/playwright/e2e/`

0 commit comments

Comments
 (0)