Skip to content

cypress-tutorial.md

Latest commit

 

History

History
844 lines (592 loc) · 24.2 KB

cypress-tutorial.md

File metadata and controls

844 lines (592 loc) · 24.2 KB

Cypress Testing Tutorial for ODH Dashboard

A practical guide to Cypress testing in ODH Dashboard. Covers test architecture, test IDs and page objects, writing and running tests, and best practices for maintainable test code.

Table of Contents

Overview

The ODH Dashboard uses multiple test types:

Test Type Purpose Speed Network
Unit Component logic and utilities (Jest + React Testing Library) Fastest None
Mocked UI integration with mocked API responses (Cypress) Fast Mocked
E2E Full integration testing against live cluster (Cypress) Slower Real

For detailed setup and file locations, see docs/testing.md.

Key Architecture Benefits

Concept Benefit
Same Repository Product code and tests live together - checkout v1.0 gets both code and tests for that version
Shared Test IDs Same data-testid attributes used across Unit, Mocked, and E2E tests
Page Objects Encapsulate selectors in reusable classes - change once, all tests work
Test Cases in Code Use cy.step() instead of external docs (e.g., Polarion) - always in sync
Shared Utilities Common functions for login, API mocking, cluster operations

Test IDs and Page Objects

The Problem: Fragile Selectors

Old way (breaks easily):

// If HTML structure changes, test breaks!
cy.get('table tbody tr:first-child td:nth-child(3) button').click();

Problems:

  • Breaks when UI changes
  • Hard to understand
  • Hard to maintain

The Solution: Test IDs

New way (stable):

<Button data-testid="create-project">Create Project</Button>
cy.findByTestId('create-project').click();

Key Concept: Shared Test IDs

Test IDs are shared across all test types (see Overview).

Example Flow:

Component Code
    ↓
data-testid="create-project"
    ↓
    ├─── Unit Test (Jest)
    │    └── screen.getByTestId('create-project')
    │
    └─── Cypress Test (Mocked or E2E)
         └── cy.findByTestId('create-project')

Who Updates Test IDs?

Both developers and QE can add or update test IDs via PR. When a test ID changes, update:

  1. Component code
  2. Unit tests
  3. Page object method (all tests using page objects automatically work)

Naming Conventions

  1. Use kebab-case: create-project-button
  2. Be descriptive: model-serving-deploy-button not btn1
  3. Include context: project-list-name-filter
  4. Always use static prefix with dynamic suffix: card ${resource.id} not just resource.id

Selection Patterns

// Array syntax for compound test IDs (e.g., data-testid="card my-model-name")
cy.findByTestId(['card', 'my-model-name']).click();

// Chained selection (scoped within parent element)
cy.findByTestId('project-row').findByTestId('edit-button').click();

Benefits

  • ✅ Survives UI changes - Test IDs don't break when CSS/HTML changes
  • ✅ Easy to understand - Clear intent vs cryptic selectors
  • ✅ Single source of truth - Defined once in component
  • ✅ Consistency - Same selector across all test types
  • ✅ Refactoring safe - Changes propagate via page objects

Page Object Pattern

Page objects encapsulate test IDs and UI interactions in reusable classes. When a test ID changes, update the page object once - all tests using it automatically work.

Location: packages/cypress/cypress/pages/ | Reference: docs/testing.md

class MyFeaturePage {
  visit() {
    cy.visitWithLogin('/my-feature');
    this.wait();
  }

  private wait() {
    cy.findByTestId('app-page-title');
    cy.testA11y();
  }

  findCreateButton() {
    return cy.findByTestId('create-button');
  }
}

export const myFeaturePage = new MyFeaturePage();

Usage in tests:

myFeaturePage.visit();
myFeaturePage.findCreateButton().click();

Shared Utility Functions

The Problem: Code Duplication

Writing the same code in every test leads to:

  • ❌ Copy-paste errors
  • ❌ Inconsistent behavior
  • ❌ Hard to maintain

The Solution: Shared Utilities

Create reusable functions once, use them everywhere.

Instead of this in every test:

cy.exec('oc new-project my-project');
cy.exec('oc label namespace my-project opendatahub.io/dashboard=true');
// ... error handling ...

We write this once:

createOpenShiftProject('my-project');

Example - Using oc Commands:

import { createOpenShiftProject, deleteOpenShiftProject } from '../../../utils/oc_commands/project';

// Create a project for testing
createOpenShiftProject('test-project', 'Test Project Display Name');

// Clean up after test
deleteOpenShiftProject('test-project');

Example - Login with Different Users:

User credentials are defined in test-variables.yml (not stored upstream - injected during CI):

HTPASSWD_CLUSTER_ADMIN_USER:
  AUTH_TYPE: htpasswd-auth
  USERNAME: admin-user
  PASSWORD: admin-password

Then imported and used in tests:

import { HTPASSWD_CLUSTER_ADMIN_USER } from '../../../utils/e2eUsers';

cy.visitWithLogin('/projects', HTPASSWD_CLUSTER_ADMIN_USER);

Available utilities:

  • OpenShift Commands (utils/oc_commands/) - Create/delete projects, verify existence, get config
  • Test Configuration (utils/testConfig.ts) - Load test variables, manage settings
  • Test Helpers (utils/) - Retry logic, UUID generation, error handling

Location: packages/cypress/cypress/utils/oc_commands/project.ts

Benefits:

  • ✅ Write once, use everywhere
  • ✅ Fix bugs in one place
  • ✅ Tests are easier to read
  • ✅ Consistent behavior

Test Cases Documentation in Code

The Old Way: External Documentation (e.g. Polarion)

Problems with Polarion:

  • ❌ Documentation gets outdated quickly
  • ❌ Separate from code (two places to maintain)
  • ❌ Manual updates required (easy to forget)
  • ❌ Extra step in release process
  • ❌ Not executable (just documentation)

The New Way: Code as Documentation

Instead of Polarion, we document tests in the code itself!

Key insight: Tests ARE the documentation. They execute and prove what they document.

How We Document Tests

1. Test Tags

it(
  'Create a project',
  {
    tags: [
      '@Smoke', // Critical test
      '@ODS-1875', // Jira ticket
      '@ci-dashboard-set-1', // CI group
    ],
  },
  () => {
    // Test code
  },
);

Benefits:

  • Run specific tests: grepTags="@Smoke"
  • Link to requirements (Jira)
  • Organize for parallel execution

2. Step Documentation

cy.step('Log into the application');
// ... login code ...

cy.step('Create a new project');
// ... create code ...

cy.step('Verify project was created');
// ... verify code ...

Benefits:

  • Clear test logs
  • Easy to understand flow
  • Self‑documenting

Why Not External Test Case Management?

External Tools (e.g., Polarion) Code Documentation
Manual updates, can become outdated Always in sync - tests ARE the docs
Extra release step to update docs No separate step - part of normal workflow
Separate from code, hard to find Integrated with code, easy to discover
Not executable Executable - proves what it documents
Developers don't use it Used by both developers and QE
PR review = code + docs separately PR review includes documentation
Manual sync check before release CI/CD generates reports automatically
Separate release documentation Test reports = release documentation

Best Practices for Writing Tests

  1. Always start with a visit - Never assume page state
  2. Use cy.step() for readability - Document test steps
  3. Wait for network requests - Always wait for API calls
  4. Assert request payloads - Verify what was sent
  5. Scope modal queries - Use page objects to avoid conflicts
  6. Clean up E2E resources - Always delete created resources
  7. Use tags for organization - Categorize tests for filtering
  8. Use fixtures for test data - Load from YAML files

Cursor AI Rules: For detailed guidelines when writing tests with AI assistance, see:

Tags and Test Options

it('should create project', { tags: ['@Smoke', '@Dashboard'], timeout: 30000 }, () => {
  // test code
});

Available Options: tags (array), timeout (milliseconds), standard Mocha options (skip, only)

Tag Purpose
@Smoke P1/2 Critical Priority
@Sanity P3 Normal Priority
@Tier1 P4 Low Priority
@ODS-1234 Test Case ID
@Dashboard / @Workbenches Team / Functional Area
@Bug / @Maintenance Tests requiring attention

Test Data in YAML

Store test data in fixture files at packages/cypress/cypress/fixtures/e2e/:

# testProjectCreation.yaml
projectDisplayName: 'Cypress Test Project'
projectResourceName: 'cypress-test-project'
invalidResourceNames:
  - Test-Project
  - test project
cy.fixture('e2e/dataScienceProjects/testProjectCreation.yaml', 'utf8').then((yamlContent) => {
  testData = yaml.load(yamlContent);
});

Setup/Teardown via CLI

Use oc commands in before/after hooks for faster resource management. Use retryableBefore for setup and wasSetupPerformed() check in cleanup:

import {
  deleteOpenShiftProject,
  verifyOpenShiftProjectExists,
} from '../../../utils/oc_commands/project';
import { retryableBefore, wasSetupPerformed } from '../../../utils/retryableHooks';

retryableBefore(() =>
  verifyOpenShiftProjectExists(projectName).then((exists) => {
    if (exists) deleteOpenShiftProject(projectName);
  }),
);

after(() => {
  if (!wasSetupPerformed()) return; // Skip cleanup if setup failed
  deleteOpenShiftProject(projectName, { wait: false, ignoreNotFound: true });
});

CLI utilities: packages/cypress/cypress/utils/oc_commands/ - project, model serving, pipelines operations.

Test Structure

it(
  'Create and delete a project',
  {
    tags: ['@Smoke', '@ODS-1234', '@Dashboard', '@Workbenches'],
  },
  () => {
    cy.step('Log into the application');
    cy.visitWithLogin('/', HTPASSWD_CLUSTER_ADMIN_USER);
    projectListPage.navigate();

    cy.step('Create a new project');
    projectListPage.findCreateProjectButton().click();
    createProjectModal.findSubmitButton().click();

    cy.step('Verify project was created');
    cy.url().should('include', '/projects/my-project');
  },
);

Mock Testing

For mocked tests, use typed intercept commands:

  • cy.interceptK8s() / cy.interceptOdh() - Mock API calls
  • cy.wsK8s() - Simulate Kubernetes watch events

Reference: See .cursor/rules/cypress-mock.mdc and packages/cypress/cypress/tests/mocked/.

Common Commands

Command Purpose
cy.visitWithLogin(url, user?) Navigate and login
cy.findByTestId(id) Select by test ID
cy.interceptK8s(model, response) Mock K8s API
cy.interceptOdh(route, response) Mock Dashboard API
cy.step(description) Add test step for logs

Parallel Test Execution in CI

Organize tests into groups using tags for parallel execution:

// Group 1
it('test 1', { tags: ['@SmokeSet1'] }, () => { ... });

// Group 2
it('test 2', { tags: ['@SmokeSet2'] }, () => { ... });

Define parallel stages from Test Tags list (e.g., "SmokeSet1,SmokeSet2")

GitHub Actions (Matrix Strategy):

strategy:
  matrix:
    testTags: ['@SmokeSet1', '@SmokeSet2']
steps:
  - run: npm run cypress:run -- --env grepTags="${{ matrix.testTags }}"

Jenkins (Parallel Stages):

def testTags = ['@SmokeSet1','@SmokeSet2']
def parallelStages = testTags.collectEntries { tag ->
    ["${tag}": {
        sh "npm run cypress:run -- --env grepTags='${tag}'"
    }]
}
parallel parallelStages

Benefits:

  • ✅ 3x faster execution (or more with more parallel jobs)
  • ✅ Isolated - each job/stage has its own resources
  • ✅ Reliable - one failure doesn't block others

Cypress Configuration Files

cypress.config.ts

Location: packages/cypress/cypress.config.ts

Configuration:

  • baseUrl: Derived from ODH_DASHBOARD_URL in test-variables.yml
  • viewportWidth/Height: 1920x1080
  • defaultCommandTimeout: 10000ms
  • retries: 2 for E2E (configurable via CY_RETRY), 0 for mocked

Reporters:

  • cypress-mochawesome-reporter: HTML reports with charts
  • mocha-junit-reporter: JUnit XML for CI/CD

Plugins registered in setupNodeEvents:

  • @cypress/grep: Test filtering by tags
  • @cypress/code-coverage: Coverage collection
  • cypress-high-resolution: Screenshot quality
  • WebSocket support for mocked K8s watch events

Spec patterns:

  • Mocked: cypress/tests/mocked/**/*.cy.ts
  • E2E: cypress/tests/e2e/**/*.cy.ts

e2e.ts (Support File)

The packages/cypress/cypress/support/e2e.ts file is automatically loaded before all test files.

Reference: docs/testing.md

What e2e.ts Does

  • Global Setup: Configures plugins (@cypress/grep, @cypress/code-coverage, cypress-mochawesome-reporter, cypress-plugin-steps), custom commands, and global test behavior
  • Test Tagging: Extends it() to support tags: it('test name', { tags: ['@Smoke'] }, () => { ... })
  • Test Filtering: Filters tests based on grepTags and skipTags, skips entire suites when all tests are filtered
  • Global Hooks:
    • before(): Sets up intercepts and timeouts
    • beforeEach(): Applies filtering, sets up mocks (CY_MOCK=1), configures module federation, logs test info
    • afterEach(): Handles skipped suites, tracks execution
    • after(): Runs soft assertions
  • Error Handling: Ignores ChunkLoadError and webpack-dev-server fallback errors
  • Command Logging: Logs cy.step(), cy.exec(), and cy.log() to terminal

Skipped Tests in JUnit Results

Tests filtered by tags use Mocha's this.skip() in beforeEach(), marking them as pending (skipped) rather than failed. The mocha-junit-reporter converts these to JUnit XML with skipped="true", allowing CI/CD systems to distinguish:

  • ✅ Passed: Test executed and passed
  • ❌ Failed: Test executed and failed
  • ⏸️ Skipped: Test was intentionally skipped (not executed)

Benefits: Skipped tests don't count as failures, remain visible in reports, and CI/CD can distinguish skipped vs failed tests.

Cypress Environment Variables

Common test configuration via environment variables and flags (full list in docs/testing.md).

Variable Purpose Default Example
CY_TEST_CONFIG Path to test config - export CY_TEST_CONFIG='./test-variables.yml'
CY_MOCK Enable mocked mode 0 CY_MOCK=1 npm run cypress:run
grepTags Filter by tags - --env grepTags="@Smoke"
skipTags Skip by tags - --env skipTags="@Bug"
CY_COVERAGE Enable coverage false CY_COVERAGE=true npm run cypress:run:mock
CY_RETRY Test retries 2 (E2E), 0 (mock) CY_RETRY=0 (disable retries)
CY_RESULTS_DIR Results directory results CY_RESULTS_DIR=my-results

Code Coverage

Uses @cypress/code-coverage plugin to track statements, branches, functions, and lines executed during tests.

# Enable coverage for mocked tests
CY_COVERAGE=true npm run cypress:run:mock

# Reports generated in packages/cypress/coverage/

Integration:

  • Merges with Jest unit test coverage for combined reports
  • Can be uploaded to Codecov/Coveralls
  • HTML reports at coverage/lcov-report/index.html

Configuration: packages/cypress/cypress.config.ts

Running Cypress Tests

Setup

git clone https://github.com/opendatahub-io/odh-dashboard.git
cd odh-dashboard
npm install

For E2E tests, create test-variables.yml from packages/cypress/test-variables.yml.example:

export CY_TEST_CONFIG=/path/to/test-variables.yml
oc login -u <username> -p <password> -s https://api.<cluster>.<domain>:6443

Configure ODH_DASHBOARD_URL in your test-variables.yml:

# Remote cluster (no dev server needed)
ODH_DASHBOARD_URL: https://rh-ai.apps.my-cluster.test.redhat.com

# Or localhost (requires dev server)
ODH_DASHBOARD_URL: http://localhost:4010

Running Mocked Tests

Run from packages/cypress directory. Requires production build on port 9001.

cd packages/cypress

# Quick: build and run in one command
npm run test:cypress-ci -- --spec '**/storageClasses.cy.ts'

# Or interactive mode
npm run open:mock

# Or headless with filters
npm run run:mock -- --spec 'cypress/tests/mocked/storageClasses/storageClasses.cy.ts' --env grepTags="@Smoke"

Tip: If switching from E2E tests, unset the config first: unset CY_TEST_CONFIG

Running E2E Tests

Run from frontend directory. Requires oc login and CY_TEST_CONFIG set.

cd frontend

# Interactive mode
npm run cypress:open

# Headless with tag filters
npm run cypress:run -- --env grepTags="@Smoke",skipTags="@Bug" --browser chrome

# Run specific spec
npm run cypress:run -- --spec "cypress/tests/e2e/testProjectCreation.cy.ts" --browser chrome

If using localhost, start the dev server first:

npm run start:dev:ext  # Proxies to your logged-in cluster

Results

  • Screenshots: packages/cypress/results/{mocked|e2e}/screenshots/
  • Videos: packages/cypress/results/{mocked|e2e}/videos/
  • HTML Reports: packages/cypress/results/{mocked|e2e}/index.html

Note: If mocked tests fail with "server not running on port 9001", use npm run test:cypress-ci which builds and serves automatically.

Cypress could not verify that this server is running:
  > http://localhost:9001

This means you need to build and serve the frontend on port 9001 (see "Start the Frontend Dev Server" above), or use npm run test:cypress-ci which handles this automatically.

Debugging Tests

Debug-Specific Tips

Useful environment variables for debugging:

export CY_WATCH=false   # Disable auto-rerun on file changes
export CY_RETRY=0       # No retries (see failures immediately)

Run with browser visible (headed mode):

npm run cypress:run -- --headed --spec '**/testName.cy.ts'

Two-Terminal Workflow

For debugging local frontend changes against a cluster:

Terminal 1 - Dev server:

lsof -ti :9001,4010 | xargs -rt kill -9  # Kill existing processes
npm --prefix frontend run start:dev:ext

Terminal 2 - Cypress:

npm --prefix frontend run cypress:open

One-Liner (Single Terminal)

lsof -ti :9001,4010 | xargs -rt kill -9 ; (npm --prefix frontend run start:dev:ext &) && npm --prefix frontend run cypress:open

For more details, see testing.md.

Hands-On Example

A walkthrough: finding a test ID in a component, tracing it to a page object, and verifying it in Cypress interactive mode.

Example: "Create project" button

Follow along by opening these files in your IDE:

Step 1: Find the test ID in the component

Open frontend/src/pages/projects/screens/projects/NewProjectButton.tsx and locate the data-testid:

<Button data-testid="create-project" variant="primary" onClick={() => setOpen(true)}>
  Create project
</Button>

Step 2: See how it's used in the page object

Open packages/cypress/cypress/pages/projects.ts and find the method that wraps this test ID:

class ProjectListPage {
  findCreateProjectButton() {
    return cy.findByTestId('create-project');
  }
}

Step 3: Run E2E tests against local dev server

To see local code changes in E2E tests, run the dev server and point tests to it:

# Terminal 1: Start local dev server (hot reload on port 4010)
cd frontend
npm run start:dev
# test-variables.yml - point to local dev server
ODH_DASHBOARD_URL: http://localhost:4010
# Terminal 2: Open Cypress UI
cd frontend
export CY_TEST_CONFIG=/path/to/test-variables.yml
npm run cypress:open
  1. Select E2E Testing → Chrome
  2. Open cypress/tests/e2e/dataScienceProjects/testProjectCreation.cy.ts
  3. Run the test - it should pass
  4. Click on findCreateProjectButton command to see the DOM snapshot
  5. Press F12 → inspect the data-testid="create-project" attribute

Step 4: Change and update the test ID

  1. In frontend/src/pages/projects/screens/projects/NewProjectButton.tsx, change the test ID:
// Change this:
<Button data-testid="create-project" ...>
// To this:
<Button data-testid="create-new-project" ...>

  1. Dev server hot reloads → rerun test in Cypress → test fails (can't find element)

  2. In packages/cypress/cypress/pages/projects.ts, update the page object:

// Change this:
return cy.findByTestId('create-project');
// To this:
return cy.findByTestId('create-new-project');

  1. Rerun test in Cypress → test passes

  2. Revert both changes back to create-project

This demonstrates: change the page object once → all tests using it work!

Contribution Guidelines

Branching Strategy

git checkout -b feature/my-new-feature

Testing Your Changes

  1. Test locally against a PSI Cluster and/or the ODH-Nightly cluster
  2. Execute headlessly - This imitates the execution in Jenkins
  3. Use Jenkins Job - Dashboard tests (Fill the values needed for verification)

Submitting Changes

  1. Run linter:

    npm run test:lint
# or auto-fix
npm run test:fix

  2. Commit and push:

    git add .
git commit -m "Add my new test"
git push origin feature/my-new-test

  3. Create Pull Request - Open a PR against the main branch with a clear description

For general contribution guidelines, see CONTRIBUTING.md.

Resources