Testing of PIE Components
The PIE components are tested using a combination of component, accessibility, and visual tests to ensure they meet the required functionality, accessibility standards, and visual consistency across different screen sizes. This documentation provides an overview of the testing strategies and technologies used for the PIE Chip component as an example.
Storybook: A sandbox environment for developing, documenting and testing UI components in isolation. Within the context of testing, we use this tool for running our component, visual and accessibility tests on isolated versions of our components.
Note
Any stories used for testing purposes should live in the pie-storybook/stories/testing directory so that they're not shown on the production Storybook deployment.
Playwright: A Node.js library to automate Chromium, Firefox, and WebKit with a single API. It is used for writing and running tests for the PIE components.
Percy: A visual testing tool that captures screenshots and compares them to baseline images to detect visual changes.
Axe Core: Used to automate the checking for accessibility issues, ensuring that components comply with Web Content Accessibility Guidelines (WCAG). In our testing setup, Axe Core is integrated with Playwright to perform accessibility checks on the PIE components.
Component tests are designed to verify the functionality and behavior of individual components in isolation. They ensure that components render correctly, handle user interactions, and emit events as expected.
Below is a basic test example from the component spec that demonstrates how to test the rendering of the PieChip component and perform a simple assertion to check its visibility.
import { test, expect } from '@playwright/test';
import { BasePage } from '@justeattakeaway/pie-webc-testing/src/helpers/page-object/base-page.ts';
import type { ChipProps } from '../../src/index.ts';
import { chip } from '../helpers/page-object/selectors.ts';
const componentSelector = '[data-test-id="pie-chip"]';
test.describe('PieChip - Component tests', () => {
test('should render successfully', async ({ page }) => {
// Arrange
const pieChipPage = new BasePage(page, 'chip--default');
await pieChipPage.load();
// Act
const chipComponent = page.locator(chip.selectors.container.dataTestId);
// Assert
await expect(chipComponent).toBeVisible();
});
});System tests aim to test components / functionality in a fully integrated system. Areas of coverage include page routing, as well as how components interact with each other, and third-party dependencies such as APIs.
This level of testing is important as it allows us to test full pages worth of functionality if required as well as test the functionality of components and their interactions between each other.
We use system tests in areas such as Pie Docs for testing pages within our documentation site but we also use system testing within PIE Aperture for testing component integration such as form pages that contain Pie components.
Below is an example of a system test for the 404 page on the Pie Docs site.
import { test, expect } from '@playwright/test';
test.describe('PIE - 404 Page - @desktop', () => {
test.beforeEach(async ({ page, baseURL }) => {
await page.goto(baseURL);
});
test('Should go to the homepage when clicking "Visit homepage" link', async ({ page, baseURL }) => {
// Arrange
await page.goto('404.html');
const visitHomepageLink = page.getByTestId('404-visit-homepage');
// Act
await Promise.all([
page.waitForResponse((resp) => resp.status() === 200),
visitHomepageLink.click()
]);
// Assert
const actualUrl = new URL(page.url()).href;
const expectedUrl = new URL(baseURL).href;
await expect(actualUrl).toBe(expectedUrl);
});
});Accessibility tests ensure that components comply with Web Content Accessibility Guidelines (WCAG) to provide an inclusive experience for all users. These tests check for issues such as missing ARIA attributes, incorrect semantic markup, and other potential accessibility barriers.
Below is an example of an accessibility test for the Pie Chip component. This test uses the Axe Core accessibility testing tool to analyze the component for any WCAG compliance violations.
import { test, expect } from '@justeattakeaway/pie-webc-testing/src/playwright/playwright-fixtures.ts';
import { BasePage } from '@justeattakeaway/pie-webc-testing/src/helpers/page-object/base-page.ts';
test.describe('PieChip - Accessibility tests', () => {
test('a11y - should test the PieChip component WCAG compliance', async ({ makeAxeBuilder, page }) => {
const pieChipPage = new BasePage(page, 'chip--default');
await pieChipPage.load();
const results = await makeAxeBuilder().analyze();
expect(results.violations).toEqual([]);
});
});Note
Our accessibility tests leverage a custom fixture built on top of the @playwright/test library. This fixture, defined in the playwright-fixtures.ts file, extends the base test functionality by providing a makeAxeBuilder function for use in accessibility test specs as well a specifying the WCAG rules we test against.
Visual tests ensure PIE components maintain visual consistency across different screen sizes and variations. These tests help detect unintended visual changes that could affect the user interface's appearance and usability.
We use Percy in conjunction with Playwright to perform visual testing. Percy captures screenshots of the components and compares them to baseline images to identify any visual discrepancies. This process helps ensure that any changes to the codebase do not inadvertently alter the visual presentation of the components. Below is an example of a visual test for the PieChip component. This test captures screenshots of the component for different prop variations and compares them against baseline images:
import { test } from '@playwright/test';
import percySnapshot from '@percy/playwright';
import { percyWidths } from '@justeattakeaway/pie-webc-testing/src/percy/breakpoints.ts';
import { BasePage } from '@justeattakeaway/pie-webc-testing/src/helpers/page-object/base-page.ts';
import { variants } from '../../src/defs.ts';
variants.forEach((variant) => test(`should render all prop variations for Variant: ${variant}`, async ({ page }) => {
const basePage = new BasePage(page, `chip--${variant}-prop-variations`);
basePage.load();
await page.waitForTimeout(5000);
await percySnapshot(page, `PIE Chip - Variant: ${variant}`, percyWidths);
}));All of our tests run using a test version of Storybook for rendering components.
Before running any test, you want to run the below command in a new terminal instance to start the Storybook test web server instance.
yarn dev:testing
Once you have done this, you can then run tests using the related commands.
You can use the built in --filter command to choose which component you want to run.
For example, --filter=pie-chip.
For component, system and accessibility tests.
yarn test:browsers --filter=pie-your-component
For visual tests.
yarn test:visual --filter=pie-your-component
We use the createVariantStory function to generate stories that render all combinations of component properties. This function is particularly useful for visual testing as it systematically displays each variation of a component, ensuring comprehensive visual coverage.
For example, in the pie-chip.test.stories.ts file, the createVariantStory function is used to create stories for different variants of the Pie Chip component:
// Define the prop options for the matrix
const sharedPropOptions = {
disabled: [true, false],
isSelected: [true, false],
isLoading: [true, false],
isDismissible: [true, false],
showIcon: [true, false],
slot: ['Hello World'],
};
const defaultPropOptions = {
...sharedPropOptions,
variant: ['default'],
};
const ghostPropOptions = {
...sharedPropOptions,
variant: ['ghost'],
};
const outlinePropOptions = {
...sharedPropOptions,
variant: ['outline'],
};
export const DefaultPropVariations = createVariantStory<Omit<ChipProps, 'aria'> >(Template, defaultPropOptions);
export const GhostPropVariations = createVariantStory<Omit<ChipProps, 'aria'>>(Template, ghostPropOptions);
export const OutlinePropVariations = createVariantStory<Omit<ChipProps, 'aria'>>(Template, outlinePropOptions);For some component variations, you may need to specify a background so the component is rendered in a way that matches its intended design.
To do this, you can pass the bgColor property.
You can see the effect this has in the Storybook example here
export const GhostVariations = createVariantStory<IconButtonProps>(Template, ghostVariantPropsMatrix, { bgColor: 'background-subtle' });
export const GhostSecondaryVariations = createVariantStory<IconButtonProps>(Template, ghostSecondaryVariantPropsMatrix, { bgColor: 'background-subtle' });
export const InverseVariations = createVariantStory<IconButtonProps>(Template, inverseVariantPropsMatrix, { bgColor: 'dark (container-dark)' });
export const GhostInverseVariations = createVariantStory<IconButtonProps>(Template, ghostInverseVariantPropsMatrix, { bgColor: 'dark (container-dark)' });For some components, you may need to verify the component renders correctly for different writing directions. To do this, you can leverage the Storybook Writing Direction control within your visual test.
import { test } from '@playwright/test';
import percySnapshot from '@percy/playwright';
import { BasePage } from '@justeattakeaway/pie-webc-testing/src/helpers/page-object/base-page.ts';
import { radio } from '../helpers/page-object/selectors.ts';
const readingDirections = ['ltr', 'rtl'];
readingDirections.forEach((dir) => {
test(`should render all prop variations for the direction: ${dir}`, async ({ page }) => {
// Arrange
const radioVariations = new BasePage(page, 'radio--variations');
await radioVariations.load({}, { writingDirection: dir });
await page.waitForSelector(radio.selectors.container.dataTestId);
// Assert
await percySnapshot(page, `PIE Radio Variations: ${dir}`, { widths: [1280] });
});
});You may notice that for more complex components with a large number of prop variations, stories that leverage createVariantStory get quite difficult to manage. You can leverage the multiColumn property to control how these stories are rendered in Storybook.
export const NeutralVariations = createVariantStory<ToastProps>(Template, neutralPropsMatrix, { ...variantPropDisplayOptions, multiColumn: true });
export const InfoVariations = createVariantStory<ToastProps>(Template, infoPropsMatrix, { ...variantPropDisplayOptions, multiColumn: true });