Add accessibility test library - #6118
Conversation
There was a problem hiding this comment.
Pull request overview
Publishes a new VA Design System Accessibility Test Library (YAML test definitions + component result files) and adds Liquid includes, styles, and contributor tooling (Copilot prompts / Claude skills) to render and maintain accessibility test results on the documentation site.
Changes:
- Adds WCAG-principle-based test library YAML files plus configuration and documentation.
- Adds Liquid includes/pages to render the public test library catalog and per-component test results.
- Adds AI workflow assets (GitHub prompts and Claude skills) intended to generate, update, and audit component test result files.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 21 comments.
Show a summary per file
| File | Description |
|---|---|
| src/assets/stylesheets/application.scss | Includes new accessibility test styling partial in the global stylesheet build. |
| src/assets/stylesheets/_components/_accessibility-tests.scss | Adds styling for test library and component test-result displays. |
| src/_includes/accessibility-test-results.html | New renderer for per-component accessibility test summaries and detailed environment results. |
| src/_includes/accessibility-test-lookup.html | Helper include to resolve test definitions (including nested subtests) by ID. |
| src/_includes/accessibility-test-library-loader.html | Loads and concatenates WCAG principle test definitions for use by other includes. |
| src/_includes/accessibility-test-library-list.html | Renders the full public test library grouped by WCAG criterion. |
| src/_includes/accessibility-test-env-lookup.html | Resolves environment IDs to display names from the shared config. |
| src/_data/accessibility-tests/README.md | Top-level documentation for the accessibility test data layout and contributor workflows. |
| src/_data/accessibility-tests/components/va-statement-of-truth.yml | Adds initial (placeholder) test coverage mapping for va-statement-of-truth. |
| src/_data/accessibility-tests/components/va-details.yml | Adds populated test results for va-details across multiple environments. |
| src/_data/accessibility-tests/components/README.md | Documentation for creating/maintaining component test result YAML files. |
| src/_data/accessibility-tests/test-library/README.md | Detailed documentation for writing and maintaining test library entries. |
| src/_data/accessibility-tests/test-library/1-perceivable.yml | Adds WCAG Principle 1 test definitions (Perceivable). |
| src/_data/accessibility-tests/test-library/2-operable.yml | Adds WCAG Principle 2 test definitions (Operable). |
| src/_data/accessibility-tests/test-library/3-understandable.yml | Adds WCAG Principle 3 test definitions (Understandable). |
| src/_data/accessibility-tests/test-library/4-robust.yml | Adds WCAG Principle 4 test definitions (Robust). |
| src/_data/accessibility-tests/test-library/_config.yml | Adds environment definitions, categories, and library metadata. |
| src/_accessibility/test-library.md | Adds the public-facing Accessibility Test Library page that renders the catalog. |
| src/_accessibility/accessibility-testing-for-design-system-components.md | Updates testing methodology guidance and links to the new test library page. |
| .github/prompts/accessibility-tests-generate-component-tests.prompt.md | Adds Copilot prompt to generate new component test YAML files. |
| .github/prompts/accessibility-tests-generate-metadata.prompt.md | Adds Copilot prompt to add/update test run metadata in component YAML files. |
| .github/prompts/accessibility-tests-audit.prompt.md | Adds Copilot prompt to audit component YAML files for compliance/coverage. |
| .github/prompts/accessibility-tests-add-to-component-page.prompt.md | Adds Copilot prompt to insert the “Accessibility tests” section on component docs pages. |
| .claude/skills/accessibility-tests/writing-entries/SKILL.md | Adds Claude skill pointer for consistent writing of test library entries. |
| .claude/skills/accessibility-tests/test-selection/SKILL.md | Adds Claude skill for selecting applicable tests for a component based on features. |
amyleadem
changed the title
|
This PR is ready for review. I'd like to get review from:
|
There was a problem hiding this comment.
Note: I did not include the component page update to include these test results on the va-details page, but that is only because I wasn't sure if it was ready and if we wanted to break it out into a follow-on PR. @jeana-adhoc let me know if you have a preference on releasing this test library with the va-details page update.
There was a problem hiding this comment.
to have a cleaner version number, I could run through the tests again on the latest release instead of an unreleased version.
There was a problem hiding this comment.
But it was legit for real. And should be releasable
There was a problem hiding this comment.
Is this just a blank test file? I don't see it loading on the statement-of-truth component page. Should this be in this PR?
| it pronounces content in the correct language. | ||
| automation_coverage: full | ||
| automation_notes: >- | ||
| Custom test recommended. Applies to page-level tests, no component-level test needed. |
There was a problem hiding this comment.
Some other time can you help me understand this yml formatting?
What are the |'s and >- doing here?
| ### Environments tested | ||
|
|
||
| Each test result shows the environments and component versions tested. We track results for each environment because accessibility behavior can vary across browsers and assistive technologies. For the complete list of environments we test in, see [How we audit components](/accessibility/accessibility-testing-for-design-system-components#how-we-audit-components). | ||
|
|
There was a problem hiding this comment.
Do you think it'd be helpful to show a generic table here of the test results to give a visual of the various parts?
|
@amyleadem - this is really good. One question I have -- and this is probably for a larger conversation. Do we need to mention/talk about, on the doc site, any of the copilot skills that are available? It seems right now that the only way someone would know about them is by being in the codebase. This might be ok, as I think the prompts, for now, are only for us DST. |
jeana-adhoc
left a comment
jeana-adhoc
left a comment
There was a problem hiding this comment.
I'm so excited about this.
LGTM!
Summary
Added the accessibility test library along with related documentation and tooling. The yaml-based accessibility test library maps granular, plain language manual accessibility tests to WCAG 2.2 AA. This reusable test data can be used to quickly build manual test suites for components, and opens the door to additional tooling.
This PR also adds an experimental prompt (
/accessibility-tests-generate-component-tests) that can help accessibility specialists identify which tests should be included for a given component. This prompt is in beta while we continue to refine our expectations and standards across component types.Stable items:
Beta:
Note
This work originated in a separate prototype PR. This PR includes the files needed for the MVP release, but there are some component files in there that might be helpful reference in the future.
Related Issue
If this PR resolves an open issue, please add the issue number here.
Closes #5819
Preview Environment Links
Open Preview Environment