From dd327b19a17305239bf2d8d29af83539dc991c53 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 13:56:03 -0700 Subject: [PATCH 01/13] Move MVP files from PR #5703 --- .../test-selection/SKILL.md | 179 +++++ .../writing-entries/SKILL.md | 36 + ...lity-tests-add-to-component-page.prompt.md | 67 ++ .../accessibility-tests-audit.prompt.md | 167 +++++ ...y-tests-generate-component-tests.prompt.md | 146 ++++ ...sibility-tests-generate-metadata.prompt.md | 110 +++ ...ty-testing-for-design-system-components.md | 240 +++---- src/_accessibility/test-library.md | 49 ++ src/_data/accessibility-tests/README.md | 92 +++ .../accessibility-tests/components/README.md | 410 +++++++++++ .../components/va-details.yml | 338 +++++++++ .../components/va-statement-of-truth.yml | 290 ++++++++ .../test-library/1-perceivable.yml | 649 ++++++++++++++++++ .../test-library/2-operable.yml | 485 +++++++++++++ .../test-library/3-understandable.yml | 322 +++++++++ .../test-library/4-robust.yml | 135 ++++ .../test-library/README.md | 418 +++++++++++ .../test-library/_config.yml | 85 +++ .../accessibility-test-env-lookup.html | 23 + .../accessibility-test-library-list.html | 84 +++ .../accessibility-test-library-loader.html | 15 + src/_includes/accessibility-test-lookup.html | 24 + src/_includes/accessibility-test-results.html | 305 ++++++++ .../_components/_accessibility-tests.scss | 100 +++ src/assets/stylesheets/application.scss | 1 + 25 files changed, 4643 insertions(+), 127 deletions(-) create mode 100644 .claude/skills/accessibility-tests/test-selection/SKILL.md create mode 100644 .claude/skills/accessibility-tests/writing-entries/SKILL.md create mode 100644 .github/prompts/accessibility-tests-add-to-component-page.prompt.md create mode 100644 .github/prompts/accessibility-tests-audit.prompt.md create mode 100644 .github/prompts/accessibility-tests-generate-component-tests.prompt.md create mode 100644 .github/prompts/accessibility-tests-generate-metadata.prompt.md create mode 100644 src/_accessibility/test-library.md create mode 100644 src/_data/accessibility-tests/README.md create mode 100644 src/_data/accessibility-tests/components/README.md create mode 100644 src/_data/accessibility-tests/components/va-details.yml create mode 100644 src/_data/accessibility-tests/components/va-statement-of-truth.yml create mode 100644 src/_data/accessibility-tests/test-library/1-perceivable.yml create mode 100644 src/_data/accessibility-tests/test-library/2-operable.yml create mode 100644 src/_data/accessibility-tests/test-library/3-understandable.yml create mode 100644 src/_data/accessibility-tests/test-library/4-robust.yml create mode 100644 src/_data/accessibility-tests/test-library/README.md create mode 100644 src/_data/accessibility-tests/test-library/_config.yml create mode 100644 src/_includes/accessibility-test-env-lookup.html create mode 100644 src/_includes/accessibility-test-library-list.html create mode 100644 src/_includes/accessibility-test-library-loader.html create mode 100644 src/_includes/accessibility-test-lookup.html create mode 100644 src/_includes/accessibility-test-results.html create mode 100644 src/assets/stylesheets/_components/_accessibility-tests.scss diff --git a/.claude/skills/accessibility-tests/test-selection/SKILL.md b/.claude/skills/accessibility-tests/test-selection/SKILL.md new file mode 100644 index 000000000..b97b78064 --- /dev/null +++ b/.claude/skills/accessibility-tests/test-selection/SKILL.md @@ -0,0 +1,179 @@ +--- +name: accessibility-test-selection +description: Determines which accessibility tests from the test library apply to a VA Design System component based on its features and implementation. +--- + +# Accessibility Test Selection + +This skill helps identify which accessibility tests from the VA Design System test library apply to a specific component. + +## Primary Documentation Source + +All test selection rules, subtest priority guidelines, and component-type checklists are documented in: + +**`src/_data/accessibility_tests/components/README.md`** + +**Required reading sections:** +- **Test Selection Guidelines** → **Subtest Preference Rule** — Rules for preferring specific subtests over parent tests, how to find subtests in YAML files +- **Choosing tests by component type** — Complete category-by-category test checklists for all component types + +## Additional Documentation + +### Test Library Structure + +Read these to understand test definitions, IDs, and configuration: + +- **`src/_data/accessibility_tests/test-library/README.md`** + - Test ID format and structure + - WCAG mappings + - Test categories and required environments + - How to read test entries + +- **`src/_data/accessibility_tests/test-library/_config.yml`** + - Environment ID definitions (see `environments:` section) + - Category definitions (see `categories:` section) + - Required environments per category (e.g., `screen_reader`, `mobile`) + +- **`src/_data/accessibility_tests/README.md`** + - High-level overview of the test library + - Directory structure + - Links to related documentation + +### Test Definition Files + +Tests are organized by WCAG principle: + +- `src/_data/accessibility_tests/test-library/1-perceivable.yml` +- `src/_data/accessibility_tests/test-library/2-operable.yml` +- `src/_data/accessibility_tests/test-library/3-understandable.yml` +- `src/_data/accessibility_tests/test-library/4-robust.yml` + +**To find subtests:** Open these YAML files and look for `subtests:` arrays under each parent test entry. + +### Example Component Test Files + +Reference these for patterns: +- `src/_data/accessibility_tests/components/va-link.yml` — Simple interactive component +- `src/_data/accessibility_tests/components/va-text-input.yml` — Form input +- `src/_data/accessibility_tests/components/va-accordion.yml` — Dynamic content + +## Agent workflow + +When invoked to select tests for a component: + +1. **Read the components README** at `src/_data/accessibility_tests/components/README.md` + +2. **Follow the "Test Selection Guidelines" section**, which includes: + - How to analyze your component + - Subtest preference rule + - Child component inheritance + - Determining required test environments + +3. **Apply the "Choosing tests by component type" guidelines** to select applicable tests based on component features + +4. **Return the complete list** of test IDs: + - Sorted numerically by test ID + - No duplicates + - Specific subtests used instead of parent tests where applicable + - Include note about required environments for tests with `screen_reader` or `mobile` categories + +## Related Prompts + +- **Generate component test files**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` + - **Depends on this skill** — Calls this skill to select tests for a new component test file + - Creates complete YAML test files with pre-populated metadata + +- **Audit component test files**: `.github/prompts/accessibility-tests-audit.prompt.md` + - **Depends on this skill** — Uses test selection guidelines to verify test coverage completeness + - Reviews existing component test files for missing or incorrect tests + +## File Paths Quick Reference + +### Component Test Files +- **Directory**: `src/_data/accessibility_tests/components/` +- **Pattern**: `va-[component-name].yml` +- **Examples**: `va-button.yml`, `va-text-input.yml`, `va-accordion.yml` + +### Test Library Files +- **Directory**: `src/_data/accessibility_tests/test-library/` +- **Test definitions**: + - `1-perceivable.yml` (WEB-1xx tests) + - `2-operable.yml` (WEB-2xx tests) + - `3-understandable.yml` (WEB-3xx tests) + - `4-robust.yml` (WEB-4xx tests) +- **Configuration**: `_config.yml` (environments, categories, required environments) +- **README**: `README.md` (test structure, writing guidelines, WCAG mappings) + +### Component Source Code +- **Repository**: `https://github.com/department-of-veterans-affairs/component-library` +- **Path**: `packages/web-components/src/components/` +- **Pattern**: `va-[component-name]/va-[component-name].tsx` + +## Implementation Notes + +### When to Use This Skill + +- Creating a new component test file +- Reviewing an existing component test file for completeness +- Answering "what tests should apply to this component?" +- Auditing test coverage across components + +### When NOT to Use This Skill + +- **Test result recording** — Use components README "Recording Test Results" section instead +- **Writing test library entries** — Use the writing-entries skill instead +- **Understanding WCAG mappings** — Read test-library README directly +- **Interpreting test results** — This skill selects tests, doesn't validate or interpret results + +### Assumptions and Edge Cases + +- **Component-specific context required**: Cannot select tests without knowing component features +- **False positives preferred**: When uncertain, include the test (missing coverage is worse than extra tests) +- **Child component inheritance**: Parent components should include all applicable tests from their children +- **Implementation-dependent tests**: Some tests (e.g., `WEB-244` Link Purpose) may need to be marked "conditional" if they can only be tested in implementation context + +### Output Format + +When returning test selections, provide: + +1. **Complete list of test IDs** (sorted numerically) +2. **Category groupings** (optional, for readability) showing which category each test came from +3. **Required environments note** for tests with `screen_reader` or `mobile` categories +4. **Rationale** (brief) for why each test applies to the component + +Example output: +``` +Test IDs for va-button: + +All components: +- WEB-134 (Orientation) +- WEB-1410 (Reflow) + +Components with text: +- WEB-143 (Contrast - Minimum) +- WEB-144 (Resize Text) +- WEB-1412 (Text Spacing) + +All interactive elements: +- VADS-211-002 (Keyboard - form inputs) [Note: button is technically a form control] +- WEB-1411 (Non-text Contrast) [has subtests - check if specific ones apply] +- WEB-212 (No Keyboard Trap) +- WEB-243 (Focus Order) +- WEB-247 (Focus Visible) +- WEB-2411 (Focus Not Obscured) +- VADS-253-001 (Label in Name - screen reader) +- VADS-258-001 (Target Size - mobile) [required environments: android-chrome, ios-safari] +- WEB-321 (On Focus) +- WEB-412 (Name, Role, Value) [has subtests - check if specific ones apply] + +Buttons: +- WEB-412-002 (State announcements) — if button has state +- WEB-412-003 (Proper element types) +- VADS-412-002 (Button actions) + +Total: [number] tests +``` + +## Questions? + +Contact the Design System team in [#platform-design-system](https://dsva.slack.com/archives/C01K37HRUAH) Slack channel. diff --git a/.claude/skills/accessibility-tests/writing-entries/SKILL.md b/.claude/skills/accessibility-tests/writing-entries/SKILL.md new file mode 100644 index 000000000..c6c40463d --- /dev/null +++ b/.claude/skills/accessibility-tests/writing-entries/SKILL.md @@ -0,0 +1,36 @@ +--- +name: writing-test-library-entries +description: Guidelines for writing accessibility test library entries including tests and subtests with proper descriptions and automation notes +--- + +# Writing Test Library Entries + +**Use this skill when:** Creating new test entries, editing existing test descriptions, or reviewing test library content for consistency. + +**Test library files:** +- `src/_data/accessibility_tests/test-library/1-perceivable.yml` +- `src/_data/accessibility_tests/test-library/2-operable.yml` +- `src/_data/accessibility_tests/test-library/3-understandable.yml` +- `src/_data/accessibility_tests/test-library/4-robust.yml` +- `src/_data/accessibility_tests/test-library/_config.yml` + +## Complete Guidelines + +All guidelines for writing and editing test library entries live in: + +**`src/_data/accessibility_tests/test-library/README.md`** + +Read the following sections before creating or editing entries: + +- **Test Structure** — Required and optional fields for each test entry +- **Writing Test and Subtest Entries** — Detailed guidelines for `description_short`, `description_full`, and `automation_notes` with good/bad examples + - Subsections: description_short, description_full, automation_notes, Plain Language Principles, Finding Good Examples, Review Checklist +- **Adding New Tests** — Step-by-step workflow for adding new test entries + +## Key Reminders for AI Agents + +1. **description_short**: Concise, plain language, present tense, no "When you..." format +2. **description_full**: ALWAYS use "When you [action], you [result]" format +3. **automation_notes**: Required for `full` and `partial` automation coverage; describe what can be automated and what requires manual testing +4. **Validate YAML**: Ensure proper YAML syntax before saving changes +5. **Cross-reference examples**: Check existing entries in the YAML files for patterns diff --git a/.github/prompts/accessibility-tests-add-to-component-page.prompt.md b/.github/prompts/accessibility-tests-add-to-component-page.prompt.md new file mode 100644 index 000000000..e42533f84 --- /dev/null +++ b/.github/prompts/accessibility-tests-add-to-component-page.prompt.md @@ -0,0 +1,67 @@ +# generate-accessibility-tests-section + +Add the accessibility tests section to a component's documentation page. This displays test results from the component's test file on the public documentation site. + +## Usage + +### 1. Collect component information + +Ask: "Which component would you like to add accessibility tests to? Please provide the component name with the `va-` prefix (e.g., `va-button`, `va-alert`)." + +**Validate:** +- Component name must start with `va-` +- Component name must be lowercase kebab-case +- Verify the component test file exists at `src/_data/accessibility_tests/components/[component-name].yml` +- If test file doesn't exist, inform the user and suggest running `/accessibility-tests-generate-component-tests` first + +### 2. Read the component documentation file + +1. Open `src/_components/[component-name].md` (convert to kebab-case if needed, e.g., `va-text-input` → `text-input.md`) +2. Parse the YAML front matter +3. Identify existing sections in the markdown body + +### 3. Update the YAML front matter + +Add `- anchor: Accessibility tests` to the `anchors:` list in the front matter. + +**Placement:** +- After "Accessibility considerations" (if present) +- Before "Related" or "Component checklist" +- If neither of those exist, add at the end of the anchors list + +### 4. Add the accessibility tests section to the markdown body + +Add the following section to the markdown: + +```markdown +## Accessibility tests + +{% include accessibility-test-results.html component_name="[component-name]" %} +``` + +Replace `[component-name]` with the component's `web-component` value from the YAML front matter (e.g., `va-accordion`, `va-details`). + +**Placement:** +1. **If "Accessibility considerations" section exists**: Add immediately after it +2. **If "Related" section exists but no accessibility considerations**: Add before "Related" +3. **If neither exists**: Add before the component checklist include (e.g., `{% include component-docs/...`) +4. **If none of the above**: Add at the end of the file before any trailing includes + +### 5. Save and confirm + +Save the updated component markdown file and display a summary: + +``` +Added accessibility tests section to [component-name].md: +- Added anchor to front matter +- Added section after [section-name] +- Using component_name: [web-component-value] +``` + +## Resources + +- **Generate Test File Prompt**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` +- **Update Metadata Prompt**: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` +- **Components README**: `src/_data/accessibility_tests/components/README.md#step-5-display-results-on-component-page` (detailed placement guidance) +- **Component test files**: `src/_data/accessibility_tests/components/va-*.yml` +- **Component documentation**: `src/_components/*.md` diff --git a/.github/prompts/accessibility-tests-audit.prompt.md b/.github/prompts/accessibility-tests-audit.prompt.md new file mode 100644 index 000000000..c94cb0b81 --- /dev/null +++ b/.github/prompts/accessibility-tests-audit.prompt.md @@ -0,0 +1,167 @@ +# audit-component-accessibility-tests + +Audit existing accessibility test files to ensure they follow current standards, skill guidelines, and structural requirements. + +## Purpose + +Review component accessibility test files to identify: +- Missing tests based on current skill guidelines +- Incorrect test ordering +- Structural issues (missing environments, metadata) +- Inconsistencies with child component tests + +## Usage + +When this prompt is invoked: + +1. **Ask for scope**: "What would you like to audit?" + - Options: + - "Single file: `va-[component-name]`" (e.g., `va-button`) + - "Multiple files: `va-component1, va-component2, va-component3`" (comma-separated) + - "All files: all" (audits all `src/_data/accessibility_tests/components/va-*.yml` files) + +2. **Wait for user response** and determine files to audit: + - Single file: `src/_data/accessibility_tests/components/va-[component-name].yml` + - Multiple files: Parse comma-separated list, prepend `src/_data/accessibility_tests/components/` and append `.yml` + - All files: Use file search for `src/_data/accessibility_tests/components/va-*.yml` + +3. **For each file, perform these checks**: + + **A. Load the component test file** + - Read the YAML file + - Note the component name, URLs, and test IDs + + **B. Review component implementation** + - Check component source at GitHub path: `https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/[component-name]/` + - Identify features: interactive elements, text, images, forms, dynamic content, etc. + - Identify child VA web components used (e.g., ``, ``) + + **C. Check child component inheritance** + - For each child component found, check if test file exists at `src/_data/accessibility_tests/components/[child-component].yml` + - Compare child component tests with parent component tests + - Flag if parent is missing tests that child components have + + **D. Invoke accessibility-test-selection skill** + - Read `.claude/skills/accessibility-tests/test-selection/SKILL.md` for the agent workflow + - The skill references `src/_data/accessibility_tests/components/README.md` for test selection rules + - Determine which tests should apply based on component features + - Combine with tests from child components + - Compare expected tests with actual tests in file + + **E. Validate test ordering** + - Load test library files in order: `1-perceivable.yml`, `2-operable.yml`, `3-understandable.yml`, `4-robust.yml` + - Check that component tests appear in the same order as they appear in library files + - Flag any out-of-order tests + + **F. Validate structure** + - Confirm each test has at least one `test_results` entry (may be empty or filled) + - Confirm each test_results entry has `environments` array with **at least one environment entry** + - Do NOT check if environment entries are empty (tests may have results filled in) + - Check that tests with `screen_reader` or `mobile` categories have required environments + - Validate YAML structure matches expected format + +4. **Generate output based on file count**: + + **If auditing fewer than 4 files: Output detailed summary** + + For each file: + ```markdown + ## va-[component-name] + + ### Status: ✓ Compliant | ⚠ Needs Review | ✗ Issues Found + + #### Missing Tests + - [test-id]: [reason - e.g., "Component uses text input, should include WEB-131-006"] + - [test-id]: [reason] + + #### Extra Tests (Review Needed) + - [test-id]: [reason - e.g., "Component doesn't have expandable content, WEB-131-010 may not apply"] + + #### Ordering Issues + - [test-id] appears at position X but should be at position Y (following library file order) + + #### Structural Issues + - Test [test-id] missing environments array + - Test [test-id] has screen_reader category but missing required environments + + #### Child Component Inheritance + - Uses va-[child]: Missing [test-id] from child component + + --- + ``` + + **If auditing 4 or more files: Output summary report** + + ```markdown + ## Accessibility Test Audit Summary + + **Audited**: [N] component files + **Date**: [current date] + + ### Issues by Category + + #### Missing Tests ([count] components) + - va-[component-1]: [count] missing tests + - va-[component-2]: [count] missing tests + + #### Ordering Issues ([count] components) + - va-[component-1]: [count] tests out of order + - va-[component-2]: [count] tests out of order + + #### Structural Issues ([count] components) + - va-[component-1]: Missing environments array in [count] tests + - va-[component-2]: Missing required environments for screen_reader tests + + #### Child Component Inheritance ([count] components) + - va-[component-1]: Missing [count] tests from child components + + ### Compliance Summary + - ✓ Compliant: [count] ([percentage]%) + - ⚠ Needs Review: [count] ([percentage]%) + - ✗ Issues Found: [count] ([percentage]%) + + ### Recommendations + 1. [Priority action if applicable] + 2. [Priority action if applicable] + + --- + + **For detailed breakdown of specific issues, audit individual component files.** + ``` + +5. **Offer next steps**: + - "Would you like me to fix issues for any specific component?" + - "Should I generate updated test files for components with issues?" + +## Validation Criteria + +### Required Checks + +- [ ] Component metadata (component, component_url, storybook_url) is present +- [ ] All tests have at least one test_results entry +- [ ] All test_results entries have environments array +- [ ] Each environments array has at least one environment entry (empty or filled) +- [ ] Tests with `screen_reader` category have required screen reader environments +- [ ] Tests with `mobile` category have required mobile environments +- [ ] Tests follow test library file order +- [ ] Tests match skill guidelines for component features +- [ ] Parent components inherit tests from child components where applicable + +### Optional Warnings + +- Extra tests that may not apply based on current skill guidelines (requires manual review) +- Tests that could use more specific subtests instead of parent tests + +## Resources + +- **Test Library**: `src/_data/accessibility_tests/test-library/` +- **Accessibility Test Selection Skill**: `.claude/skills/accessibility-tests/test-selection/SKILL.md` +- **Component Test Files**: `src/_data/accessibility_tests/components/va-*.yml` +- **GitHub Component Library**: `https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/` + +## Notes + +- This audit does NOT check whether test results are complete (pass/fail/not applicable) +- This audit focuses on test selection, ordering, and structure +- Manual review may be needed for "extra tests" — some components may have legitimate edge cases +- Components may intentionally omit certain tests if they don't apply to their specific implementation diff --git a/.github/prompts/accessibility-tests-generate-component-tests.prompt.md b/.github/prompts/accessibility-tests-generate-component-tests.prompt.md new file mode 100644 index 000000000..de60c6b6e --- /dev/null +++ b/.github/prompts/accessibility-tests-generate-component-tests.prompt.md @@ -0,0 +1,146 @@ +# generate-component-accessibility-tests + +Generate an accessibility test file for a VA Design System web component. This file documents which accessibility tests from the test library apply to a specific component. + +## Usage + +### 1. Collect component information + +Ask for the component name: "Which component would you like to create a test file for? Please provide the web component element name with the `va-` prefix (e.g., `va-button`, `va-alert`, `va-text-input`)." + +**Validate the format:** +- Component name must start with `va-` +- Component name must be lowercase kebab-case +- If incorrect format provided (e.g., `button`, `VaButton`, `Button`), prompt for the correct format +- Examples: `va-button`, `va-alert`, `va-text-input`, `va-accordion` + +### 2. Ask about metadata + +Ask: "Do you want to include test metadata now (tester name, version, date)? You can add or update this later using the `/accessibility-tests-generate-metadata` prompt." + +**If yes:** +- Follow the metadata collection process described in step 3 of `generate-metadata-for-component-accessibility-tests.prompt.md`: + - Read [`https://github.com/department-of-veterans-affairs/vets-design-system-documentation/blob/main/json_data_cache/component_library_releases.json`](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/blob/main/json_data_cache/component_library_releases.json) for the current version + - Get today's date in YYYY-MM-DD format + - Ask user for tester name, component version (default: latest), and test date (default: today) +- Use the collected values in step 4 below + +**If no:** +- Skip metadata collection +- Fields will be created with empty values + +### 3. Select applicable tests + +**Invoke the accessibility-test-selection skill** (`.claude/skills/accessibility-tests/test-selection/SKILL.md`) + +The skill will: +- Analyze the component's features (interactive elements, text, images, dynamic content, etc.) +- Check for child VA web components and inherit their tests +- Apply test selection guidelines from the components README +- Determine which tests have required environments +- Return a complete list of applicable test IDs + +### 4. Generate the YAML file + +Create the component test file with this structure: + +**For each test** returned by the test-selection skill: +1. Add a comment above the test with format: `# {wcag_criterion} - {description_short}` + - Get `wcag_criterion` and `description_short` from the test definition in the test library YAML files + - Strip any extra whitespace or newlines from `description_short` + - Example: `# 1.1.1 - Screen readers don't announce decorative images.` +2. Add the test `id` +3. Add a `test_results` entry with: + - `version:` — From user input or empty + - `tester:` — From user input or empty + - `date:` — From user input or empty + - `environments:` array with: + - **If test category is `screen_reader` or `mobile`**: Pre-populate with required environment IDs from `_config.yml`, each with empty `result:` field + - **All other tests**: Single empty list item (`- `) + +**YAML formatting:** +- Use 2-space indentation +- Order tests as they appear in test library files (1-perceivable.yml, 2-operable.yml, 3-understandable.yml, 4-robust.yml) +- Component metadata at top (component, component_url, storybook_url) +- Include descriptive comment above each test entry + +### 5. Validate and save + +**Before saving, verify:** +- Component name matches web component tag (e.g., `va-button`, not `Button`) +- URLs follow correct pattern for component type +- All test IDs exist in test library files +- All tests have `test_results` entry with `version`, `tester`, `date`, `environments` fields +- Environments array properly formatted (required environments for screen_reader/mobile tests, single empty item for others) +- 2-space indentation used throughout + +**Save to:** `src/_data/accessibility_tests/components/[component-name].yml` + +### 6. Offer to add to component page + +Ask: "Would you like to add the accessibility tests section to the component documentation page now? This will update `src/_components/[component].md` to display the test results." + +**If yes:** +- Invoke the `/accessibility-tests-add-to-component-page` prompt +- Pass the component name to continue the workflow + +**If no:** +- Inform the user they can add it later using `/accessibility-tests-add-to-component-page` + +## File Format Examples + +**Without metadata values:** +```yaml +component: va-button +component_url: /components/button/ +storybook_url: /storybook/?path=/docs/components-va-button--docs + +tests: + # 1.3.4 - Content is available in both portrait and landscape orientations. + - id: WEB-134 + test_results: + - version: + tester: + date: + environments: + - +``` + +**With metadata values:** +```yaml +component: va-button +component_url: /components/button/ +storybook_url: /storybook/?path=/docs/components-va-button--docs + +tests: + # 1.3.4 - Content is available in both portrait and landscape orientations. + - id: WEB-134 + test_results: + - version: v56.2.1 + tester: Jane Smith + date: 2026-04-10 + environments: + - +``` + +**Example files for reference:** +- `src/_data/accessibility_tests/components/va-link.yml` +- `src/_data/accessibility_tests/components/va-text-input.yml` +- `src/_data/accessibility_tests/components/va-accordion.yml` + +## Next Steps + +After creating the test file: +- **To add test metadata later**: Use `/accessibility-tests-generate-metadata` if you skipped metadata collection +- **To add to component page**: If you didn't add it in step 6, use `/accessibility-tests-add-to-component-page` when ready to publish +- **To perform testing**: Follow the test procedures in each test's definition and record results in the YAML file + +## Resources + +- **Components README**: `src/_data/accessibility_tests/components/README.md` +- **Test Selection Skill**: `.claude/skills/accessibility-tests/test-selection/SKILL.md` +- **Update Metadata Prompt**: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` +- **Add to Component Page Prompt**: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` +- **Test Library**: `src/_data/accessibility_tests/test-library/` +- **Test Library Config**: `src/_data/accessibility_tests/test-library/_config.yml` +- **Component Library Source**: `https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/` diff --git a/.github/prompts/accessibility-tests-generate-metadata.prompt.md b/.github/prompts/accessibility-tests-generate-metadata.prompt.md new file mode 100644 index 000000000..e8b03d1ff --- /dev/null +++ b/.github/prompts/accessibility-tests-generate-metadata.prompt.md @@ -0,0 +1,110 @@ +# generate-metadata-for-component-accessibility-tests + +Add or update test metadata (tester name, version, date, and environments) in an existing component accessibility test file. + +## Purpose + +This prompt helps you: +- Add metadata to component test files that were created without it +- Update metadata for a new test run on an existing component +- Prepare test files for actual accessibility testing + +## Usage + +When this prompt is invoked: + +### 1. Collect component information + +Ask: "Which component would you like to update? Please provide the component name (e.g., `va-button`, `va-alert`, `va-text-input`)." + +**Validate:** +- Component name must start with `va-` +- Component name must be lowercase kebab-case +- Check that the file exists at `src/_data/accessibility_tests/components/[component-name].yml` +- If the file doesn't exist, inform the user and suggest using `/accessibility-tests-generate-component-tests` first + +### 2. Read existing file + +1. Open `src/_data/accessibility_tests/components/[component-name].yml` +2. Check if tests already have `test_results` entries +3. Parse the component metadata (component_url, storybook_url) +4. Get the list of test IDs + +### 3. Collect test metadata + +1. Read [`https://github.com/department-of-veterans-affairs/vets-design-system-documentation/blob/main/json_data_cache/component_library_releases.json`](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/blob/main/json_data_cache/component_library_releases.json) to get the current version from the first item's `name` field +2. Get today's date in YYYY-MM-DD format +3. Ask the user: + - **Tester name**: "What is your name (as you'd like it to appear in test results)?" + - **Component version**: "What component library version are you testing?" (default: latest version from releases.json) + - **Test date**: "What is the date you performed/will perform testing?" (default: today in YYYY-MM-DD format) + +### 4. Determine action + +**If tests already have `test_results` entries:** +- Ask: "This file already has test metadata. Do you want to add a new test run (adds additional test_results entries) or replace the existing metadata?" + - **Add new test run**: Keep existing `test_results` and append new entries + - **Replace**: Remove existing `test_results` and add new entries + +**If tests do NOT have `test_results` entries:** +- Proceed with adding metadata to all tests + +### 5. Add metadata to tests + +For each test in the file: + +1. Look up the test definition in the test library to check its `category` field +2. Create a `test_results` entry with: + - `version`: From user input + - `tester`: From user input + - `date`: From user input +3. Add an `environments` array: + - **If the test has `screen_reader` or `mobile` category**: + - Read `src/_data/accessibility_tests/test-library/_config.yml` + - Find the category's `required_environments` + - Pre-populate the environments array with those environment IDs + - Each environment gets an empty `result:` field + - **For all other tests**: + - Include a single empty list item: `- ` (dash space, no properties) + +**Formatting:** +- If adding a new test run, append to the existing `test_results` array +- If replacing, replace the entire `test_results` array +- Maintain 2-space indentation +- Keep tests in the same order as they appear in the file + +### 6. Save the updated file + +Write the updated YAML back to `src/_data/accessibility_tests/components/[component-name].yml` + +### 7. Summary + +Display a summary: +``` +Updated va-[component-name].yml: +- Tester: [name] +- Version: [version] +- Date: [date] +- Tests updated: [count] +- [Added new test run / Replaced existing metadata] +``` + +## Next Steps + +After updating metadata: +- **To display results on the component page**: Use `/accessibility-tests-add-to-component-page` when ready to publish +- **To perform testing**: Follow the test procedures in each test's definition and record results in the YAML file + +## Resources + +- **Generate Test File Prompt**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` +- **Add to Component Page Prompt**: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` +- **Components README**: `src/_data/accessibility_tests/components/README.md` (file structure, recording test results) +- **Test Library Config**: `src/_data/accessibility_tests/test-library/_config.yml` (required environments by category) +- **Test Library Files**: `src/_data/accessibility_tests/test-library/*.yml` (test definitions with categories) + +## Notes + +- This prompt only updates metadata structure, not actual test results (pass/fail/conditional) +- Testers will fill in actual `result` values as they perform tests +- The `environments` array is pre-populated based on test categories to guide testers on what environments to test diff --git a/src/_accessibility/accessibility-testing-for-design-system-components.md b/src/_accessibility/accessibility-testing-for-design-system-components.md index d14d1cc8d..decf5aab0 100644 --- a/src/_accessibility/accessibility-testing-for-design-system-components.md +++ b/src/_accessibility/accessibility-testing-for-design-system-components.md @@ -4,200 +4,186 @@ title: Accessibility testing for design system components permalink: /accessibility/accessibility-testing-for-design-system-components anchors: - anchor: Overview - - anchor: VFS team responsibilities - - anchor: Testing principles - - anchor: Testing methodology + - anchor: How components are tested + - anchor: Using components in your product + - anchor: Scope and limitations + - anchor: Test results on component pages + --- ## Overview -Accessibility specialists on the [VA.gov](http://va.gov/) Platform team thoroughly test components before they are added or updated in the Design System. This accessibility testing plan outlines a comprehensive strategy to ensure that both our website and mobile application prioritize accessibility throughout the development lifecycle, incorporating best practices and industry standards to create an inclusive digital environment. - -## Responsibilities for teams building products on VA.gov - -Using design system components alone will not guarantee that your product is accessible. - -### Accessibility test your product +Accessibility specialists on the [VA.gov](http://va.gov/) design system team test components before they are added or updated in the Design System. Our goal is to establish a reliable baseline of accessible behavior for each component, including expected interactions, compatibility with common assistive technologies, and known limitations. Testing is an ongoing process that's revisited as components evolve. -VFS teams are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). Moreover, teams are required to complete [foundational accessibility testing](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review#foundational-testing) as they prepare for a staging review, and are highly encouraged to complete [advanced accessibility testing](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review#advanced-testing) as well. +For the specific tests we run and how to interpret test results on component pages, see the [Accessibility Test Library](/accessibility/test-library). -### Evaluate components' accessibility in your product +### What the design system provides -Design system components are tested in isolation. While we try to ensure that no accessibility barriers exist in any given component, barriers can still be introduced when components are used in an unanticipated combination, or when components are placed in an unanticipated context within the page. +The design system establishes baseline accessible behavior for individual components, including: -#### Readability +- Expected interaction patterns (for example: focus management, announcements, and state changes) +- Semantic structure and labeling +- Compatibility with common assistive technologies and input methods -Readability of an individual component is highly dependent on the context around that component. When using a component, VFS teams should check: +### What the design system does not guarantee -- Headings and sub-headings - - Do component headings fit into the overall heading hierarchy of the page? - - Is content in close proximity to the component grouped logically? -- Buttons and link text - - Is any text hardcoded into the component appropriate for your user flow? - - When adding your own text via component properties, is the text meaningful? Does it convey a purpose? -- Labels - - Are any labels hardcoded into the component appropriate for your user flow? - - When adding your own label text via component properties, are the labels clear, concise, and easy to understand? -- Plain language - - Do error messages clearly describe the error and provide a clear path for resolving the error? +Using design system components alone won't guarantee that your product is accessible. -#### Color considerations +Accessibility issues can still be introduced when: -When using a component, VFS teams should check: +- Components are combined in complex ways +- Content or labels aren't clear or are misleading +- Components are used in contexts that weren't anticipated during testing -- Color contrast against against backgrounds and nearby or adjacent elements. +Product teams are responsible for evaluating accessibility in the context of their full experience. -### Report accessibility defects +## How components are tested -If you identify an accessibility defect in a component, please [submit an issue](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new/choose). +We evaluate accessibility through a combination of code review, automated tooling, and manual testing across assistive technologies and input methods. Each method contributes a different perspective, and no single approach is sufficient on its own. Accessibility issues often emerge through interaction and state changes, which is why manual testing across assistive technologies is a core part of this process. -## Testing principles +### Testing principles -### POUR +#### POUR -Each component is tested for accessibility in the design system based on the four principles of accessibility. +Each component is evaluated against the four principles of accessibility: - **Perceivable** - Information and user interface components must be presentable to users in ways they can perceive. - **Operable** - User interface components and navigation must be operable. - **Understandable** - Information and the operation of user interface must be understandable. - **Robust** - Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies. -### WCAG 2.2 AA +#### WCAG 2.2 AA -These principles are the foundation of [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/), which incorporates Web Content Accessibility Guidelines (WCAG) 2.0 Level AA success criteria. We aim to adhere to the latest version of WCAG Level AA, which at the time of writing is [WCAG 2.2](https://www.w3.org/TR/WCAG22/). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we creating an inclusive and equitable experience for everyone. +These principles are the foundation of [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/), which incorporates Web Content Accessibility Guidelines (WCAG) 2.0 Level AA success criteria. We aim to adhere to the latest version of WCAG Level AA, which is currently [WCAG 2.2](https://www.w3.org/TR/WCAG22/). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we're creating an inclusive and equitable experience for everyone. -### Multiple types of assistive technologies +#### Behavior-based testing -Each component is also tested to ensure compatibility with common assistive technologies covering a broad set of common interaction modalities, including: +Accessibility tests are derived from expected user interactions and outcomes. For example: -- Screen readers (text to speech) -- Voice command/speech recognition (speech to text) -- Screen magnification -- Browser display settings (eg. zoom, text size) -- Device contrast themes -- Keyboard-only -- Mouse-only -- Touch-only -- Alternative input devices +- When a user completes an action, the result is communicated clearly +- When content updates dynamically, the change is announced appropriately +- When an error occurs, the user is informed and can recover -## Testing methodology +This approach helps ensure components behave consistently across assistive technologies and input methods. -### Code review +#### Multiple types of assistive technologies -Code reviews are essential for building inclusive and usable digital products for everyone. Well-written semantic HTML ensures that browsers faithfully convey design system components to users, and ensures maximum compatibility with assistive technologies. Here's what we look for in code: +Components are evaluated across a range of assistive technologies and interaction modalities, including: -- [Valid HTML usage](https://developer.mozilla.org/en-US/docs/Learn/Accessibility/HTML "https://developer.mozilla.org/en-US/docs/Learn/Accessibility/HTML") - - Are semantic HTML elements used properly? -- [Valid ARIA usage](https://developer.mozilla.org/en-US/docs/Learn/Accessibility/WAI-ARIA_basics "https://developer.mozilla.org/en-US/docs/Learn/Accessibility/WAI-ARIA_basics") - - Are ARIA roles and attributes used properly? -- [Controls are labeled](https://www.w3.org/WAI/tutorials/forms/labels/ "https://www.w3.org/WAI/tutorials/forms/labels/") - - Are elements labeled using appropriate techniques? +- Screen readers +- Voice command software +- Screen magnification tools +- Browser display settings +- Keyboard, mouse, and touch input -### Automated scans +### Testing methodology -Automated accessibility testing tools are crucial for building accessible digital products because they provide a quick and efficient way to identify potential issues. Currently, each design system component goes through: +#### Code review -- An aXe DevTools scan via Cypress for the default variation of the component. -- An aXe DevTools scan via the browser extension for each additional variation of the component. +We review component code to ensure semantic HTML, appropriate ARIA (Accessible Rich Internet Applications) usage, and clear labeling. This helps ensure components are correctly interpreted by browsers and assistive technologies. -### Readability evaluation +#### Automated scans -Readability is a key part of accessibility because it directly impacts how easily users can understand the content. We ensure the following are easy to understand: +We use automated tools, such as aXe DevTools, to identify common accessibility issues. Automated testing supports, but does not replace, manual evaluation. -- [Headings and sub-headings](https://design.va.gov/content-style-guide/page-titles-and-section-titles "https://design.va.gov/content-style-guide/page-titles-and-section-titles") - - Are headings used in a hierarchical manner? - - Is the content grouped logically? -- [Button](https://design.va.gov/content-style-guide/button-labels "https://design.va.gov/content-style-guide/button-labels") and [link](https://design.va.gov/content-style-guide/links "https://design.va.gov/content-style-guide/links") text - - Is the text meaningful? - - Does the text convey a purpose? -- Labels - - Are labels clear, concise, and easy to understand? -- [Plain language](https://design.va.gov/content-style-guide/plain-language/ "https://design.va.gov/content-style-guide/plain-language/") - - Is all text consistent with our plain language standards? +#### Readability evaluation -### Use of color +Components and patterns are evaluated for clear and understandable content, including heading structure, link and button text, labels, and adherence to [plain language standards](https://design.va.gov/content-style-guide/plain-language/). -Color plays a crucial role in accessibility because it significantly impacts how users perceive and interact with digital content, especially those with visual impairments. Here is what we test for: +#### Use of color -- Color contrast meets or exceeds [WCAG Level AA contrast ratios](https://www.w3.org/WAI/WCAG22/Understanding/contrast-minimum.html). -- Information is not communicated through color alone. Relying solely on color for communication is an accessibility barrier for individuals with visual impairments. -- Use of [Windows contrast themes](https://learn.microsoft.com/en-us/windows/apps/design/accessibility/high-contrast-themes) (formerly High Contrast Mode) does not result in any use of color regressions from the default color presentation. +Components are tested for sufficient color contrast and to ensure information is not communicated through color alone. -### Text resizing, zoom, and magnification +#### Text resizing, zoom, and magnification -Testing text resizing, browser zoom levels, and screen magnifications is crucial for accessibility because it ensures that your website or application remains usable for individuals who need to adjust their browser's display settings or magnify parts of their screen. Here's what we test: +Components are tested at up to 400% browser zoom and with screen magnification tools to ensure layouts remain usable and content remains readable. -- Browser font size options correctly resize the text of a component. -- Zoom levels of 200%, 300%, and 400% do not affect the usability of the component. -- Features are visible or discoverable when using [MacOS Zoom](https://support.apple.com/guide/mac-help/change-zoom-settings-for-accessibility-mh40579/mac), [Windows Magnifier](https://support.microsoft.com/en-us/windows/use-magnifier-to-make-things-on-the-screen-easier-to-see-414948ba-8b1c-d3bd-8615-0e5e32204198), and/or similar magnification tools. +- [MacOS Zoom](https://support.apple.com/guide/mac-help/change-zoom-settings-for-accessibility-mh40579/mac) +- [Windows Magnifier](https://support.microsoft.com/en-us/windows/use-magnifier-to-make-things-on-the-screen-easier-to-see-414948ba-8b1c-d3bd-8615-0e5e32204198) -### Screen readers +#### Screen readers -Screen reader testing is important for accessibility because it ensures each component is usable for users of assistive technology. We test with the following screen reader and browser combinations to ensure a consistent experience across devices: +Components are tested with the following screen reader and browser combinations: - [JAWS](https://www.freedomscientific.com/products/software/jaws/ "https://www.freedomscientific.com/products/software/jaws/") + Chrome on Windows -- [NVDA](https://www.nvaccess.org/download/ "https://www.nvaccess.org/download/") + Firefox on Windows -- [Narrator](https://support.microsoft.com/en-us/windows/complete-guide-to-narrator-e4397a0d-ef4f-b386-d8ae-c172f109bdb1 "https://support.microsoft.com/en-us/windows/complete-guide-to-narrator-e4397a0d-ef4f-b386-d8ae-c172f109bdb1") + Edge on Windows +- [NVDA](https://www.nvaccess.org/download/ "https://www.nvaccess.org/download/") + Edge on Windows - [VoiceOver](https://support.apple.com/guide/voiceover/welcome/mac "https://support.apple.com/guide/voiceover/welcome/mac") + Safari on MacOS - [TalkBack](https://support.google.com/accessibility/android/answer/6283677?hl=en "https://support.google.com/accessibility/android/answer/6283677?hl=en") + Chrome on Android - [VoiceOver](https://support.apple.com/guide/iphone/turn-on-and-practice-voiceover-iph3e2e415f/ios "https://support.apple.com/guide/iphone/turn-on-and-practice-voiceover-iph3e2e415f/ios") + Safari on iOS -With each screen reader, here's what we test: +We verify that content is announced in logical order, interactive elements have appropriate accessible names and states, and common screen reader interaction patterns are supported. + +#### Input and interaction methods + +Components are tested across different input methods, including keyboard, mouse, touch, and voice control. + +- [Voice Control](https://support.apple.com/en-us/102225) + Safari on MacOS +- [Dragon](https://www.nuance.com/dragon.html) + Edge or Chrome on Windows + +We verify keyboard accessibility (focus management, keyboard operability, and visible focus indicators), touch and mouse compatibility (appropriate target sizes and gesture support), and voice command compatibility (accessible names that match visible labels). -- All meaningful content is announced to screen reader users in a logical order consistent with the visual presentation. -- All interactive elements have a unique accessible name. -- All interactive elements are announced with their element type, current value, and current state (as appropriate). -- Common screen reader interaction patterns are supported (eg. navigating a page by heading, navigating a page by landmark, etc). +Alternative input devices such as sip-and-puff switches, eye-tracking software, and refreshable Braille displays are not directly tested but are supported through robust keyboard and mouse support, along with standards-based code and proper ARIA usage. -### Input and interaction methods +## Using components in your product -Each component is tested to ensure it is usable for individuals who rely on different input methods. Input methods tested include: +We test design system components in isolation. While these tests provide a baseline of accessible behavior, issues may emerge when components are used together or placed within a broader page context. -- Keyboard-only -- Mouse-only -- Touch-only (as appropriate, where interactions may be distinct from mouse-only) -- [Voice Control](https://support.apple.com/en-us/102225 "https://support.apple.com/en-us/102225") + Safari on MacOS -- [Dragon](https://www.nuance.com/dragon.html "https://www.nuance.com/dragon.html") + Edge or Chrome on Windows +### Accessibility testing expectations -#### Alternative input devices +Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). Teams are also required to complete an [accessibility testing artifact](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review) as they prepare for a staging review. -Alternative input devices such as sip-and-puff switches or eye-tracking software are not directly tested. These devices typically map user interactions to specific keystrokes or mouse movements, so robust support for keyboard and mouse combined with well-tested standards-based code should ensure support for these devices. +### What to evaluate -Likewise, we do not directly test with refreshable Braille displays or other nonvisual displays, but appropriate use of ARIA and semantic HTML should ensure support for these devices as well. +When using a component, teams should evaluate how it functions within the full experience. -#### Testing input methods +#### Readability and structure + +Readability depends on surrounding context. Evaluate: + +- Headings and sub-headings + - Do component headings fit into the overall heading hierarchy of the page? + - Is nearby content grouped logically? + +- Buttons and link text + - Is hardcoded text appropriate for your user flow? + - When adding your own text, is it meaningful and descriptive? + +- Labels + - Are labels appropriate and clearly understood? + - Are they concise and easy to follow? + +- Plain language + - Do error messages clearly describe the issue and provide a path for resolution? + +#### Color considerations + +When using a component, teams should check: + +- Color contrast against backgrounds and nearby or adjacent elements. + +### Report accessibility defects -For each input method, we test: +If you identify an accessibility defect in a component, please [submit an issue on GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new/choose) describing the defect. -- Touch and mouse compatibility, including: +## Scope and limitations - - Target sizes meet or exceed [WCAG Level AA target size requirements](https://www.w3.org/WAI/WCAG22/Understanding/target-size-minimum.html "https://www.w3.org/WAI/WCAG22/Understanding/target-size-minimum.html"). - - Controls do not require multipoint or path-based gestures. - - Controls do not require specific click or tap patterns or timing. +Accessibility testing for design system components is intentionally scoped. -- Keyboard compatibility, including: - - All interactive elements are focusable using the keyboard. - - Focus indicators are fully visible and meet color contrast requirements. - - Form elements such as dropdowns, radio buttons, and sliders can be utilized with arrow keys. - - Interactive elements can be activated with the keyboard. - - Users can exit all interactive elements without being trapped. -- Voice command/speech recognition compatibility, including: +- We test components in isolation, not within full product flows +- We don't test all assistive technology and browser combinations +- Some accessibility issues may only emerge in real-world product contexts - - The accessible name of elements matches the visible label for the element; or, if not, the visible label supports a clear interaction path. - - Labels have a straightforward pronunciation. +The design system provides a tested baseline, but accessibility must be validated at the product level. -## Component checklists +## Test results on component pages -For each component, the results of our accessibility testing are displayed in the component checklist section, organized by the categories described in our testing methodology. Each category may be labeled as: +The [Accessibility Test Library](/accessibility/test-library) defines the specific tests, expected behaviors, and environments used to evaluate components. -- Pass -- Conditional pass -- Pass with exceptions -- Fail -- Not completed +On each component page, the “Accessibility tests” section shows: -Additional notes are provided where appropriate. For example, if automated scans for a component indicate a color contrast violation but manual testing for use of color finds that the automated scan result was a false positive, the component checklist may read: +- Which tests apply to the component +- The status of those tests (Pass, Failed, Conditional) +- The environments in which testing was performed +- When the tests were last completed -> **Automated scans** -> -> Pass with exceptions - Axe testing shows a false positive on color contrast for the default variation +This helps teams understand what has been validated and what should be verified within their own implementation. diff --git a/src/_accessibility/test-library.md b/src/_accessibility/test-library.md new file mode 100644 index 000000000..3a885dc15 --- /dev/null +++ b/src/_accessibility/test-library.md @@ -0,0 +1,49 @@ +--- +layout: documentation +title: Accessibility Test Library +permalink: /accessibility/test-library +anchors: + - anchor: About this library + - anchor: How to read test results + - anchor: Environment-specific results + - anchor: Test library +--- + +## About this library + +_**This test library is in beta and may change.** As this library matures, we welcome your feedback on the tests, structure, and results. Share feedback in the [#platform-design-system Slack channel](https://dsva.slack.com/archives/C01DBGX4P45) or on [GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new?template=DST-basic_issue_template.md)._ + +This library provides the specific tests we use to evaluate design system components for accessibility. Each test maps to [WCAG 2.2 success criteria](https://www.w3.org/WAI/WCAG22/quickref/) and builds on the [VA Platform team's accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual). + +For details on our overall testing approach, principles, and methodology, see [Accessibility testing for design system components](/accessibility/accessibility-testing-for-design-system-components). + +We're actively auditing components, so not all components have test results yet. On each component page, the "Accessibility tests" section shows the status of each test across each environment. + +## How to read test results + +### Test status +Component test results use three statuses: + +- **Passed**: The component meets the test requirement. +- **Failed**: The component does not meet the test requirement. +- **Conditional**: The test can only be evaluated within a specific product or workflow. Teams using the component are responsible for verifying these tests in their own context. + +### Environments tested + +Each test result shows the environments and component versions tested. We track results for each environment because accessibility behavior can vary across browsers and assistive technologies. Per-environment results help us: + +- **Establish a baseline** so we can catch when component updates change in a specific environment. +- **Share what we've tested** so teams can focus their testing on areas we haven't covered, like product-specific workflows. +- **Surface gaps** by displaying what we haven't tested yet. + +### Understanding test IDs + +Test IDs show how each test relates to WCAG 2.2. + +- **WEB-* tests**: General VA Platform tests aligned with WCAG. They apply across many contexts, and failures likely indicate a WCAG violation. +- **VADS-* tests**: VA Design System–specific tests that map to WCAG criteria. They define more specific expectations for components, including standardized pattern choices. + +**Example:** WEB-141 requires color is not the only way to convey information. VADS-141-001 defines how links provide a consistent non-color indicator in VA Design System components. + +## Test library +{% include accessibility-test-library-list.html %} diff --git a/src/_data/accessibility-tests/README.md b/src/_data/accessibility-tests/README.md new file mode 100644 index 000000000..d49309d91 --- /dev/null +++ b/src/_data/accessibility-tests/README.md @@ -0,0 +1,92 @@ +# Accessibility Tests + +> **Note:** This test library is in beta and may change as it matures. We welcome feedback on the tests, structure, and results in the [#platform-design-system Slack channel](https://dsva.slack.com/archives/C01DBGX4P45). + +This directory contains accessibility test documentation for VA Design System components. Test results are documented in YAML files organized into subdirectories. + +## Overview + +The VA Design System Accessibility Test Library is a comprehensive catalog of plain language accessibility tests organized by WCAG 2.2 principles. Each test includes specific success criteria, testing procedures, and mappings to WCAG standards and Section 508 requirements. + +This library: +- Builds on the [VA Platform team's accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual) +- Provides plain language, component-specific tests with standardized test IDs +- Enables consistent accessibility testing across the design system +- Documents test results over time to track component accessibility + +**Public-facing documentation:** [/accessibility/test-library](/accessibility/test-library) + +## Directory Structure + +- **`components/`** — Component test files (one YAML file per component) + - Example: `components/va-button.yml` + - See [components README](components/README.md) for details +- **`test-library/`** — Test definitions, WCAG mappings, and writing guidelines + - Test files: `1-perceivable.yml`, `2-operable.yml`, `3-understandable.yml`, `4-robust.yml` + - Configuration: `_config.yml` (environment IDs and test categories) + - See [test-library README](test-library/README.md) for details + +## Getting Started + +### Working with Component Test Files + +To create or edit component test files, see the **[components README](components/README.md)**, which covers: +- Creating new component test files (AI-assisted or manual) +- Choosing which tests apply to your component +- Recording test results (formats, fields, examples) +- File structure and examples +- How test results are displayed + +### Working with Test Definitions + +To understand or edit test definitions in the test library, see the **[test-library README](test-library/README.md)**, which covers: +- Test ID format and WCAG mappings +- Test structure and required fields +- Writing guidelines for descriptions +- Plain language principles + +## AI-Assisted Tooling + +### GitHub Copilot Prompts + +Use these prompts in GitHub Copilot Chat to work with accessibility tests: + +- **`/accessibility-tests-generate-component-tests`** — Create a component test file with applicable tests + - Available at: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` + - Optionally includes test metadata (tester name, version, date) + - Generates YAML file in `src/_data/accessibility_tests/components/` +- **`/accessibility-tests-generate-metadata`** — Add or update test metadata in an existing component test file + - Available at: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` + - Useful for adding metadata to files created without it, or for adding new test runs + - Updates YAML files with tester, version, date, and environment structure +- **`/accessibility-tests-add-to-component-page`** — Add accessibility tests section to a component documentation page + - Available at: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` + - Updates component markdown files (e.g., `src/_components/button.md`) + - Adds Jekyll include to display test results on the public site +- **Audit prompt** — Review component test files for completeness + - Available at: `.github/prompts/accessibility-tests-audit.prompt.md` + +### Claude Code Skills + +When using Claude Code, these skills are automatically available: + +- **Test Selection** (`.claude/skills/accessibility-tests/test-selection/`) — Determines which tests apply to a component based on its features +- **Writing Test Entries** (`.claude/skills/accessibility-tests/writing-entries/`) — Guidelines for writing test library entries with proper descriptions + +These skills are referenced by the prompts above and help maintain consistency across test files. + +## Related Resources + +### VA Design System Documentation +- **[Accessibility guidance](/accessibility/)** — VADS accessibility standards and best practices +- **[Component documentation](/components/)** — Individual component pages display test results +- **[Test library catalog](/accessibility/test-library)** — Public-facing test definitions and WCAG mappings + +### External Standards +- **[WCAG 2.2 Guidelines](https://www.w3.org/TR/WCAG22/)** — Web Content Accessibility Guidelines +- **[Section 508 Standards](https://www.section508.gov/)** — Federal accessibility requirements +- **[VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual)** — VA-wide testing guidance + +## Questions? + +Contact the Design System team in [#platform-design-system](https://dsva.slack.com/archives/C01K37HRUAH) Slack channel. diff --git a/src/_data/accessibility-tests/components/README.md b/src/_data/accessibility-tests/components/README.md new file mode 100644 index 000000000..93f1ff420 --- /dev/null +++ b/src/_data/accessibility-tests/components/README.md @@ -0,0 +1,410 @@ +# Component Accessibility Test Files + +This directory contains accessibility test documentation for individual VA Design System components. Each YAML file maps a component to its applicable tests from the [test library](../test-library/) and records test results over time. + +## How this documentation is organized + +- **This README** — Creating component test files, choosing tests, and recording results +- **[Parent README](../README.md)** — Directory overview and AI tooling references +- **[Test Library README](../test-library/README.md)** — Test definitions, WCAG mappings, and writing guidelines + +## File naming + +Files follow the format `va-[component-name].yml`, matching the web component tag name (e.g., `va-accordion.yml`, `va-text-input.yml`). + +## Creating a new component test file + +### Recommended: Use the AI prompt + +**Use the AI prompt** at `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` to automatically generate a component test file. + +The prompt will: +- Analyze the component to understand its features +- Select applicable tests from the test library based on component type +- Use specific subtests instead of parent tests where appropriate +- Optionally pre-populate test metadata (tester name, version, date) if you're ready to test +- Create a properly formatted YAML file + +**To use the prompt:** +1. In GitHub Copilot Chat, use the slash command: `/accessibility-tests-generate-component-tests` +2. Follow the prompts to provide component name +3. Choose whether to include metadata now or add it later +4. Review the generated file and save it to `src/_data/accessibility_tests/components/[component-name].yml` + +**To add or update test metadata later:** +- Use the `/accessibility-tests-generate-metadata` prompt (`.github/prompts/accessibility-tests-generate-metadata.prompt.md`) +- This is useful when you're ready to start testing but created the file without metadata initially + +**To add the accessibility tests section to the component page:** +- Use the `/accessibility-tests-add-to-component-page` prompt (`.github/prompts/accessibility-tests-add-to-component-page.prompt.md`) +- This displays test results on the public documentation site +- See [Step 5 below](#step-5-display-results-on-component-page) for manual instructions + +**Additional AI tools:** +- **Audit prompt**: `.github/prompts/accessibility-tests-audit.prompt.md` — Review component test files for completeness +- **Test selection skill**: `.claude/skills/accessibility-tests/test-selection/` — Determines which tests apply based on component features +- **For other tools**: See [../README.md](../README.md#ai-assisted-tooling) for complete AI assistance information + +### Manual creation + +If you prefer to work manually or need to understand the file structure: + +#### Step 1: Create the YAML file + +Create a new file in this directory using the format: `va-[component-name].yml` + +Example: `va-button.yml` for the Button component + +#### Step 2: Add component metadata + +```yaml +component: va-button +component_url: /components/button/ +storybook_url: /storybook/?path=/docs/components-va-button--docs + +tests: + # Test results will go here +``` + +**Metadata fields:** +- `component`: Component name (e.g., `va-button`, `va-alert`) — must match the web component tag name +- `component_url`: Relative URL to the component's documentation page +- `storybook_url`: Relative URL to the component's Storybook documentation + +#### Step 3: Add test IDs + +Add applicable test IDs under `tests:` in your file. Use the [test selection guide](#choosing-tests-by-component-type) below to determine which tests apply, or browse existing files like `va-link.yml`, `va-text-input.yml`, or `va-accordion.yml` for examples. + +You can add tests with or without results. Tests without results will show as "Untested" on the documentation page. + +#### Step 4: Add test results + +For guidance on recording test results, see [Recording Test Results](#recording-test-results) below. + +#### Step 5: Display results on component page + +Add the accessibility tests section to the component's documentation page at `src/_components/[component-name].md`. + +**Update the YAML front matter:** +1. Find the `anchors:` list in the front matter +2. Add `- anchor: Accessibility tests` +3. Place it after "Accessibility considerations" (if present) and before "Related" or "Component checklist" + +**Add the section to the markdown body:** +1. Place the new section after "Accessibility considerations" if it exists +2. Otherwise, place it before the "Related" section +3. If neither exists, place it before the component checklist include +4. Use this format: + ```markdown + ## Accessibility tests + + {% include accessibility-test-results.html component_name="[component-name]" %} + ``` +5. Replace `[component-name]` with the component's `web-component` value from the front matter (e.g., `va-accordion`, `va-details`) + +## Workflow Overview + +1. **Create or update** a component test file in this directory +2. **Add test IDs** using the [test selection guide](#choosing-tests-by-component-type) below +3. **Perform accessibility testing** for each test ID +4. **Record results** in the component YAML file (see [Recording Test Results](#recording-test-results)) +5. **Add the Jekyll include** to display results on the component documentation page +6. **Update and maintain** results as new component versions are released + +## File structure + +```yaml +component: va-button +component_url: /components/button/ +storybook_url: /storybook/?path=/docs/components-va-button--docs + +tests: + - id: WEB-211 # Test ID from the test library + test_results: + - version: 54.6.1 # Component library version + tester: Jane Smith + date: 2026-03-26 + environments: + - name: mac-chrome + result: pass +``` + +**Required fields:** +- `component`: Component name matching the web component tag +- `component_url`: Relative URL to component documentation +- `storybook_url`: Relative URL to Storybook documentation +- `tests`: Array of test IDs with optional test results + +## Recording Test Results + +### Basic Structure + +Each test can have multiple test result entries (for different versions, testers, or test sessions). + +```yaml +tests: + - id: WEB-1410 + test_results: + - version: 54.6.1 + tester: John Doe + date: 2024-11-15 + environments: + - name: mac-chrome + result: pass +``` + +### Required Fields +- **`date`**: Date the test was performed (format: `YYYY-MM-DD`) +- **`version`**: Component library version tested (e.g., `54.6.1`, `55.0.0`) +- **`tester`**: Name of the person who performed the test +- **`environments`**: List of environments tested (at least one required) + - **`name`**: Environment ID (see [Environment IDs](#environment-ids) below) + - **`result`**: Test result value (see [Test Results](#test-results) below) + +### Optional Fields + +- **`notes`**: Additional context about the test results + - **Required** for `conditional` or `fail` results to explain the issue or limitation + - Optional for `pass` results to provide additional context + +### Example: Pass + +```yaml +tests: + - id: WEB-1410 + test_results: + - version: 54.6.1 + tester: Jane Smith + environments: + - name: mac-chrome + result: pass +``` + +### Example: Conditional result with notes + +```yaml +tests: + - id: WEB-244 + test_results: + - version: 54.6.1 + tester: Jane Smith + date: 2024-11-20 + environments: + - name: mac-chrome + result: conditional + notes: Link text is provided by implementations and must be tested in each implementation context. +``` + +### Test Results + +Valid result values: +- **`pass`**: Test passed without issues +- **`fail`**: Test failed (must include `notes` explaining the failure) +- **`conditional`**: Test passes with conditions or caveats (must include `notes` explaining the limitation) + +**Note:** Do not use `untested` as a result value. If a test hasn't been performed, simply omit the `test_results` entry for that test. The component page will automatically show it as "Untested." + +### Environment IDs + +Environment IDs use the format: `{os}-{browser}` or `{os}-{browser}-{assistive-technology}` + +Examples: `mac-chrome`, `windows-edge-nvda`, `ios-safari-voiceover`, `mac-voice-control` + +**For the complete list of valid environment IDs**, see the [Environment IDs section](../test-library/README.md#environment-ids) in the test library README. + +## How test results are displayed + +Test results appear on each component's documentation page in an "Accessibility tests" section via the `accessibility-test-results.html` Jekyll include (see Step 5 of [manual creation](#manual-creation) above). + +## Test Selection Guidelines + +### How to analyze your component + +Before selecting tests, review your component to identify its features: + +- **Interactive elements**: Is it a button, link, form input, or does it contain these? +- **Content types**: Does it have text, headings, images, icons, or color-coded information? +- **Dynamic behavior**: Does it show/hide content, expand/collapse, trap focus, or manage state? +- **Child components**: Does it use other VA web components (e.g., ``, ``)? +- **Primary purpose**: What is this component's main function? + +Use these features to determine which test categories apply in the "Choosing tests by component type" section below. + +### Subtest Preference Rule + +**Always prefer the most specific subtest over the parent test** when available. + +- **Subtest format**: `PREFIX-YYY-ZZZ` (e.g., `WEB-111-001`, `VADS-211-002`) +- **Use specific subtests** when a parent test has them (e.g., use `WEB-111-001` for images, not `WEB-111`) +- **Include multiple subtests** from the same parent when relevant +- **Use parent test only** when no specific subtest applies + +**How to find subtests:** + +1. **Determine which file contains the test** based on the test ID prefix: + - `WEB-1xx` tests → `1-perceivable.yml` + - `WEB-2xx` tests → `2-operable.yml` + - `WEB-3xx` tests → `3-understandable.yml` + - `WEB-4xx` tests → `4-robust.yml` + - `VADS-xxx` tests → Search all files (these are VADS-specific subtests) + +2. **Open the corresponding YAML file** at `src/_data/accessibility_tests/test-library/` + +3. **Find the test entry** by searching for `id: [test-id]` + +4. **Look for a `subtests:` array** under that test entry + +5. **If subtests exist**: + - Review each subtest's `description_short` to determine if it applies to your component + - Replace the parent test ID with applicable subtest IDs (e.g., replace `WEB-111` with `WEB-111-001` and `WEB-111-003`) + - Include multiple subtests if they apply to different aspects of the component + +6. **If no subtests exist**, keep the parent test ID + +See [test library README](../test-library/README.md) for test structure and ID format details. + +### Child component inheritance + +If your component uses child VA web components (e.g., a form component that uses `` or ``), you should inherit applicable tests from those child components: + +1. **Identify child components** in your component's source code +2. **Open each child's test file** at `src/_data/accessibility_tests/components/va-[child-name].yml` +3. **Review the child's test list** under the `tests:` array +4. **Add relevant tests** from the child to your component's test list +5. **Remove duplicates** after combining with your category-based tests + +This ensures parent components maintain the same accessibility coverage as their children. + +### Determining required test environments + +Some tests require specific environments (e.g., screen readers, mobile devices) for test results: + +1. **Open the test definition** in the appropriate test library YAML file +2. **Check the `category:` field** (e.g., `screen_reader`, `mobile`, `keyboard`) +3. **Open** `src/_data/accessibility_tests/test-library/_config.yml` +4. **Find the category definition** under `categories:` +5. **Check for `required_environments:`** — if present, those environments must be tested and reported + +For example, tests with the `mobile` category require results from `android-chrome` and `ios-safari` environments. + +See the [Environment IDs section](../test-library/README.md#environment-ids) in the test library README for the complete list of valid environment IDs. + +## Choosing tests by component type + +Use the guidelines below to determine which tests apply to a component. Start with "All components," then add tests for each applicable category. + +### All components + +Include these tests for every component: +- `WEB-134` — Orientation (content orientation is not restricted) +- `WEB-1410` — Reflow (content doesn't require two-directional scrolling at 400% zoom) + +### Most components with multiple elements + +- `WEB-132` — Meaningful sequence (content is presented in a meaningful order) + +### Components with text + +- `WEB-143` — Contrast (Minimum) — text has sufficient contrast +- `WEB-144` — Resize Text — text resizes to 200% without loss of content or functionality +- `WEB-1412` — Text Spacing — text remains readable when spacing is adjusted + +### Components with headings + +- `WEB-131-001` — Headings are used to organize content +- `WEB-131-002` — Headings are used in a logical order +- `WEB-246` or applicable subtests — Headings and labels describe topic or purpose (`WEB-246-001` for heading descriptions, `WEB-246-002` for input labels) + +### All interactive elements + +Applies to all buttons, links, and form controls. + +- `WEB-211` or applicable subtests — Keyboard operability (`VADS-211-001` for links, `VADS-211-002` for form inputs) +- `WEB-1411` or applicable subtests — Non-text Contrast +- `WEB-212` or applicable subtests — No Keyboard Trap +- `WEB-243` — Focus Order (tab order follows a logical sequence) +- `WEB-247` — Focus Visible +- `WEB-2411` — Focus Not Obscured (focused elements are fully visible) +- `WEB-253` or applicable subtests — Label in Name (`VADS-253-001` for screen reader announcement matching, `VADS-253-002` for voice control activation) +- `WEB-258` or applicable subtests — Target Size +- `WEB-321` — On Focus +- `WEB-412` or applicable subtests — Name, Role, Value + +### Links + +In addition to all interactive element tests above. + +- `WEB-141` or applicable subtests — Use of Color (`VADS-141-001` for links indicated by color) +- `WEB-244` — Link Purpose (In Context) — link text is clear and descriptive +- `WEB-412` or applicable subtests — Name, Role, Value (`WEB-412-003` for proper element types, `VADS-412-001` for link navigation) + +### Buttons + +In addition to all interactive element tests above. + +- `WEB-412` or applicable subtests — Name, Role, Value (`WEB-412-002` for state announcements if stateful, `WEB-412-003` for proper element types, `VADS-412-002` for button actions) + +### Form inputs + +Applies to text-input, select, checkbox, radio, textarea. In addition to all interactive element tests above. + +- `WEB-131` or applicable subtests — Info and Relationships (`WEB-131-005` for grouped elements, `WEB-131-006` for input labels and instructions, `WEB-131-007` for required fields) +- `WEB-135` — Identify input purpose (if applicable) +- `VADS-1411-002` — Non-text Contrast for form input borders +- `WEB-1413` or applicable subtests — Content on Hover or Focus (`VADS-1413-001` for hover content persistence, `VADS-1413-002` for Escape key dismissal) +- `WEB-246` or applicable subtests — Headings and labels (`WEB-246-002` for input labels) +- `WEB-321` — On Focus +- `WEB-322` or applicable subtests — On Input +- `WEB-331` or applicable subtests — Error Identification (`VADS-331-001` for errors on blur, `VADS-331-002` for error descriptions, `VADS-331-003` for error styles, `VADS-331-004` for screen reader announcements, `VADS-331-005` for error placement) +- `WEB-332` or applicable subtests — Labels or Instructions (`WEB-332-001` for visible labels, `WEB-332-002` for format instructions, `WEB-332-003` for required field marking) +- `WEB-333` — Error Suggestion + +### Components with images or icons + +Applies to components with any visual non-text content: ``, ``, icon fonts, CSS background images, `::before`/`::after` pseudo-elements, or video/audio content. + +- `WEB-111` or applicable subtests — Non-text content (`WEB-111-001` for informative images, `WEB-111-003` for decorative images/icons, `WEB-111-005` for video/audio descriptions; include multiple if component has different image types) +- `WEB-145` — Images of Text (text is not presented as images) +- `VADS-1411-001` — Icons have sufficient contrast (if icons convey meaning or are interactive) + +### Table components + +Applies to data tables, sortable tables. + +- `WEB-131-009` — Table header and data cell relationships (screen readers announce table structure) + +### Dynamic content + +Applies to alerts, modals, accordions, expandable content. + +- `WEB-243` or applicable subtests — Focus Order (`VADS-243-001` for focus trapping inside modals, `VADS-243-002` for focus returning to trigger on close) +- `WEB-253` or applicable subtests — Label in Name (`VADS-253-001` if component label/content changes dynamically) +- `WEB-412` or applicable subtests — Name, Role, Value (`WEB-412-002` for screen reader state announcements when content expands/collapses or shows/hides) +- `WEB-413` or applicable subtests — Status Messages (`VADS-413-001` for loading status announcements, `VADS-413-002` for alert announcements) + +### Components with color coding + +Applies to components that use color to convey information or status, such as alerts, status badges, or color-coded indicators. + +- `WEB-141` or applicable subtests — Use of Color (`VADS-141-001` for links, `VADS-141-002` for error states, `VADS-141-003` for status indicators, `VADS-141-004` for required fields, `VADS-141-005` for interactive state changes, `VADS-141-006` for charts/graphs) +- `WEB-1411` or applicable subtests — Non-text Contrast (`VADS-1411-001` for icons, `VADS-1411-002` for form input borders, `WEB-1411-001` for interactive elements, `WEB-1411-002` for meaningful graphics) + +### Navigation components + +Applies to breadcrumbs, pagination, menus. These are typically collections of links, so apply the "Links" category above. Also include: + +- `WEB-323` — Consistent Navigation + +If the component includes skip navigation functionality, also include: +- `WEB-241` or `VADS-241-001` — Bypass Blocks + +## Resources + +- **Test Library**: `src/_data/accessibility_tests/test-library/` +- **Test Library README**: [../test-library/README.md](../test-library/README.md) — Test definitions, WCAG mappings, and writing guidelines +- **Parent README**: [../README.md](../README.md) — Directory overview and AI assistance +- **Component Library source**: [department-of-veterans-affairs/component-library](https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/) + +## Questions? + +Contact the Design System team in [#platform-design-system](https://dsva.slack.com/archives/C01K37HRUAH) Slack channel. diff --git a/src/_data/accessibility-tests/components/va-details.yml b/src/_data/accessibility-tests/components/va-details.yml new file mode 100644 index 000000000..81c210ff4 --- /dev/null +++ b/src/_data/accessibility-tests/components/va-details.yml @@ -0,0 +1,338 @@ +component: va-details +component_url: /components/details/ +storybook_url: /storybook/?path=/docs/components-va-details--docs + +tests: + + # 1.1.1 - Decorative chevron icon not announced by screen readers + - id: WEB-111-003 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-03 + environments: + - name: mac-safari-voiceover + result: pass + - name: windows-chrome-jaws + result: pass + - name: windows-edge-nvda + result: pass + - name: ios-safari-voiceover + result: pass + - name: android-chrome-talkback + result: pass + notes: Chrome announces "disclosure triangle" as the role description for the native summary element — this is browser-native behavior, not an announcement of the decorative chevron icon. The icon itself is not announced. + + # 1.3.1 - Expanded/collapsed state conveyed through code, not just visually + - id: WEB-131 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-safari-voiceover + result: pass + - name: windows-chrome-jaws + result: pass + - name: windows-edge-nvda + result: pass + - name: ios-safari-voiceover + result: pass + - name: android-chrome-talkback + result: pass + + # 1.3.4 - Component available in both portrait and landscape orientations + - id: WEB-134 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: android-chrome + result: pass + - name: ios-safari + result: pass + + # 1.4.1 - Interactivity not conveyed by color alone (chevron provides non-color affordance) + - id: WEB-141 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 1.4.3 - Summary label text contrast (4.5:1) + - id: WEB-143 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 1.4.4 - Text resizes to 200% without clipping or loss of content + - id: WEB-144 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 1.4.10 - Reflow at 400% / 1280px viewport + - id: WEB-1410 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 1.4.11 subtest - Chevron icon contrast (3:1) + - id: VADS-1411-001 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + notes: Even though the chevron is decorative for screen reader users, it is still visually meaningful for sighted users. Therefore it must have sufficient contrast. + + # 1.4.12 - Text spacing doesn't break or clip content + - id: WEB-1412 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 2.1.1 - Summary fully operable by keyboard (Space/Enter to toggle) + - id: WEB-211 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 2.1.2 - Focus doesn't get trapped inside open details + - id: WEB-212 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 2.4.3 - Focus order is logical (summary before slotted content) + - id: WEB-243 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + notes: This has to do with keyboard focus, different than WEB-132 which is about content structure + + # 2.4.7 - Summary has visible focus indicator + - id: WEB-247 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 2.5.2 - Summary toggle executes on pointer up, not pointer down + - id: WEB-252 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + notes: This should always pass with native implementation. But if someone tries to change how the component works, this test may fail. + + # 2.5.3 - Accessible name matches visible label text + - id: WEB-253 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-safari-voice-control + result: pass + - name: windows-edge-voice-recognition + result: pass + + # 2.5.8 - Summary touch target at least 24x24px + - id: WEB-258 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: android-chrome + result: pass + - name: ios-safari + result: pass + + # 3.2.1 - Focusing on summary does not automatically expand/collapse details + - id: WEB-321 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-chrome + result: pass + - name: mac-safari + result: pass + - name: windows-chrome + result: pass + - name: windows-edge + result: pass + + # 4.1.2 subtest - Summary has clear accessible name from label prop + - id: WEB-412-001 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-03 + environments: + - name: mac-safari-voiceover + result: pass + - name: windows-chrome-jaws + result: pass + - name: windows-edge-nvda + result: pass + - name: ios-safari-voiceover + result: pass + notes: Accessible name is derived from the label prop and is announced correctly across all environments including iOS VoiceOver. + + # 4.1.2 subtest - Screen reader announces expanded/collapsed state + - id: WEB-412-002 + test_results: + - version: unreleased (base 56.0.0) + tester: Jeana Clark + date: 2026-03-04 + environments: + - name: mac-safari-voiceover + result: pass + - name: windows-chrome-jaws + result: pass + - name: windows-edge-nvda + result: pass + - name: android-chrome-talkback + result: pass + - name: ios-safari-voiceover + result: pass + notes: iOS VoiceOver prevents native details interaction in the shadow DOM. We are overriding the default details event handling with manual open attribute management to enable functionality in iOS VoiceOver. Additionally, we added a short timeout to prevent the VoiceOver announcement from getting cut off. + + # Implementation-level tests + # 4.1.2 subtest - Summary has clear and unique accessible name from label prop + - id: VADS-412-003 + category: implementation + test_results: + - version: unreleased (base 56.0.0) + notes: Implementation-level test. Teams using this component are responsible for verifying content sequence in context. When there are multiple instances on a page, each has a unique label + + # 1.3.2 - Content is presented in a logical order + - id: WEB-132 + category: implementation + test_results: + - version: unreleased (base 56.0.0) + notes: Implementation-level test. Teams using this component are responsible for verifying content sequence in context. In this test, the Summary is indeed a summary of the content contained within. + + # 2.4.11 - Focused summary not obscured by sticky headers or overlays + - id: WEB-2411 + category: implementation + test_results: + - version: unreleased (base 56.0.0) + notes: Implementation-level test. Teams using this component are responsible for verifying that the details component is not obscured by sticky headers or overlays. + + # 2.4.6 - Summary label text is descriptive of the content it reveals + - id: WEB-246 + category: implementation + test_results: + - version: unreleased (base 56.0.0) + notes: Implementation-level test. Teams using this component are responsible for verifying that the summary label is descriptive of the content it reveals diff --git a/src/_data/accessibility-tests/components/va-statement-of-truth.yml b/src/_data/accessibility-tests/components/va-statement-of-truth.yml new file mode 100644 index 000000000..0bbb7a46b --- /dev/null +++ b/src/_data/accessibility-tests/components/va-statement-of-truth.yml @@ -0,0 +1,290 @@ +component: va-statement-of-truth +component_url: /components/form/statement-of-truth +storybook_url: /storybook/?path=/docs/uswds-va-statement-of-truth--docs + +tests: + # 1.3.1 - Screen readers announce input labels and instructions. + - id: WEB-131-006 + test_results: + - version: + tester: + date: + environments: + - mac-safari-voiceover + - windows-chrome-jaws + - windows-edge-nvda + + # 1.3.1 - Screen readers announce required fields. + - id: WEB-131-007 + test_results: + - version: + tester: + date: + environments: + - mac-safari-voiceover + - windows-chrome-jaws + - windows-edge-nvda + + # 1.3.2 - Content is presented in a logical order. + - id: WEB-132 + test_results: + - version: + tester: + date: + environments: + - + + # 1.3.4 - Content is available in both portrait and landscape orientations. + - id: WEB-134 + test_results: + - version: + tester: + date: + environments: + - android-chrome + - ios-safari + + # 1.3.5 - Form fields support autofill for common information. + - id: WEB-135 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.3 - Text has sufficient contrast against its background. + - id: WEB-143 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.4 - Text resizes to 200% without loss of content or functionality. + - id: WEB-144 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.10 - Content adapts to small screens and zoom. + - id: WEB-1410 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.12 - Text remains readable when spacing is adjusted. + - id: WEB-1412 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.13 - Users can dismiss content that appears on hover or focus. + - id: WEB-1413 + test_results: + - version: + tester: + date: + environments: + - + + # 2.1.2 - Keyboard focus doesn't get trapped. + - id: WEB-212 + test_results: + - version: + tester: + date: + environments: + - + + # 2.4.3 - Tab order follows a logical sequence. + - id: WEB-243 + test_results: + - version: + tester: + date: + environments: + - + + # 2.4.6 - Input labels are descriptive. + - id: WEB-246-002 + test_results: + - version: + tester: + date: + environments: + - + + # 2.4.7 - Interactive elements display a clear focus indicator. + - id: WEB-247 + test_results: + - version: + tester: + date: + environments: + - + + # 2.4.11 - Focused elements are fully visible. + - id: WEB-2411 + test_results: + - version: + tester: + date: + environments: + - + + # 2.5.3 - Accessible name matches visible text. + - id: WEB-253 + test_results: + - version: + tester: + date: + environments: + - + + # 2.5.8 - Interactive element size is at least 24 pixels. + - id: WEB-258 + test_results: + - version: + tester: + date: + environments: + - + + # 3.2.1 - Focusing on an element doesn't trigger unexpected changes. + - id: WEB-321 + test_results: + - version: + tester: + date: + environments: + - + + # 3.2.2 - Changing the input value doesn't trigger unexpected changes. + - id: WEB-322 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.2 - Form inputs display persistent visible labels. + - id: WEB-332-001 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.2 - Form inputs that require specific formats include instructions. + - id: WEB-332-002 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.2 - Form inputs visibly identify required fields. + - id: WEB-332-003 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.3 - Error messages explain how to fix the error. + - id: WEB-333 + test_results: + - version: + tester: + date: + environments: + - + + # 4.1.2 - Every interactive element has a clear name and purpose. + - id: WEB-412 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.1 - Errors are identified on blur. + - id: VADS-331-001 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.1 - Error messages describe the reason for the error. + - id: VADS-331-002 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.1 - Invalid inputs display visible error styles. + - id: VADS-331-003 + test_results: + - version: + tester: + date: + environments: + - + + # 3.3.1 - Screen readers announce errors when they occur. + - id: VADS-331-004 + test_results: + - version: + tester: + date: + environments: + - mac-safari-voiceover + - windows-chrome-jaws + - windows-edge-nvda + + # 3.3.1 - Error messages are visually associated with the invalid element. + - id: VADS-331-005 + test_results: + - version: + tester: + date: + environments: + - + + # 2.1.1 - Form inputs are fully operable using only a keyboard. + - id: VADS-211-002 + test_results: + - version: + tester: + date: + environments: + - + + # 1.4.11 - Form inputs have sufficient contrast against their surroundings. + - id: VADS-1411-002 + test_results: + - version: + tester: + date: + environments: + - diff --git a/src/_data/accessibility-tests/test-library/1-perceivable.yml b/src/_data/accessibility-tests/test-library/1-perceivable.yml new file mode 100644 index 000000000..eef8e6a92 --- /dev/null +++ b/src/_data/accessibility-tests/test-library/1-perceivable.yml @@ -0,0 +1,649 @@ +# WCAG Principle 1: Perceivable +# Information and user interface components must be presentable to users in ways they can perceive +# Tests: WEB-1xx series + +tests: + - id: WEB-111 + category: screen_reader + wcag_criterion: 1.1.1 + wcag_name: Non-text content + wcag_level: A + 508_severity: Critical + governance_severity: Medium-Critical + wcag_url: https://www.w3.org/TR/WCAG22/#non-text-content + description_short: | + Non-text content such as images, audio, and video include equivalent text alternatives. + description_full: | + When you view or interact with non-text content on the page, + you hear a text summary that helps you understand its purpose. + automation_coverage: partial + automation_notes: >- + Parent test. See subtests for specific automated checks. + subtests: + - id: WEB-111-001 + description_short: | + Screen readers announce descriptions for meaningful images. + description_full: | + When you navigate to an informative image with a screen reader, + you hear a description of the image that conveys the content and function of the image. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck 'image-alt', which catches missing or empty alt text on images. Manual + review still needed to determine if the image is meaningful vs. decorative and if alt text + accurately describes the content. + - id: WEB-111-002 + description_short: | + Screen readers announce both brief and detailed descriptions for graphs, maps, and charts. + description_full: | + When you view a complex visual element, such as a chart, graph, or diagram, + you hear both a concise alt text description and a longer description that together provide + all relevant information conveyed by the visual element. + automation_coverage: partial + automation_notes: >- + Custom test needed. Applies when component contains complex visuals like charts, graphs, or + diagrams. Assert aria-describedby or longdesc attribute exists. Manual review still needed for + description completeness and accuracy. + - id: WEB-111-003 + description_short: | + Screen readers don't announce decorative images. + description_full: | + When you navigate to a decorative image with a screen reader, + you don't hear any information about the image. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck 'image-alt', which catches images missing alt handling (empty alt, + aria-hidden, or role='presentation'). Manual review still needed to confirm images are truly + decorative. + - id: WEB-111-004 + description_short: | + Background images don't convey important information. + description_full: | + When you view a background image on a page, + the image is purely decorative and doesn't convey important information or functionality. + automation_coverage: manual + - id: WEB-111-005 + description_short: | + Screen readers announce brief descriptions of videos and audio files. + description_full: | + When you navigate to video or audio content, + you hear a short description that conveys the purpose of the content. + automation_coverage: partial + automation_notes: >- + Custom test needed. Applies when component contains video, audio, or iframe-embedded media. Assert + an accessible name exists via aria-label, aria-labelledby, title, or a wrapping figure with + figcaption. axe-core 'frame-title' catches iframes missing a title attribute, covering embedded + players (YouTube, Vimeo). Manual review still needed for description quality and screen reader + announcement. + - id: WEB-121 + category: general + wcag_criterion: 1.2.1 + wcag_name: Audio-only and Video-only (Prerecorded) + wcag_level: A + 508_severity: Critical + governance_severity: Critical + wcag_url: https://www.w3.org/TR/WCAG22/#audio-only-and-video-only-prerecorded + description_short: | + Audio and video-only content include a text transcript. + description_full: | + When you interact with audio-only or video-only media, + you can find a transcript that provides the same information as presented in the original media content. + automation_coverage: manual + - id: WEB-122 + category: general + wcag_criterion: 1.2.2 + wcag_name: Captions (Prerecorded) + wcag_level: A + 508_severity: Critical + governance_severity: Critical + wcag_url: https://www.w3.org/TR/WCAG22/#captions-prerecorded + description_short: | + Prerecorded videos provide captions. + description_full: | + When you view prerecorded videos, + you find synchronized captions for dialogue, sound effects, and relevant audio. + automation_coverage: manual + - id: WEB-123 + category: general + wcag_criterion: 1.2.3 + wcag_name: Audio Description or Media Alternative (Prerecorded) + wcag_level: A + 508_severity: High + governance_severity: High + wcag_url: https://www.w3.org/TR/WCAG22/#audio-description-or-media-alternative-prerecorded + description_short: | + Videos with audio include transcripts or audio descriptions. + description_full: | + When you interact with video content, + you have access to a full descriptive transcript or an audio description. + automation_coverage: manual + - id: WEB-124 + category: general + wcag_criterion: 1.2.4 + wcag_name: Captions (Live) + wcag_level: AA + 508_severity: Critical + governance_severity: Critical + wcag_url: https://www.w3.org/TR/WCAG22/#captions-live + description_short: | + Live videos include real-time captions. + description_full: | + When you interact with live video content, + you find synchronized captions generated in real-time. + automation_coverage: manual + - id: WEB-125 + category: general + wcag_criterion: 1.2.5 + wcag_name: Audio Description (Prerecorded) + wcag_level: AA + 508_severity: High + governance_severity: High + wcag_url: https://www.w3.org/TR/WCAG22/#audio-description-prerecorded + description_short: | + Videos with important visual information include audio descriptions. + description_full: | + When you watch a video that includes important visual information, + you can turn on an audio description that conveys that visual information. + automation_coverage: manual + - id: WEB-131 + category: general + wcag_criterion: 1.3.1 + wcag_name: Info and Relationships + wcag_level: A + 508_severity: Critical + governance_severity: Medium-Critical + wcag_url: https://www.w3.org/TR/WCAG22/#info-and-relationships + description_short: | + Information, structure, and relationships are conveyed through code, not just presentation. + description_full: | + When you interact with the page, + you find that information, structure, and relationships are conveyed through code, not just visual presentation. + automation_coverage: partial + automation_notes: >- + Parent test. See subtests for specific automated checks. + subtests: + - id: WEB-131-001 + category: screen_reader + description_short: | + Headings are used to convey structure. + description_full: | + When you use a screen reader to navigate the page by heading, + you find that all elements that act as content headings are included in the heading structure. + automation_coverage: manual + - id: WEB-131-002 + description_short: | + Headings follow a logical order. + description_full: | + When you navigate the page by headings, + you find that heading levels follow a logical order and don't skip levels. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck 'heading-order', which catches skipped heading levels (e.g., h2 to h4 + without h3). Manual review still needed to verify headings are logically ordered in the + context of the full page. + - id: WEB-131-003 + description_short: | + Page includes one H1. + description_full: | + When you inspect the page, + you find a single H1 exists for the page or screen. + automation_coverage: full + automation_notes: >- + Custom page-level test needed. No component-level test needed. Use + querySelectorAll('h1') and assert exactly one h1 exists. axe-core 'page-has-heading-one' is + disabled in the component-library axeCheck helper because components are tested in isolation. + - id: WEB-131-004 + description_short: | + Screen readers announce list items. + description_full: | + When you use a screen reader to navigate a list, + you hear an announcement that indicates the number of items in the list and the position of each item within the list. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck 'list, listitem', which catches invalid list structure (e.g., li outside ul/ol) + but can't verify screen readers announce item count and position. + Applies when component contains list content. + Assert ul/ol elements contain li children and no orphaned li elements exist. + Manual review still needed to verify screen readers announce list count and item position correctly. + - id: WEB-131-005 + category: screen_reader + description_short: | + Screen readers announce related form elements as a group. + description_full: | + When you use a screen reader to interact with a group of related form elements, such as a set of radio buttons, checkboxes or grouped text inputs, + you hear an announcement that includes the group label, the item label, and indicates the elements are related. + automation_coverage: partial + automation_notes: >- + Custom test recommended. axe-core 'fieldset-no-legend' catches fieldset elements missing a + legend but can't verify screen readers announce group labels and relationships. Applies when + component groups related form elements (radio buttons, checkboxes, grouped inputs). Assert + fieldset has legend, or role='group' with aria-labelledby. For radio groups, verify all radio + inputs within the group share the same name attribute. Manual review still needed to verify + screen readers announce group label and relationship. + - id: WEB-131-006 + category: screen_reader + description_short: | + Screen readers announce input labels and instructions. + description_full: | + When you interact with the form input with a screen reader, + you hear an announcement that includes the label, hint text, required state, and error message when applicable. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck 'label', which catches inputs missing accessible labels. + Applies when component contains form inputs. + Assert input has associated label via for/id or aria-labelledby. + Check label includes hint text and required state. + Manual review still needed to verify screen reader announces complete label including hints. + - id: WEB-131-007 + category: screen_reader + description_short: | + Screen readers announce required fields. + description_full: | + When you interact with required fields using a screen reader, + you hear an announcement indicating that the field is required. + automation_coverage: partial + automation_notes: >- + Custom test recommended. axe-core 'aria-required-attr' catches missing required ARIA attributes + but can't verify screen readers consistently announce 'required' across assistive technologies. + Applies when component contains required form fields. Assert required attribute or + aria-required='true' is present when required prop is set. Manual review still needed to verify + screen readers announce 'required' consistently across NVDA, JAWS, and VoiceOver. + - id: WEB-131-008 + category: screen_reader + description_short: | + Screen reader users can navigate by landmarks. + description_full: | + When you use a screen reader to navigate the page by landmarks, + you find that you can easily identify and navigate to key sections of the page, such as the main content, navigation, and complementary content. + automation_coverage: partial + automation_notes: >- + Custom test needed. For components that serve a landmark purpose (e.g., navigation, header, + footer), assert the component uses the correct landmark element or role. For page-level + testing, assert the complete set of landmark roles (main, navigation, complementary, banner, + contentinfo) exist for key page sections. Manual review still needed to verify screen reader + landmark navigation works correctly. axe-core 'landmark-one-main, region' is disabled in the + component-library axeCheck helper because components are tested in isolation. + - id: WEB-131-009 + category: screen_reader + description_short: | + Screen readers announce relationship between table headers and data cells. + description_full: | + When you use a screen reader to interact with a table, + you hear announcements that include table headers and associations between headers and data cells, + allowing you to understand the structure and content of the table. + automation_coverage: partial + automation_notes: >- + Covered by axeCheck. Fully validates that table headers are properly associated with data cells via + scope and headers attributes. (axe rules: th-has-data-cells, td-headers-attr). + Manual review still needed to verify screen readers announce header-cell relationships correctly, + especially for complex tables with multi-level headers. + - id: WEB-131-010 + category: screen_reader + description_short: | + Screen readers announce relationships between parent and child elements. + description_full: | + When you use a screen reader to navigate elements with parent-child relationships, + you hear announcements that correctly identify the relationship structure and required elements. + automation_coverage: partial + automation_notes: >- + Custom tests needed. Applies to all components with required parent/child markup. + Assert proper semantic roles and containment (e.g., ul contains li, role='menu' contains + role='menuitem'). Manual review still needed to verify + screen readers announce relationships correctly. + - id: VADS-131-001 + category: screen_reader + description_short: | + Screen readers identify navigation menus and announce their purpose. + description_full: | + When you use a screen reader to navigate a menu (such as breadcrumbs, a site menu, or page navigation), + you hear an announcement that tells you it's a navigation region and describes what kind of navigation it provides. + automation_coverage: partial + automation_notes: >- + Custom test needed. Applies when component serves a navigation purpose (e.g., breadcrumbs, + side navigation, pagination, step indicators, table of contents). Assert component uses +

+ WCAG {{ criterion }} - {{ criterion_test.wcag_name }} +

+ +
    + {% for test in test_library %} + {% assign test_id_parts = test.id | split: "-" %} + {% comment %} Only display parent tests (2-part IDs) - subtests are nested under them {% endcomment %} + {% if test_id_parts.size == 2 and test.wcag_criterion == criterion %} +
  • +

    + {{ test.description_short | strip }} +

    +

    + {{ test.description_full | strip }} +

    + + + {% comment %} Display subtests nested under this parent test {% endcomment %} + {% if test.subtests %} +
      + {% for subtest in test.subtests %} + {% assign subtest_wcag_criterion = subtest.wcag_criterion | default: test.wcag_criterion %} + {% if subtest_wcag_criterion == criterion %} +
    • +

      + {{ subtest.description_short | strip }} +

      +

      + {{ subtest.description_full | strip }} +

      +
        +
      • + {{ subtest.id }} +
        • +
      +
      • + {% endif %} + {% endfor %} +
    + {% endif %} +
    • + {% endif %} + {% endfor %} +
+{% endfor %} + +

+ Total tests in library: {{ test_library.size }} +

diff --git a/src/_includes/accessibility-test-library-loader.html b/src/_includes/accessibility-test-library-loader.html new file mode 100644 index 000000000..f35468a58 --- /dev/null +++ b/src/_includes/accessibility-test-library-loader.html @@ -0,0 +1,15 @@ + +{% comment %} + Loads all WCAG principle-based test files and merges into single test_library array. + Also loads required CSS for accessibility test displays. + Usage: {% include accessibility-test-library-loader.html %} + Output: Sets test_library variable with all tests from 4 WCAG principle files +{% endcomment %} +{% assign test_library = "" | split: "" %} +{% assign perceivable_tests = site.data.accessibility_tests.test-library["1-perceivable"].tests %} +{% assign operable_tests = site.data.accessibility_tests.test-library["2-operable"].tests %} +{% assign understandable_tests = site.data.accessibility_tests.test-library["3-understandable"].tests %} +{% assign robust_tests = site.data.accessibility_tests.test-library["4-robust"].tests %} +{% assign test_library = test_library | concat: perceivable_tests | concat: operable_tests | concat: understandable_tests | concat: robust_tests %} + + diff --git a/src/_includes/accessibility-test-lookup.html b/src/_includes/accessibility-test-lookup.html new file mode 100644 index 000000000..ed50fdafd --- /dev/null +++ b/src/_includes/accessibility-test-lookup.html @@ -0,0 +1,24 @@ +{% comment %} + Looks up a test definition from the test library by ID. + Handles both top-level tests and subtests nested under a parent. + + Input: include.test_id — the test ID to look up + test_library — must already be set (via accessibility-test-library-loader.html) + Output: test_def — the matched test definition (or nil) + parent_def — the parent test definition if this is a subtest (or nil) +{% endcomment %} +{% assign test_def = test_library | where: "id", include.test_id | first %} +{% assign parent_def = nil %} + +{% unless test_def %} + {% for parent in test_library %} + {% if parent.subtests %} + {% assign subtest = parent.subtests | where: "id", include.test_id | first %} + {% if subtest %} + {% assign test_def = subtest %} + {% assign parent_def = parent %} + {% break %} + {% endif %} + {% endif %} + {% endfor %} +{% endunless %} diff --git a/src/_includes/accessibility-test-results.html b/src/_includes/accessibility-test-results.html new file mode 100644 index 000000000..b92f1fe40 --- /dev/null +++ b/src/_includes/accessibility-test-results.html @@ -0,0 +1,305 @@ +{% assign component_name = include.component_name %} +{% assign component_tests = site.data.accessibility_tests[component_name] %} + +{% include accessibility-test-library-loader.html %} + +{% if component_tests %} + +{% assign keyboard_tests = "" | split: "" %} +{% assign screen_reader_tests = "" | split: "" %} +{% assign general_tests = "" | split: "" %} +{% assign voice_control_tests = "" | split: "" %} +{% assign structural_tests = "" | split: "" %} +{% assign mobile_tests = "" | split: "" %} +{% assign zoom_tests = "" | split: "" %} +{% assign visual_tests = "" | split: "" %} +{% assign implementation_tests = "" | split: "" %} + +{% comment %} Calculate summary statistics {% endcomment %} +{% assign total_tests = 0 %} +{% assign passed_count = 0 %} +{% assign failed_count = 0 %} +{% assign conditional_count = 0 %} +{% assign untested_count = 0 %} +{% assign latest_version = "" %} + +{% comment %} Categorize tests and calculate statistics {% endcomment %} +{% for test in component_tests.tests %} + {% assign total_tests = total_tests | plus: 1 %} + + {% include accessibility-test-lookup.html test_id=test.id %} + + {% if test_def %} + {% comment %} Determine category (check override, then subtest, then parent) {% endcomment %} + {% assign test_category = test.category | default: test_def.category %} + {% if parent_def and test_category == nil %} + {% assign test_category = parent_def.category %} + {% endif %} + + {% comment %} Add to appropriate category array {% endcomment %} + {% case test_category %} + {% when "general" %}{% assign general_tests = general_tests | push: test %} + {% when "keyboard" %}{% assign keyboard_tests = keyboard_tests | push: test %} + {% when "mobile" %}{% assign mobile_tests = mobile_tests | push: test %} + {% when "screen_reader" %}{% assign screen_reader_tests = screen_reader_tests | push: test %} + {% when "structural" %}{% assign structural_tests = structural_tests | push: test %} + {% when "visual" %}{% assign visual_tests = visual_tests | push: test %} + {% when "voice_control" %}{% assign voice_control_tests = voice_control_tests | push: test %} + {% when "zoom" %}{% assign zoom_tests = zoom_tests | push: test %} + {% when "implementation" %}{% assign implementation_tests = implementation_tests | push: test %} + {% endcase %} + {% endif %} + + {% comment %} Calculate result counts and find latest version {% endcomment %} + {% if test.test_results and test.test_results.size > 0 %} + {% assign has_pass = false %} + {% assign has_fail = false %} + {% assign has_conditional = false %} + + {% for run in test.test_results %} + {% comment %} Track latest version {% endcomment %} + {% if latest_version == "" or run.version > latest_version %} + {% assign latest_version = run.version %} + {% endif %} + + {% comment %} Check all environments for this test run {% endcomment %} + {% for env in run.environments %} + {% if env.result == "pass" %} + {% assign has_pass = true %} + {% elsif env.result == "fail" %} + {% assign has_fail = true %} + {% elsif env.result == "conditional" %} + {% assign has_conditional = true %} + {% endif %} + {% endfor %} + {% endfor %} + + {% comment %} Count test based on worst result {% endcomment %} + {% if has_fail %} + {% assign failed_count = failed_count | plus: 1 %} + {% elsif has_conditional %} + {% assign conditional_count = conditional_count | plus: 1 %} + {% elsif has_pass %} + {% assign passed_count = passed_count | plus: 1 %} + {% else %} + {% assign untested_count = untested_count | plus: 1 %} + {% endif %} + {% else %} + {% assign untested_count = untested_count | plus: 1 %} + {% endif %} +{% endfor %} + +{% comment %} Render categorized tests {% endcomment %} +{% if component_tests.size > 0 %} + +{% comment %} Summary Dashboard {% endcomment %} +{% assign display_component_name = component_name | replace: "va-", "" | replace: "-", " " | capitalize %} + +{% assign context_dependent_count = conditional_count | plus: implementation_tests.size %} + + + +

Accessibility test sections are a newer addition to the VA Design System and are being added to components over time. Not all components have been evaluated using this rubric yet. These tests are based on the and establish a baseline for expected component behavior.

+ + + + Version tested + Passed + Failed + Conditional + + + {{ latest_version }} + {{ passed_count }} + {{ failed_count }} + {{ context_dependent_count }} + + + +{% comment %} Define test categories in display order {% endcomment %} +{% assign test_categories = "general_tests,General tests|keyboard_tests,Keyboard tests|screen_reader_tests,Screen reader tests|voice_control_tests,Voice control tests|mobile_tests,Mobile tests|zoom_tests,Zoom tests|structural_tests,Structural tests|visual_tests,Visual tests|implementation_tests,Conditional tests" | split: "|" %} + + +{% for category_config in test_categories %} + {% assign config_parts = category_config | split: "," %} + {% assign category_var = config_parts[0] %} + {% assign category_title = config_parts[1] %} + + {% comment %} Map category variable name to test array {% endcomment %} + {% case category_var %} + {% when "general_tests" %}{% assign category_tests = general_tests %} + {% when "keyboard_tests" %}{% assign category_tests = keyboard_tests %} + {% when "screen_reader_tests" %}{% assign category_tests = screen_reader_tests %} + {% when "implementation_tests" %}{% assign category_tests = implementation_tests %} + {% when "voice_control_tests" %}{% assign category_tests = voice_control_tests %} + {% when "mobile_tests" %}{% assign category_tests = mobile_tests %} + {% when "zoom_tests" %}{% assign category_tests = zoom_tests %} + {% when "structural_tests" %}{% assign category_tests = structural_tests %} + {% when "visual_tests" %}{% assign category_tests = visual_tests %} + {% endcase %} + + {% if category_tests.size > 0 %} + + + {% if category_var == "implementation_tests" %} +

Teams are responsible for verifying these tests in their own context.

+ {% endif %} +
    + {% for test in category_tests %} + {% include accessibility-test-lookup.html test_id=test.id %} + + {% if test_def %} + {% comment %} Get WCAG metadata (use parent values if this is a subtest) {% endcomment %} + {% assign wcag_source = parent_def | default: test_def %} +
  • +

    + {{ test_def.description_short | strip }} +

    +

    + {{ test_def.description_full | strip }} +

    + + + + {% comment %} Test Results Table {% endcomment %} + {% assign test_runs = test.test_results %} + + {% comment %} Get category for this test {% endcomment %} + {% if parent_def %} + {% assign test_category = test_def.category | default: parent_def.category %} + {% else %} + {% assign test_category = test_def.category %} + {% endif %} + {% assign category_envs = site.data.accessibility_tests.test-library._config.environment_definitions[test_category] %} + + {% comment %} Build environment list: always include required environments, then add any from test results {% endcomment %} + {% assign all_environments = "" | split: "" %} + + {% comment %} Step 1: Add required environments from category config {% endcomment %} + {% if category_envs %} + {% for env in category_envs %} + {% if env.required %} + {% assign all_environments = all_environments | push: env.id %} + {% endif %} + {% endfor %} + {% endif %} + + {% comment %} Step 2: Merge in any additional environments from test results {% endcomment %} + {% if test_runs and test_runs.size > 0 %} + {% for run in test_runs %} + {% for env in run.environments %} + {% assign env_exists = false %} + {% for existing_env in all_environments %} + {% if existing_env == env.name %} + {% assign env_exists = true %} + {% break %} + {% endif %} + {% endfor %} + {% unless env_exists or env.name == blank %} + {% assign all_environments = all_environments | push: env.name %} + {% endunless %} + {% endfor %} + {% endfor %} + {% endif %} + + + + Environment + Result + Last tested + + {% if all_environments.size > 0 %} + {% comment %} Display environment rows - either with results or as untested {% endcomment %} + {% for env_id in all_environments %} + {% include accessibility-test-env-lookup.html env_id=env_id %} + + {% comment %} Find the latest result for this environment {% endcomment %} + {% assign latest_env_result = nil %} + {% assign latest_env_version = "" %} + {% assign latest_env_date = nil %} + {% if test_runs and test_runs.size > 0 %} + {% for run in test_runs %} + {% assign matching_env = run.environments | where: "name", env_id | first %} + {% if matching_env and run.version > latest_env_version %} + {% assign latest_env_result = matching_env.result %} + {% assign latest_env_version = run.version %} + {% assign latest_env_date = run.date %} + {% endif %} + {% endfor %} + {% endif %} + + {% if latest_env_version != "" %} + + {{ env_display_name }} + + {% if latest_env_result == "pass" %} + + {% elsif latest_env_result == "fail" %} + + {% elsif latest_env_result == "conditional" %} + + {% else %} + Untested + {% endif %} + + + {% if latest_env_date %}{{ latest_env_date | date: "%B %-d, %Y" }}
    {% endif %} + Version: {{ latest_env_version }} +     +     + {% else %} + {% comment %} No test results - show environment as untested {% endcomment %} + + {{ env_display_name }} + Untested + — + + {% endif %} + {% endfor %} + {% else %} + {% comment %} No environments defined at all {% endcomment %} + + No test results yet + + {% endif %} +     + {% comment %} Display notes separately if they exist {% endcomment %} + {% if test_runs %} + {% for run in test_runs %} + {% if run.notes %} +

    + Notes (Version {{ run.version }}): {{ run.notes }} +

    + {% endif %} + {% endfor %} + {% endif %} +
    • + {% endif %} + {% endfor %} +
+ + {% endif %} +{% endfor %} + +{% endif %} +{% else %} +

No accessibility test results available for the {{ component_name }} component.

+{% endif %} diff --git a/src/assets/stylesheets/_components/_accessibility-tests.scss b/src/assets/stylesheets/_components/_accessibility-tests.scss new file mode 100644 index 000000000..d3d7c3a54 --- /dev/null +++ b/src/assets/stylesheets/_components/_accessibility-tests.scss @@ -0,0 +1,100 @@ +@use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/stylesheets/functions' as *; +@use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/tokens/scss/variables' as *; + +// Accessibility Test Display Styles +// Used in component test results and test library pages + +// Main test list container +.a11y-test-list { + list-style: none; + margin: 0 0 units(3) 0; + padding: units(1) 0 0; + + > li { + border-bottom: 1px solid var(--vads-color-base-light); + padding: units(1.5) 0 units(1); + max-width: 100%; + + &:first-of-type { + padding-top: 0; + } + + &:last-of-type { + border: none; + padding-bottom: 0; + } + + > p { + margin-block: 0; + max-width: 77ch; + + &.a11y-subtest-tag { + margin-top: units(0.5); + } + } + } + + // Nested subtests + .a11y-test-subtest-list { + list-style: none; + margin: units(1) 0 0 units(1); + padding: 0; + border-left: 2px solid var(--vads-color-base-lighter); + + > li { + border-bottom: none; + padding: units(1) 0 units(1) units(1); + border-top: 1px solid var(--vads-color-base-lighter); + + &:first-child { + border-top: none; + padding-top: units(0.5); + } + } + } + + // Accessibility test results table styles + va-table-row { + display: none; + } +} + +// Test details metadata list +.a11y-test-details-list { + list-style: none; + display: flex; + flex-direction: row; + margin: units(0.5) 0; + padding: 0; + flex-wrap: wrap; + + li { + margin-right: units(1); + } +} + +// Table header with icons +.a11y-table-header { + display: flex; + align-items: center; + gap: units(0.25); +} + +// Icon colors for test results +.a11y-icon-success { + color: var(--vads-color-success-darker); +} + +.a11y-icon-info { + color: var(--vads-color-info-darker); +} + +.a11y-icon-error { + color: var(--vads-color-error-dark); +} + +// WCAG criterion headers +h3.a11y-wcag-criterion-header { + border-bottom: 2px solid var(--vads-color-base-lighter); + padding-bottom: units(0.5); +} diff --git a/src/assets/stylesheets/application.scss b/src/assets/stylesheets/application.scss index dd2be4a3e..370710b7a 100644 --- a/src/assets/stylesheets/application.scss +++ b/src/assets/stylesheets/application.scss @@ -42,6 +42,7 @@ front-matter: yay @use '_layout/site-content'; @use '_layout/content-wrappers'; @use '_layout/metrics'; +@use '_components/accessibility-tests'; @use '_components/anchors'; @use '_components/badge'; @use '_components/code-snippets'; From 8b2d53bb9fae7d7579a868dd4927bf9ef95ad2e0 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 14:21:59 -0700 Subject: [PATCH 02/13] Update accessibility_tests to accessibility-tests --- .../test-selection/SKILL.md | 28 +++---- .../writing-entries/SKILL.md | 12 +-- ...lity-tests-add-to-component-page.prompt.md | 6 +- .../accessibility-tests-audit.prompt.md | 16 ++-- ...y-tests-generate-component-tests.prompt.md | 14 ++-- ...sibility-tests-generate-metadata.prompt.md | 14 ++-- src/_data/accessibility-tests/README.md | 2 +- .../accessibility-tests/components/README.md | 81 +++++++++++++++++-- .../accessibility-test-env-lookup.html | 2 +- .../accessibility-test-library-loader.html | 8 +- src/_includes/accessibility-test-results.html | 4 +- 11 files changed, 128 insertions(+), 59 deletions(-) diff --git a/.claude/skills/accessibility-tests/test-selection/SKILL.md b/.claude/skills/accessibility-tests/test-selection/SKILL.md index b97b78064..edcecd52b 100644 --- a/.claude/skills/accessibility-tests/test-selection/SKILL.md +++ b/.claude/skills/accessibility-tests/test-selection/SKILL.md @@ -11,7 +11,7 @@ This skill helps identify which accessibility tests from the VA Design System te All test selection rules, subtest priority guidelines, and component-type checklists are documented in: -**`src/_data/accessibility_tests/components/README.md`** +**`src/_data/accessibility-tests/components/README.md`** **Required reading sections:** - **Test Selection Guidelines** → **Subtest Preference Rule** — Rules for preferring specific subtests over parent tests, how to find subtests in YAML files @@ -23,18 +23,18 @@ All test selection rules, subtest priority guidelines, and component-type checkl Read these to understand test definitions, IDs, and configuration: -- **`src/_data/accessibility_tests/test-library/README.md`** +- **`src/_data/accessibility-tests/test-library/README.md`** - Test ID format and structure - WCAG mappings - Test categories and required environments - How to read test entries -- **`src/_data/accessibility_tests/test-library/_config.yml`** +- **`src/_data/accessibility-tests/test-library/_config.yml`** - Environment ID definitions (see `environments:` section) - Category definitions (see `categories:` section) - Required environments per category (e.g., `screen_reader`, `mobile`) -- **`src/_data/accessibility_tests/README.md`** +- **`src/_data/accessibility-tests/README.md`** - High-level overview of the test library - Directory structure - Links to related documentation @@ -43,25 +43,25 @@ Read these to understand test definitions, IDs, and configuration: Tests are organized by WCAG principle: -- `src/_data/accessibility_tests/test-library/1-perceivable.yml` -- `src/_data/accessibility_tests/test-library/2-operable.yml` -- `src/_data/accessibility_tests/test-library/3-understandable.yml` -- `src/_data/accessibility_tests/test-library/4-robust.yml` +- `src/_data/accessibility-tests/test-library/1-perceivable.yml` +- `src/_data/accessibility-tests/test-library/2-operable.yml` +- `src/_data/accessibility-tests/test-library/3-understandable.yml` +- `src/_data/accessibility-tests/test-library/4-robust.yml` **To find subtests:** Open these YAML files and look for `subtests:` arrays under each parent test entry. ### Example Component Test Files Reference these for patterns: -- `src/_data/accessibility_tests/components/va-link.yml` — Simple interactive component -- `src/_data/accessibility_tests/components/va-text-input.yml` — Form input -- `src/_data/accessibility_tests/components/va-accordion.yml` — Dynamic content +- `src/_data/accessibility-tests/components/va-link.yml` — Simple interactive component +- `src/_data/accessibility-tests/components/va-text-input.yml` — Form input +- `src/_data/accessibility-tests/components/va-accordion.yml` — Dynamic content ## Agent workflow When invoked to select tests for a component: -1. **Read the components README** at `src/_data/accessibility_tests/components/README.md` +1. **Read the components README** at `src/_data/accessibility-tests/components/README.md` 2. **Follow the "Test Selection Guidelines" section**, which includes: - How to analyze your component @@ -90,12 +90,12 @@ When invoked to select tests for a component: ## File Paths Quick Reference ### Component Test Files -- **Directory**: `src/_data/accessibility_tests/components/` +- **Directory**: `src/_data/accessibility-tests/components/` - **Pattern**: `va-[component-name].yml` - **Examples**: `va-button.yml`, `va-text-input.yml`, `va-accordion.yml` ### Test Library Files -- **Directory**: `src/_data/accessibility_tests/test-library/` +- **Directory**: `src/_data/accessibility-tests/test-library/` - **Test definitions**: - `1-perceivable.yml` (WEB-1xx tests) - `2-operable.yml` (WEB-2xx tests) diff --git a/.claude/skills/accessibility-tests/writing-entries/SKILL.md b/.claude/skills/accessibility-tests/writing-entries/SKILL.md index c6c40463d..5cab5142d 100644 --- a/.claude/skills/accessibility-tests/writing-entries/SKILL.md +++ b/.claude/skills/accessibility-tests/writing-entries/SKILL.md @@ -8,17 +8,17 @@ description: Guidelines for writing accessibility test library entries including **Use this skill when:** Creating new test entries, editing existing test descriptions, or reviewing test library content for consistency. **Test library files:** -- `src/_data/accessibility_tests/test-library/1-perceivable.yml` -- `src/_data/accessibility_tests/test-library/2-operable.yml` -- `src/_data/accessibility_tests/test-library/3-understandable.yml` -- `src/_data/accessibility_tests/test-library/4-robust.yml` -- `src/_data/accessibility_tests/test-library/_config.yml` +- `src/_data/accessibility-tests/test-library/1-perceivable.yml` +- `src/_data/accessibility-tests/test-library/2-operable.yml` +- `src/_data/accessibility-tests/test-library/3-understandable.yml` +- `src/_data/accessibility-tests/test-library/4-robust.yml` +- `src/_data/accessibility-tests/test-library/_config.yml` ## Complete Guidelines All guidelines for writing and editing test library entries live in: -**`src/_data/accessibility_tests/test-library/README.md`** +**`src/_data/accessibility-tests/test-library/README.md`** Read the following sections before creating or editing entries: diff --git a/.github/prompts/accessibility-tests-add-to-component-page.prompt.md b/.github/prompts/accessibility-tests-add-to-component-page.prompt.md index e42533f84..ac960f625 100644 --- a/.github/prompts/accessibility-tests-add-to-component-page.prompt.md +++ b/.github/prompts/accessibility-tests-add-to-component-page.prompt.md @@ -11,7 +11,7 @@ Ask: "Which component would you like to add accessibility tests to? Please provi **Validate:** - Component name must start with `va-` - Component name must be lowercase kebab-case -- Verify the component test file exists at `src/_data/accessibility_tests/components/[component-name].yml` +- Verify the component test file exists at `src/_data/accessibility-tests/components/[component-name].yml` - If test file doesn't exist, inform the user and suggest running `/accessibility-tests-generate-component-tests` first ### 2. Read the component documentation file @@ -62,6 +62,6 @@ Added accessibility tests section to [component-name].md: - **Generate Test File Prompt**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` - **Update Metadata Prompt**: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` -- **Components README**: `src/_data/accessibility_tests/components/README.md#step-5-display-results-on-component-page` (detailed placement guidance) -- **Component test files**: `src/_data/accessibility_tests/components/va-*.yml` +- **Components README**: `src/_data/accessibility-tests/components/README.md#step-5-display-results-on-component-page` (detailed placement guidance) +- **Component test files**: `src/_data/accessibility-tests/components/va-*.yml` - **Component documentation**: `src/_components/*.md` diff --git a/.github/prompts/accessibility-tests-audit.prompt.md b/.github/prompts/accessibility-tests-audit.prompt.md index c94cb0b81..b590a44d3 100644 --- a/.github/prompts/accessibility-tests-audit.prompt.md +++ b/.github/prompts/accessibility-tests-audit.prompt.md @@ -18,12 +18,12 @@ When this prompt is invoked: - Options: - "Single file: `va-[component-name]`" (e.g., `va-button`) - "Multiple files: `va-component1, va-component2, va-component3`" (comma-separated) - - "All files: all" (audits all `src/_data/accessibility_tests/components/va-*.yml` files) + - "All files: all" (audits all `src/_data/accessibility-tests/components/va-*.yml` files) 2. **Wait for user response** and determine files to audit: - - Single file: `src/_data/accessibility_tests/components/va-[component-name].yml` - - Multiple files: Parse comma-separated list, prepend `src/_data/accessibility_tests/components/` and append `.yml` - - All files: Use file search for `src/_data/accessibility_tests/components/va-*.yml` + - Single file: `src/_data/accessibility-tests/components/va-[component-name].yml` + - Multiple files: Parse comma-separated list, prepend `src/_data/accessibility-tests/components/` and append `.yml` + - All files: Use file search for `src/_data/accessibility-tests/components/va-*.yml` 3. **For each file, perform these checks**: @@ -37,13 +37,13 @@ When this prompt is invoked: - Identify child VA web components used (e.g., ``, ``) **C. Check child component inheritance** - - For each child component found, check if test file exists at `src/_data/accessibility_tests/components/[child-component].yml` + - For each child component found, check if test file exists at `src/_data/accessibility-tests/components/[child-component].yml` - Compare child component tests with parent component tests - Flag if parent is missing tests that child components have **D. Invoke accessibility-test-selection skill** - Read `.claude/skills/accessibility-tests/test-selection/SKILL.md` for the agent workflow - - The skill references `src/_data/accessibility_tests/components/README.md` for test selection rules + - The skill references `src/_data/accessibility-tests/components/README.md` for test selection rules - Determine which tests should apply based on component features - Combine with tests from child components - Compare expected tests with actual tests in file @@ -154,9 +154,9 @@ When this prompt is invoked: ## Resources -- **Test Library**: `src/_data/accessibility_tests/test-library/` +- **Test Library**: `src/_data/accessibility-tests/test-library/` - **Accessibility Test Selection Skill**: `.claude/skills/accessibility-tests/test-selection/SKILL.md` -- **Component Test Files**: `src/_data/accessibility_tests/components/va-*.yml` +- **Component Test Files**: `src/_data/accessibility-tests/components/va-*.yml` - **GitHub Component Library**: `https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/` ## Notes diff --git a/.github/prompts/accessibility-tests-generate-component-tests.prompt.md b/.github/prompts/accessibility-tests-generate-component-tests.prompt.md index de60c6b6e..5640b567f 100644 --- a/.github/prompts/accessibility-tests-generate-component-tests.prompt.md +++ b/.github/prompts/accessibility-tests-generate-component-tests.prompt.md @@ -74,7 +74,7 @@ Create the component test file with this structure: - Environments array properly formatted (required environments for screen_reader/mobile tests, single empty item for others) - 2-space indentation used throughout -**Save to:** `src/_data/accessibility_tests/components/[component-name].yml` +**Save to:** `src/_data/accessibility-tests/components/[component-name].yml` ### 6. Offer to add to component page @@ -124,9 +124,9 @@ tests: ``` **Example files for reference:** -- `src/_data/accessibility_tests/components/va-link.yml` -- `src/_data/accessibility_tests/components/va-text-input.yml` -- `src/_data/accessibility_tests/components/va-accordion.yml` +- `src/_data/accessibility-tests/components/va-link.yml` +- `src/_data/accessibility-tests/components/va-text-input.yml` +- `src/_data/accessibility-tests/components/va-accordion.yml` ## Next Steps @@ -137,10 +137,10 @@ After creating the test file: ## Resources -- **Components README**: `src/_data/accessibility_tests/components/README.md` +- **Components README**: `src/_data/accessibility-tests/components/README.md` - **Test Selection Skill**: `.claude/skills/accessibility-tests/test-selection/SKILL.md` - **Update Metadata Prompt**: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` - **Add to Component Page Prompt**: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` -- **Test Library**: `src/_data/accessibility_tests/test-library/` -- **Test Library Config**: `src/_data/accessibility_tests/test-library/_config.yml` +- **Test Library**: `src/_data/accessibility-tests/test-library/` +- **Test Library Config**: `src/_data/accessibility-tests/test-library/_config.yml` - **Component Library Source**: `https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/` diff --git a/.github/prompts/accessibility-tests-generate-metadata.prompt.md b/.github/prompts/accessibility-tests-generate-metadata.prompt.md index e8b03d1ff..368eb0bc2 100644 --- a/.github/prompts/accessibility-tests-generate-metadata.prompt.md +++ b/.github/prompts/accessibility-tests-generate-metadata.prompt.md @@ -20,12 +20,12 @@ Ask: "Which component would you like to update? Please provide the component nam **Validate:** - Component name must start with `va-` - Component name must be lowercase kebab-case -- Check that the file exists at `src/_data/accessibility_tests/components/[component-name].yml` +- Check that the file exists at `src/_data/accessibility-tests/components/[component-name].yml` - If the file doesn't exist, inform the user and suggest using `/accessibility-tests-generate-component-tests` first ### 2. Read existing file -1. Open `src/_data/accessibility_tests/components/[component-name].yml` +1. Open `src/_data/accessibility-tests/components/[component-name].yml` 2. Check if tests already have `test_results` entries 3. Parse the component metadata (component_url, storybook_url) 4. Get the list of test IDs @@ -60,7 +60,7 @@ For each test in the file: - `date`: From user input 3. Add an `environments` array: - **If the test has `screen_reader` or `mobile` category**: - - Read `src/_data/accessibility_tests/test-library/_config.yml` + - Read `src/_data/accessibility-tests/test-library/_config.yml` - Find the category's `required_environments` - Pre-populate the environments array with those environment IDs - Each environment gets an empty `result:` field @@ -75,7 +75,7 @@ For each test in the file: ### 6. Save the updated file -Write the updated YAML back to `src/_data/accessibility_tests/components/[component-name].yml` +Write the updated YAML back to `src/_data/accessibility-tests/components/[component-name].yml` ### 7. Summary @@ -99,9 +99,9 @@ After updating metadata: - **Generate Test File Prompt**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` - **Add to Component Page Prompt**: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` -- **Components README**: `src/_data/accessibility_tests/components/README.md` (file structure, recording test results) -- **Test Library Config**: `src/_data/accessibility_tests/test-library/_config.yml` (required environments by category) -- **Test Library Files**: `src/_data/accessibility_tests/test-library/*.yml` (test definitions with categories) +- **Components README**: `src/_data/accessibility-tests/components/README.md` (file structure, recording test results) +- **Test Library Config**: `src/_data/accessibility-tests/test-library/_config.yml` (required environments by category) +- **Test Library Files**: `src/_data/accessibility-tests/test-library/*.yml` (test definitions with categories) ## Notes diff --git a/src/_data/accessibility-tests/README.md b/src/_data/accessibility-tests/README.md index d49309d91..4f841aa7f 100644 --- a/src/_data/accessibility-tests/README.md +++ b/src/_data/accessibility-tests/README.md @@ -54,7 +54,7 @@ Use these prompts in GitHub Copilot Chat to work with accessibility tests: - **`/accessibility-tests-generate-component-tests`** — Create a component test file with applicable tests - Available at: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` - Optionally includes test metadata (tester name, version, date) - - Generates YAML file in `src/_data/accessibility_tests/components/` + - Generates YAML file in `src/_data/accessibility-tests/components/` - **`/accessibility-tests-generate-metadata`** — Add or update test metadata in an existing component test file - Available at: `.github/prompts/accessibility-tests-generate-metadata.prompt.md` - Useful for adding metadata to files created without it, or for adding new test runs diff --git a/src/_data/accessibility-tests/components/README.md b/src/_data/accessibility-tests/components/README.md index 93f1ff420..5fe3de2b1 100644 --- a/src/_data/accessibility-tests/components/README.md +++ b/src/_data/accessibility-tests/components/README.md @@ -29,7 +29,7 @@ The prompt will: 1. In GitHub Copilot Chat, use the slash command: `/accessibility-tests-generate-component-tests` 2. Follow the prompts to provide component name 3. Choose whether to include metadata now or add it later -4. Review the generated file and save it to `src/_data/accessibility_tests/components/[component-name].yml` +4. Review the generated file and save it to `src/_data/accessibility-tests/components/[component-name].yml` **To add or update test metadata later:** - Use the `/accessibility-tests-generate-metadata` prompt (`.github/prompts/accessibility-tests-generate-metadata.prompt.md`) @@ -230,6 +230,55 @@ Before selecting tests, review your component to identify its features: Use these features to determine which test categories apply in the "Choosing tests by component type" section below. +#### Source of truth for component features + +**Always check these sources** before selecting tests: + +1. **Component documentation page** (`src/_components/form/[component-name].md`) + - Read the "Examples" section — what UI elements are shown? + - Read the "Usage" section — what is the component's purpose? + - Look for screenshots/examples showing all possible states + +2. **Component source code** (if needed for clarification) + - Repository: `https://github.com/department-of-veterans-affairs/component-library` + - Path: `packages/web-components/src/components/va-[component-name]/` + - Look for: child components used, interactive elements, ARIA attributes + +3. **Storybook examples** + - URL: `https://design.va.gov/storybook/?path=/docs/components-va-[component-name]--docs` + - Check all variants and states shown + +#### Create a feature checklist + +Before selecting tests, explicitly list what the component contains: + +**Example for va-statement-of-truth:** +- ✓ Text input field (for signature) +- ✓ Checkbox (for certification) +- ✓ Heading ("Veteran's statement of truth") +- ✓ Link (privacy policy) +- ✓ Static text/instructions +- ✓ Error states (validation) +- ✓ Icons (if prefix/suffix used) + +Then map each feature to test categories: +- Text input → Form inputs category +- Checkbox → Form inputs category +- Heading → Components with headings category +- Link → Links category +- Static text → Components with text category +- etc. + +#### Critical rule for test coverage + +**⚠️ Component has a feature = Apply that category's tests** + +If uncertain whether a feature exists: +- ✅ **Include the tests** (safer to over-test than miss coverage) +- ✅ Read the component page examples to verify +- ✅ Check Storybook for all variants +- ❌ Don't assume features don't exist without checking sources + ### Subtest Preference Rule **Always prefer the most specific subtest over the parent test** when available. @@ -248,7 +297,7 @@ Use these features to determine which test categories apply in the "Choosing tes - `WEB-4xx` tests → `4-robust.yml` - `VADS-xxx` tests → Search all files (these are VADS-specific subtests) -2. **Open the corresponding YAML file** at `src/_data/accessibility_tests/test-library/` +2. **Open the corresponding YAML file** at `src/_data/accessibility-tests/test-library/` 3. **Find the test entry** by searching for `id: [test-id]` @@ -267,10 +316,17 @@ See [test library README](../test-library/README.md) for test structure and ID f If your component uses child VA web components (e.g., a form component that uses `` or ``), you should inherit applicable tests from those child components: -1. **Identify child components** in your component's source code -2. **Open each child's test file** at `src/_data/accessibility_tests/components/va-[child-name].yml` +1. **Identify child components**: + - Check the component documentation page examples for `` elements + - Look in the component source code at `packages/web-components/src/components/va-[component-name]/` + - Check Storybook examples for rendered child components + +2. **Open each child's test file** at `src/_data/accessibility-tests/components/va-[child-name].yml` + 3. **Review the child's test list** under the `tests:` array + 4. **Add relevant tests** from the child to your component's test list + 5. **Remove duplicates** after combining with your category-based tests This ensures parent components maintain the same accessibility coverage as their children. @@ -281,7 +337,7 @@ Some tests require specific environments (e.g., screen readers, mobile devices) 1. **Open the test definition** in the appropriate test library YAML file 2. **Check the `category:` field** (e.g., `screen_reader`, `mobile`, `keyboard`) -3. **Open** `src/_data/accessibility_tests/test-library/_config.yml` +3. **Open** `src/_data/accessibility-tests/test-library/_config.yml` 4. **Find the category definition** under `categories:` 5. **Check for `required_environments:`** — if present, those environments must be tested and reported @@ -293,6 +349,19 @@ See the [Environment IDs section](../test-library/README.md#environment-ids) in Use the guidelines below to determine which tests apply to a component. Start with "All components," then add tests for each applicable category. +**Before using these guidelines:** +1. Complete the component analysis steps in ["How to analyze your component"](#how-to-analyze-your-component) +2. Create your feature checklist listing all interactive elements, content types, and behaviors +3. Check for child components and plan to inherit their tests +4. When in doubt about whether a category applies, **include it** (false positives are safer than missing coverage) + +**How to use this section:** +- Start with "All components" tests (everyone gets these) +- For each feature in your checklist, find the matching category below +- Add ALL tests from each matching category +- Apply the [subtest preference rule](#subtest-preference-rule) to use specific subtests instead of parents +- Add tests inherited from [child components](#child-component-inheritance) + ### All components Include these tests for every component: @@ -400,7 +469,7 @@ If the component includes skip navigation functionality, also include: ## Resources -- **Test Library**: `src/_data/accessibility_tests/test-library/` +- **Test Library**: `src/_data/accessibility-tests/test-library/` - **Test Library README**: [../test-library/README.md](../test-library/README.md) — Test definitions, WCAG mappings, and writing guidelines - **Parent README**: [../README.md](../README.md) — Directory overview and AI assistance - **Component Library source**: [department-of-veterans-affairs/component-library](https://github.com/department-of-veterans-affairs/component-library/tree/main/packages/web-components/src/components/) diff --git a/src/_includes/accessibility-test-env-lookup.html b/src/_includes/accessibility-test-env-lookup.html index 88187d33f..6b01eb9ba 100644 --- a/src/_includes/accessibility-test-env-lookup.html +++ b/src/_includes/accessibility-test-env-lookup.html @@ -8,7 +8,7 @@ {% endcomment %} {% assign env_display_name = include.env_id %} {% assign env_is_required = false %} -{% assign env_definitions = site.data.accessibility_tests.test-library._config.environment_definitions %} +{% assign env_definitions = site.data.accessibility-tests.test-library._config.environment_definitions %} {% for group in env_definitions %} {% for env in group[1] %} diff --git a/src/_includes/accessibility-test-library-loader.html b/src/_includes/accessibility-test-library-loader.html index f35468a58..d81f9a27c 100644 --- a/src/_includes/accessibility-test-library-loader.html +++ b/src/_includes/accessibility-test-library-loader.html @@ -6,10 +6,10 @@ Output: Sets test_library variable with all tests from 4 WCAG principle files {% endcomment %} {% assign test_library = "" | split: "" %} -{% assign perceivable_tests = site.data.accessibility_tests.test-library["1-perceivable"].tests %} -{% assign operable_tests = site.data.accessibility_tests.test-library["2-operable"].tests %} -{% assign understandable_tests = site.data.accessibility_tests.test-library["3-understandable"].tests %} -{% assign robust_tests = site.data.accessibility_tests.test-library["4-robust"].tests %} +{% assign perceivable_tests = site.data.accessibility-tests.test-library["1-perceivable"].tests %} +{% assign operable_tests = site.data.accessibility-tests.test-library["2-operable"].tests %} +{% assign understandable_tests = site.data.accessibility-tests.test-library["3-understandable"].tests %} +{% assign robust_tests = site.data.accessibility-tests.test-library["4-robust"].tests %} {% assign test_library = test_library | concat: perceivable_tests | concat: operable_tests | concat: understandable_tests | concat: robust_tests %} diff --git a/src/_includes/accessibility-test-results.html b/src/_includes/accessibility-test-results.html index b92f1fe40..32eb2b802 100644 --- a/src/_includes/accessibility-test-results.html +++ b/src/_includes/accessibility-test-results.html @@ -1,5 +1,5 @@ {% assign component_name = include.component_name %} -{% assign component_tests = site.data.accessibility_tests[component_name] %} +{% assign component_tests = site.data.accessibility-tests[component_name] %} {% include accessibility-test-library-loader.html %} @@ -188,7 +188,7 @@ {% else %} {% assign test_category = test_def.category %} {% endif %} - {% assign category_envs = site.data.accessibility_tests.test-library._config.environment_definitions[test_category] %} + {% assign category_envs = site.data.accessibility-tests.test-library._config.environment_definitions[test_category] %} {% comment %} Build environment list: always include required environments, then add any from test results {% endcomment %} {% assign all_environments = "" | split: "" %} From 561cc3ca7dbffd3b50520f3377c59bb365538efd Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 14:51:40 -0700 Subject: [PATCH 03/13] Fix copilot review items --- ...sibility-tests-generate-metadata.prompt.md | 7 +-- .../accessibility-tests/components/README.md | 2 +- .../test-library/README.md | 4 +- .../test-library/_config.yml | 21 +-------- .../accessibility-test-library-loader.html | 3 -- src/_includes/accessibility-test-results.html | 46 +++++++++++++++---- .../_components/_accessibility-tests.scss | 28 +++++------ 7 files changed, 59 insertions(+), 52 deletions(-) diff --git a/.github/prompts/accessibility-tests-generate-metadata.prompt.md b/.github/prompts/accessibility-tests-generate-metadata.prompt.md index 368eb0bc2..1cf1af699 100644 --- a/.github/prompts/accessibility-tests-generate-metadata.prompt.md +++ b/.github/prompts/accessibility-tests-generate-metadata.prompt.md @@ -61,8 +61,9 @@ For each test in the file: 3. Add an `environments` array: - **If the test has `screen_reader` or `mobile` category**: - Read `src/_data/accessibility-tests/test-library/_config.yml` - - Find the category's `required_environments` - - Pre-populate the environments array with those environment IDs + - Look up the category in `environment_definitions` (e.g., `mobile`, `screen_reader`) + - Filter environments where `required: true` + - Pre-populate the environments array with those required environment IDs - Each environment gets an empty `result:` field - **For all other tests**: - Include a single empty list item: `- ` (dash space, no properties) @@ -100,7 +101,7 @@ After updating metadata: - **Generate Test File Prompt**: `.github/prompts/accessibility-tests-generate-component-tests.prompt.md` - **Add to Component Page Prompt**: `.github/prompts/accessibility-tests-add-to-component-page.prompt.md` - **Components README**: `src/_data/accessibility-tests/components/README.md` (file structure, recording test results) -- **Test Library Config**: `src/_data/accessibility-tests/test-library/_config.yml` (required environments by category) +- **Test Library Config**: `src/_data/accessibility-tests/test-library/_config.yml` (environment definitions with required flags by category) - **Test Library Files**: `src/_data/accessibility-tests/test-library/*.yml` (test definitions with categories) ## Notes diff --git a/src/_data/accessibility-tests/components/README.md b/src/_data/accessibility-tests/components/README.md index 5fe3de2b1..f66bf67b7 100644 --- a/src/_data/accessibility-tests/components/README.md +++ b/src/_data/accessibility-tests/components/README.md @@ -208,7 +208,7 @@ Valid result values: Environment IDs use the format: `{os}-{browser}` or `{os}-{browser}-{assistive-technology}` -Examples: `mac-chrome`, `windows-edge-nvda`, `ios-safari-voiceover`, `mac-voice-control` +Examples: `mac-chrome`, `windows-edge-nvda`, `ios-safari-voiceover`, `mac-safari-voice-control` **For the complete list of valid environment IDs**, see the [Environment IDs section](../test-library/README.md#environment-ids) in the test library README. diff --git a/src/_data/accessibility-tests/test-library/README.md b/src/_data/accessibility-tests/test-library/README.md index cb8fb96a4..b77abc250 100644 --- a/src/_data/accessibility-tests/test-library/README.md +++ b/src/_data/accessibility-tests/test-library/README.md @@ -159,8 +159,8 @@ Environment IDs are defined in `_config.yml` and use the format: `{os}-{browser} - `ios-safari-voiceover` → iOS Safari/VoiceOver ### Voice Control -- `mac-voice-control` → macOS Voice Control -- `windows-voice-recognition` → Windows Voice Recognition +- `mac-safari-voice-control` → macOS Safari/Voice Control +- `windows-edge-voice-recognition` → Windows Edge/Voice Recognition ## Writing Test and Subtest Entries diff --git a/src/_data/accessibility-tests/test-library/_config.yml b/src/_data/accessibility-tests/test-library/_config.yml index 8d395b977..3a8cd7c81 100644 --- a/src/_data/accessibility-tests/test-library/_config.yml +++ b/src/_data/accessibility-tests/test-library/_config.yml @@ -50,9 +50,9 @@ environment_definitions: name: iOS Safari/VoiceOver # Voice control voice_control: - - id: mac-voice-control + - id: mac-safari-voice-control name: macOS Voice Control - - id: windows-voice-recognition + - id: windows-edge-voice-recognition name: Windows Voice Recognition # Test categories @@ -66,20 +66,3 @@ categories: - voice_control - zoom - implementation - -# Metadata -metadata: - version: "1.0" - created: "2024-02-05" - last_updated: "2026-02-10" - total_tests: 55 - total_subtests: 30 - test_breakdown: - general: 36 - keyboard: 9 - screen_reader: 8 - structural: 2 - mobile: 2 - visual: 0 - voice_control: 0 - zoom: 0 diff --git a/src/_includes/accessibility-test-library-loader.html b/src/_includes/accessibility-test-library-loader.html index d81f9a27c..80b8a5c6c 100644 --- a/src/_includes/accessibility-test-library-loader.html +++ b/src/_includes/accessibility-test-library-loader.html @@ -1,7 +1,6 @@ {% comment %} Loads all WCAG principle-based test files and merges into single test_library array. - Also loads required CSS for accessibility test displays. Usage: {% include accessibility-test-library-loader.html %} Output: Sets test_library variable with all tests from 4 WCAG principle files {% endcomment %} @@ -11,5 +10,3 @@ {% assign understandable_tests = site.data.accessibility-tests.test-library["3-understandable"].tests %} {% assign robust_tests = site.data.accessibility-tests.test-library["4-robust"].tests %} {% assign test_library = test_library | concat: perceivable_tests | concat: operable_tests | concat: understandable_tests | concat: robust_tests %} - - diff --git a/src/_includes/accessibility-test-results.html b/src/_includes/accessibility-test-results.html index 32eb2b802..dd8dab54e 100644 --- a/src/_includes/accessibility-test-results.html +++ b/src/_includes/accessibility-test-results.html @@ -57,9 +57,17 @@ {% assign has_conditional = false %} {% for run in test.test_results %} - {% comment %} Track latest version {% endcomment %} - {% if latest_version == "" or run.version > latest_version %} + {% comment %} Track latest version by date {% endcomment %} + {% if latest_version == "" %} {% assign latest_version = run.version %} + {% assign latest_date = run.date %} + {% elsif run.date and latest_date %} + {% assign run_timestamp = run.date | date: "%s" %} + {% assign latest_timestamp = latest_date | date: "%s" %} + {% if run_timestamp > latest_timestamp %} + {% assign latest_version = run.version %} + {% assign latest_date = run.date %} + {% endif %} {% endif %} {% comment %} Check all environments for this test run {% endcomment %} @@ -103,7 +111,7 @@ } -

Accessibility test sections are a newer addition to the VA Design System and are being added to components over time. Not all components have been evaluated using this rubric yet. These tests are based on the and establish a baseline for expected component behavior.

+

Accessibility test sections are a newer addition to the VA Design System and are being added to components over time. Not all components have been evaluated using this rubric yet. These tests are based on the and establish a baseline for expected component behavior.

{% if category_var == "implementation_tests" %}

Teams are responsible for verifying these tests in their own context.

@@ -188,7 +196,15 @@ {% else %} {% assign test_category = test_def.category %} {% endif %} - {% assign category_envs = site.data.accessibility-tests.test-library._config.environment_definitions[test_category] %} + + {% comment %} Map test category to environment definition group {% endcomment %} + {% assign env_group = test_category %} + {% case test_category %} + {% when "general" or "keyboard" or "structural" or "visual" or "zoom" %} + {% assign env_group = "browser" %} + {% endcase %} + + {% assign category_envs = site.data.accessibility-tests.test-library._config.environment_definitions[env_group] %} {% comment %} Build environment list: always include required environments, then add any from test results {% endcomment %} {% assign all_environments = "" | split: "" %} @@ -231,17 +247,27 @@ {% for env_id in all_environments %} {% include accessibility-test-env-lookup.html env_id=env_id %} - {% comment %} Find the latest result for this environment {% endcomment %} + {% comment %} Find the latest result for this environment by date {% endcomment %} {% assign latest_env_result = nil %} {% assign latest_env_version = "" %} {% assign latest_env_date = nil %} {% if test_runs and test_runs.size > 0 %} {% for run in test_runs %} {% assign matching_env = run.environments | where: "name", env_id | first %} - {% if matching_env and run.version > latest_env_version %} - {% assign latest_env_result = matching_env.result %} - {% assign latest_env_version = run.version %} - {% assign latest_env_date = run.date %} + {% if matching_env %} + {% if latest_env_version == "" %} + {% assign latest_env_result = matching_env.result %} + {% assign latest_env_version = run.version %} + {% assign latest_env_date = run.date %} + {% elsif run.date and latest_env_date %} + {% assign run_timestamp = run.date | date: "%s" %} + {% assign latest_timestamp = latest_env_date | date: "%s" %} + {% if run_timestamp > latest_timestamp %} + {% assign latest_env_result = matching_env.result %} + {% assign latest_env_version = run.version %} + {% assign latest_env_date = run.date %} + {% endif %} + {% endif %} {% endif %} {% endfor %} {% endif %} diff --git a/src/assets/stylesheets/_components/_accessibility-tests.scss b/src/assets/stylesheets/_components/_accessibility-tests.scss index d3d7c3a54..906d22ce6 100644 --- a/src/assets/stylesheets/_components/_accessibility-tests.scss +++ b/src/assets/stylesheets/_components/_accessibility-tests.scss @@ -1,18 +1,18 @@ @use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/stylesheets/functions' as *; @use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/tokens/scss/variables' as *; - +@use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/stylesheets/mixins' as *; // Accessibility Test Display Styles // Used in component test results and test library pages // Main test list container .a11y-test-list { list-style: none; - margin: 0 0 units(3) 0; - padding: units(1) 0 0; + margin: 0 0 units(6) 0; + padding: units(2) 0 0; > li { border-bottom: 1px solid var(--vads-color-base-light); - padding: units(1.5) 0 units(1); + padding: units(3) 0 units(2); max-width: 100%; &:first-of-type { @@ -23,32 +23,32 @@ border: none; padding-bottom: 0; } + } - > p { + li > p { margin-block: 0; max-width: 77ch; &.a11y-subtest-tag { - margin-top: units(0.5); + margin-top: units(1); } } - } // Nested subtests .a11y-test-subtest-list { list-style: none; - margin: units(1) 0 0 units(1); + margin: units(2) 0 0 units(2); padding: 0; border-left: 2px solid var(--vads-color-base-lighter); > li { border-bottom: none; - padding: units(1) 0 units(1) units(1); + padding: units(2) 0 units(2) units(2); border-top: 1px solid var(--vads-color-base-lighter); &:first-child { border-top: none; - padding-top: units(0.5); + padding-top: units(1); } } } @@ -64,12 +64,12 @@ list-style: none; display: flex; flex-direction: row; - margin: units(0.5) 0; + margin: units(1) 0; padding: 0; flex-wrap: wrap; li { - margin-right: units(1); + margin-right: units(2); } } @@ -77,7 +77,7 @@ .a11y-table-header { display: flex; align-items: center; - gap: units(0.25); + gap: units(0.5); } // Icon colors for test results @@ -96,5 +96,5 @@ // WCAG criterion headers h3.a11y-wcag-criterion-header { border-bottom: 2px solid var(--vads-color-base-lighter); - padding-bottom: units(0.5); + padding-bottom: units(1); } From 8290a6cec26208b4608c5f0ee030fa408aafe424 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 14:54:52 -0700 Subject: [PATCH 04/13] Include subtests in test count --- src/_includes/accessibility-test-library-list.html | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/src/_includes/accessibility-test-library-list.html b/src/_includes/accessibility-test-library-list.html index 3c2c7cd55..c2a6bfc27 100644 --- a/src/_includes/accessibility-test-library-list.html +++ b/src/_includes/accessibility-test-library-list.html @@ -79,6 +79,15 @@

{% endfor %} +{% comment %} Calculate total test count including subtests {% endcomment %} +{% assign total_test_count = 0 %} +{% for test in test_library %} + {% assign total_test_count = total_test_count | plus: 1 %} + {% if test.subtests %} + {% assign total_test_count = total_test_count | plus: test.subtests.size %} + {% endif %} +{% endfor %} +

- Total tests in library: {{ test_library.size }} + Total tests in library: {{ total_test_count }} ({{ test_library.size }} parent tests, {{ total_test_count | minus: test_library.size }} subtests)

From 3d903c55362913ff84b318d8a43116f2e7275e3c Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 15:01:00 -0700 Subject: [PATCH 05/13] Format code --- src/_includes/accessibility-test-library-list.html | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/src/_includes/accessibility-test-library-list.html b/src/_includes/accessibility-test-library-list.html index c2a6bfc27..27561947f 100644 --- a/src/_includes/accessibility-test-library-list.html +++ b/src/_includes/accessibility-test-library-list.html @@ -29,7 +29,6 @@

    {% for test in test_library %} {% assign test_id_parts = test.id | split: "-" %} - {% comment %} Only display parent tests (2-part IDs) - subtests are nested under them {% endcomment %} {% if test_id_parts.size == 2 and test.wcag_criterion == criterion %}

  • @@ -40,14 +39,11 @@

    {% comment %} Display subtests nested under this parent test {% endcomment %} From c2fdedb96e19e8837f3a0783248308ded88d7a6c Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 15:14:00 -0700 Subject: [PATCH 06/13] Hide va-table-row in accessibility results table --- .../stylesheets/_components/_accessibility-tests.scss | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/assets/stylesheets/_components/_accessibility-tests.scss b/src/assets/stylesheets/_components/_accessibility-tests.scss index 906d22ce6..1f59434be 100644 --- a/src/assets/stylesheets/_components/_accessibility-tests.scss +++ b/src/assets/stylesheets/_components/_accessibility-tests.scss @@ -1,6 +1,5 @@ @use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/stylesheets/functions' as *; @use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/tokens/scss/variables' as *; -@use '../../../node_modules/@department-of-veterans-affairs/css-library/dist/stylesheets/mixins' as *; // Accessibility Test Display Styles // Used in component test results and test library pages @@ -52,8 +51,10 @@ } } } +} - // Accessibility test results table styles +.accessibility-test-summary-table, +.a11y-test-list { va-table-row { display: none; } @@ -98,3 +99,8 @@ h3.a11y-wcag-criterion-header { border-bottom: 2px solid var(--vads-color-base-lighter); padding-bottom: units(1); } + +// Tag status alignment in test result tables +.va-tag-status { + vertical-align: middle; +} From b37c05c27f90aab47e94ca979f05bc24712abfc5 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 15:17:31 -0700 Subject: [PATCH 07/13] Remove unused code, directional language --- src/_includes/accessibility-test-results.html | 36 ++++++------------- 1 file changed, 11 insertions(+), 25 deletions(-) diff --git a/src/_includes/accessibility-test-results.html b/src/_includes/accessibility-test-results.html index dd8dab54e..e295c3108 100644 --- a/src/_includes/accessibility-test-results.html +++ b/src/_includes/accessibility-test-results.html @@ -1,5 +1,5 @@ {% assign component_name = include.component_name %} -{% assign component_tests = site.data.accessibility-tests[component_name] %} +{% assign component_tests = site.data.accessibility-tests.components[component_name] %} {% include accessibility-test-library-loader.html %} @@ -16,16 +16,15 @@ {% assign implementation_tests = "" | split: "" %} {% comment %} Calculate summary statistics {% endcomment %} -{% assign total_tests = 0 %} {% assign passed_count = 0 %} {% assign failed_count = 0 %} {% assign conditional_count = 0 %} {% assign untested_count = 0 %} {% assign latest_version = "" %} +{% assign latest_date = nil %} {% comment %} Categorize tests and calculate statistics {% endcomment %} {% for test in component_tests.tests %} - {% assign total_tests = total_tests | plus: 1 %} {% include accessibility-test-lookup.html test_id=test.id %} @@ -97,26 +96,22 @@ {% endif %} {% endfor %} -{% comment %} Render categorized tests {% endcomment %} -{% if component_tests.size > 0 %} - {% comment %} Summary Dashboard {% endcomment %} {% assign display_component_name = component_name | replace: "va-", "" | replace: "-", " " | capitalize %} {% assign context_dependent_count = conditional_count | plus: implementation_tests.size %} - - -

    Accessibility test sections are a newer addition to the VA Design System and are being added to components over time. Not all components have been evaluated using this rubric yet. These tests are based on the and establish a baseline for expected component behavior.

    +

    + The VA Design System tests this component in isolation using the . + The results in this section show baseline testing across different browsers and assistive technologies. + You need to test the {{ display_component_name }} component in the context of your own product to ensure accessibility compliance. +

    Version tested @@ -187,19 +182,11 @@

- {% comment %} Test Results Table {% endcomment %} {% assign test_runs = test.test_results %} - {% comment %} Get category for this test {% endcomment %} - {% if parent_def %} - {% assign test_category = test_def.category | default: parent_def.category %} - {% else %} - {% assign test_category = test_def.category %} - {% endif %} - - {% comment %} Map test category to environment definition group {% endcomment %} - {% assign env_group = test_category %} - {% case test_category %} + {% comment %} Map display category to environment definition group {% endcomment %} + {% assign env_group = category_var | remove: "_tests" %} + {% case env_group %} {% when "general" or "keyboard" or "structural" or "visual" or "zoom" %} {% assign env_group = "browser" %} {% endcase %} @@ -325,7 +312,6 @@ {% endif %} {% endfor %} -{% endif %} {% else %}

No accessibility test results available for the {{ component_name }} component.

{% endif %} From 2bd4b83ddf9c1767a8ff77547f31e5dda5430e08 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 16:06:02 -0700 Subject: [PATCH 08/13] Update content on accessibility testing page --- ...ty-testing-for-design-system-components.md | 195 +++++------------- 1 file changed, 46 insertions(+), 149 deletions(-) diff --git a/src/_accessibility/accessibility-testing-for-design-system-components.md b/src/_accessibility/accessibility-testing-for-design-system-components.md index decf5aab0..0b7286668 100644 --- a/src/_accessibility/accessibility-testing-for-design-system-components.md +++ b/src/_accessibility/accessibility-testing-for-design-system-components.md @@ -4,186 +4,83 @@ title: Accessibility testing for design system components permalink: /accessibility/accessibility-testing-for-design-system-components anchors: - anchor: Overview - - anchor: How components are tested - - anchor: Using components in your product - - anchor: Scope and limitations - - anchor: Test results on component pages + - anchor: What we do + - anchor: What teams must do + - anchor: Report accessibility defects --- ## Overview -Accessibility specialists on the [VA.gov](http://va.gov/) design system team test components before they are added or updated in the Design System. Our goal is to establish a reliable baseline of accessible behavior for each component, including expected interactions, compatibility with common assistive technologies, and known limitations. Testing is an ongoing process that's revisited as components evolve. +The VA Design System is built on a foundation of accessibility. We build testing into every phase of the component development lifecycle, from defining requirements to design to development and release. -For the specific tests we run and how to interpret test results on component pages, see the [Accessibility Test Library](/accessibility/test-library). +Our testing meets [WCAG 2.2 Level AA standards](https://www.w3.org/TR/WCAG22/) and [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/). We audit every component before it is released using the [Accessibility Test Library](/accessibility/test-library), which includes tests from the [VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we creating an inclusive and equitable experience for everyone. -### What the design system provides +**However, using an accessible Design System component does not guarantee your product is accessible.** We only test components in isolation. Accessibility issues can emerge from the context surrounding a component, such as page structure, content, or how multiple components work together. Teams are responsible for testing their implementations in the full context of their product. -The design system establishes baseline accessible behavior for individual components, including: +## What we do -- Expected interaction patterns (for example: focus management, announcements, and state changes) -- Semantic structure and labeling -- Compatibility with common assistive technologies and input methods +We start the component development process by defining accessibility requirements. The design then goes through accessibility review to ensure it meets those requirements before development begins. During and after development, the component goes through multiple rounds of accessibility audits, with issues addressed before the component is released. -### What the design system does not guarantee +Before components are released, they must pass a final audit by our accessibility specialists. Beginning in April 2026, we use the [Accessibility Test Library](/accessibility/test-library) to define and run these audits. -Using design system components alone won't guarantee that your product is accessible. +### How we audit components -Accessibility issues can still be introduced when: +We test components in isolation across multiple assistive technologies and environments: -- Components are combined in complex ways -- Content or labels aren't clear or are misleading -- Components are used in contexts that weren't anticipated during testing +{% assign env_defs = site.data.accessibility-tests.test-library._config.environment_definitions -%} -Product teams are responsible for evaluating accessibility in the context of their full experience. +**Screen readers:** +{%- for env in env_defs.screen_reader %} +- {{ env.name }}{% if env.required %} (required){% endif %} +{%- endfor %} -## How components are tested +**Voice control:** +{%- for env in env_defs.voice_control %} +- {{ env.name }}{% if env.required %} (required){% endif %} +{%- endfor %} -We evaluate accessibility through a combination of code review, automated tooling, and manual testing across assistive technologies and input methods. Each method contributes a different perspective, and no single approach is sufficient on its own. Accessibility issues often emerge through interaction and state changes, which is why manual testing across assistive technologies is a core part of this process. +**Mobile browsers:** +{%- for env in env_defs.mobile %} +- {{ env.name }}{% if env.required %} (required){% endif %} +{%- endfor %} -### Testing principles +**Desktop browsers:** +{%- for env in env_defs.browser %} +- {{ env.name }}{% if env.required %} (required){% endif %} +{%- endfor %} -#### POUR +See the [Accessibility Test Library](/accessibility/test-library) for complete test procedures and expected behaviors. -Each component is evaluated against the four principles of accessibility: +## What teams must do -- **Perceivable** - Information and user interface components must be presentable to users in ways they can perceive. -- **Operable** - User interface components and navigation must be operable. -- **Understandable** - Information and the operation of user interface must be understandable. -- **Robust** - Content must be robust enough that it can be interpreted reliably by a wide variety of user agents, including assistive technologies. +Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). -#### WCAG 2.2 AA +### Test in your product context -These principles are the foundation of [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/), which incorporates Web Content Accessibility Guidelines (WCAG) 2.0 Level AA success criteria. We aim to adhere to the latest version of WCAG Level AA, which is currently [WCAG 2.2](https://www.w3.org/TR/WCAG22/). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we're creating an inclusive and equitable experience for everyone. +Some accessibility criteria can only be evaluated in the context of a full page or user flow. When using components, test how they function within your complete experience. -#### Behavior-based testing +**Common areas to evaluate:** -Accessibility tests are derived from expected user interactions and outcomes. For example: +- **Heading hierarchy** — Do component headings fit into the overall heading hierarchy of the page? Is nearby content grouped logically? +- **Buttons and link text** — Is hardcoded text appropriate for your user flow? When adding your own text, is it meaningful and descriptive? +- **Labels** — Are labels appropriate and clearly understood? Are they concise and easy to follow? +- **Plain language** — Do error messages clearly describe the issue and provide a path for resolution? +- **Color contrast** — Check contrast against backgrounds and nearby or adjacent elements -- When a user completes an action, the result is communicated clearly -- When content updates dynamically, the change is announced appropriately -- When an error occurs, the user is informed and can recover +### Use automated tools -This approach helps ensure components behave consistently across assistive technologies and input methods. +Make sure axe-core is set up in your implementation to catch common automated issues. Automated testing should be part of your continuous integration process. -#### Multiple types of assistive technologies +### Conduct manual testing -Components are evaluated across a range of assistive technologies and interaction modalities, including: +We recommend a mix of automated, semi-automated, and manual testing. When possible, conduct tests with people who need assistive accommodations to interact with services effectively. -- Screen readers -- Voice command software -- Screen magnification tools -- Browser display settings -- Keyboard, mouse, and touch input +### Complete required artifacts -### Testing methodology +Teams must complete an [accessibility testing artifact](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review) as part of the staging review process. -#### Code review - -We review component code to ensure semantic HTML, appropriate ARIA (Accessible Rich Internet Applications) usage, and clear labeling. This helps ensure components are correctly interpreted by browsers and assistive technologies. - -#### Automated scans - -We use automated tools, such as aXe DevTools, to identify common accessibility issues. Automated testing supports, but does not replace, manual evaluation. - -#### Readability evaluation - -Components and patterns are evaluated for clear and understandable content, including heading structure, link and button text, labels, and adherence to [plain language standards](https://design.va.gov/content-style-guide/plain-language/). - -#### Use of color - -Components are tested for sufficient color contrast and to ensure information is not communicated through color alone. - -#### Text resizing, zoom, and magnification - -Components are tested at up to 400% browser zoom and with screen magnification tools to ensure layouts remain usable and content remains readable. - -- [MacOS Zoom](https://support.apple.com/guide/mac-help/change-zoom-settings-for-accessibility-mh40579/mac) -- [Windows Magnifier](https://support.microsoft.com/en-us/windows/use-magnifier-to-make-things-on-the-screen-easier-to-see-414948ba-8b1c-d3bd-8615-0e5e32204198) - -#### Screen readers - -Components are tested with the following screen reader and browser combinations: - -- [JAWS](https://www.freedomscientific.com/products/software/jaws/ "https://www.freedomscientific.com/products/software/jaws/") + Chrome on Windows -- [NVDA](https://www.nvaccess.org/download/ "https://www.nvaccess.org/download/") + Edge on Windows -- [VoiceOver](https://support.apple.com/guide/voiceover/welcome/mac "https://support.apple.com/guide/voiceover/welcome/mac") + Safari on MacOS -- [TalkBack](https://support.google.com/accessibility/android/answer/6283677?hl=en "https://support.google.com/accessibility/android/answer/6283677?hl=en") + Chrome on Android -- [VoiceOver](https://support.apple.com/guide/iphone/turn-on-and-practice-voiceover-iph3e2e415f/ios "https://support.apple.com/guide/iphone/turn-on-and-practice-voiceover-iph3e2e415f/ios") + Safari on iOS - -We verify that content is announced in logical order, interactive elements have appropriate accessible names and states, and common screen reader interaction patterns are supported. - -#### Input and interaction methods - -Components are tested across different input methods, including keyboard, mouse, touch, and voice control. - -- [Voice Control](https://support.apple.com/en-us/102225) + Safari on MacOS -- [Dragon](https://www.nuance.com/dragon.html) + Edge or Chrome on Windows - -We verify keyboard accessibility (focus management, keyboard operability, and visible focus indicators), touch and mouse compatibility (appropriate target sizes and gesture support), and voice command compatibility (accessible names that match visible labels). - -Alternative input devices such as sip-and-puff switches, eye-tracking software, and refreshable Braille displays are not directly tested but are supported through robust keyboard and mouse support, along with standards-based code and proper ARIA usage. - -## Using components in your product - -We test design system components in isolation. While these tests provide a baseline of accessible behavior, issues may emerge when components are used together or placed within a broader page context. - -### Accessibility testing expectations - -Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). Teams are also required to complete an [accessibility testing artifact](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review) as they prepare for a staging review. - -### What to evaluate - -When using a component, teams should evaluate how it functions within the full experience. - -#### Readability and structure - -Readability depends on surrounding context. Evaluate: - -- Headings and sub-headings - - Do component headings fit into the overall heading hierarchy of the page? - - Is nearby content grouped logically? - -- Buttons and link text - - Is hardcoded text appropriate for your user flow? - - When adding your own text, is it meaningful and descriptive? - -- Labels - - Are labels appropriate and clearly understood? - - Are they concise and easy to follow? - -- Plain language - - Do error messages clearly describe the issue and provide a path for resolution? - -#### Color considerations - -When using a component, teams should check: - -- Color contrast against backgrounds and nearby or adjacent elements. - -### Report accessibility defects +## Report accessibility defects If you identify an accessibility defect in a component, please [submit an issue on GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new/choose) describing the defect. -## Scope and limitations - -Accessibility testing for design system components is intentionally scoped. - -- We test components in isolation, not within full product flows -- We don't test all assistive technology and browser combinations -- Some accessibility issues may only emerge in real-world product contexts - -The design system provides a tested baseline, but accessibility must be validated at the product level. - -## Test results on component pages - -The [Accessibility Test Library](/accessibility/test-library) defines the specific tests, expected behaviors, and environments used to evaluate components. - -On each component page, the “Accessibility tests” section shows: - -- Which tests apply to the component -- The status of those tests (Pass, Failed, Conditional) -- The environments in which testing was performed -- When the tests were last completed - -This helps teams understand what has been validated and what should be verified within their own implementation. From bdefdf5d2ec54ddce0ad6853e06ac610ae10529d Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 16:38:31 -0700 Subject: [PATCH 09/13] Update copy on accessibility testing page; add resources --- ...ty-testing-for-design-system-components.md | 32 +++++++++++++------ 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/src/_accessibility/accessibility-testing-for-design-system-components.md b/src/_accessibility/accessibility-testing-for-design-system-components.md index 0b7286668..3989f95c4 100644 --- a/src/_accessibility/accessibility-testing-for-design-system-components.md +++ b/src/_accessibility/accessibility-testing-for-design-system-components.md @@ -16,7 +16,8 @@ The VA Design System is built on a foundation of accessibility. We build testing Our testing meets [WCAG 2.2 Level AA standards](https://www.w3.org/TR/WCAG22/) and [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/). We audit every component before it is released using the [Accessibility Test Library](/accessibility/test-library), which includes tests from the [VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we creating an inclusive and equitable experience for everyone. -**However, using an accessible Design System component does not guarantee your product is accessible.** We only test components in isolation. Accessibility issues can emerge from the context surrounding a component, such as page structure, content, or how multiple components work together. Teams are responsible for testing their implementations in the full context of their product. +### Working together to ensure accessibility +The Design System team tests components in isolation, which means we can't account for accessibility issues that emerge from surrounding context, like page structure, content, or how multiple components interact. Building accessible products for Veterans requires a partnership between the Design System team and the product teams who implement our components. This page outlines each team's role in that effort. ## What we do @@ -50,8 +51,16 @@ We test components in isolation across multiple assistive technologies and envir - {{ env.name }}{% if env.required %} (required){% endif %} {%- endfor %} + + See the [Accessibility Test Library](/accessibility/test-library) for complete test procedures and expected behaviors. +### Test results on component pages + +We publish audit results on each component's documentation page in an "Accessibility tests" section. These results show which tests were run, the outcome in each environment, and when the tests were last performed. This helps teams understand what has been validated and where they should focus their own testing. + +We're actively auditing components, so not all component pages have test results yet. As we complete audits, results will be added to each component page. + ## What teams must do Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). @@ -62,11 +71,11 @@ Some accessibility criteria can only be evaluated in the context of a full page **Common areas to evaluate:** -- **Heading hierarchy** — Do component headings fit into the overall heading hierarchy of the page? Is nearby content grouped logically? -- **Buttons and link text** — Is hardcoded text appropriate for your user flow? When adding your own text, is it meaningful and descriptive? -- **Labels** — Are labels appropriate and clearly understood? Are they concise and easy to follow? -- **Plain language** — Do error messages clearly describe the issue and provide a path for resolution? -- **Color contrast** — Check contrast against backgrounds and nearby or adjacent elements +- **Heading hierarchy:** Do component headings fit into the overall heading hierarchy of the page? Is nearby content grouped logically? +- **Buttons and link text:** Is hardcoded text appropriate for your user flow? When adding your own text, is it meaningful and descriptive? +- **Labels:** Are labels appropriate and clearly understood? Are they concise and easy to follow? +- **Plain language:** Do error messages clearly describe the issue and provide a path for resolution? +- **Color contrast:** Check contrast against backgrounds and nearby or adjacent elements ### Use automated tools @@ -76,11 +85,16 @@ Make sure axe-core is set up in your implementation to catch common automated is We recommend a mix of automated, semi-automated, and manual testing. When possible, conduct tests with people who need assistive accommodations to interact with services effectively. -### Complete required artifacts +### Testing resources + +For guidance on how to test with assistive technologies: -Teams must complete an [accessibility testing artifact](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review) as part of the staging review process. +- **[VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual)** — General testing procedures and requirements +- **[Accessibility Test Library](/accessibility/test-library)** — Specific test procedures for each component test +- **[WebAIM screen reader testing](https://webaim.org/articles/screenreader_testing/)** — How to test with screen readers +- **[WebAIM keyboard testing](https://webaim.org/articles/keyboard/)** — How to test keyboard accessibility ## Report accessibility defects -If you identify an accessibility defect in a component, please [submit an issue on GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new/choose) describing the defect. +If you identify an accessibility defect in a component, please [submit an issue on GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new?template=5_Accessibility-issue.yml) describing the defect. From d0d55a9da7490d31203466788293cee7029cc1d8 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 16:38:43 -0700 Subject: [PATCH 10/13] Remove Firefox from test environments --- src/_data/accessibility-tests/test-library/_config.yml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/_data/accessibility-tests/test-library/_config.yml b/src/_data/accessibility-tests/test-library/_config.yml index 3a8cd7c81..1e46f5ac0 100644 --- a/src/_data/accessibility-tests/test-library/_config.yml +++ b/src/_data/accessibility-tests/test-library/_config.yml @@ -11,16 +11,12 @@ environment_definitions: browser: - id: mac-chrome name: macOS Chrome - - id: mac-firefox - name: macOS Firefox - id: mac-edge name: macOS Edge - id: mac-safari name: macOS Safari - id: windows-chrome name: Windows Chrome - - id: windows-firefox - name: Windows Firefox - id: windows-edge name: Windows Edge # Mobile browsers From 4272d1d17b845ccdef248319d54048e1b3ddecd5 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Fri, 10 Apr 2026 16:39:10 -0700 Subject: [PATCH 11/13] Fix capitalization; add sub-page; remove firefox --- src/_accessibility/index.md | 1 + src/_accessibility/test-library.md | 4 ++-- src/_data/accessibility-tests/test-library/README.md | 2 -- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/src/_accessibility/index.md b/src/_accessibility/index.md index 03c9412fe..6f776f1fb 100644 --- a/src/_accessibility/index.md +++ b/src/_accessibility/index.md @@ -6,6 +6,7 @@ index: true intro-text: How to follow accessibility standards when using or contributing to the VA Design System (VADS) sub-pages: - sub-page: Accessibility testing for design system components + - sub-page: Accessibility test library - sub-page: Accessibility annotations - sub-page: Focus management - sub-page: When a screen reader needs to announce content diff --git a/src/_accessibility/test-library.md b/src/_accessibility/test-library.md index 3a885dc15..1b17a8eeb 100644 --- a/src/_accessibility/test-library.md +++ b/src/_accessibility/test-library.md @@ -1,6 +1,6 @@ --- layout: documentation -title: Accessibility Test Library +title: Accessibility test library permalink: /accessibility/test-library anchors: - anchor: About this library @@ -8,7 +8,7 @@ anchors: - anchor: Environment-specific results - anchor: Test library --- - + ## About this library _**This test library is in beta and may change.** As this library matures, we welcome your feedback on the tests, structure, and results. Share feedback in the [#platform-design-system Slack channel](https://dsva.slack.com/archives/C01DBGX4P45) or on [GitHub](https://github.com/department-of-veterans-affairs/vets-design-system-documentation/issues/new?template=DST-basic_issue_template.md)._ diff --git a/src/_data/accessibility-tests/test-library/README.md b/src/_data/accessibility-tests/test-library/README.md index b77abc250..be77961ce 100644 --- a/src/_data/accessibility-tests/test-library/README.md +++ b/src/_data/accessibility-tests/test-library/README.md @@ -139,11 +139,9 @@ Environment IDs are defined in `_config.yml` and use the format: `{os}-{browser} ### Desktop Browsers - `mac-chrome` → macOS Chrome -- `mac-firefox` → macOS Firefox - `mac-edge` → macOS Edge - `mac-safari` → macOS Safari - `windows-chrome` → Windows Chrome -- `windows-firefox` → Windows Firefox - `windows-edge` → Windows Edge ### Mobile Browsers From 691022d3821bc43c66853c1fb3d13ac09df5fb3a Mon Sep 17 00:00:00 2001 From: amyleadem Date: Mon, 13 Apr 2026 10:05:43 -0700 Subject: [PATCH 12/13] Update copy on accessibility test pages --- ...ty-testing-for-design-system-components.md | 50 +++++++++++++++---- src/_accessibility/test-library.md | 38 ++++++++------ 2 files changed, 62 insertions(+), 26 deletions(-) diff --git a/src/_accessibility/accessibility-testing-for-design-system-components.md b/src/_accessibility/accessibility-testing-for-design-system-components.md index 3989f95c4..eadbcfc44 100644 --- a/src/_accessibility/accessibility-testing-for-design-system-components.md +++ b/src/_accessibility/accessibility-testing-for-design-system-components.md @@ -12,22 +12,41 @@ anchors: ## Overview -The VA Design System is built on a foundation of accessibility. We build testing into every phase of the component development lifecycle, from defining requirements to design to development and release. - -Our testing meets [WCAG 2.2 Level AA standards](https://www.w3.org/TR/WCAG22/) and [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/). We audit every component before it is released using the [Accessibility Test Library](/accessibility/test-library), which includes tests from the [VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual). We strive to not only meet those standards, but to go beyond compliance with thorough testing to ensure that we creating an inclusive and equitable experience for everyone. +The VA Design System is built on a foundation of accessibility. We test components to meet [WCAG 2.2 Level AA standards](https://www.w3.org/TR/WCAG22/) and [Section 508 of the Rehabilitation Act](https://www.access-board.gov/ict/), and we strive to go beyond compliance to ensure an inclusive and equitable experience for everyone. ### Working together to ensure accessibility + The Design System team tests components in isolation, which means we can't account for accessibility issues that emerge from surrounding context, like page structure, content, or how multiple components interact. Building accessible products for Veterans requires a partnership between the Design System team and the product teams who implement our components. This page outlines each team's role in that effort. ## What we do -We start the component development process by defining accessibility requirements. The design then goes through accessibility review to ensure it meets those requirements before development begins. During and after development, the component goes through multiple rounds of accessibility audits, with issues addressed before the component is released. +We review accessibility throughout the component development lifecycle: -Before components are released, they must pass a final audit by our accessibility specialists. Beginning in April 2026, we use the [Accessibility Test Library](/accessibility/test-library) to define and run these audits. +1. **Define requirements** — Before design begins, we document accessibility expectations and concerns in an architecture document +2. **Review design** — Visual and code designs are reviewed against these accessibility requirements +3. **Test during development** — Components are evaluated during and after development to ensure they meet requirements, which we refine as needed +4. **Audit before release** — Our accessibility specialists conduct a final audit using the [Accessibility Test Library](/accessibility/test-library) before the component goes to staging review, where the VA Platform accessibility team conducts an independent review ### How we audit components -We test components in isolation across multiple assistive technologies and environments: +Our audit process includes: + +**Code review** — We review component code for valid HTML usage, proper ARIA implementation, and correct labeling techniques to ensure semantic markup and maximum compatibility with assistive technologies. + +**Automated scans** — Each component variation is scanned with axe-core via Cypress and browser extensions to identify potential accessibility issues. + +**Manual testing** — We test across multiple assistive technologies and input methods: + +- Screen readers (text to speech) +- Voice command (speech to text) +- Zoom and screen magnification +- Keyboard-only navigation +- Mouse-only navigation +- Touch-only navigation + +**Note:** Alternative input devices such as sip-and-puff switches, eye-tracking software, and refreshable Braille displays are not directly tested. However, these devices typically map to keyboard or mouse interactions, so robust support for keyboard and mouse combined with semantic HTML and proper ARIA usage ensures compatibility. + +We test with these assistive technologies on real devices in the following environments: {% assign env_defs = site.data.accessibility-tests.test-library._config.environment_definitions -%} @@ -57,23 +76,30 @@ See the [Accessibility Test Library](/accessibility/test-library) for complete t ### Test results on component pages -We publish audit results on each component's documentation page in an "Accessibility tests" section. These results show which tests were run, the outcome in each environment, and when the tests were last performed. This helps teams understand what has been validated and where they should focus their own testing. +We publish audit results on each component's documentation page in an "Accessibility tests" section. These results show which tests were run, the outcome in each environment (Passed, Failed, or Conditional), and when the tests were last performed. See the [Accessibility Test Library](/accessibility/test-library#test-status) for details on what each status means. + +This helps teams understand what has been validated and where they should focus their own testing. We're actively auditing components, so not all component pages have test results yet. As we complete audits, results will be added to each component page. ## What teams must do -Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). +Using Design System components alone does not guarantee that your product is accessible. Teams building products on VA.gov are responsible for testing their own products for accessibility and meeting the [VA.gov Experience Standards](https://design.va.gov/about/experience-standards/). + +Our component-level testing helps catch issues early, but your product must still pass a holistic accessibility review before launch. ### Test in your product context Some accessibility criteria can only be evaluated in the context of a full page or user flow. When using components, test how they function within your complete experience. +In the [Accessibility Test Library](/accessibility/test-library), these context-dependent tests are marked as "conditional" in component test results, meaning they can only be evaluated in your implementation of the component. The following areas are typically marked as conditional in the test library, so be sure to test for these in your products: + **Common areas to evaluate:** - **Heading hierarchy:** Do component headings fit into the overall heading hierarchy of the page? Is nearby content grouped logically? - **Buttons and link text:** Is hardcoded text appropriate for your user flow? When adding your own text, is it meaningful and descriptive? - **Labels:** Are labels appropriate and clearly understood? Are they concise and easy to follow? +- **Form grouping:** Are related form inputs properly grouped using fieldsets with descriptive legends? - **Plain language:** Do error messages clearly describe the issue and provide a path for resolution? - **Color contrast:** Check contrast against backgrounds and nearby or adjacent elements @@ -81,16 +107,20 @@ Some accessibility criteria can only be evaluated in the context of a full page Make sure axe-core is set up in your implementation to catch common automated issues. Automated testing should be part of your continuous integration process. +Write end-to-end (e2e) tests to cover accessibility tests that can be automated. While many accessibility tests must be performed manually, many accessibility-focused code checks can be validated programmatically. Incorporating these into your e2e test suite ensures consistent accessibility coverage as your product evolves. + ### Conduct manual testing -We recommend a mix of automated, semi-automated, and manual testing. When possible, conduct tests with people who need assistive accommodations to interact with services effectively. +We recommend a mix of automated, semi-automated, and manual testing. + +Whenever possible, conduct tests with real users of assistive technologies. People who rely on assistive technologies daily can identify usability issues that may not be caught by manual testing alone, even when following established procedures. Their feedback provides invaluable insights into the actual experience of navigating your product. ### Testing resources For guidance on how to test with assistive technologies: - **[VA Platform accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual)** — General testing procedures and requirements -- **[Accessibility Test Library](/accessibility/test-library)** — Specific test procedures for each component test +- **[VA Platform guidance to prepare for an accessibility staging review](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/prepare-for-an-accessibility-staging-review#foundational-testing)** — Instructions for completing accessibility testing prior to the collaboration cycle's staging review - **[WebAIM screen reader testing](https://webaim.org/articles/screenreader_testing/)** — How to test with screen readers - **[WebAIM keyboard testing](https://webaim.org/articles/keyboard/)** — How to test keyboard accessibility diff --git a/src/_accessibility/test-library.md b/src/_accessibility/test-library.md index 1b17a8eeb..3dd7dcc11 100644 --- a/src/_accessibility/test-library.md +++ b/src/_accessibility/test-library.md @@ -4,8 +4,8 @@ title: Accessibility test library permalink: /accessibility/test-library anchors: - anchor: About this library + - anchor: How to read the test library - anchor: How to read test results - - anchor: Environment-specific results - anchor: Test library --- @@ -15,13 +15,32 @@ _**This test library is in beta and may change.** As this library matures, we we This library provides the specific tests we use to evaluate design system components for accessibility. Each test maps to [WCAG 2.2 success criteria](https://www.w3.org/WAI/WCAG22/quickref/) and builds on the [VA Platform team's accessibility testing manual](https://depo-platform-documentation.scrollhelp.site/collaboration-cycle/accessibility-testing-manual). +We're actively auditing components using these tests. On each component page, the "Accessibility tests" section shows the status of each test across each environment. Not all components have test results yet. + For details on our overall testing approach, principles, and methodology, see [Accessibility testing for design system components](/accessibility/accessibility-testing-for-design-system-components). -We're actively auditing components, so not all components have test results yet. On each component page, the "Accessibility tests" section shows the status of each test across each environment. +## How to read the test library + +### Understanding test IDs + +Test IDs show the test source and which [POUR principle](https://www.w3.org/WAI/fundamentals/accessibility-principles/) and WCAG criterion they test. + +**Format:** `WEB/VADS-CRITERION-SUBTEST` + +- **WEB/VADS:** Test source + - **WEB**: General VA Platform tests aligned with WCAG that apply across many contexts + - **VADS**: VA Design System–specific tests that define more specific expectations for components, including standardized pattern choices +- **CRITERION**: WCAG criterion number (1XX = Perceivable, 2XX = Operable, 3XX = Understandable, 4XX = Robust) +- **SUBTEST**: Subtest number when a WCAG criterion requires multiple tests + +**Example:** WEB-141 is a general VA Platform test for WCAG 1.4.1. VADS-141-001 is a VADS-specific test for WCAG 1.4.1 that defines how links must provide a consistent non-color indicator in VA Design System components. ## How to read test results +Component pages show test results in the "Accessibility tests" section. + ### Test status + Component test results use three statuses: - **Passed**: The component meets the test requirement. @@ -30,20 +49,7 @@ Component test results use three statuses: ### Environments tested -Each test result shows the environments and component versions tested. We track results for each environment because accessibility behavior can vary across browsers and assistive technologies. Per-environment results help us: - -- **Establish a baseline** so we can catch when component updates change in a specific environment. -- **Share what we've tested** so teams can focus their testing on areas we haven't covered, like product-specific workflows. -- **Surface gaps** by displaying what we haven't tested yet. - -### Understanding test IDs - -Test IDs show how each test relates to WCAG 2.2. - -- **WEB-* tests**: General VA Platform tests aligned with WCAG. They apply across many contexts, and failures likely indicate a WCAG violation. -- **VADS-* tests**: VA Design System–specific tests that map to WCAG criteria. They define more specific expectations for components, including standardized pattern choices. - -**Example:** WEB-141 requires color is not the only way to convey information. VADS-141-001 defines how links provide a consistent non-color indicator in VA Design System components. +Each test result shows the environments and component versions tested. We track results for each environment because accessibility behavior can vary across browsers and assistive technologies. For the complete list of environments we test in, see [How we audit components](/accessibility/accessibility-testing-for-design-system-components#how-we-audit-components). ## Test library {% include accessibility-test-library-list.html %} From d35c15539a9b500cb06ab2afd5eb64600a51bfd7 Mon Sep 17 00:00:00 2001 From: amyleadem Date: Mon, 13 Apr 2026 10:55:01 -0700 Subject: [PATCH 13/13] Set required browsers; reorder them --- src/_data/accessibility-tests/test-library/_config.yml | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/src/_data/accessibility-tests/test-library/_config.yml b/src/_data/accessibility-tests/test-library/_config.yml index 1e46f5ac0..09dfd109a 100644 --- a/src/_data/accessibility-tests/test-library/_config.yml +++ b/src/_data/accessibility-tests/test-library/_config.yml @@ -9,14 +9,16 @@ environment_definitions: # Desktop browsers (used for general, keyboard, structural, visual, and zoom tests) browser: - - id: mac-chrome - name: macOS Chrome - - id: mac-edge - name: macOS Edge - id: mac-safari name: macOS Safari + required: true - id: windows-chrome name: Windows Chrome + required: true + - id: mac-chrome + name: macOS Chrome + - id: mac-edge + name: macOS Edge - id: windows-edge name: Windows Edge # Mobile browsers