Skip to content

Latest commit

 

History

History
429 lines (319 loc) · 12 KB

accessibility-testing.md

File metadata and controls

429 lines (319 loc) · 12 KB

Accessibility Testing

This document outlines Bayat's standards and procedures for comprehensive accessibility testing across all digital products.

Table of Contents

Introduction

Purpose

Accessibility testing ensures our products are usable by people with disabilities and meets regulatory requirements. This includes verifying that products are:

  • Perceivable: Information is presented in ways all users can perceive
  • Operable: Interface components are navigable and usable by all
  • Understandable: Information and operation are clear and consistent
  • Robust: Content is compatible with current and future assistive technologies

Business Importance

  • Legal Compliance: Meet requirements like ADA, Section 508, and AODA
  • Market Reach: Serve the 15-20% of the population with disabilities
  • Brand Reputation: Demonstrate commitment to inclusivity
  • Better UX for Everyone: Accessibility improvements benefit all users
  • Cost Efficiency: Early detection reduces remediation costs

Accessibility Standards

Primary Standards

Standard Applicability Compliance Level
WCAG 2.1 All digital products AA (minimum)
Section 508 US federal agencies and contractors Required for government work
EN 301 549 EU digital products Required for EU operations
ADA US-based products and services Required for US operations

Standard Interpretation

  • Refer to WCAG Understanding Documents for official interpretations
  • Apply the POUR principles (Perceivable, Operable, Understandable, Robust)
  • When conflicts arise between standards, apply the more stringent requirement

Testing Approach

Testing Pyramid

Our accessibility testing follows a pyramid approach:

  1. Foundation: Automated testing for high coverage of basic issues
  2. Middle: Manual expert testing for more complex issues
  3. Top: Assistive technology testing and testing with users with disabilities

Risk-Based Prioritization

Prioritize testing based on:

  • Criticality of the feature
  • User impact of potential accessibility issues
  • Complexity of the interface
  • Likelihood of accessibility issues based on technology used

Testing Frequency

Development Stage Testing Type Frequency
Design Accessibility reviews Every major design iteration
Development Automated testing Continuous/daily
Development Developer manual tests For complex components
Sprint completion Accessibility testing Each sprint
Pre-release Full accessibility audit Before each major release
Post-release Ongoing monitoring Monthly

Automated Testing

Tools by Platform

Platform Primary Tools Secondary Tools
Web Axe-core, Lighthouse WAVE, Pa11y
iOS Xcode Accessibility Inspector XCUITest with accessibility
Android Accessibility Scanner, Espresso TalkBack automated tests
Windows Accessibility Insights Windows Narrator API tests
PDF PAC 3, Adobe Acrobat Pro PDF Accessibility Checker

Automation in CI/CD

  • Integrate accessibility testing in CI/CD pipelines
  • Break builds for critical accessibility issues (WCAG Level A)
  • Flag warnings for Level AA issues
  • Generate accessibility reports with each build
  • Track accessibility debt over time

Automated Test Coverage

Ensure automated tests check for:

  • Proper heading structure
  • Alternative text for images
  • Color contrast ratios
  • Form labels and instructions
  • Keyboard focus indicators
  • ARIA usage and landmark regions
  • Reading order and tab order
  • Frame titles
  • Language settings

Manual Testing

Test Protocols

Develop systematic manual test protocols for:

  1. Keyboard Navigation:

    • Tab order follows logical sequence
    • Focus is clearly visible at all times
    • No keyboard traps
    • All functionality is keyboard accessible
    • Shortcut keys don't conflict with assistive technology

  2. Content Structure:

    • Proper heading hierarchy
    • Meaningful link text
    • Lists used appropriately
    • Tables have proper headers
    • Reading order is logical

  3. Visual Design:

    • Text resizes without loss of content
    • Content reflows on zoom
    • Interface works in different orientations
    • Content readable at high contrast

  4. Multimedia:

    • Captions for video content
    • Transcripts for audio
    • Audio descriptions when needed
    • Controls are accessible

Test Cases

Document specific test cases for:

  • User login and authentication
  • Form submission processes
  • Navigation systems
  • Search functionality
  • Media players
  • Interactive elements (dropdowns, tabs, modals)
  • Data visualization components
  • Notification systems

Checklists

Use detailed checklists by component type:

  • Forms and form controls
  • Navigation menus
  • Modal dialogs
  • Carousels
  • Auto-complete widgets
  • Date pickers
  • Custom components

Assistive Technology Testing

Required Assistive Technologies

Platform Primary AT Secondary AT
Windows NVDA JAWS, Windows Narrator
macOS VoiceOver
iOS VoiceOver Switch Control
Android TalkBack Switch Access
All Keyboard only Voice control

Testing Methodology

For each assistive technology:

  1. Create user flows for key tasks
  2. Document expected behavior
  3. Test using assistive technology
  4. Document actual behavior
  5. Identify and report discrepancies

Assistive Technology Matrix

Maintain a compatibility matrix showing:

  • Which assistive technologies are supported
  • Level of support for each feature
  • Known limitations
  • Workarounds for issues

User Testing

Participant Recruitment

  • Partner with disability organizations
  • Build a diverse panel of users with disabilities
  • Ensure representation across disability types:
    • Visual impairments
    • Mobility impairments
    • Cognitive disabilities
    • Hearing impairments
    • Speech disabilities

Test Protocols

  1. Planning:

    • Define clear test objectives
    • Create realistic scenarios
    • Select appropriate participants
    • Prepare accessible test materials

  2. Execution:

    • Set up accessible testing environment
    • Provide multiple communication options
    • Allow extra time if needed
    • Record sessions (with consent)

  3. Analysis:

    • Document barriers encountered
    • Categorize issues by severity
    • Map issues to WCAG criteria
    • Identify patterns across participants

Remote vs. In-person Testing

Guidelines for both approaches:

  • Remote:

    • Ensure testing platform is accessible
    • Test connection and AT compatibility before session
    • Have backup communication channels
    • Provide clear instructions in advance

  • In-person:

    • Ensure physical accessibility of testing location
    • Allow participants to use their own AT when possible
    • Be prepared to provide accommodations
    • Create a comfortable environment

Reporting and Documentation

Issue Documentation

Document each issue with:

  • Description of the issue
  • Steps to reproduce
  • Impact on users
  • Related WCAG criteria
  • Severity rating
  • Screenshots/recordings
  • Assistive technology affected
  • Recommended solution

Severity Classification

Severity Definition Response Time
Critical Prevents task completion for users with disabilities Immediate fix required
High Significantly impairs user experience but has workarounds Fix in current sprint
Medium Causes difficulty but doesn't prevent task completion Schedule for near-term fix
Low Minor inconvenience or technical violation with minimal impact Add to backlog

Report Templates

Standard report sections:

  1. Executive Summary:

    • Overall compliance status
    • Key findings
    • Risk assessment
    • Recommended priorities

  2. Methodology:

    • Standards applied
    • Testing tools used
    • Testing approach
    • Environments tested

  3. Detailed Findings:

    • Categorized by severity
    • Mapped to WCAG criteria
    • Affected user groups
    • Impact descriptions

  4. Remediation Plan:

    • Recommended fixes
    • Resource requirements
    • Proposed timeline
    • Success criteria

Remediation Process

Prioritization Framework

Prioritize remediation based on:

  1. Impact on users
  2. Severity of the issue
  3. Frequency of feature use
  4. Technical complexity of the fix
  5. Regulatory requirements

Fix Verification

For each remediated issue:

  1. Retest using the same method that identified the issue
  2. Verify fix works across all relevant platforms
  3. Ensure fix doesn't introduce new accessibility issues
  4. Document the resolution and update status

Knowledge Sharing

  • Document common issues and solutions
  • Create accessible pattern library
  • Share learnings in design and development reviews
  • Maintain accessibility knowledge base

Integration with Development Lifecycle

Design Phase

  • Use accessible design components
  • Review wireframes and mockups for accessibility
  • Create accessibility annotations in design files
  • Perform early accessibility evaluations

Development Phase

  • Use accessible coding patterns
  • Implement automated accessibility testing
  • Perform developer self-testing
  • Include accessibility acceptance criteria in stories

QA Phase

  • Include accessibility in test plans
  • Perform specialized accessibility testing
  • Block release for critical accessibility issues
  • Track accessibility metrics

Release Phase

  • Include accessibility status in release notes
  • Update accessibility conformance documentation
  • Communicate known issues and workarounds
  • Plan for addressing accessibility debt

Testing Environments

Browser and Device Coverage

Platform Minimum Coverage Preferred Coverage
Desktop browsers Latest 2 versions of Chrome, Firefox, Safari, Edge Add IE11 if required
Mobile browsers Latest 2 versions of Safari iOS, Chrome Android Add Samsung Internet
Screen sizes 320px, 768px, 1024px, 1440px Add additional breakpoints
Operating systems Windows 10, macOS, iOS, Android Add Windows 11, Linux

Testing Tools Environment

Standardize testing environments with:

  • Consistent browser extensions
  • Calibrated displays
  • Standardized assistive technology versions
  • Documented tool configurations
  • Virtual machine templates for testing

Platform-Specific Considerations

Web Applications

  • Test with and without JavaScript
  • Verify WAI-ARIA implementation
  • Check responsive behavior
  • Test with screen magnification
  • Verify print stylesheets

Mobile Applications

  • Test with platform accessibility services
  • Verify custom controls have proper accessibility implementations
  • Test in different orientations
  • Check touch target sizes
  • Verify compatibility with external keyboards

Native Desktop Applications

  • Test with platform accessibility APIs
  • Verify keyboard interface and shortcuts
  • Check high contrast themes
  • Test with screen magnification
  • Verify proper focus handling

Document Accessibility

  • Check tag structure
  • Verify reading order
  • Test bookmarks and navigation
  • Verify form fields accessibility
  • Check for OCR on scanned content