|
| 1 | +# Accessibility verification |
| 2 | + |
| 3 | +The website accessibility baseline combines shared server-rendered patterns, a deterministic |
| 4 | +Playwright/axe gate, and an independent manual review. Automated results are necessary, but they do |
| 5 | +not establish WCAG conformance by themselves. |
| 6 | + |
| 7 | +## Register a route or state |
| 8 | + |
| 9 | +Every product issue that adds or materially changes a user-facing state must add a `CriticalState` |
| 10 | +to `core/accessibility_registry.py`. Give it a stable identifier, product group, rendered fixture |
| 11 | +surface, and an executable scenario key that proves the route, authorization, privacy, and business |
| 12 | +contract. Set `js_required`, `core_smoke`, or `route_contract` only when those properties apply. |
| 13 | + |
| 14 | +Add the corresponding deterministic `Surface` and state assertion to the scenario executor in |
| 15 | +`playwright_tests/test_accessibility.py`. The complete matrix resolves every scenario key, runs each |
| 16 | +real workflow, and compares the exact set of reported states with the registry. A state is reported |
| 17 | +only after its distinct setup, behavioral assertion, and comprehensive page scan; route contracts |
| 18 | +are reported only after their status and destination are asserted. The test never deduplicates by |
| 19 | +rendered URL. |
| 20 | + |
| 21 | +Fixtures must use isolated SQLite records, source-derived public slugs, synthetic copy, and the |
| 22 | +repository's factory/runtime namespace. They must not call email, provider, AWS, or other network |
| 23 | +services. A denial or redirect whose accepted response is deliberately plain text can remain a |
| 24 | +route contract without pretending it is a rendered page. |
| 25 | + |
| 26 | +The registry and fixture map fail closed: duplicate identifiers, missing rendered surfaces, |
| 27 | +unclassified code-owned templates, missing behavior-test references, and incomplete exception |
| 28 | +records fail the test suite. |
| 29 | + |
| 30 | +## Run the gates |
| 31 | + |
| 32 | +Use the uv-backed targets from the repository root: |
| 33 | + |
| 34 | +```console |
| 35 | +make test-accessibility |
| 36 | +make test-playwright-core |
| 37 | +``` |
| 38 | + |
| 39 | +`make test-accessibility` selects both the representative core smoke and the complete scheduled/full |
| 40 | +matrix. For focused diagnosis, use: |
| 41 | + |
| 42 | +```console |
| 43 | +DJANGO_SETTINGS_MODULE=website.settings.test DJANGO_ALLOW_ASYNC_UNSAFE=true \ |
| 44 | + uv run --frozen pytest playwright_tests/test_accessibility.py \ |
| 45 | + -m 'accessibility and core' -v |
| 46 | + |
| 47 | +DJANGO_SETTINGS_MODULE=website.settings.test DJANGO_ALLOW_ASYNC_UNSAFE=true \ |
| 48 | + uv run --frozen pytest playwright_tests/test_accessibility.py \ |
| 49 | + -m 'accessibility and full' -v |
| 50 | +``` |
| 51 | + |
| 52 | +The harness injects the locally installed axe-core 4.12.1 payload from the pinned |
| 53 | +`axe-playwright-python==0.1.8` package. It never downloads test code at runtime. The explicit checks |
| 54 | +cover landmarks/headings, skip behavior, keyboard order/traps, computed focus, status/error |
| 55 | +relationships, value preservation, target size/spacing, overflow, dates/timezones, media |
| 56 | +alternatives, text spacing, zoom/reflow, reduced motion, forced colors, and JavaScript-off public |
| 57 | +reading. The home-page event selection is frozen at `DEFAULT_FROZEN_AT`, and each evidence PNG is |
| 58 | +accepted only when a second immediate capture has identical bytes. |
| 59 | + |
| 60 | +The full lane also reads the platform accessibility tree from the named local |
| 61 | +Chromium/HeadlessChrome 145 browser through Blink's `Accessibility.getFullAXTree` protocol. It |
| 62 | +checks representative landmarks, headings, navigation, table, form-error, date/timezone, |
| 63 | +transcript/media, and polite copy-status semantics. The exact runtime version and checked states are |
| 64 | +written to `.tmp/screenshots/issue-65/named-browser-engine-evidence.json`. This is reproducible |
| 65 | +browser-engine evidence, not screen-reader evidence and not a WCAG conformance claim. |
| 66 | + |
| 67 | +The transactional-email fixture renders every active definition in |
| 68 | +`course_management/datamailer_templates/definitions/registry.py` with that definition's maintained |
| 69 | +example context. It wraps the exact current subject/HTML fragment for document-level checks and |
| 70 | +validates the exact current plain-text alternative. It does not add a message purpose or send an |
| 71 | +email. |
| 72 | + |
| 73 | +## Add a bounded axe exception |
| 74 | + |
| 75 | +Fix the page whenever possible. If axe is demonstrably wrong or remediation would break a safer |
| 76 | +accepted contract, add one `AxeException` in `core/accessibility_registry.py` with all of: |
| 77 | + |
| 78 | +- the exact axe rule; |
| 79 | +- the exact selector emitted for the affected node; |
| 80 | +- one exact registered state identifier; |
| 81 | +- a concrete reason automation is wrong or remediation is unsafe; |
| 82 | +- an accountable owner; and |
| 83 | +- a future expiry date. |
| 84 | + |
| 85 | +Blanket selectors, impact filtering, disabled WCAG rules, and unexplained exclusions are not |
| 86 | +accepted. The exception test rejects missing fields, broad selectors, unknown states, and expired |
| 87 | +records. There are no accepted exceptions in the issue-65 baseline. |
| 88 | + |
| 89 | +## Manual gate and evidence |
| 90 | + |
| 91 | +A tester who did not implement the candidate must complete |
| 92 | +`_docs/accessibility/manual-evidence-template.md`. Record the exact candidate fingerprint, date, |
| 93 | +browser/OS, viewport or zoom, assistive technology and version, route/state, result, evidence path, |
| 94 | +and defect or approved exception. |
| 95 | + |
| 96 | +The named real screen-reader/browser rows must remain unpassed until a tester actually performs |
| 97 | +them. In particular, the Chromium/Blink tree artifact above must never be copied into the assistive |
| 98 | +technology column as if it were NVDA, JAWS, VoiceOver, Orca, or another real screen reader. |
| 99 | + |
| 100 | +Store screenshots only under `.tmp/screenshots/issue-65/`. Use synthetic data, open every original |
| 101 | +image, and reject evidence containing an email address, profile value/link, provider payload, |
| 102 | +registration row, token, credential, or one-time secret. The engineer report must say that keyboard, |
| 103 | +screen-reader, contrast, and visual inspection remain unperformed until the independent tester has |
| 104 | +actually recorded them. |
0 commit comments