Skip to content

Commit 58932b4

Browse files
Merge issue 65: establish accessibility patterns (#65)
2 parents 4f8e6c8 + 815ee08 commit 58932b4

48 files changed

Lines changed: 3972 additions & 205 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Makefile

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
test-core test test-ci test-ci-focused test-compatibility compatibility-source-artifacts-check \
33
compatibility-artifacts-check check-links check-seo compatibility-real-gate-blocked-check \
44
test-content test-factories test-migrations test-playwright-core test-playwright test-browser \
5+
test-accessibility \
56
test-remote-readonly test-remote-mutation test-live-email test-live-provider test-all migrate run worker \
67
terraform-seo-source-check terminology-check check-openapi check-management-parity \
78
database-portability-check verify-dtc-content review-data review-data-dry-run \
@@ -170,6 +171,12 @@ test-playwright:
170171
uv run --frozen pytest playwright_tests \
171172
-m '(core or full) and not remote_readonly and not remote_mutation and not live_email and not live_provider' -v
172173

174+
test-accessibility:
175+
DTC_TEST_RUN_ID="$${DTC_TEST_RUN_ID:-make-$${PPID}}" \
176+
DJANGO_SETTINGS_MODULE=website.settings.test DJANGO_ALLOW_ASYNC_UNSAFE=true \
177+
uv run --frozen pytest playwright_tests/test_accessibility.py \
178+
-m 'accessibility and not remote_readonly and not remote_mutation and not live_email and not live_provider' -v
179+
173180
test-browser: test-playwright
174181

175182
test-remote-readonly:

_docs/accessibility/README.md

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# Accessibility verification
2+
3+
The website accessibility baseline combines shared server-rendered patterns, a deterministic
4+
Playwright/axe gate, and an independent manual review. Automated results are necessary, but they do
5+
not establish WCAG conformance by themselves.
6+
7+
## Register a route or state
8+
9+
Every product issue that adds or materially changes a user-facing state must add a `CriticalState`
10+
to `core/accessibility_registry.py`. Give it a stable identifier, product group, rendered fixture
11+
surface, and an executable scenario key that proves the route, authorization, privacy, and business
12+
contract. Set `js_required`, `core_smoke`, or `route_contract` only when those properties apply.
13+
14+
Add the corresponding deterministic `Surface` and state assertion to the scenario executor in
15+
`playwright_tests/test_accessibility.py`. The complete matrix resolves every scenario key, runs each
16+
real workflow, and compares the exact set of reported states with the registry. A state is reported
17+
only after its distinct setup, behavioral assertion, and comprehensive page scan; route contracts
18+
are reported only after their status and destination are asserted. The test never deduplicates by
19+
rendered URL.
20+
21+
Fixtures must use isolated SQLite records, source-derived public slugs, synthetic copy, and the
22+
repository's factory/runtime namespace. They must not call email, provider, AWS, or other network
23+
services. A denial or redirect whose accepted response is deliberately plain text can remain a
24+
route contract without pretending it is a rendered page.
25+
26+
The registry and fixture map fail closed: duplicate identifiers, missing rendered surfaces,
27+
unclassified code-owned templates, missing behavior-test references, and incomplete exception
28+
records fail the test suite.
29+
30+
## Run the gates
31+
32+
Use the uv-backed targets from the repository root:
33+
34+
```console
35+
make test-accessibility
36+
make test-playwright-core
37+
```
38+
39+
`make test-accessibility` selects both the representative core smoke and the complete scheduled/full
40+
matrix. For focused diagnosis, use:
41+
42+
```console
43+
DJANGO_SETTINGS_MODULE=website.settings.test DJANGO_ALLOW_ASYNC_UNSAFE=true \
44+
uv run --frozen pytest playwright_tests/test_accessibility.py \
45+
-m 'accessibility and core' -v
46+
47+
DJANGO_SETTINGS_MODULE=website.settings.test DJANGO_ALLOW_ASYNC_UNSAFE=true \
48+
uv run --frozen pytest playwright_tests/test_accessibility.py \
49+
-m 'accessibility and full' -v
50+
```
51+
52+
The harness injects the locally installed axe-core 4.12.1 payload from the pinned
53+
`axe-playwright-python==0.1.8` package. It never downloads test code at runtime. The explicit checks
54+
cover landmarks/headings, skip behavior, keyboard order/traps, computed focus, status/error
55+
relationships, value preservation, target size/spacing, overflow, dates/timezones, media
56+
alternatives, text spacing, zoom/reflow, reduced motion, forced colors, and JavaScript-off public
57+
reading. The home-page event selection is frozen at `DEFAULT_FROZEN_AT`, and each evidence PNG is
58+
accepted only when a second immediate capture has identical bytes.
59+
60+
The full lane also reads the platform accessibility tree from the named local
61+
Chromium/HeadlessChrome 145 browser through Blink's `Accessibility.getFullAXTree` protocol. It
62+
checks representative landmarks, headings, navigation, table, form-error, date/timezone,
63+
transcript/media, and polite copy-status semantics. The exact runtime version and checked states are
64+
written to `.tmp/screenshots/issue-65/named-browser-engine-evidence.json`. This is reproducible
65+
browser-engine evidence, not screen-reader evidence and not a WCAG conformance claim.
66+
67+
The transactional-email fixture renders every active definition in
68+
`course_management/datamailer_templates/definitions/registry.py` with that definition's maintained
69+
example context. It wraps the exact current subject/HTML fragment for document-level checks and
70+
validates the exact current plain-text alternative. It does not add a message purpose or send an
71+
email.
72+
73+
## Add a bounded axe exception
74+
75+
Fix the page whenever possible. If axe is demonstrably wrong or remediation would break a safer
76+
accepted contract, add one `AxeException` in `core/accessibility_registry.py` with all of:
77+
78+
- the exact axe rule;
79+
- the exact selector emitted for the affected node;
80+
- one exact registered state identifier;
81+
- a concrete reason automation is wrong or remediation is unsafe;
82+
- an accountable owner; and
83+
- a future expiry date.
84+
85+
Blanket selectors, impact filtering, disabled WCAG rules, and unexplained exclusions are not
86+
accepted. The exception test rejects missing fields, broad selectors, unknown states, and expired
87+
records. There are no accepted exceptions in the issue-65 baseline.
88+
89+
## Manual gate and evidence
90+
91+
A tester who did not implement the candidate must complete
92+
`_docs/accessibility/manual-evidence-template.md`. Record the exact candidate fingerprint, date,
93+
browser/OS, viewport or zoom, assistive technology and version, route/state, result, evidence path,
94+
and defect or approved exception.
95+
96+
The named real screen-reader/browser rows must remain unpassed until a tester actually performs
97+
them. In particular, the Chromium/Blink tree artifact above must never be copied into the assistive
98+
technology column as if it were NVDA, JAWS, VoiceOver, Orca, or another real screen reader.
99+
100+
Store screenshots only under `.tmp/screenshots/issue-65/`. Use synthetic data, open every original
101+
image, and reject evidence containing an email address, profile value/link, provider payload,
102+
registration row, token, credential, or one-time secret. The engineer report must say that keyboard,
103+
screen-reader, contrast, and visual inspection remain unperformed until the independent tester has
104+
actually recorded them.
Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Accessibility manual evidence — candidate template
2+
3+
Candidate fingerprint:
4+
5+
Base commit:
6+
7+
Worktree and branch:
8+
9+
Tester and date:
10+
11+
This template is intentionally blank until an independent tester performs the checks. Do not mark a
12+
row passed from automation or from screenshot existence alone.
13+
14+
Automated prerequisite (not a manual screen-reader result): the full lane writes the named local
15+
Chromium/HeadlessChrome version, Blink accessibility-tree protocol, and checked representative
16+
states to `.tmp/screenshots/issue-65/named-browser-engine-evidence.json`. Record that artifact in the
17+
engineering report only. A real screen-reader/browser pairing is still mandatory below and remains
18+
pending when no named screen reader is available.
19+
20+
| Area | Browser / OS | Assistive technology and version | Viewport / zoom / mode | Route and state | Result | Evidence path | Defect or bounded exception |
21+
| --- | --- | --- | --- | --- | --- | --- | --- |
22+
| Keyboard: skip, order, activation, Escape, restoration, no trap | | | | | | | |
23+
| Screen reader: landmarks, headings, navigation, tables | | | | | | | |
24+
| Screen reader: form instructions, errors, status, copy result | | | | | | | |
25+
| Normal desktop and mobile visual review | | | | | | | |
26+
| 200% browser zoom and 320 CSS-pixel reflow | | | | | | | |
27+
| WCAG text-spacing override | | | | | | | |
28+
| Contrast, focus visibility, and target spacing | | | | | | | |
29+
| Reduced motion and forced colors | | | | | | | |
30+
| Long, empty, success, validation, stale/conflict, denied, table/code states | | | | | | | |
31+
| HTML/plain email at narrow/wide widths and with images disabled | | | | | | | |
32+
33+
Screenshot inspection log:
34+
35+
| Original image path | Shared pattern or changed surface | Synthetic/private-data check | Visual result | Notes |
36+
| --- | --- | --- | --- | --- |
37+
| | | | | |
38+
39+
Unperformed or blocked checks:
40+
41+
Named real screen-reader/browser status (must say pending until actually performed):
42+
43+
Defects filed:
44+
45+
Tester conclusion:

0 commit comments

Comments
 (0)