Attribute findings to WCAG success criteria, without claiming compliance

[georgekaran]

Blocked by #61 — the success criteria come from axe's tags, which we currently discard.

Context

Everyone evaluating an accessibility tool arrives with the same question: am I compliant? tabstop deliberately refuses to answer it. DECISIONS.md is explicit that the score is for trending, not judging, and #27 exists to explain the formula rather than dress it up as a grade. That decision should hold.

But there is a wide gap between "we grade your compliance" and "we won't tell you which criterion this fails", and we are currently sitting at the far end of it. A finding today reads:

color-contrast — Elements must meet minimum contrast ratio thresholds

when axe already told us it is WCAG 2.1 AA, 1.4.3 Contrast (Minimum).

Naming the criterion is a fact. Grading the site is a claim. Shipping the first while continuing to refuse the second is a differentiator rather than a compromise — it answers what the person is actually asking, without pretending to a completeness no automated tool has.

Scope

In

• Parse tags into structured attribution: the standard and level (wcag2aa, wcag21aa), and the success criteria (wcag143 → 1.4.3).
• Show the criterion on each finding, with its proper name.
• Group or filter findings by criterion or by level.
• Say plainly, wherever this appears, what automated testing does and does not cover.

Out

• A compliance badge, percentage, or pass/fail. That is the line DECISIONS.md draws.
• Section 508 / EAA / AODA mapping. They are largely derivative of WCAG, and asserting them is a legal interpretation we are not in a position to make.
• Any claim about criteria axe does not test.

The honesty problem, which is the hard part of this issue

Automated testing catches a minority of WCAG failures — the machine-checkable ones. Contrast, alt text presence, ARIA validity and heading order are testable; whether alt text is meaningful, whether focus order is logical, whether an error message is helpful are not.

So the risk of criterion attribution is not that it is wrong — axe's mapping is good — but that it implies coverage we do not have. A UI listing "WCAG 2.1 AA criteria" invites the reading "and the rest passed", which is false and is precisely the misreading that gets organisations sued after buying a clean report.

Whatever this ships must make the shape of the claim unmistakable: these criteria are ones we can test and your page fails; silence here is not a pass. That sentence, or its equivalent, is a requirement of the feature and not a footnote to it.

A useful precision available cheaply: axe distinguishes best-practice rules from standards-mapped ones. Findings that map to no criterion should be presented as good practice rather than smuggled in alongside the ones that do.

Implementation notes

• Criterion numbers come free from the tags (wcag143 → 1.4.3); criterion names ("Contrast (Minimum)") do not, and need a small static map. Keep it to the criteria axe can actually produce, and treat an unmapped tag as unmapped rather than guessing.
• Level is in the tags too (wcag2a / wcag2aa / wcag2aaa), so "AA failures only" is a filter rather than a computation.
• This is also the natural place for the help text from Keep the axe fields we already fetch and discard: tags, help, failureSummary #61: criterion, plus what to do about it.

Acceptance criteria

• Each finding shows its WCAG version, level and success criterion where axe provides one.
• Findings with no standard mapping are labelled as best practice, not as criterion failures.
• Filtering or grouping by level and criterion works.
• The limits of automated coverage are stated wherever criteria are shown — not only in the docs.
• No compliance grade, badge or percentage ships with this, and DECISIONS.md records that as a deliberate limit rather than an omission.

[georgekaran]

Tagged v2. Mapping findings to WCAG success criteria is a meaningful trust signal, but it is additive to the core loop (audit → see what is broken → fix → confirm) rather than part of it, and the type:decision on how to attribute without implying a compliance claim deserves more care than a pre-launch slot allows.

Getting this wrong in the direction of over-claiming is a legal-adjacent risk, which is a further argument for not rushing it.