Skip to content

Commit d135ad6

Browse files
arberxclaude
andauthored
feat: critical per-page defects, agent-native audit output (2.0.0) (#44)
* feat: surface critical per-page defects, make audit output agent-native (2.0.0) Resolves #42. Binary structural defects (an H1 count other than one, a missing <title>, a missing meta description) were detected per page but lost in sitemap aggregation: prioritizedFixes ranked by prevalence only, the factor score averaged them away to a passing grade, and crossCuttingIssues was keyed by factor. A homepage split across four H1s appeared nowhere in the summary. - Detect critical defects straight from the DOM, independent of scoring, so no existing score, grade, or exit code changes. New criticalDefects rollup on sitemap/static reports plus a Critical Defects section in the text and markdown output, grouped by defect with every affected page named (homepage and high sitemap-priority pages first). Shown even with --top-issues. - Make the output agent-native: prioritizedFixes is now a structured PrioritizedFix[] (stable id, kind, severity, full affectedPages, affectsHomepage, prevalencePct, summary) instead of prose strings, and every report carries a schemaVersion so parsers detect shape drift. - No truncation in the end-of-report sections: every issue and every affected page is listed. BREAKING CHANGE: SitemapAuditReport.prioritizedFixes is now PrioritizedFix[] (read .summary for the previous one-liner); reports gain a required schemaVersion field. Bumped to 2.0.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: add --format agent, a slim agent-native decision output Returns { schemaVersion, tool, mode, url, score, grade, pass, criticalDefectCount, issues } as JSON, where issues is the ranked PrioritizedFix[] and the per-factor/per-page detail is omitted, so an agent can act on the decision without averaging and re-ranking scores. Single-page reuses the sitemap critical-defect and cross-cutting aggregation over a one-page site; --detect-platform falls back to JSON. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: stable codes on every finding (2.1.0) (#45) * feat: add stable codes to every finding (2.1.0) Every AuditFinding now carries a `code` namespaced as <factor-id>.<check>[.<variant>] (e.g. technical-seo.h1.multiple), so agents and integrations key on a stable machine identifier instead of regex-matching the human message (which can change between releases). 212 codes across all 19 analyzers; the registry is in docs/finding-codes.md. `code` is required on AuditFinding, so the compiler guarantees coverage, and a test enforces the convention and global uniqueness. hasMissingMetaDescription (the --require-meta gate) now keys on the technical-seo.meta-description.missing code rather than a message prefix — the first consumer migrated to codes. schemaVersion bumped to 1.1 (additive: findings gained the code field). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: correct schemaVersion example value to 1.1 The api.md runSitemapAudit example printed '1.0', but SCHEMA_VERSION was bumped to '1.1' in 2.1.0. Align the doc with the actual value. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> * fix: cross-cutting reach unions all recommendations, not just the top buildPrioritizedFixes derived a cross-cutting fix's affectedPages, prevalencePct, and affectsHomepage from only the top recommendation's affectedUrls. The entry is identified by the factor (factorId / factorName), so its reach must cover the whole factor: a homepage hit only by a secondary recommendation read as affectsHomepage=false, and prevalence undercounted. Union every recommendation's pages instead, sorted homepage-first, so the three fields stay consistent and factor-level. Adds a regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent c71e743 commit d135ad6

43 files changed

Lines changed: 1808 additions & 229 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,27 @@
11
# Changelog
22

3+
## 2.1.0 (2026-06-03)
4+
5+
### Added
6+
- **Stable finding codes.** Every `AuditFinding` now carries a `code` namespaced as `<factor-id>.<check>[.<variant>]` (e.g. `technical-seo.h1.multiple`, `schema-validity.singleton.duplicate`), so agents and integrations key on a stable machine identifier instead of regex-matching the human `message` (which can change between releases). 212 codes across all 19 analyzers; the full registry is in [docs/finding-codes.md](docs/finding-codes.md). Codes follow a documented convention and are unique across the tool (enforced by a test). `AuditFinding.code` is required, so the compiler guarantees no finding ships without one.
7+
- `hasMissingMetaDescription` (the `--require-meta` gate) now keys on `technical-seo.meta-description.missing` rather than a message prefix — the first consumer migrated to codes.
8+
9+
### Changed
10+
- **`schemaVersion` bumped to `1.1`** (additive: findings gained the `code` field). Report shapes are otherwise unchanged.
11+
12+
## 2.0.0 (2026-06-03)
13+
14+
### Breaking
15+
- **`SitemapAuditReport.prioritizedFixes` is now a structured `PrioritizedFix[]`, not `string[]`.** Each entry is a typed object — `{ kind, id, title, recommendation, severity?, affectedPages, affectsHomepage, prevalencePct, avgGrade?, summary }` — so an AI agent can act on the ranked to-do list without regex-parsing prose. The human-readable one-liner is preserved on `.summary`; migrate by reading `prioritizedFixes.map(f => f.summary)`. The text/markdown reports are unchanged in spirit (they render the structured fixes, now spelling out every affected page).
16+
- **New `schemaVersion` field on `AuditReport` and `SitemapAuditReport`** (exported `SCHEMA_VERSION`, currently `"1.0"`). It versions the report's JSON shape independently of the npm package version so agent parsers can detect breaking drift instead of failing silently. Treat the absence of the field as "pre-2.0 / legacy shape."
17+
18+
### Added
19+
- **`--format agent` — a slim, agent-native decision output.** Returns `{ schemaVersion, tool, mode, url, score, grade, pass, criticalDefectCount, issues }` as JSON, where `issues` is the ranked `PrioritizedFix[]`, omitting the per-factor and per-page detail an agent would otherwise have to average and re-rank. Works for single-URL, sitemap, and static-output audits (single-page reuses the same critical-defect and cross-cutting aggregation over a one-page "site"); `--detect-platform` falls back to structured JSON. New `agentSummaryFromAudit()` / `agentSummaryFromSitemap()` exports, `AgentSummary` type, and `formatAgent` / `formatSitemapAgent` formatters.
20+
- **Critical per-page defects surfaced by impact, not prevalence (#42).** Sitemap and static-directory reports now include a `criticalDefects` rollup and a **Critical Defects** section (text + markdown) that lists binary, one-line-fix structural defects — an `<h1>` count other than one, a missing `<title>`, a missing meta description — **regardless of how few pages exhibit them**. Previously these were detected per page but lost in aggregation: `prioritizedFixes` ranked only by prevalence (so a defect on a single page was structurally excluded), the factor score averaged the defect away to a passing grade, and `crossCuttingIssues` was keyed by factor, never the specific defect. An unambiguous, high-impact defect on the most important page (e.g. a homepage split across four `<h1>`s, or a `/contact-us` page with none) appeared nowhere in the top-level summary. Now each defect names **every** offending page (homepage and high sitemap-`priority` pages first), and critical-severity defects are promoted to the **top** of `prioritizedFixes`. Shown even with `--top-issues`.
21+
- The end-of-report summaries no longer truncate: the Critical Defects block and each prioritized fix list **every** affected page (no "+N more"), and `prioritizedFixes` reports every cross-cutting issue ordered by prevalence rather than a top-5 slice — a fix the audit computed always reaches the report.
22+
- New `detectCriticalDefects()`, `buildCriticalDefects()`, and `SCHEMA_VERSION` exports plus `CriticalDefect`, `CriticalDefectGroup`, `CriticalDefectAffectedPage`, `CriticalDefectId`, `CriticalDefectSeverity`, and `PrioritizedFix` types. `AuditReport` gains `criticalDefects` and `schemaVersion`; `SitemapAuditReport` gains `criticalDefects` and `schemaVersion`; `SitemapPageResult` gains the page's sitemap `priority`.
23+
- Detection is independent of the weighted factor scores, so **no existing audit scores or grades change** (and exit codes are unaffected).
24+
325
## 1.13.0 (2026-05-31)
426

527
### Added

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@
99
- Audit **built HTML offline** in CI: a `next export` / `dist` / `out` directory, no network. [Static output](docs/cli.md#static-output-mode)
1010
- Detect the **platform / CMS / framework**: WordPress, Webflow, Shopify, Next.js, Vercel. [Platform detection](docs/cli.md#platform-detection)
1111
- Opt in to **Lighthouse, geographic, and agent-skill** factors. [Optional factors](docs/scoring.md#optional-factors)
12-
- `text`, `json`, and `markdown` output with **CI-friendly exit codes**. [CLI reference](docs/cli.md)
12+
- `text`, `json`, `markdown`, and `agent` output with **CI-friendly exit codes**. [CLI reference](docs/cli.md)
13+
- **Agent-native output**: a versioned `schemaVersion`, a slim `--format agent` decision, ranked structured fixes, and stable [finding codes](docs/finding-codes.md) so integrations key on codes, not prose. [API](docs/api.md#machine-readable-output-for-ai-agents)
1314
- Use as a **library** ([API](docs/api.md)) or from Claude Code via the **`/aeo` skill** ([skill](docs/skill.md)).
1415

1516
Website: [canonry.ai](https://canonry.ai)

docs/api.md

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,14 +33,26 @@ const report = await runSitemapAudit('https://example.com', {
3333
factors: ['schema-validity', 'structured-data'], // Optional subset
3434
})
3535

36-
console.log(report.aggregateGrade) // 'B+'
37-
console.log(report.pagesAudited) // 22
36+
console.log(report.schemaVersion) // '1.1', JSON shape version (see "Machine-readable output")
37+
console.log(report.aggregateGrade) // 'B+'
38+
console.log(report.pagesAudited) // 22
39+
console.log(report.criticalDefects) // Binary per-page defects (multiple/missing H1, missing title/meta), grouped by defect
3840
console.log(report.crossCuttingIssues) // Per-factor rollup with affectedUrls for every recommendation
39-
console.log(report.prioritizedFixes) // Top 5 fixes ranked by site-wide impact
41+
console.log(report.prioritizedFixes) // Ranked PrioritizedFix[]: critical defects first, then cross-cutting by impact
4042
```
4143

4244
Each entry in `crossCuttingIssues[].topIssues` carries a `recommendation` plus the exact `affectedUrls` so you can attribute each problem to specific pages, e.g. "FAQPage duplicate" pointing at every blog post that has it.
4345

46+
`criticalDefects` surfaces **binary structural defects by impact, not prevalence**. The cross-cutting rollup ranks by how many pages a factor affects, so an unambiguous one-line-fix defect on a single important page (a homepage split across four `<h1>`s, or a `/contact-us` page with none) would otherwise be averaged into a passing factor grade and excluded from `prioritizedFixes`. Each group names the offending pages (homepage and high sitemap-`priority` pages first), and the critical-severity ones lead `prioritizedFixes`.
47+
48+
### Machine-readable output (for AI agents)
49+
50+
`--format json` and these return values are the contract for programmatic use. The report is built to be acted on, not just rendered:
51+
52+
- **`schemaVersion`** (on `AuditReport` and `SitemapAuditReport`, exported as `SCHEMA_VERSION`) versions the JSON shape independently of the npm version. Pin to it and treat a major bump as breaking; treat its absence as a pre-2.0 report.
53+
- **`prioritizedFixes: PrioritizedFix[]`** is the ranked, pre-computed to-do list, so an agent need not average factor scores and re-rank. Each fix carries a stable `id` (a defect id like `"multiple-h1"` or a factor id like `"technical-seo"`), `kind`, an optional `severity`, the complete `affectedPages` array (never truncated), `affectsHomepage`, `prevalencePct`, and a human `summary`.
54+
- **Stable identifiers** everywhere: the decision surface (`criticalDefects[].id`, `prioritizedFixes[].id` / `kind`) and every individual factor finding (`factors[].findings[].code`, e.g. `technical-seo.h1.multiple`) carry stable codes, so integrations key on codes, not on matching message strings. The full code registry is in [finding-codes.md](finding-codes.md).
55+
4456
## Static output (offline, from disk)
4557

4658
```ts
@@ -55,6 +67,7 @@ if (result.kind === 'single') {
5567
console.log(result.report.overallGrade) // single .html file → AuditReport
5668
} else {
5769
console.log(result.report.aggregateGrade) // directory → SitemapAuditReport shape
70+
console.log(result.report.criticalDefects)
5871
console.log(result.report.crossCuttingIssues)
5972
}
6073
```

docs/cli.md

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,8 +19,15 @@ npx @ainyc/aeo-audit https://example.com --format json
1919

2020
# Markdown report
2121
npx @ainyc/aeo-audit https://example.com --format markdown
22+
23+
# Agent summary: the slim JSON decision, not the full report
24+
npx @ainyc/aeo-audit https://example.com --sitemap --format agent
2225
```
2326

27+
`--format json` is the contract for programmatic and agent consumers: every report carries a `schemaVersion` (so a parser can detect breaking shape drift) and sitemap reports expose a `criticalDefects` rollup plus a ranked `prioritizedFixes` array of structured objects. See [api.md](api.md#machine-readable-output-for-ai-agents) for the field shapes.
28+
29+
`--format agent` returns just the decision, not the report: `{ schemaVersion, tool, mode, url, score, grade, pass, criticalDefectCount, issues }`, where `issues` is the ranked `PrioritizedFix[]` (critical defects first, then cross-cutting by prevalence). It omits the per-factor and per-page detail so an agent can act without averaging and re-ranking scores itself. Works for single-URL, sitemap, and static-output audits; in `--detect-platform` mode it falls back to the structured JSON.
30+
2431
## Running a subset of factors
2532

2633
```bash
@@ -76,7 +83,7 @@ npx @ainyc/aeo-audit https://example.com --sitemap https://example.com/sitemap.x
7683
# Cap the number of pages (default 200, sorted by sitemap priority)
7784
npx @ainyc/aeo-audit https://example.com --sitemap --limit 50
7885

79-
# Skip per-page output and show only cross-cutting issues
86+
# Skip per-page output and show only the cross-cutting issues and critical defects
8087
npx @ainyc/aeo-audit https://example.com --sitemap --top-issues
8188

8289
# Rewrite each <loc>'s origin to the target you named (audit staging with prod's sitemap)
@@ -92,6 +99,8 @@ Auto-discovery checks `/sitemap.xml` → `/sitemap-index.xml` → `Sitemap:` dir
9299

93100
When the sitemap has more URLs than `--limit`, the run audits the highest-priority pages and prints a notice to stderr listing how many were skipped and how to audit them all.
94101

102+
A **Critical Defects** section lists binary, one-line-fix structural defects (an `<h1>` count other than one, a missing `<title>`, a missing meta description) surfaced **regardless of how few pages they affect**, with the offending pages named (homepage and high sitemap-`priority` pages first). These would otherwise be averaged into a passing factor grade and excluded from the prevalence-ranked fixes; the critical-severity ones also lead the prioritized fix list. The section is shown even with `--top-issues`. See the machine-readable shapes in [api.md](api.md#machine-readable-output-for-ai-agents).
103+
95104
The optional in-process factors are honored per page: pass `--include-geo` and/or `--include-agent-skills` to add them to every audited page. `--lighthouse` is the exception: it cannot be combined with `--sitemap` because each PageSpeed Insights call takes 15-30s.
96105

97106
## Static-output mode
@@ -184,14 +193,14 @@ When fetching `/llms.txt`, `/llms-full.txt`, `/robots.txt`, and `/sitemap.xml` t
184193

185194
| Flag | Description |
186195
|------|-------------|
187-
| `--format <type>` | Output format: `text` (default), `json`, `markdown` |
196+
| `--format <type>` | Output format: `text` (default), `json`, `markdown`, `agent`. `agent` emits the slim JSON decision (score, pass gate, `criticalDefectCount`, ranked `issues`) for AI agents. |
188197
| `--factors <list>` | Comma-separated factor IDs to run (runs all if omitted) |
189198
| `--include-geo` | Include the optional geographic signals factor |
190199
| `--include-agent-skills` | Include the optional agent skill exposure factor |
191200
| `--lighthouse` | Include the optional Lighthouse factor (Performance + Accessibility + Best Practices, mobile strategy) via Google PageSpeed Insights. Single-URL only; cannot combine with `--sitemap` or `--detect-platform`. Adds ~15-30s. Set `PAGESPEED_API_KEY` env var to lift anonymous rate limits. |
192201
| `--sitemap [url]` | Audit all pages from the sitemap. Auto-discovery tries `/sitemap.xml`, then `/sitemap-index.xml`, then `Sitemap:` directives in `/robots.txt`. Pass an explicit URL to override. |
193202
| `--limit <n>` | Max pages to audit in sitemap mode (default 200, sorted by sitemap priority) |
194-
| `--top-issues` | In sitemap mode, skip per-page output and show only cross-cutting issues |
203+
| `--top-issues` | In sitemap mode, skip per-page output and show only the cross-cutting issues and critical defects |
195204
| `--detect-platform` | Identify the platform/CMS/framework powering the site instead of running an audit |
196205
| `--urls <src>` | In `--detect-platform` mode, run on multiple URLs. `<src>` is a file path (one URL per line), a comma-separated list, or `-` for stdin |
197206
| `--concurrency <n>` | In `--detect-platform` batch mode, max in-flight fetches (default 5) |

0 commit comments

Comments
 (0)