Skip to content

Commit 7789685

Browse files
author
tuanductran
committed
feat(packages): add audit and build drift checks
1 parent ba60b8d commit 7789685

19 files changed

Lines changed: 262 additions & 11 deletions

File tree

AGENTS.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ statistics.
129129
| `check-duplicates` | Duplicate title detection |
130130
| `check-tags` | Tag hygiene validation |
131131
| `generate-stats` | Statistics report (`--text`) |
132+
| `audit` | Combined structured maintenance audit (`--json`) |
132133

133134
**Package scripts:**
134135

@@ -167,7 +168,7 @@ Also exposes a programmatic API.
167168

168169
| Command | Description |
169170
| :--- | :--- |
170-
| `build` | Build AGENTS.md (`--all` or `--skill=<name>`) |
171+
| `build` | Build AGENTS.md (`--all`, `--skill=<name>`, or `--check`) |
171172
| `validate` | Validate rule frontmatter and structure |
172173
| `search` | Search rules (`--query=`, `--tag=`, `--skill=`, `--impact=`, `--json`) |
173174
| `export` | Export rules to JSON/CSV (`--format=`, `--out=`, `--skill=`) |
@@ -180,6 +181,7 @@ Also exposes a programmatic API.
180181
| :--- | :--- |
181182
| `build` | `tsdown` — compile to `dist/` |
182183
| `build-all` | Build AGENTS.md for all skills |
184+
| `build-check` | Verify all generated AGENTS.md files are up to date |
183185
| `build-api` | Build `nextdns-api` only |
184186
| `build-cli` | Build `nextdns-cli` only |
185187
| `build-frontend` | Build `nextdns-frontend` only |

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,14 @@ This file records user-facing, contributor-facing, and distribution-relevant cha
1111
- Public architecture, contribution, and documentation standards.
1212
- Root contribution, security, and code-of-conduct entry points.
1313
- GitHub issue and pull-request templates.
14+
- A structured `audit` command with JSON output for maintenance quality gates.
15+
- `build --check` and root `pnpm build:check` for generated-output drift detection.
16+
- `--version` support and version helpers for both package CLIs.
1417

1518
### Changed
1619

1720
- Documentation now distinguishes public project guidance, agent context, agent workflows, source rules, and generated output.
21+
- Package documentation now covers audit, public API metadata, and generated-output checks.
1822

1923
### Maintenance notes
2024

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,9 @@ Rules are categorized into two types to guide AI precision:
8585
| **Check broken links** | `pnpm run lint:links` |
8686
| **Check duplicate titles** | `pnpm check-duplicates` |
8787
| **Check tag hygiene** | `pnpm check-tags` |
88+
| **Run package audit** | `pnpm run audit` |
8889
| **Build all skills** | `pnpm build:skills` |
90+
| **Check generated output** | `pnpm build:check` |
8991
| **Sync rule counts** | `pnpm update-counts` |
9092
| **Statistics report** | `pnpm stats` |
9193
| **Search rules** | `pnpm rule-search -- --query=<text>` |

docs/ARCHITECTURE.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ The diagram is conceptual. It describes ownership and dependency direction, not
4545
| Skill manifests | `skills/*/SKILL.md` | Define activation metadata, categories, versions, and rule registration | Update with any rule addition or removal |
4646
| Domain rules | `skills/*/rules/` | Provide the knowledge injected into agents | Edit directly using the rule template |
4747
| Generated output | `skills/*/AGENTS.md` | Aggregate rules for agent consumption | Never edit by hand; run `pnpm build:skills` |
48-
| Validation packages | `packages/nextdns-skills-build`, `packages/nextdns-scripts` | Build, validate, search, export, test, and check content quality | Change code with tests and type-checking |
48+
| Validation packages | `packages/nextdns-skills-build`, `packages/nextdns-scripts` | Build, validate, search, export, audit, test, and check content quality | Change code with tests and type-checking |
4949
| Structural schema | `data/schemas/profile.json` | Define profile data shape used by relevant guidance | Update together with structural changes |
5050

5151
## Skill categories
@@ -70,9 +70,10 @@ The root package uses pnpm scripts and Turbo to coordinate the two workspace pac
7070
| :--- | :--- |
7171
| Public docs only | `pnpm lint:md`, `pnpm lint:links`, `git diff --check` |
7272
| Rule content | `pnpm build:skills`, `pnpm lint:rules`, `pnpm lint:all`, `pnpm test` |
73-
| Manifest or counts | `pnpm build:skills`, `pnpm update-counts`, `pnpm check-duplicates`, `pnpm check-tags` |
73+
| Manifest or counts | `pnpm build:skills`, `pnpm build:check`, `pnpm update-counts`, `pnpm check-duplicates`, `pnpm check-tags` |
7474
| TypeScript or package code | `pnpm lint`, `pnpm types:check`, `pnpm test` |
7575
| CI or workflow changes | Applicable checks plus a review of changed paths and permissions |
76+
| Package or generated-output API | `pnpm run audit`, `pnpm build:check`, package tests, and type-check |
7677

7778
A link check is evidence about an external service at a point in time, not a guarantee that a URL will remain available. Treat protected pages, rate limits, downloads, redirects, and transient server errors as distinct categories. Replace a genuinely stale canonical link; document an expected exception instead of deleting useful source context.
7879

docs/CONTRIBUTING.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ After editing, run the smallest useful checks first. For rule or manifest change
4747

4848
```bash
4949
pnpm build:skills
50+
pnpm build:check
51+
pnpm run audit
5052
pnpm lint:fix
5153
pnpm lint:all
5254
pnpm check-duplicates
@@ -56,7 +58,7 @@ pnpm test
5658
git diff --check
5759
```
5860

59-
For documentation-only changes, `pnpm lint:md`, `pnpm lint:links`, and `git diff --check` are the minimum checks. Add `pnpm lint:all` when the change touches referenced URLs or CI. If a remote service returns 403, 429, 5xx, or a download response, record the exception and do not treat the response as proof that the documentation is invalid without further review.
61+
For documentation-only changes, `pnpm lint:md`, `pnpm lint:links`, and `git diff --check` are the minimum checks. For package or generated-output changes, also run `pnpm run audit` and `pnpm build:check`. Add `pnpm lint:all` when the change touches referenced URLs or CI. If a remote service returns 403, 429, 5xx, or a download response, record the exception and do not treat the response as proof that the documentation is invalid without further review.
6062

6163
## Pull request expectations
6264

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,9 +41,11 @@
4141
"build:frontend": "turbo run build-frontend",
4242
"build:integrations": "turbo run build-integrations",
4343
"build:skills": "turbo run build-all",
44+
"build:check": "nextdns-skills-build build --all --check",
4445
"build:ui": "turbo run build-ui",
4546
"check-duplicates": "turbo run check-duplicates",
4647
"check-tags": "turbo run check-tags",
48+
"audit": "nextdns-skills-scripts audit",
4749
"dev": "turbo run dev",
4850
"format": "vp fmt",
4951
"format:check": "vp fmt --check",

packages/nextdns-scripts/package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
{
22
"name": "nextdns-skills-scripts",
33
"version": "1.0.0",
4-
"description": "Maintenance scripts for NextDNS Skills: validate rule integrity, sync counts, check duplicates/tags, and generate stats",
4+
"description": "Maintenance scripts for NextDNS Skills: audit, validate rule integrity, sync counts, check duplicates/tags, and generate stats",
55
"keywords": [
66
"maintenance",
77
"nextdns",
@@ -27,6 +27,7 @@
2727
"build": "vp pack",
2828
"check-duplicates": "nextdns-skills-scripts check-duplicates",
2929
"check-tags": "nextdns-skills-scripts check-tags",
30+
"audit": "nextdns-skills-scripts audit",
3031
"dev": "vp pack --watch",
3132
"generate-stats": "nextdns-skills-scripts generate-stats",
3233
"prepublishOnly": "pnpm build",
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
import { describe, expect, it } from 'vitest';
2+
3+
import { formatAuditText, runAudit } from '../audit.js';
4+
import { getPackageVersion } from '../version.js';
5+
6+
describe('runAudit', () => {
7+
it('returns a passing structured report for the repository', () => {
8+
const report = runAudit();
9+
10+
expect(report.passed).toBe(true);
11+
expect(report.ruleCount).toBeGreaterThan(0);
12+
expect(report.checks).toHaveLength(5);
13+
expect(report.checks.every((check) => check.passed)).toBe(true);
14+
expect(report.statistics.totalRules).toBe(report.ruleCount);
15+
});
16+
17+
it('formats a concise human-readable summary', () => {
18+
const report = runAudit();
19+
const output = formatAuditText(report);
20+
21+
expect(output).toContain('NextDNS Skills audit');
22+
expect(output).toContain('PASS referential-integrity');
23+
expect(output).toContain('Audit passed.');
24+
});
25+
});
26+
27+
describe('getPackageVersion', () => {
28+
it('returns a semantic package version', () => {
29+
expect(getPackageVersion()).toMatch(/^\d+\.\d+\.\d+$/);
30+
});
31+
});
Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
#!/usr/bin/env node
2+
/**
3+
* Run the complete repository maintenance audit.
4+
*
5+
* Usage:
6+
* nextdns-skills-scripts audit
7+
* nextdns-skills-scripts audit --json
8+
*/
9+
10+
import path from 'node:path';
11+
import { fileURLToPath } from 'node:url';
12+
13+
import { checkDuplicateTags, checkDuplicateTitles, loadAllRules } from './check-duplicates.js';
14+
import { buildReport, type StatsReport } from './generate-stats.js';
15+
import { validateFrontmatter, validateReferentialIntegrity } from './validate-rules.js';
16+
import { validateTags } from './check-tags.js';
17+
18+
const __dirname = path.dirname(fileURLToPath(import.meta.url));
19+
const REPO_ROOT = path.join(__dirname, '../../..');
20+
const SKILLS_DIR = path.join(REPO_ROOT, 'skills');
21+
22+
export interface AuditCheck {
23+
name: 'referential-integrity' | 'frontmatter' | 'tags' | 'duplicate-titles' | 'duplicate-tags';
24+
passed: boolean;
25+
errors: number;
26+
warnings: number;
27+
}
28+
29+
export interface AuditReport {
30+
generatedAt: string;
31+
passed: boolean;
32+
ruleCount: number;
33+
checks: AuditCheck[];
34+
statistics: StatsReport;
35+
}
36+
37+
function suppressOutput<T>(fn: () => T): T {
38+
const log = console.log;
39+
const error = console.error;
40+
console.log = () => undefined;
41+
console.error = () => undefined;
42+
try {
43+
return fn();
44+
} finally {
45+
console.log = log;
46+
console.error = error;
47+
}
48+
}
49+
50+
export function runAudit(skillsDir: string = SKILLS_DIR, repoRoot: string = REPO_ROOT): AuditReport {
51+
const rules = suppressOutput(() => loadAllRules(skillsDir));
52+
const duplicateTitles = suppressOutput(() => checkDuplicateTitles(rules));
53+
const duplicateTags = suppressOutput(() => checkDuplicateTags(rules));
54+
const referentialIntegrity = suppressOutput(() => validateReferentialIntegrity(skillsDir));
55+
const frontmatter = suppressOutput(() => validateFrontmatter(skillsDir));
56+
const tags = suppressOutput(() => validateTags(skillsDir));
57+
58+
const checks: AuditCheck[] = [
59+
{
60+
name: 'referential-integrity',
61+
passed: referentialIntegrity,
62+
errors: referentialIntegrity ? 0 : 1,
63+
warnings: 0,
64+
},
65+
{
66+
name: 'frontmatter',
67+
passed: frontmatter,
68+
errors: frontmatter ? 0 : 1,
69+
warnings: 0,
70+
},
71+
{
72+
name: 'tags',
73+
passed: tags,
74+
errors: tags ? 0 : 1,
75+
warnings: 0,
76+
},
77+
{
78+
name: 'duplicate-titles',
79+
passed: duplicateTitles.errors === 0,
80+
errors: duplicateTitles.errors,
81+
warnings: duplicateTitles.warnings,
82+
},
83+
{
84+
name: 'duplicate-tags',
85+
passed: duplicateTags === 0,
86+
errors: 0,
87+
warnings: duplicateTags,
88+
},
89+
];
90+
91+
return {
92+
generatedAt: new Date().toISOString(),
93+
passed: checks.every((check) => check.passed),
94+
ruleCount: rules.length,
95+
checks,
96+
statistics: buildReport(skillsDir, repoRoot),
97+
};
98+
}
99+
100+
export function formatAuditText(report: AuditReport): string {
101+
const lines = [
102+
`NextDNS Skills audit — ${report.generatedAt}`,
103+
`Rules checked: ${report.ruleCount}`,
104+
'',
105+
...report.checks.map(
106+
(check) =>
107+
`${check.passed ? 'PASS' : 'FAIL'} ${check.name} ` +
108+
`(errors: ${check.errors}, warnings: ${check.warnings})`
109+
),
110+
'',
111+
report.passed ? 'Audit passed.' : 'Audit failed.',
112+
];
113+
return lines.join('\n');
114+
}
115+
116+
export function run(): void {
117+
const args = process.argv.slice(2);
118+
const report = runAudit();
119+
if (args.includes('--json')) {
120+
console.log(JSON.stringify(report, null, 2));
121+
} else {
122+
console.log(formatAuditText(report));
123+
}
124+
process.exitCode = report.passed ? 0 : 1;
125+
}
126+
127+
if (fileURLToPath(import.meta.url) === process.argv[1]) run();

packages/nextdns-scripts/src/cli.ts

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,12 +12,15 @@
1212
* nextdns-skills-scripts <command> [options]
1313
*/
1414

15+
import { getPackageVersion } from './version.js';
16+
1517
const COMMANDS: Record<string, string> = {
1618
'validate-rules': './validate-rules.mjs',
1719
'update-counts': './update-counts.mjs',
1820
'check-duplicates': './check-duplicates.mjs',
1921
'check-tags': './check-tags.mjs',
2022
'generate-stats': './generate-stats.mjs',
23+
audit: './audit.mjs',
2124
};
2225

2326
const HELP = `Usage: nextdns-skills-scripts <command> [options]
@@ -28,12 +31,22 @@ Commands:
2831
check-duplicates Check for duplicate/identical tag sets across rules
2932
check-tags Check tag hygiene across rules
3033
generate-stats Print repository-wide rule statistics (--text)
34+
audit Run the complete maintenance audit (--json)
35+
36+
Options:
37+
--version Print the package version
38+
--help Show this help message
3139
32-
Run without a command, or with --help, to see this message.`;
40+
Run without a command to see this message.`;
3341

3442
async function main(): Promise<void> {
3543
const [subcommand, ...rest] = process.argv.slice(2);
3644

45+
if (subcommand === '--version' || subcommand === '-v') {
46+
console.log(getPackageVersion());
47+
return;
48+
}
49+
3750
if (!subcommand || subcommand === '--help' || subcommand === '-h') {
3851
console.log(HELP);
3952
process.exit(subcommand ? 0 : 1);

0 commit comments

Comments
 (0)