Skip to content

Commit f8af379

Browse files
authored
feat(skills): add architecture and development workflow skills
Add 7 new skills for architecture decisions and development workflows including system design review, API design, database schema design, CI/CD pipeline setup, code organization, and dev environment configuration.
1 parent 0ee3179 commit f8af379

7 files changed

Lines changed: 1684 additions & 0 deletions

File tree

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
---
2+
name: audit-api-consistency
3+
description: Audit existing API endpoints for consistency when the user asks to check API quality, review API patterns, audit endpoints, or find API inconsistencies
4+
owner: chalk
5+
version: "1.0.0"
6+
metadata-version: "1"
7+
allowed-tools: Read, Glob, Grep, Bash
8+
argument-hint: "[optional: specific area or endpoint pattern to audit]"
9+
---
10+
11+
# Audit API Consistency
12+
13+
## Overview
14+
15+
Scan the codebase for API route definitions and analyze them for consistency across naming, HTTP method usage, error response shapes, pagination, auth patterns, and versioning. Produce a structured report with severity-rated findings and concrete fix recommendations. This is an audit skill -- it analyzes what exists, it does not create new APIs.
16+
17+
## Workflow
18+
19+
1. **Read project context** -- Read `.chalk/docs/engineering/` for:
20+
- Existing API design documents that define the intended conventions
21+
- Architecture docs describing the API layer
22+
- Any API style guides or conventions documents
23+
- Prior audit reports to check if previously flagged issues were resolved
24+
25+
2. **Discover route definitions** -- Scan the codebase to find all API endpoints:
26+
- Use Grep to search for route registration patterns:
27+
- Express/Koa: `router.get`, `router.post`, `app.use`, etc.
28+
- Django: `path(`, `url(`, `@api_view`
29+
- Spring: `@GetMapping`, `@PostMapping`, `@RequestMapping`
30+
- FastAPI: `@app.get`, `@router.post`
31+
- Rails: `resources :`, `get '`, `post '`
32+
- Next.js: files in `app/api/` or `pages/api/`
33+
- Build a complete inventory of endpoints with: method, path, handler file, and handler function
34+
35+
3. **Analyze each consistency category** -- For each category below, compare every endpoint against the dominant pattern. The dominant pattern is the convention used by the majority of endpoints.
36+
37+
4. **Classify findings by severity** -- Each inconsistency gets one of:
38+
- **Critical**: Will cause client errors or security issues (e.g., missing auth on a protected endpoint)
39+
- **High**: Breaks client assumptions or developer experience (e.g., different error shapes)
40+
- **Medium**: Inconsistency that causes confusion but no runtime issues (e.g., mixed naming conventions)
41+
- **Low**: Minor deviation that could be addressed opportunistically (e.g., inconsistent sort defaults)
42+
43+
5. **Generate fix recommendations** -- For each finding, provide:
44+
- The current state (what is inconsistent)
45+
- The expected state (what it should be, based on dominant pattern)
46+
- A concrete code change or migration path
47+
- Whether it is a breaking change for existing clients
48+
49+
6. **Output the report** -- Present the report in conversation or write to `.chalk/docs/engineering/<n>_api_audit.md` if the user requests a persisted report.
50+
51+
7. **Summarize** -- Provide an executive summary with total findings by severity, the most impactful issues, and a recommended prioritization for fixes.
52+
53+
## Consistency Categories
54+
55+
### 1. URL Naming Patterns
56+
57+
Check for:
58+
- **Plural vs. singular nouns**: `/users` vs. `/user` (pick one, apply everywhere)
59+
- **Casing consistency**: `/user-profiles` vs. `/userProfiles` vs. `/user_profiles`
60+
- **Nesting depth**: Are some resources nested 3+ levels while others are flat?
61+
- **Verb usage**: `/api/getUsers` (RPC-style) mixed with `/api/users` (REST-style)
62+
- **ID format in paths**: `:id` vs. `:userId` vs. `{id}` -- consistent parameter naming
63+
- **Trailing slashes**: Some paths with `/`, some without
64+
65+
### 2. HTTP Method Usage
66+
67+
Check for:
68+
- **GET with side effects**: GET endpoints that modify data (should be POST/PATCH/DELETE)
69+
- **POST for retrieval**: POST used to fetch data that should be a GET with query params
70+
- **PUT vs. PATCH confusion**: PUT used for partial updates (should be PATCH) or PATCH used for full replacement (should be PUT)
71+
- **DELETE semantics**: Some DELETEs return the deleted resource, others return 204 No Content -- pick one
72+
- **Correct status codes**: POST returning 200 instead of 201, DELETE returning 200 instead of 204
73+
74+
### 3. Error Response Format
75+
76+
Check for:
77+
- **Shape consistency**: Do all endpoints return errors in the same JSON structure?
78+
- **Error code presence**: Do all errors include a machine-readable error code?
79+
- **Validation error format**: Are field-level validation errors structured consistently?
80+
- **HTTP status code accuracy**: 400 vs. 422 for validation, 401 vs. 403 for auth
81+
- **Error middleware**: Is error formatting centralized or scattered across handlers?
82+
- **Stack traces in production**: Are internal details leaked in error responses?
83+
84+
### 4. Pagination Approach
85+
86+
Check for:
87+
- **Pagination presence**: Are all list endpoints paginated? Flag any that return unbounded results
88+
- **Pagination style**: Cursor vs. offset -- is it consistent across all list endpoints?
89+
- **Parameter naming**: `page`/`limit` vs. `offset`/`count` vs. `cursor`/`size`
90+
- **Default values**: Are default page sizes consistent?
91+
- **Maximum limits**: Do all paginated endpoints enforce a maximum page size?
92+
- **Response meta shape**: Is pagination metadata structured the same way everywhere?
93+
94+
### 5. Authentication and Authorization
95+
96+
Check for:
97+
- **Auth middleware coverage**: Are all non-public endpoints protected by auth middleware?
98+
- **Auth header format**: Consistent use of `Authorization: Bearer <token>` or API key headers
99+
- **Missing auth on sensitive endpoints**: POST/PUT/PATCH/DELETE without auth checks
100+
- **Role/scope checking**: Is authorization granularity consistent?
101+
- **Public endpoint documentation**: Are intentionally public endpoints clearly marked?
102+
103+
### 6. Versioning
104+
105+
Check for:
106+
- **Version presence**: Is API versioning used at all? If so, is it consistent?
107+
- **Version format**: URL-based (`/api/v1/`) vs. header-based (`Accept: application/vnd.api.v1+json`)
108+
- **Unversioned endpoints**: Endpoints that bypass the versioning scheme
109+
- **Deprecated versions**: Are old versions still active without deprecation headers?
110+
111+
### 7. Response Envelope
112+
113+
Check for:
114+
- **Wrapper consistency**: Do all endpoints use the same response wrapper (`{ data }`, `{ data, meta }`, or raw)?
115+
- **Single vs. collection distinction**: Single resources returning arrays or collections returning unwrapped objects
116+
- **Null handling**: `null` vs. absent key vs. empty string for missing optional fields
117+
- **Timestamp format**: ISO 8601 everywhere or mixed formats?
118+
119+
## Report Format
120+
121+
```markdown
122+
# API Consistency Audit Report
123+
124+
Date: <YYYY-MM-DD>
125+
Scope: <All endpoints | Specific area>
126+
Total endpoints scanned: <count>
127+
128+
## Executive Summary
129+
130+
<2-3 sentences. Overall consistency score, most impactful issues, recommended priority.>
131+
132+
### Findings by Severity
133+
134+
| Severity | Count |
135+
|----------|-------|
136+
| Critical | <n> |
137+
| High | <n> |
138+
| Medium | <n> |
139+
| Low | <n> |
140+
141+
## Dominant Patterns (Established Conventions)
142+
143+
<Document the patterns used by the majority of endpoints. These are the "correct" baseline.>
144+
145+
| Category | Dominant Pattern | Adoption Rate |
146+
|----------|-----------------|---------------|
147+
| URL casing | kebab-case | 85% (34/40) |
148+
| Pluralization | Plural nouns | 90% (36/40) |
149+
| Error shape | `{ error: { code, message, status, details } }` | 75% (30/40) |
150+
| Pagination | Offset with `page`/`limit` | 100% (8/8 list endpoints) |
151+
| Auth | Bearer token via middleware | 92% (37/40) |
152+
153+
## Findings
154+
155+
### Critical
156+
157+
**[C-1]** Missing auth on `POST /api/v1/admin/settings`
158+
- **File**: `src/routes/admin.ts:45`
159+
- **Issue**: Endpoint modifies system settings but has no auth middleware
160+
- **Expected**: Auth middleware with `admin` role check
161+
- **Fix**: Add `requireAuth('admin')` middleware
162+
- **Breaking**: No
163+
164+
### High
165+
166+
**[H-1]** Inconsistent error shape in billing endpoints
167+
- **File**: `src/routes/billing.ts`
168+
- **Issue**: Returns `{ "message": "error" }` instead of standard `{ "error": { "code": "...", "message": "..." } }`
169+
- **Expected**: Use shared error middleware
170+
- **Fix**: Replace manual error returns with `throw new AppError('BILLING_ERROR', message)`
171+
- **Breaking**: Yes -- clients parsing billing errors will need to update
172+
173+
### Medium
174+
175+
...
176+
177+
### Low
178+
179+
...
180+
181+
## Migration Recommendations
182+
183+
### Priority 1: Critical and High (do now)
184+
185+
<Ordered list of fixes with estimated effort>
186+
187+
### Priority 2: Medium (next sprint)
188+
189+
<Ordered list>
190+
191+
### Priority 3: Low (opportunistic)
192+
193+
<Fixes to apply when touching these files for other reasons>
194+
195+
## Legacy Endpoints
196+
197+
<List endpoints that are intentionally inconsistent due to backwards compatibility. Document why they are exempt and whether a migration is planned.>
198+
```
199+
200+
## Scanning Strategy
201+
202+
When scanning a large codebase, follow this order for efficiency:
203+
204+
1. **Find the router/route files first** -- Grep for the framework's routing pattern to locate all route files
205+
2. **Extract the endpoint inventory** -- Build the full list before analyzing
206+
3. **Check error middleware/handler** -- Find the centralized error handling to understand the intended pattern
207+
4. **Spot-check endpoints** -- Read 3-5 endpoint handlers in full to understand the actual implementation pattern
208+
5. **Compare outliers** -- Focus analysis time on endpoints that deviate from the dominant pattern
209+
210+
## Anti-patterns
211+
212+
- **Only checking names, not behavior** -- URL naming is the easiest thing to audit but the least impactful. Inconsistent error shapes and missing auth are far more dangerous. Always audit behavior (error handling, auth, pagination) before cosmetic naming.
213+
- **Ignoring legacy endpoints** -- Old endpoints that predate current conventions should still be cataloged. Document them as "legacy, migration planned" or "legacy, exempt" -- but do not pretend they do not exist.
214+
- **Not suggesting a migration path** -- Flagging inconsistencies without explaining how to fix them is not useful. Every finding must include a concrete fix and whether it is a breaking change.
215+
- **Auditing against an ideal, not the project's own conventions** -- The correct convention is whatever the majority of the codebase uses, not what a blog post says. If the project uses `snake_case` URLs, do not flag them as wrong because REST guides prefer `kebab-case`.
216+
- **Missing auth gaps** -- The single most valuable finding in an API audit is an endpoint that should require auth but does not. Always prioritize auth coverage analysis.
217+
- **One-time audit with no follow-up** -- An audit is only valuable if issues get fixed. Include a prioritized action plan and suggest re-running the audit after fixes are applied.

0 commit comments

Comments
 (0)