Skip to content

Commit e110ad9

Browse files
committed
Add API benchmark suite for issue 30
1 parent 9d5dd45 commit e110ad9

12 files changed

Lines changed: 2348 additions & 2 deletions

.env.benchmark.example

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Copy to .env.benchmark or .env.benchmark.local and adjust for local/staging runs.
2+
3+
# If unset, the benchmark runner starts the local Express app on a random loopback port.
4+
# BENCHMARK_TARGET_URL=http://localhost:4000
5+
6+
# Use a dedicated staging benchmark token for auth-protected routes.
7+
# If unset, local mode creates a short-lived token from JWT_SECRET.
8+
# BENCHMARK_AUTH_TOKEN=
9+
10+
# Local JWT secret used only when auto-generating a benchmark token.
11+
JWT_SECRET=development-secret
12+
13+
# Default full-run load profile. CLI flags override these values.
14+
BENCHMARK_REQUESTS_PER_ENDPOINT=8
15+
BENCHMARK_CONCURRENCY=2
16+
BENCHMARK_WARMUP_REQUESTS=1
17+
BENCHMARK_TIMEOUT_MS=10000
18+
19+
# Optional metadata included in reports.
20+
BENCHMARK_MACHINE_TYPE=local workstation or CI runner
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: API benchmark smoke
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "apps/api/**"
7+
- "benchmarks/**"
8+
- "package.json"
9+
- "package-lock.json"
10+
- ".github/workflows/api-benchmark-smoke.yml"
11+
workflow_dispatch:
12+
13+
jobs:
14+
smoke-benchmark:
15+
runs-on: ubuntu-latest
16+
steps:
17+
- uses: actions/checkout@v4
18+
- uses: actions/setup-node@v4
19+
with:
20+
node-version: 20
21+
cache: npm
22+
- run: npm ci
23+
- run: npm run benchmark:smoke
24+
env:
25+
JWT_SECRET: benchmark-ci-secret

apps/api/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"scripts": {
66
"dev": "node src/server.js",
77
"start": "node src/server.js",
8-
"test": "node --test src/tests"
8+
"test": "node --test \"src/tests/**/*.js\""
99
},
1010
"dependencies": {
1111
"cors": "^2.8.5",

benchmarks/README.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# API Benchmark Suite
2+
3+
This directory contains a reproducible benchmark suite for the platform API.
4+
5+
## Coverage
6+
7+
The suite covers every route mounted under `/api/` in `apps/api/src/app.js` plus `/health` for server readiness:
8+
9+
- `POST /api/auth/register`
10+
- `POST /api/auth/login`
11+
- `GET /api/auth/oauth/:provider/callback`
12+
- `POST /api/auth/refresh`
13+
- `GET /api/users`
14+
- `POST /api/users`
15+
- `GET /api/jobs`
16+
- `POST /api/jobs`
17+
- `GET /api/proposals`
18+
- `POST /api/proposals`
19+
- `POST /api/payments`
20+
- `GET /api/reviews`
21+
- `POST /api/reviews`
22+
- `GET /api/messages`
23+
- `POST /api/messages`
24+
- `GET /api/notifications`
25+
- `POST /api/notifications`
26+
- `POST /api/uploads`
27+
- `GET /api/search?q=...`
28+
- `GET /api/admin/metrics` with a benchmark token
29+
30+
Payloads live in `routes.mjs` so reviewers can inspect and tune realistic request bodies without changing the runner.
31+
32+
## Commands
33+
34+
From the repository root:
35+
36+
```bash
37+
npm run benchmark
38+
```
39+
40+
Smoke mode for CI:
41+
42+
```bash
43+
npm run benchmark:smoke
44+
```
45+
46+
Optional overrides:
47+
48+
```bash
49+
npm run benchmark -- --requests 20 --concurrency 4 --warmup 2
50+
npm run benchmark:smoke -- --requests 3 --concurrency 1
51+
```
52+
53+
## Target selection
54+
55+
If `BENCHMARK_TARGET_URL` is unset, the runner starts the local Express app on a random loopback port and benchmarks that process.
56+
57+
If `BENCHMARK_TARGET_URL` is set, the runner benchmarks that local/staging host instead:
58+
59+
```bash
60+
BENCHMARK_TARGET_URL=http://localhost:4000 npm run benchmark
61+
```
62+
63+
Do not run this against production unless the service owner has explicitly approved the load profile.
64+
65+
## Auth-protected routes
66+
67+
`GET /api/admin/metrics` uses a dedicated benchmark token.
68+
69+
- Set `BENCHMARK_AUTH_TOKEN` to use a real staging benchmark token.
70+
- If omitted, the runner creates a short-lived HS256 token using `JWT_SECRET` or the local development default.
71+
72+
## Output
73+
74+
Every run writes both machine-readable and human-readable output:
75+
76+
```text
77+
benchmarks/results/<mode>-<timestamp>.json
78+
benchmarks/results/<mode>-<timestamp>.md
79+
```
80+
81+
Captured per endpoint:
82+
83+
- p50, p95, p99 latency in milliseconds
84+
- p50, p95, p99 time-to-first-byte in milliseconds
85+
- sustained requests per second
86+
- peak requests-per-second estimate
87+
- error rate percentage
88+
- status-code counts
89+
90+
## Regression gate
91+
92+
`thresholds.json` stores reviewable limits. CI runs smoke mode and fails if any endpoint exceeds configured p99 latency or error-rate thresholds.
93+
94+
The default thresholds are intentionally conservative for local/CI smoke checks. Tighten them after collecting representative staging baselines.

0 commit comments

Comments
 (0)