Skip to content

Commit 96624c5

Browse files
author
Bounty Contributor
committed
Add API benchmark suite
1 parent 8af3f5d commit 96624c5

11 files changed

Lines changed: 873 additions & 1 deletion

File tree

‎.github/workflows/api-benchmark-smoke.yml‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: API benchmark smoke
2+
3+
on:
4+
pull_request:
5+
paths:
6+
- "apps/api/**"
7+
- "benchmarks/**"
8+
- "package.json"
9+
- "package-lock.json"
10+
11+
permissions:
12+
contents: read
13+
14+
jobs:
15+
benchmark-smoke:
16+
runs-on: ubuntu-latest
17+
18+
steps:
19+
- name: Checkout
20+
uses: actions/checkout@v4
21+
22+
- name: Setup Node.js
23+
uses: actions/setup-node@v4
24+
with:
25+
node-version: 20
26+
cache: npm
27+
28+
- name: Install dependencies
29+
run: npm ci
30+
31+
- name: Verify benchmark endpoint coverage
32+
run: npm run benchmark:coverage
33+
34+
- name: Start API
35+
env:
36+
JWT_SECRET: benchmark-ci-secret
37+
PORT: 4100
38+
run: |
39+
npm run start -w apps/api > /tmp/freelanceflow-api.log 2>&1 &
40+
echo "$!" > /tmp/freelanceflow-api.pid
41+
for attempt in $(seq 1 20); do
42+
if curl -fsS http://127.0.0.1:4100/health; then
43+
exit 0
44+
fi
45+
sleep 1
46+
done
47+
cat /tmp/freelanceflow-api.log
48+
exit 1
49+
50+
- name: Run smoke benchmark
51+
env:
52+
BENCHMARK_TARGET: http://127.0.0.1:4100
53+
BENCHMARK_CONCURRENCY: 1
54+
BENCHMARK_REQUESTS_PER_ENDPOINT: 2
55+
BENCHMARK_RESULTS_DIR: benchmarks/results/ci
56+
JWT_SECRET: benchmark-ci-secret
57+
run: npm run benchmark:smoke
58+
59+
- name: Stop API
60+
if: always()
61+
run: |
62+
if [ -f /tmp/freelanceflow-api.pid ]; then
63+
kill "$(cat /tmp/freelanceflow-api.pid)" || true
64+
fi

‎apps/api/package.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@
55
"scripts": {
66
"dev": "node src/server.js",
77
"start": "node src/server.js",
8-
"test": "node --test src/tests"
8+
"test": "node --test \"src/tests/**/*.test.js\""
99
},
1010
"dependencies": {
1111
"cors": "^2.8.5",

‎benchmarks/.env.benchmark.example‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
BENCHMARK_TARGET=http://127.0.0.1:4000
2+
BENCHMARK_CONCURRENCY=4
3+
BENCHMARK_REQUESTS_PER_ENDPOINT=8
4+
BENCHMARK_RESULTS_DIR=benchmarks/results
5+
JWT_SECRET=development-secret
6+
7+
# Use this when benchmarking a staging host that cannot share JWT_SECRET.
8+
# BENCHMARK_AUTH_TOKEN=

‎benchmarks/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# API Benchmark Suite
2+
3+
This directory contains a reproducible benchmark suite for every mounted API endpoint.
4+
5+
## Setup
6+
7+
1. Install dependencies from the repository root.
8+
2. Start the API server.
9+
3. Copy `benchmarks/.env.benchmark.example` to `benchmarks/.env.benchmark` and adjust the target if needed.
10+
4. Run the suite.
11+
12+
```bash
13+
npm install
14+
npm run dev -w apps/api
15+
npm run benchmark
16+
```
17+
18+
The runner writes JSON and Markdown reports to `benchmarks/results/`.
19+
20+
## Commands
21+
22+
```bash
23+
npm run benchmark:coverage
24+
npm run benchmark:smoke
25+
npm run benchmark
26+
```
27+
28+
- `benchmark:coverage` verifies that `benchmarks/endpoints.json` covers every route mounted in `apps/api/src/app.js`.
29+
- `benchmark:smoke` runs a low-concurrency suite suitable for CI and enforces `thresholds.json`.
30+
- `benchmark` runs the default local suite and also enforces the configured thresholds.
31+
32+
## Environment
33+
34+
The runner reads `benchmarks/.env.benchmark` when present. Environment variables set in the shell take precedence.
35+
36+
| Variable | Default | Purpose |
37+
| --- | --- | --- |
38+
| `BENCHMARK_TARGET` | `http://127.0.0.1:4000` | API host to benchmark. |
39+
| `BENCHMARK_CONCURRENCY` | `4` | Concurrent requests per endpoint. |
40+
| `BENCHMARK_REQUESTS_PER_ENDPOINT` | `8` | Total requests sent to each endpoint. |
41+
| `BENCHMARK_RESULTS_DIR` | `benchmarks/results` | Output directory for reports. |
42+
| `JWT_SECRET` | `development-secret` | Signs the local benchmark token for protected routes. |
43+
| `BENCHMARK_AUTH_TOKEN` | unset | Preissued token for staging or remote hosts. |
44+
45+
For auth-protected routes, the runner uses `BENCHMARK_AUTH_TOKEN` when provided. Otherwise it signs a short-lived benchmark token with `JWT_SECRET`.

‎benchmarks/endpoints.json‎

Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
[
2+
{
3+
"name": "health",
4+
"method": "GET",
5+
"path": "/health",
6+
"description": "Service health check",
7+
"includeInApiCoverage": false
8+
},
9+
{
10+
"name": "auth-register",
11+
"method": "POST",
12+
"path": "/api/auth/register",
13+
"description": "Register a freelancer account",
14+
"json": {
15+
"email": "benchmark.freelancer@example.com",
16+
"password": "benchmark-pass-123",
17+
"role": "freelancer"
18+
}
19+
},
20+
{
21+
"name": "auth-login",
22+
"method": "POST",
23+
"path": "/api/auth/login",
24+
"description": "Log in an existing client",
25+
"json": {
26+
"email": "benchmark.client@example.com",
27+
"password": "benchmark-pass-123"
28+
}
29+
},
30+
{
31+
"name": "auth-oauth-callback",
32+
"method": "GET",
33+
"path": "/api/auth/oauth/github/callback",
34+
"coveragePath": "/api/auth/oauth/:provider/callback",
35+
"description": "OAuth provider callback"
36+
},
37+
{
38+
"name": "auth-refresh",
39+
"method": "POST",
40+
"path": "/api/auth/refresh",
41+
"description": "Refresh an access token"
42+
},
43+
{
44+
"name": "users-list",
45+
"method": "GET",
46+
"path": "/api/users",
47+
"description": "List users"
48+
},
49+
{
50+
"name": "users-create",
51+
"method": "POST",
52+
"path": "/api/users",
53+
"description": "Create a user profile",
54+
"json": {
55+
"email": "benchmark.user@example.com",
56+
"name": "Benchmark User",
57+
"role": "client",
58+
"company": "Benchmark Studio"
59+
}
60+
},
61+
{
62+
"name": "jobs-list",
63+
"method": "GET",
64+
"path": "/api/jobs",
65+
"description": "List jobs"
66+
},
67+
{
68+
"name": "jobs-create",
69+
"method": "POST",
70+
"path": "/api/jobs",
71+
"description": "Create a job with realistic payload size",
72+
"json": {
73+
"title": "Build a secure onboarding dashboard",
74+
"description": "Benchmark payload for a medium-sized marketplace job brief with scope, acceptance criteria, milestones, and preferred collaboration details.",
75+
"budgetMin": 1500,
76+
"budgetMax": 4200,
77+
"categoryId": "cat_web_apps",
78+
"skills": ["node", "express", "react", "security-review", "performance"]
79+
}
80+
},
81+
{
82+
"name": "proposals-list",
83+
"method": "GET",
84+
"path": "/api/proposals",
85+
"description": "List proposals"
86+
},
87+
{
88+
"name": "proposals-create",
89+
"method": "POST",
90+
"path": "/api/proposals",
91+
"description": "Create a proposal",
92+
"json": {
93+
"jobId": "job_benchmark",
94+
"freelancerId": "usr_benchmark_freelancer",
95+
"coverLetter": "I can deliver a secure dashboard with clear milestones, weekly demos, and performance baselines.",
96+
"bidAmount": 2800,
97+
"estimatedDays": 14
98+
}
99+
},
100+
{
101+
"name": "payments-create",
102+
"method": "POST",
103+
"path": "/api/payments",
104+
"description": "Create a payment intent",
105+
"json": {
106+
"jobId": "job_benchmark",
107+
"amount": 50000,
108+
"currency": "usd",
109+
"milestone": "deposit"
110+
}
111+
},
112+
{
113+
"name": "reviews-list",
114+
"method": "GET",
115+
"path": "/api/reviews",
116+
"description": "List reviews"
117+
},
118+
{
119+
"name": "reviews-create",
120+
"method": "POST",
121+
"path": "/api/reviews",
122+
"description": "Create a review",
123+
"json": {
124+
"jobId": "job_benchmark",
125+
"rating": 5,
126+
"comment": "Clear communication, secure delivery, and strong documentation.",
127+
"reviewerId": "usr_benchmark_client",
128+
"revieweeId": "usr_benchmark_freelancer"
129+
}
130+
},
131+
{
132+
"name": "messages-list",
133+
"method": "GET",
134+
"path": "/api/messages",
135+
"description": "List messages"
136+
},
137+
{
138+
"name": "messages-create",
139+
"method": "POST",
140+
"path": "/api/messages",
141+
"description": "Send a project message",
142+
"json": {
143+
"threadId": "thread_benchmark",
144+
"senderId": "usr_benchmark_client",
145+
"recipientId": "usr_benchmark_freelancer",
146+
"body": "Can you confirm the milestone handoff checklist and next demo date?"
147+
}
148+
},
149+
{
150+
"name": "notifications-list",
151+
"method": "GET",
152+
"path": "/api/notifications",
153+
"description": "List notifications"
154+
},
155+
{
156+
"name": "notifications-create",
157+
"method": "POST",
158+
"path": "/api/notifications",
159+
"description": "Create a notification",
160+
"json": {
161+
"userId": "usr_benchmark_freelancer",
162+
"type": "milestone_due",
163+
"message": "Your benchmark milestone is due tomorrow.",
164+
"metadata": {
165+
"jobId": "job_benchmark",
166+
"severity": "info"
167+
}
168+
}
169+
},
170+
{
171+
"name": "uploads-create",
172+
"method": "POST",
173+
"path": "/api/uploads",
174+
"description": "Upload a small handoff document",
175+
"multipart": {
176+
"fieldName": "file",
177+
"filename": "benchmark-handoff.txt",
178+
"contentType": "text/plain",
179+
"content": "Benchmark upload payload for the freelance marketplace API.\nIncludes milestone notes and delivery checklist.\n"
180+
}
181+
},
182+
{
183+
"name": "search",
184+
"method": "GET",
185+
"path": "/api/search",
186+
"description": "Search jobs and users",
187+
"query": {
188+
"q": "secure react dashboard"
189+
}
190+
},
191+
{
192+
"name": "admin-metrics",
193+
"method": "GET",
194+
"path": "/api/admin/metrics",
195+
"description": "Read admin metrics with a benchmark token",
196+
"auth": "benchmarkToken"
197+
}
198+
]

‎benchmarks/results/.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
*
2+
!.gitignore
3+
!.gitkeep

‎benchmarks/results/.gitkeep‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+

0 commit comments

Comments
 (0)