Skip to content

Add API benchmark suite#513

Open
JPL-Jarvis wants to merge 1 commit into
SecureBananaLabs:mainfrom
JPL-Jarvis:codex/api-benchmark-suite
Open

Add API benchmark suite#513
JPL-Jarvis wants to merge 1 commit into
SecureBananaLabs:mainfrom
JPL-Jarvis:codex/api-benchmark-suite

Conversation

@JPL-Jarvis
Copy link
Copy Markdown

Summary

  • add a dependency-free benchmark suite under benchmarks/ that covers /health plus every mounted /api/* route
  • capture p50/p95/p99 latency, p99 TTFB, RPS, request count, and error rate per endpoint
  • write JSON and Markdown reports to benchmarks/results/, add reviewable thresholds, root benchmark scripts, env template, and a GitHub Actions smoke gate
  • fix the API workspace test script so Node runs the actual test files on current Node versions

/claim #30

Demo

  • demos/api-benchmark-demo.mp4

Latest local benchmark

  • 21 endpoints covered
  • 105 requests
  • 0 errors
  • max p99 latency: 19.07 ms
  • max p99 TTFB: 18.47 ms

Verification

  • node --test benchmarks/benchmark-suite.test.mjs
  • node --check benchmarks/run-benchmarks.mjs && node --check benchmarks/endpoints.mjs
  • npm test
  • npm run test:benchmarks
  • npm run benchmark:smoke
  • npm run benchmark
  • git diff --check --cached

Benchmark Environment

Hardware

  • CPU model & core count: captured in benchmarks/results/latest.json
  • RAM: captured in benchmarks/results/latest.json
  • Storage type: local Apple SSD / APFS
  • Network interface: loopback (127.0.0.1)
  • Machine type: local workstation
  • OS & version: captured in benchmarks/results/latest.json

Runtime

  • Node.js version: captured in benchmarks/results/latest.json
  • Any resource limits applied: none intentionally applied
  • Other significant processes running during benchmark: normal local desktop/Codex tooling

If submitted by or with an AI agent

  • Agent/tool name: OpenAI Codex desktop app
  • Underlying model/version: GPT-5.5 high, as requested by Josh
  • Inference provider: OpenAI
  • Orchestration framework: none beyond Codex shell/GitHub tooling
  • Execution mode: Josh-approved autonomous bounty run
  • Shell/tool access: yes
  • Internet access: yes
  • Benchmark commands run by: agent directly
  • Known constraints/sandboxing: local loopback benchmark only; no production or staging target was load-tested and no secrets are included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant