Skip to content

Add API benchmark suite#733

Open
s0584273828-ctrl wants to merge 3 commits into
SecureBananaLabs:mainfrom
s0584273828-ctrl:bounty-30-api-benchmark-suite
Open

Add API benchmark suite#733
s0584273828-ctrl wants to merge 3 commits into
SecureBananaLabs:mainfrom
s0584273828-ctrl:bounty-30-api-benchmark-suite

Conversation

@s0584273828-ctrl
Copy link
Copy Markdown

@s0584273828-ctrl s0584273828-ctrl commented May 27, 2026

/claim #30

Summary

  • Add a dependency-light API benchmark suite under benchmarks/ covering /health and every mounted /api/* route.
  • Add realistic synthetic payloads, generated benchmark admin auth, threshold gates, JSON output, and Markdown output.
  • Add root commands: npm run benchmark and npm run benchmark:smoke.
  • Add a GitHub Actions smoke gate at .github/workflows/api-benchmark-smoke.yml.
  • Add short demo proof at demos/api-benchmark-smoke-demo.mp4.

Smoke benchmark summary

Committed full report: benchmarks/results/latest.md and benchmarks/results/latest.json.

Endpoint coverage Requests each Error rate Gate
21 endpoints (/health + all /api/*) 3 0% pass

Slowest smoke p99 in the committed run: health at 15.13 ms; upload p99 was 2.09 ms.

Benchmark Environment

Hardware

  • CPU model & core count: Apple M5, 10 cores reported by Node.js
  • RAM: 32 GB total, about 5.2 GB free before implementation
  • Storage type: local Apple SSD
  • Network interface: loopback for the benchmark target
  • Machine type: local workstation
  • OS & version: macOS 26.4.1

Runtime

  • Node.js version: v24.14.1
  • Resource limits: no explicit benchmark resource limits
  • Other significant processes: normal Codex desktop and shell background activity

AI-assisted execution disclosure

  • Agent/tool: OpenAI Codex desktop
  • Model: GPT-5-based Codex model
  • Provider: OpenAI
  • Orchestration framework: none beyond Codex shell/GitHub tooling
  • Execution mode: human-initiated, agent-executed
  • Shell/tool access: yes
  • Internet access: yes, through approved GitHub/network commands
  • Benchmark commands: run directly by the agent against local loopback
  • Known constraints: sandboxed commands cannot bind sockets without approved escalation, so benchmark validation used approved local loopback execution.

Validation

  • npm run benchmark:smoke -> all 21 endpoint gates passed
  • GitHub Actions API benchmark smoke / smoke -> passing
  • node --check benchmarks/run-api-benchmarks.mjs
  • node --check benchmarks/scenarios.mjs
  • ffprobe demos/api-benchmark-smoke-demo.mp4 -> 1280x720, 8s, 200 frames
  • git diff --check

Prepared with AI assistance and manually reviewed before submission.

@s0584273828-ctrl
Copy link
Copy Markdown
Author

Follow-up: added a GitHub Actions smoke workflow for the API benchmark suite after refreshing the token with the required workflow scope.\n\nValidation:\n- npm run benchmark:smoke\n- git diff --check

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant