Skip to content

Add API benchmark suite#38

Open
gma788-creator wants to merge 2 commits into
SecureBananaLabs:mainfrom
gma788-creator:bounty-30-api-benchmarks
Open

Add API benchmark suite#38
gma788-creator wants to merge 2 commits into
SecureBananaLabs:mainfrom
gma788-creator:bounty-30-api-benchmarks

Conversation

@gma788-creator
Copy link
Copy Markdown

@gma788-creator gma788-creator commented May 17, 2026

Closes #30
/claim #30

Summary:

  • Added a reusable API benchmark suite under /benchmarks.
  • Covers all 20 routes mounted under /api/ with realistic JSON and upload payloads.
  • Adds JSON and Markdown benchmark outputs under /benchmarks/results/.
  • Adds reviewable thresholds and a GitHub Actions smoke benchmark gate.
  • Adds .env.benchmark.example, benchmark documentation, and unit tests for metric calculations.

Validation:

  • npm run benchmark:test
  • npm run benchmark:smoke
  • npm run benchmark
  • npm run test -w apps/api

Benchmark summary from benchmarks/results/latest-full.md:

Metric Value
Target local-auto-start Express server
Endpoints covered 20
Requests per endpoint 6
Concurrency 2
Error rate 0% across all endpoints
Outputs benchmarks/results/latest-full.json, benchmarks/results/latest-full.md, smoke equivalents

Slowest full-run p99 endpoints:

Endpoint p99 ms Statuses
auth.register 107.12 201:6
uploads.create 13.67 201:6
auth.login 8.03 200:6

Benchmark Environment

Hardware

  • CPU model & core count: AMD Ryzen 5 7530U, 6 cores / 12 logical processors
  • RAM (total & available during benchmark): 16 GB total, about 3.2 GB free during environment check
  • Storage type (SSD / NVMe / HDD): NVMe SSD
  • Network interface (Ethernet / WiFi / loopback): WiFi, benchmark target used loopback
  • Machine type (local workstation / cloud VM / CI runner - include instance type if cloud): local workstation
  • OS & version: Microsoft Windows 11 Home, 10.0.26200

Runtime

  • Node.js version (or relevant runtime): v24.12.0
  • Any resource limits applied (Docker memory cap, cgroup limits, etc.): none known
  • Other significant processes running during benchmark (yes / no - if yes, describe): yes, normal desktop/Codex background processes

If submitted by or with an AI agent

  • Agent or tool name: Codex
  • Underlying model and version: GPT-5 Codex
  • Inference provider: OpenAI
  • Orchestration framework if any: none
  • Execution mode: human-initiated, agent-executed
  • Did the agent have shell/tool access during execution: yes
  • Did the agent have internet access during execution: yes
  • Were benchmark commands run by the agent directly or handed off to the human to run: run directly by agent
  • Any known agent constraints or sandboxing that may have affected execution: local Windows environment; benchmark used loopback local server

@BossChaos
Copy link
Copy Markdown

Code Review — Bounty #30 ($750)

PR: Add API benchmark suite by @gma788-creator

Wallet: 0xdaE5d307339074A24F579dB48e7c639359D94904

Code review under Bounty #30 — API Benchmark Suite ($750)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Benchmark APIs with p50, p95, p99 latency, RPS, error rate and TTFB

2 participants