I propose a new approach for CI based performance testing using a dedicated Docker image that supports multi-version Node.js benchmarks, based on a template mechanism (e.g., autocannon, but pluggable).

This solution allows libraries and frameworks to:

• Run on CI via GitHub Actions and optionally post PR comments
• Run consistent performance tests in isolated environments
• Compare latest vs candidate commits using controlled templates
• Extract and analyze key metrics like req/sec, latency, and throughput
• Keep test dependencies isolated with per-test package.jsons

How It Works

Each test case is a subfolder under a expf-tests directory. ($TEST_DIR, default = "/expf-tests")
It includes:

expf-tests/
├── autocannon.mjs (Not a folder -> template)
├── my-test/
│   ├── run-test.mjs (Your test logic)
│   └── package.json (Your isolated deps)

A template file (default: autocannon.mjs) runs the test logic.

Then, inside the Docker container:

• Each folder is executed independently
• Metrics are collected via the chosen template
• Results are compared: latest vs candidate
• Final output is shown as Markdown, and optionally sent as a GitHub PR comment

Result Example: (whereas I put 1ms sleep in bodyParser.json() function)

[This comment is auto-generated by the perf runner]

## Performance Comparison for PR #1, Node.js 20

### Test Folder: json

| Metric              | Latest          | Candidate      | Difference       | Ratio       | Status        |
|---------------------|-----------------|----------------|------------------|-------------|---------------|
| Execution Time      | 10248.56 ms     | 10275.02 ms    | -26.46 ms        | ×1.00       | ❌ Regressed  |
| Average Latency     | 1.71 ms         | 9.34 ms        | -7.63 ms         | ×0.18       | ❌ Regressed  |
| Requests/Second     | 4556387.23 rps  | 989520.96 rps  | -3566866.27 rps  | ×0.22       | ❌ Regressed  |
| Errors              | 0               | 0              | 0                | N/A         | ✅ Unchanged  |

Pros

• Reproducible, consistent test runs across environments
• Isolates test dependencies (doesn’t pollute your repo's root package.json)
• Enables comparisons between commits or PRs
• Easily installable from Docker, CI-friendly
• Markdown reports & potential dashboard integration #28
• Clean developer experience with reusable templates
• Lightweight Dockerfile (~55MB, thanks to @mertssmnoglu)
• Multi-version Node.js test support (e.g., 16, 18, 20)

Repositories

Main repository: https://github.com/GroophyLifefor/expf
Testing purposed sample usage repository: https://github.com/GroophyLifefor/expf-lib
Real usage case example with body-parser: https://github.com/GroophyLifefor/body-parser