Web Framework Benchmark

A comprehensive benchmark suite comparing Nexios with other popular Python web frameworks including Flask, FastAPI, Sanic, and Quart.

Frameworks Tested

• Nexios - Modern ASGI web framework with Rust-powered performance
• Flask - Traditional WSGI microframework
• FastAPI - Modern ASGI framework with automatic API documentation
• Sanic - ASGI web server and framework
• Quart - Async version of Flask

Test Scenarios

Each framework implements 5 identical endpoints:

1. / - Simple GET request returning a JSON message
2. /ping - Basic health check endpoint
3. /echo - POST endpoint that echoes back JSON data
4. /compute - CPU-intensive endpoint (calculates sum of squares)
5. /async - Async endpoint with simulated I/O delay

Middleware Stack

All frameworks implement identical middleware:

• Timing Middleware - Adds X-Response-Time header
• Authentication Middleware - Requires authorization header for protected endpoints

Installation

1. Clone or download this benchmark suite

2. Install dependencies:

   pip install -r requirements.txt

3. Install wrk benchmarking tool:

   • macOS: brew install wrk
   • Linux: Download from https://github.com/wg/wrk/releases
   • Windows: Download Windows binary or use WSL

Usage

Method 1: Automatic Server Management

Start all servers at once:

python start_servers.py

Then run the benchmark in another terminal:

# On Linux/macOS
./benchmark.sh

# On Windows
benchmark.bat

Method 2: Manual Server Management

Start each server in separate terminals:

# Terminal 1
python nexios_app.py      # Port 8000

# Terminal 2
python flask_app.py       # Port 8001

# Terminal 3
python fastapi_app.py     # Port 8002

# Terminal 4
python sanic_app.py       # Port 8003

# Terminal 5
python quart_app.py       # Port 8004

Then run the benchmark:

# Linux/macOS
./benchmark.sh

# Windows
benchmark.bat

Benchmark Configuration

The benchmark uses the following wrk settings:

• Threads: 12
• Connections: 400
• Duration: 30 seconds per test
• Timeout: 10 seconds
• Latency tracking: Enabled

Expected Results

The benchmark will test:

• Raw throughput (requests per second)
• Latency distribution
• Error rates
• Performance under load

Notes

• All servers run on different ports (8000-8004) to avoid conflicts
• The benchmark includes both GET and POST requests
• Authentication middleware is applied to /echo, /compute, and /async endpoints
• CPU-intensive test uses the same algorithm across all frameworks
• Async test simulates I/O-bound operations

Troubleshooting

1. Port conflicts: Make sure ports 8000-8004 are available
2. Missing wrk: Install wrk benchmarking tool first
3. Permission errors: On some systems, you may need to run with appropriate permissions
4. Dependencies: Ensure all Python packages are installed from requirements.txt

Interpreting Results

Look for:

• Higher RPS (Requests Per Second) = Better throughput
• Lower latency = Faster response times
• Lower error rates = Better reliability
• Consistent performance across different test types

The async frameworks (Nexios, FastAPI, Sanic, Quart) should generally perform better on the async endpoint, while CPU-bound performance may vary based on implementation efficiency.