VLLM Proxy

A proxy for vLLM.

Requirements

• Python 3.12+

Run for development

# Run production server
uvicorn main:app --host 0.0.0.0 --reload

# Run development server
fastapi dev main.py --host 0.0.0.0

Update Dependencies

# Compile requirements.txt into requirements.lock.txt with locked dependencies
bash compile-requirements.sh

Production

Build for production

# Build reproducible docker image
bash build-image.sh

Run for production

cd docker
docker compose up -d

Tests

Quick Start

python3 -m venv .venv
. .venv/bin/activate
pip install -r requirements.txt
pip install -r test-requirements.txt
./run_tests.sh

For detailed testing documentation, see TESTING.md.