Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
253 lines (187 loc) · 5.8 KB

CONTRIBUTING.md

File metadata and controls

253 lines (187 loc) · 5.8 KB

Contributing to vLLM Semantic Router

Thank you for your interest in contributing to the vLLM Semantic Router project! This guide will help you get started with development and contributing to the project.

Table of Contents

Development Setup

Prerequisites

Before you begin, ensure you have the following installed:

  • Rust (latest stable version)
  • Go 1.24.1 or later
  • Hugging Face CLI (pip install huggingface_hub)
  • Make (for build automation)
  • Python 3.8+ (Optiona: for training and testing)

Initial Setup

  1. Clone the repository:

    git clone <repository-url>
cd semantic-router

  2. Download required models:

    make download-models

    This downloads the pre-trained classification models from Hugging Face.

  3. Install Python dependencies(Optional):

    # For training and development
pip install -r requirements.txt

# For end-to-end testing
pip install -r e2e-tests/requirements.txt

Building the Project

The project consists of multiple components that need to be built in order:

Build Everything

make build

Build Individual Components

  1. Rust library (Candle binding):

    make rust

  2. Go router:

    make build-router

Running the System

  1. Start Envoy proxy (in one terminal):

    make run-envoy

  2. Start the semantic router (in another terminal):

    make run-router

Running Tests

Unit Tests

  1. Test Rust bindings:

    make test-binding

  2. Test Go semantic router:

    make test-semantic-router

  3. Test individual classifiers:

    make test-category-classifier
make test-pii-classifier
make test-jailbreak-classifier

Manual Testing

Test different routing scenarios:

# Test model auto-selection
make test-prompt

# Test PII detection
make test-pii

# Test prompt guard (jailbreak detection)
make test-prompt-guard

# Test tools auto-selection
make test-tools

End-to-End Tests

Ensure both Envoy and the router are running, then:

# Run all e2e tests
python e2e-tests/run_all_tests.py

# Run specific test
python e2e-tests/00-client-request-test.py

# Run tests matching a pattern
python e2e-tests/run_all_tests.py --pattern "0*-*.py"

# Check if services are running
python e2e-tests/run_all_tests.py --check-only

The test suite includes:

  • Basic client request tests
  • Envoy ExtProc interaction tests
  • Router classification tests
  • Semantic cache tests
  • Category-specific tests
  • Metrics validation tests

Development Workflow

Making Changes

  1. Create a feature branch:

    git checkout -b feature/your-feature-name

  2. Make your changes following the project structure and coding standards.

  3. Build and test:

    make clean
make build
make test

  4. Run end-to-end tests:

    # Start services
make run-envoy &
make run-router &

# Run tests
python e2e-tests/run_all_tests.py

  5. Commit your changes:

    git add .
git commit -m "feat: add your feature description"

Debugging

  • Envoy logs: Check the terminal running make run-envoy for detailed request/response logs
  • Router logs: Check the terminal running make run-router for classification and routing decisions
  • Rust library: Use RUST_LOG=debug environment variable for detailed Rust logs

Code Style and Standards

Go Code

  • Follow standard Go formatting (gofmt)
  • Use meaningful variable and function names
  • Add comments for exported functions and types
  • Write unit tests for new functionality

Rust Code

  • Follow Rust formatting (cargo fmt)
  • Use cargo clippy for linting
  • Handle errors appropriately with Result types
  • Document public APIs

Python Code

  • Follow PEP 8 style guidelines
  • Use type hints where appropriate
  • Write docstrings for functions and classes

Submitting Changes

  1. Ensure all tests pass:

    make test
python e2e-tests/run_all_tests.py

  2. Create a pull request with:

    • Clear description of changes
    • Reference to any related issues
    • Test results and validation steps

  3. Address review feedback promptly

Project Structure

├── candle-binding/          # Rust library for BERT classification
├── src/semantic-router/     # Go implementation of the router
├── src/training/           # Model training scripts
├── e2e-tests/              # End-to-end test suite
├── config/                 # Configuration files
├── docs/                   # Documentation
├── deploy/                 # Deployment configurations
├── Makefile               # Build automation
└── requirements.txt       # Python dependencies

Key Components

  • Candle Binding: Rust library providing BERT-based classification
  • Semantic Router: Go service implementing the Envoy ExtProc interface
  • Training Scripts: Python scripts for fine-tuning classification models
  • Configuration: YAML files defining routing rules and model endpoints

Getting Help

  • Check the documentation
  • Review existing issues and pull requests
  • Ask questions in discussions or create a new issue

License

By contributing to this project, you agree that your contributions will be licensed under the same license as the project (Apache 2.0).