Skip to content

@vllm-project vLLM

All

    Repositories list

    47 repositories

    • semantic-router

      Public
      A programmable Mixture-of-Models router for heterogeneous LLM inference
      kubernetesinferencepytorch
      kubernetesinferencepytorchtransformerguardrailsmixture-of-modelsllmvllmai-gatewaysemantic-router
      Go
      •
      Apache License 2.0
      •921•5.7k•325•165•Updated Sep 8, 2026Sep 8, 2026

    • vllm-metal

      Public
      Community maintained hardware plugin for vLLM on Apple Silicon
      Python
      •
      Apache License 2.0
      •247•1.7k•14•21•Updated Sep 8, 2026Sep 8, 2026

    • vllm-omni

      Public
      A framework for efficient model inference with omni-modality models
      inferencepytorchtransformer
      inferencepytorchtransformerimage-generationdiffusionmodel-servingmultimodalvideo-generationaudio-generationworld-model
      Python
      •
      Apache License 2.0
      •1.7k•6.7k•811•1.1k•Updated Sep 8, 2026Sep 8, 2026

    • afd-plugin

      Public
      vLLM plugin for attention-ffn disaggregation support
      Python
      •
      Apache License 2.0
      •41•208•20•10•Updated Sep 8, 2026Sep 8, 2026

    • vllm-project.github.io

      Public
      HTML
      •139•68•4•17•Updated Sep 8, 2026Sep 8, 2026

    • vllm-ascend

      Public
      Community maintained hardware plugin for vLLM on Ascend
      inferencetransformermodel-serving
      inferencetransformermodel-servingmlopsascendllmllmopsllm-servingvllm
      C++
      •
      Apache License 2.0
      •2.2k•2.8k•1.4k•1.6k•Updated Sep 8, 2026Sep 8, 2026

    • vllm-xpu-kernels

      Public
      The vLLM XPU kernels for Intel GPU
      C++
      •
      Apache License 2.0
      •108•69•19•58•Updated Sep 8, 2026Sep 8, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      amdcudainference
      amdcudainferencepytorchtransformeropenaimoellamagptmodel-serving
      Python
      •
      Apache License 2.0
      •22k•91k•2.4k•5.3k•Updated Sep 8, 2026Sep 8, 2026

    • aibrix

      Public
      Cost-efficient and pluggable Infrastructure components for GenAI inference
      Go
      •
      Apache License 2.0
      •683•5.1k•333•43•Updated Sep 7, 2026Sep 7, 2026

    • tpu-inference

      Public
      TPU inference for vLLM, with unified JAX and PyTorch support.
      Python
      •
      Apache License 2.0
      •308•425•84•393•Updated Sep 7, 2026Sep 7, 2026

    • ci-infra

      Public
      This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
      Python
      •
      Apache License 2.0
      •82•51•0•68•Updated Sep 7, 2026Sep 7, 2026

    • speculators

      Public
      A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
      Python
      •
      Apache License 2.0
      •211•813•46•98•Updated Sep 7, 2026Sep 7, 2026

    • recipes

      Public
      Common recipes to run vLLM
      JavaScript
      •
      Apache License 2.0
      •402•1k•54•126•Updated Sep 7, 2026Sep 7, 2026

    • vllm-gaudi

      Public
      Community maintained hardware plugin for vLLM on Intel Gaudi
      Python
      •
      Apache License 2.0
      •151•53•8•43•Updated Sep 7, 2026Sep 7, 2026

    • vime

      Public
      An LLM post-training framework with vLLM for RL Scaling
      Python
      •
      Apache License 2.0
      •88•455•37•21•Updated Sep 7, 2026Sep 7, 2026

    • agentic-api

      Public
      Stateful API logic for agentic applications using vLLM
      Rust
      •
      Apache License 2.0
      •59•219•46•22•Updated Sep 7, 2026Sep 7, 2026

    • guidellm

      Public
      Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
      Python
      •
      Apache License 2.0
      •228•1.6k•50•29•Updated Sep 7, 2026Sep 7, 2026

    • llm-compressor

      Public
      Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
      compressionquantization
      compressionquantization
      Python
      •
      Apache License 2.0
      •652•3.8k•50•86•Updated Sep 6, 2026Sep 6, 2026

    • vllm-dashboard

      Public
      TypeScript
      •13•13•1•11•Updated Sep 6, 2026Sep 6, 2026

    • perf-eval

      Public
      Performance benchmark & accuracy evaluation for vLLM
      Python
      •19•17•0•26•Updated Sep 6, 2026Sep 6, 2026

    • compressed-tensors

      Public
      A safetensors extension to efficiently store sparse quantized tensors on disk
      Python
      •
      Apache License 2.0
      •121•322•10•33•Updated Sep 5, 2026Sep 5, 2026

    • vllm-daily

      Public
      vLLM Daily Summarization of Merged PRs
      8•54•1•0•Updated Sep 5, 2026Sep 5, 2026

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •3k•135•0•47•Updated Sep 4, 2026Sep 4, 2026

    • production-stack

      Public
      vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
      Python
      •
      Apache License 2.0
      •484•2.6k•105•107•Updated Sep 2, 2026Sep 2, 2026

    • FlashKDA

      Public
      Cuda
      •
      MIT License
      •3•12•0•2•Updated Sep 2, 2026Sep 2, 2026

    • vllm-gguf-plugin

      Public
      vLLM Quantization plugin for GGUF
      Python
      •
      Apache License 2.0
      •45•46•6•19•Updated Aug 31, 2026Aug 31, 2026

    • router

      Public
      A high-performance and light-weight router for vLLM large scale deployment
      Rust
      •
      Apache License 2.0
      •148•398•25•58•Updated Aug 31, 2026Aug 31, 2026

    • FlashMLA

      Public
      C++
      •
      MIT License
      •1.1k•15•0•4•Updated Aug 27, 2026Aug 27, 2026

    • vllm-bnb-plugin

      Public
      vLLM Quantization plugin for bitsandbytes
      Python
      •
      Apache License 2.0
      •3•3•0•0•Updated Aug 24, 2026Aug 24, 2026

    • vllm-neuron

      Public
      Community maintained hardware plugin for vLLM on AWS Neuron
      awsinferentiaaws-neuron
      awsinferentiaaws-neurontrainium
      Python
      •
      Apache License 2.0
      •35•51•2•10•Updated Aug 17, 2026Aug 17, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.