Skip to content
@llm-d

llm-d

llm-d is a Kubernetes-native high-performance distributed LLM inference framework

Popular repositories Loading

  1. llm-d llm-d Public

    llm-d is a Kubernetes-native high-performance distributed LLM inference framework

    Makefile 103 6

  2. llm-d-inference-scheduler llm-d-inference-scheduler Public

    Inference scheduler for llm-d

    Go 22 1

  3. llm-d-deployer llm-d-deployer Public

    Helm charts for llm-d

    Shell 18 4

  4. llm-d-kv-cache-manager llm-d-kv-cache-manager Public

    Distributed KV cache coordinator

    Go 14 1

  5. llm-d-inference-sim llm-d-inference-sim Public

    A light weight vLLM simulator, for mocking out replicas.

    Go 7 1

  6. llm-d-model-service llm-d-model-service Public

    Incubating model service CRDs for llm-d

    Go 7 2

Repositories

Showing 9 of 9 repositories
  • llm-d Public

    llm-d is a Kubernetes-native high-performance distributed LLM inference framework

    llm-d/llm-d’s past year of commit activity
    Makefile 103 Apache-2.0 6 5 7 Updated May 20, 2025
  • llm-d-benchmark Public

    llm-d benchmark scripts and tooling

    llm-d/llm-d-benchmark’s past year of commit activity
    Shell 6 Apache-2.0 1 0 0 Updated May 20, 2025
  • llm-d-deployer Public

    Helm charts for llm-d

    llm-d/llm-d-deployer’s past year of commit activity
    Shell 18 Apache-2.0 4 20 4 Updated May 20, 2025
  • llm-d-routing-sidecar Public

    Incubating P/D sidecar for llm-d

    llm-d/llm-d-routing-sidecar’s past year of commit activity
    Go 6 Apache-2.0 1 5 (1 issue needs help) 2 Updated May 20, 2025
  • llm-d-kv-cache-manager Public

    Distributed KV cache coordinator

    llm-d/llm-d-kv-cache-manager’s past year of commit activity
    Go 14 1 8 (2 issues need help) 0 Updated May 20, 2025
  • llm-d-inference-scheduler Public

    Inference scheduler for llm-d

    llm-d/llm-d-inference-scheduler’s past year of commit activity
    Go 22 Apache-2.0 1 32 (2 issues need help) 0 Updated May 20, 2025
  • llm-d-inference-sim Public

    A light weight vLLM simulator, for mocking out replicas.

    llm-d/llm-d-inference-sim’s past year of commit activity
    Go 7 1 2 0 Updated May 20, 2025
  • llm-d-pd-utils Public
    llm-d/llm-d-pd-utils’s past year of commit activity
    Makefile 2 1 0 1 Updated May 20, 2025
  • llm-d-model-service Public

    Incubating model service CRDs for llm-d

    llm-d/llm-d-model-service’s past year of commit activity
    Go 7 Apache-2.0 2 25 2 Updated May 19, 2025

Most used topics

incubating ai