Skip to content

@moreh-dev Moreh Team

Change the repository type filter

All

    Repositories list

    65 repositories

    • mif

      Public
      MIF: MoAI Inference Framework
      mif
      Go
      •0•0•0•1•Updated Jan 22, 2026Jan 22, 2026

    • SpecForge

      Public
      Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
      Python
      •
      MIT License
      •143•0•0•0•Updated Jan 22, 2026Jan 22, 2026

    • vllm-pcp

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •13k•0•0•2•Updated Jan 22, 2026Jan 22, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      Python
      •
      Apache License 2.0
      •13k•0•0•5•Updated Jan 22, 2026Jan 22, 2026

    • mori

      Public
      Modular RDMA Interface
      C++
      •
      MIT License
      •17•0•0•2•Updated Jan 22, 2026Jan 22, 2026

    • llm-d-kv-cache-manager

      Public
      Distributed KV cache coordinator
      Go
      •
      Apache License 2.0
      •78•0•0•0•Updated Jan 21, 2026Jan 21, 2026

    • EPLB

      Public
      Expert Parallelism Load Balancer
      Python
      •
      MIT License
      •198•0•0•0•Updated Jan 21, 2026Jan 21, 2026

    • helm-charts

      Public
      Helm Chart Repository
      helm
      Makefile
      •0•0•0•0•Updated Jan 20, 2026Jan 20, 2026

    • llm-d-inference-scheduler

      Public
      Inference scheduler for llm-d
      Go
      •
      Apache License 2.0
      •114•0•0•0•Updated Jan 19, 2026Jan 19, 2026

    • gateway-api-inference-extension

      Public
      Gateway API Inference Extension
      Go
      •
      Apache License 2.0
      •222•0•0•0•Updated Jan 16, 2026Jan 16, 2026

    • sglang

      Public
      SGLang is a fast serving framework for large language models and vision language models.
      Python
      •
      Apache License 2.0
      •4.1k•0•0•2•Updated Jan 14, 2026Jan 14, 2026

    • skypilot

      Public
      SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 16+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
      Python
      •
      Apache License 2.0
      •922•0•0•0•Updated Jan 1, 2026Jan 1, 2026

    • inference-perf

      Public
      GenAI inference performance benchmarking tool
      Python
      •
      Apache License 2.0
      •61•0•0•0•Updated Dec 18, 2025Dec 18, 2025

    • mi355_benchmark

      Public
      repo for MI355X benchmark (TensorWave)
      Shell
      •
      Apache License 2.0
      •71•0•0•0•Updated Nov 26, 2025Nov 26, 2025

    • transformers-sp-prefill

      Public
      🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
      Python
      •
      Apache License 2.0
      •32k•0•0•0•Updated Nov 26, 2025Nov 26, 2025

    • flash-attention-sinks

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2.3k•0•0•0•Updated Nov 17, 2025Nov 17, 2025

    • lws

      Public
      LeaderWorkerSet: An API for deploying a group of pods as a unit of replication
      Go
      •
      Apache License 2.0
      •127•0•0•0•Updated Nov 3, 2025Nov 3, 2025

    • transformers

      Public
      🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
      Python
      •
      Apache License 2.0
      •32k•2•0•0•Updated Oct 31, 2025Oct 31, 2025

    • kgateway

      Public
      The Cloud-Native API Gateway and AI Gateway
      Go
      •
      Apache License 2.0
      •647•0•0•0•Updated Oct 20, 2025Oct 20, 2025

    • LMCache

      Public
      Redis for LLMs
      Python
      •
      Apache License 2.0
      •867•0•0•0•Updated Oct 13, 2025Oct 13, 2025

    • gpt-oss

      Public
      Inference gpt-oss in one file of pure C
      Python
      •2.4k•1•0•0•Updated Oct 12, 2025Oct 12, 2025

    • k0s

      Public
      k0s - The Zero Friction Kubernetes
      Go
      •
      Other
      •467•0•0•0•Updated Oct 2, 2025Oct 2, 2025

    • opentelemetry-operator

      Public
      Kubernetes Operator for OpenTelemetry Collector
      Go
      •
      Apache License 2.0
      •585•0•0•0•Updated Sep 27, 2025Sep 27, 2025

    • genai-bench

      Public
      Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems.
      Python
      •
      MIT License
      •46•0•0•0•Updated Sep 25, 2025Sep 25, 2025

    • moreh-grafana-public-dashboard

      Public
      0•0•0•0•Updated Sep 5, 2025Sep 5, 2025

    • DeepEP

      Public
      DeepEP: an efficient expert-parallel communication library
      Cuda
      •
      MIT License
      •1.1k•0•0•0•Updated Aug 1, 2025Aug 1, 2025

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2.3k•0•0•0•Updated Jul 8, 2025Jul 8, 2025

    • pre-commit-hooks

      Public
      git pre-commit hook for automation
      pre-commitpre-commit-hooks
      Shell
      •0•0•0•0•Updated Jun 17, 2025Jun 17, 2025

    • moai-examples

      Public
      Python
      •1•3•0•1•Updated Jun 11, 2025Jun 11, 2025

    • tt-umd

      Public
      User-Mode Driver for Tenstorrent hardware
      C++
      •
      Apache License 2.0
      •23•0•0•0•Updated Jun 10, 2025Jun 10, 2025