Name		Name	Last commit message	Last commit date
.buildkite		.buildkite
assets/poster		assets/poster
benchmarks		benchmarks
csrc		csrc
docs		docs
examples		examples
rocm_patch		rocm_patch
technical_report		technical_report
tests		tests
vllm		vllm
.dockerignore		.dockerignore
.gitignore		.gitignore
.readthedocs.yaml		.readthedocs.yaml
CONTRIBUTING.md		CONTRIBUTING.md
Dockerfile		Dockerfile
Dockerfile.rocm		Dockerfile.rocm
LICENSE		LICENSE
MANIFEST.in		MANIFEST.in
README.md		README.md
format.sh		format.sh
mypy.ini		mypy.ini
patch_xformers.rocm.sh		patch_xformers.rocm.sh
pyproject.toml		pyproject.toml
requirements-build.txt		requirements-build.txt
requirements-dev.txt		requirements-dev.txt
requirements-neuron.txt		requirements-neuron.txt
requirements-rocm.txt		requirements-rocm.txt
requirements.txt		requirements.txt
setup.py		setup.py

Repository files navigation

Mil

Official implementation of Mil: Cost-guided Minimum Makespan Scheduling for Applications of Multiple LLMs. Mil is built on top of vLLM.

Paper · DOI · Poster PDF · Editable poster

Poster

Click the poster to open the full-resolution PDF.
View PDF · Download editable PowerPoint

Installation

Install the version of vLLM included in this repository before running Mil.

Repository structure

benchmarks/ contains the core implementation of Mil.
benchmarks/MY_README.md describes the code structure.
benchmarks/add_a_new_model.md explains how to build the cost model for a new model.
technical_report/technical_report.pdf contains the technical report.

Reproducing the paper's experiments

Use benchmarks/run_end2end_exp.sh to run the end-to-end experiments reported in the paper.

About

No description, website, or topics provided.

Apache-2.0 license

0 stars

1 watching

Report repository

Releases

Packages

Contributors

Languages