feat(rollout): add continuous batching generation prototype - #8368

Merged

delock merged 17 commits into

deepspeedai:masterfrom

nathon-lee:perf/opsd-continuous-batching-prototype

Sep 11, 2026

Commits on Aug 30, 2026

feat: add continuous batching rollout prototype
nathon-lee
committed
14e3ad2
feat: integrate continuous batching rollout prototype
nathon-lee
committed
f80610a
fix: support StaticCache batch API variants
nathon-lee
committed
bb5f660
fix: support legacy Transformers StaticCache configs
nathon-lee
committed
8233436
fix: support legacy model forward cache arguments
nathon-lee
committed
a434080
fix: support legacy KV cache in continuous batching
nathon-lee
committed
67b246e

Commits on Sep 2, 2026

Commits on Sep 6, 2026

Merge branch 'master' into perf/opsd-continuous-batching-prototype
nathon-lee
authored
5e142e3

Commits on Sep 7, 2026

feat(rollout): unify continuous batching generation interface
nathon-lee
committed
9510db6

Commits on Sep 8, 2026

chore(rollout): align license headers
nathon-lee
committed
2987844
fix(rollout): address continuous batching review feedback
nathon-lee
committed
765a22b

Commits on Sep 9, 2026

fix(rollout): reclaim cache space after staggered EOS
nathon-lee
committed
3b0713e

Commits on Sep 10, 2026

Commits on Sep 11, 2026

Merge branch 'master' into perf/opsd-continuous-batching-prototype
delock
authored
5730ddb