Skip to content

Use vLLM Native Simulator for KV-Cache simulation #599

Description

@mayabar

The vLLM Native Simulator is currently available only as an unmerged PR

At the moment, inference-sim supports two modes:

  • No KV‑cache simulation
  • Internal KV‑cache simulation using an LRU eviction strategy

Add a new option to allow using the vLLM Native Simulator as the KV‑cache implementation.

Ensure that this new mode correctly supports:

  • Data parallelism (DP)
  • KV‑events replay

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions