The vLLM Native Simulator is currently available only as an unmerged PR
At the moment, inference-sim supports two modes:
- No KV‑cache simulation
- Internal KV‑cache simulation using an LRU eviction strategy
Add a new option to allow using the vLLM Native Simulator as the KV‑cache implementation.
Ensure that this new mode correctly supports:
- Data parallelism (DP)
- KV‑events replay
The vLLM Native Simulator is currently available only as an unmerged PR
At the moment, inference-sim supports two modes:
Add a new option to allow using the vLLM Native Simulator as the KV‑cache implementation.
Ensure that this new mode correctly supports: