Skip to content

[Bug]: 使用内置mooncake client方式(ModeA)卸载大量请求的KV到DRAM和SSD后,VLLM PD实例重启,重启后重新压测之前的请求,无法在DRAM和SSD中命中,导致Prefix Cache命中率降低ttft增加 #2254

@xuyucheng220

Description

@xuyucheng220

Bug Report

[Bug]: 使用内置mooncake client方式(ModeA)卸载大量请求的KV到DRAM和SSD后,VLLM PD实例重启,重启后重新压测之前的请求,无法在DRAM和SSD中命中,导致Prefix Cache命中率降低ttft增加

Before submitting...

  • Ensure you searched for relevant issues and read the [documentation]

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions