feat:add support to invalidate KV cache via AllBlocksCleared event

[yash9263]

Issue: #396

This PR adds support to Invalidate KV cache via the AllBlocksCleared event.

• Updated event processing in vllm_adapter.go and pool.go to parse and handle the AllBlocksCleared event.
• Added Clear method to the Index interface and implemented it for all the index types (CostAwareMemoryIndex, InMemoryIndex, RedisIndex).
• Introduced podToRequestKeys reverse index: podIdentifier -> []engineKeyToRequestKeyMapping for all the indexes.
  • Clear: Looks up only the request keys for the target pod via podToRequestKeys
  • Iterates those keys, and removes matching entries, prunes empty hashes + their engine-key mappings
  • Updates the reverse index: drops the pod entirely if DeviceTier is empty, or trims to still-alive request keys if a specific tier was targeted

Testing

• pkg/kvcache/kvblock/index_test.go

Examples:

• Updated the kv_events/offline and valkey_example to simulate the Clear event.

[github-actions]

Unsigned commits detected! Please sign your commits.

For instructions on how to set up GPG/SSH signing and verify your commits, please see GitHub Documentation.

[vMaroon]

Hi @yash9263 - thank you for the contribution. One detail that may have not been clear from the issue: an event coming from a certain endpoint should only clear the latter's entries.

E.g., if the index state maps pod A having blocks [x, y, z], and pod B having blocks [x, y], then an AllBlocksCleared event from pod A should remove the mappings of [x, y] -> pod A while keeping pod B, and completely remove z since now it maps to no one.

[yash9263]

E.g., if the index state maps pod A having blocks [x, y, z], and pod B having blocks [x, y], then an AllBlocksCleared event from pod A should remove the mappings of [x, y] -> pod A while keeping pod B, and completely remove z since now it maps to no one.

Hi @vMaroon, apologies for the oversight and not clarifying this earlier.

Given the indexes are structured engineKey -> requestKey -> podIdentifier, to remove all the blocks associated with a pod, it will require traversing each block to remove the pod mappings and remove the engineKey if the requestKey is empty or removed. For the Redis index, it may require multiple round-trips.

Should we also consider deviceTier while evicting blocks for the given podIdentifier here as well? Since it was part of the original event struct:

• only remove blocks for the specified device tier
• Remove blocks across gpu and cpu, if no device tier is provided?

Would it be viable to introduce a reverse index pod -> engineKey to make evicting all blocks for a pod more efficient?
Or I can proceed with the O(n) scan and evict approach.

[yash9263]

@vMaroon, ready for review.

[gyliu513]

@yash9263 I think you need rebase your PR to latest code to resolve conflict?

[gyliu513]

Good progress, thanks @yash9263 !

[gyliu513]

@vMaroon @sagearc @yankay I think this is in good shape, it is great if you guys can check if this can be merged, thanks!

[gyliu513]

/lgtm

Thanks @yash9263 !