[TensorRT-LLM] KVCacheManager.add_sequence_with_eviction API compatibility issue

[ShaobinChen-AH]

Description

I'm using tensorrt_llm.bindings.internal.batch_manager.KVCacheManager for KV cache management in a recommendation system. There seems to be an API compatibility issue with the add_sequence_with_eviction method.

Environment

• TensorRT-LLM version: 0.19.0 (pip show tensorrt-llm)
• CUDA version: [12.4]
• Python version: 3.9
• GPU: A6000

Problem

The following code worked in an older version of TensorRT-LLM but now fails:

from tensorrt_llm.bindings.internal.batch_manager import KVCacheManager

manager = KVCacheManager(...)
user_id = 867
start_pos = 0
new_history_length = 28
user_ids_set = {867}

# Old API (worked before):
manager.add_sequence_with_eviction(user_id, start_pos, new_history_length, 1, None, user_ids_set)

# Error:
TypeError: add_sequence_with_eviction(): incompatible function arguments.
The following argument types are supported:
    1. (self: tensorrt_llm.bindings.internal.batch_manager.BaseKVCacheManager, 
         arg0: int, arg1: int, arg2: int, arg3: int, 
         arg4: tensorrt_llm.bindings.internal.batch_manager.LlmRequest) -> None
         
        

[ShaobinChen-AH]

How to solve this ? Should I remove final argument i.e., user_ids_set ?