What would you like to be added:
Add video support with video token estimation for multimodal approximate prefix cache plugin
Why is this needed:
Currently multimodal approximate prefix cache plugin just support token estimation with image, we also want to let multimodal approximate prefix cache plugin support video with token estimation.
Please note as approximate prefix cache plugin use the unified token producer for token estimation, such video token estimation logic should be included in the unified token producer, see https://github.com/llm-d/llm-d-router/tree/main/pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer with estimate backend
What would you like to be added:
Add video support with video token estimation for multimodal approximate prefix cache plugin
Why is this needed:
Currently multimodal approximate prefix cache plugin just support token estimation with image, we also want to let multimodal approximate prefix cache plugin support video with token estimation.
Please note as approximate prefix cache plugin use the unified token producer for token estimation, such video token estimation logic should be included in the unified token producer, see https://github.com/llm-d/llm-d-router/tree/main/pkg/epp/framework/plugins/requestcontrol/dataproducer/tokenizer with
estimatebackend