Update precise-prefix-cache-scorer to use tokens-based GetPodScores

[acardace]

Summary

Updates the precise-prefix-cache-scorer to perform tokenization in the scheduler and pass pre-computed tokens to GetPodScores, rather than delegating tokenization to the kv-cache indexer.

Related to llm-d/llm-d-kv-cache#244
Related to #530

Note: This PR depends on llm-d/llm-d-kv-cache#266 and must be merged after it.

Changes

• Build: Update Makefile PYTHONPATH to reference llm-d-kv-cache module
• Scorer: Tokenize the prompt in the scheduler, then pass tokens to GetPodScores
• Tests: Adapt to updated signatures and reuse tokenizer's built-in chat templater

[elevran]

/hold for post 0.5

[acardace]

@elevran what's the release cadence for llm-d-kv-cache? Of course the corresponding PR in kv-cache must be merged first and have a tag before merging this.

[elevran]

I believe it's 6w give or take. @vMaroon can give you a more exact answer. From inference scheduler point of view the hold can be removed as we cut the 0.5 RC in the next few days

[elevran]

@acardace @vMaroon @kfswain
does it make sense to do toeknziation as part of the scorer or should this be more of an "infra" service (perhaps as part of an explicit data preparation phase)?

[acardace]

@acardace @vMaroon @kfswain does it make sense to do toeknziation as part of the scorer or should this be more of an "infra" service (perhaps as part of an explicit data preparation phase)?

My take is that this is just prepping in order to move tokenization as a service, possibly inside GAIE. I'm actually working on a RFC to introduce tokenization as a service inside the IGW.

[github-actions]

This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the lifecycle/stale label.