Chat Completions support; WIP

[guygir]

1. Dockerfile

• Minimal Python-enabled runtime to support chat templating.
• Builder: installs Python 3.12 headers; fetches preprocessing requirements and render_jinja_template_wrapper.py.
• Runtime: use of microdnf because a package manager is required to install Python 3.12 and zeromq at runtime, and microdnf is lighter than dnf (fewer dependencies), so it keeps the image smaller.; copies the wrapper into /usr/local/lib/python3.12/site-packages/.
• Python deps installed to a single site-packages, --no-cache-dir to keep image size smaller; torch filtered out manually for now.
• Env set for reliable imports and HF cache (PYTHONPATH, HF_HOME).

2. Precise Prefix Cache (scorer)

• Adds real chat-completions preprocessing before scoring.
• Builds RenderJinjaTemplateRequest from messages (uses msg.Content.Raw), initializes processor, fetches model chat template, renders, returns flattened prompt to KV-cache scoring.
• Regular completions use prompt directly.
• Includes PREPROCESSING: INFO/DEBUG logs; will be trimmed post-WIP.

3. Submodules

• Adds empty placeholders for llm-d-kv-cache-manager and gateway-api-inference-extension.
• Purpose: keep upstream references during integration without pulling large repos.
• Final build relies on Go modules and fetches Python files during Docker build; submodule content not required.
• Documents where things came from and makes it easier to work with if needed later; can be removed if it becomes unnecessary later.

[elevran]

@guygir also note that DCO check is failing and should be fixed

[vMaroon]

This PR should mimic as closely as possible the work done in this kv-cache-manager one: llm-d/llm-d-kv-cache#92

This includes:

• Build changes: CI, Makefile and Dockerfile updates
• The precise-prefix-cache-scorer plugin to operate similarly to the online kvevents example
• Tests and documentation

/hold

[guygir]

Updated the PR, and all previous comments have been addressed.
Regarding the last comment by @vMaroon - I will re-run the benchmark (including a more reliable workload, as discussed with @elevran) to produce a detailed doc similar to the one in kv-cache manager.

Note on DCO: Two commits in the branch (ec6d849 by learner0810, c40cc15 by Morgan Foster) are currently failing DCO checks. These are upstream commits that came into this branch by merging upstream, and they have the same signoff issues in upstream/main itself (missing/incorrect signoffs). Guidance on how to handle these upstream commits - should they be addressed in upstream/main, or is there another approach that I should take in my branch?

[elevran]

@guygir would you kindly

• rebase your work on top of main (do NOT merge commit from main which pulls in the all changes and makes it impossible to know which files yout changed and which changed on main. Doubt that you need to change 83 files and 5000 lines...)
• resolve any remaining conflicts after the rebase
  Thanks!

[elevran]

closing this in favor of clean (rebased) PR