Update requester template to work with latest FMA

[manoelmarques]

Added new resources to the requester template. They use custom CRDs that need to be pre-installed separately:

InferenceServerConfig: https://raw.githubusercontent.com/llm-d-incubation/llm-d-fast-model-actuation/main/config/crd/fma.llm-d.ai_inferenceserverconfigs.yaml

LauncherConfig: https://raw.githubusercontent.com/llm-d-incubation/llm-d-fast-model-actuation/main/config/crd/fma.llm-d.ai_launcherconfigs.yaml

[rubambiza]

LGTM. Leave some general comments without explicit approval to make sure others get a chance to weigh in.

[rubambiza]

I presume the requester will always be launched on GPU-enabled nodes. However, if it is launched using CPU, I wanted to flag that there is an upcoming PR from Jun that will set the VLLM_CPU_KV_CACHE_SPACE variable to avoid a kill by OOM for the launcher pod. This is just an FYI in case we need to make changes in the near future.

[rubambiza]

@aavarghese Just a general pondering, do I understand it correctly that the tag the launcher images use follows whatever semver we are using for a (test) release that is availed to llmd-benchmark? 👇

https://github.com/aavarghese/llm-d-fast-model-actuation/blob/5d92470760aa3825cab8dc1624fee6f33b65228d/.github/workflows/publish-release.yaml#L132

If so, then I think whatever is being output here should not necessarily be latest, right?
It would be great to make sure that all these discussions we are going through process-wise are actually useful.

CC: @diegocastanibm

[aavarghese]

I think latest is the best default right now?! And when Manoel will be running the benchmark with a stable FMA release or release candidate, he will use values-requester.yaml to specify the launcher/requester tag that will override latest...

[kalantar]

I lack understanding of the FMA approach. It appears to me that the modelServerConfig is similar in many ways to a prefill or decode pod. I see these defined as well. Could you explain the relationship or point me to a document explaining I could look at.

[rubambiza]

I lack understanding of the FMA approach. It appears to me that the modelServerConfig is similar in many ways to a prefill or decode pod. I see these defined as well. Could you explain the relationship or point me to a document explaining I could look at.

@kalantar We are working on updating the documentation for FMA as it evolves. For now, you can get an overview in this open PR: https://github.com/rubambiza/llm-d-fast-model-actuation/blob/202dc3691615f9677de9578d11e0b470815ee33d/README.md