[Usage] Why is no leaderTemplate generated during multi-node deployment?

[yyzxw]

if set multinode=true, the leaderWorkerSet resource will be rendered, but it does not contain the leaderTemplate: https://github.com/llm-d-incubation/llm-d-modelservice/blob/main/charts/llm-d-modelservice/templates/decode-lws.yaml#L36

The data parallelism in vLLM requires distinguishing between leader nodes and worker nodes, see: https://docs.vllm.ai/en/latest/serving/data_parallel_deployment

May I ask why it is not supported? If necessary, I will submit a PR to add it. Looking forward to your reply

[nicole-lihui]

At the moment, the current multinode implementation in llm-d-modelservice mainly renders a single workerTemplate in LWS, so it cannot cleanly differentiate leaderTemplate and workerTemplate configuration.
In real multi-node production deployments, leader and worker roles usually need different runtime settings.

It is possible to work around this in a single template by using scripts and checking LWS_WORKER_INDEX (for example in the wide-ep guide: decode.yaml), but completing first-class leaderTemplate support is more aligned with the LeaderWorkerSet design.

[kalantar]

This has generally been solved by a command that runs bash to distinguish between the nodes. Something like https://github.com/llm-d/llm-d/blob/main/guides/wide-ep-lws/manifests/modelserver/base/decode.yaml#L77-L89. Note that this is a kustomize template, not a modelservice template. The version that I've seen with modelservice does this:

    modelCommand: custom
    command:
      - /bin/sh
      - '-c'
    args:
     - |
        python3 /setup/preprocess/set_llmdbench_environment.py; source $HOME/llmdbench_env.sh; exec vllm serve /model-cache/models/Qwen/Qwen3-30B-A3B \
        --served-model-name Qwen/Qwen3-30B-A3B \

where /setup/preprocess/set_llmdbench_environment.py is on a mounted volume. It does some n/w discovery and sets env variables. It is not necessary to use a mounted volume, it could have been inlined.

TL;DR the differences were minor and could be determined at runtime; seemed better to specify it only once even if slightly more complex.