Deployment Fails with Sim Preset

[danehans]

Component

I don't know

Describe the bug

Model service pods (vLLM Sim) are stuck in Pending after successful install using the vLLM Simulator preset.

Steps to reproduce

I set baseConfigMapRefName: basic-sim-preset in Helm values and run ./llmd-installer.sh --gateway kgateway -m --namespace default -f my-values.yaml. The installation succeeded, but the vLLM Sim pods are stuck in pending due to:

        resources:
          limits:
            nvidia.com/gpu: "1"
          requests:
            nvidia.com/gpu: "1"

Additional context or screenshots

Manually removing the nvidia.com/gpu: "1" resources causes the pods to no longer be in Pending.

[danehans]

Pods then go into CrashLoopBackOff. xref: #298

[nerdalert]

Hi @danehans I haven't validated vllm-sim. I need to for sure and we should probably get a sample values override file with that in https://github.com/llm-d/llm-d-deployer/tree/main/quickstart/examples. I will block some time this week(end) to take a look with you as that has been neglected 😭 Ty!

[maugustosilva]

I believe this to be a duplicate #298

[danehans]

@maugustosilva the 2 issues do overlap. The difference is with #298, the vllm sim pod is stuck in Pending.

[nerdalert]

Hey frens, pushed a patch to get --served-model-name in llm-d-inference-sim. I still need to add the extra-args for --model but it comes up and works with the following and minimally invasive in the sim repo. There are config changes coming next week there so this might be the workaround until then. I will push an example values override file if that merges. llm-d/llm-d-inference-sim#69

sampleApplication:
  baseConfigMapRefName: basic-sim-preset
  model:
    modelArtifactURI: hf://Qwen/Qwen3-0.6B
    modelName: "Qwen/Qwen3-0.6B"
  prefill:
    replicas: 0
  decode:
    replicas: 1
    extraArgs:
      - --model
      - Qwen/Qwen3-0.6B

[danehans]

@nerdalert thanks for the ^ update. Do you know if this issue is still open?

Never mind, I see this repo is deprecated in favor of https://github.com/llm-d-incubation/llm-d-modelservice.