Skip to content

Bug: GenerateHelmTemplate ignores --set-file options, breaking --config-file #1434

@Leoyzen

Description

@Leoyzen

What happened?

When using arena serve kserve with the --config-file option, the deployment fails with a Helm template rendering error:

Error: template: kserve/templates/inferenceservice.yaml:423:36: executing "kserve/templates/inferenceservice.yaml" at <4>: invalid value; expected string

Command that failed:

arena serve kserve \
  --name=test-model \
  --image=kserve/sklearnserver:v0.11.0 \
  --model-name=iris \
  --model-uri=s3://model-bucket/iris \
  --config-file /host/path/config.yaml:/container/path/config.yaml

Root Cause:
The bug was introduced in commit 19b5133e ("refactor: use helm lib instead of helm binary #1207") when Arena migrated from using Helm CLI to Helm Go SDK.

In pkg/util/helm/util.go, the GenerateHelmTemplate() function accepts an options parameter (which contains --set-file options generated from --config-file), but never uses these options. The function only reads values from the values file and ignores all --set-file parameters.

As a result:

  1. Config file paths are passed as --set-file configFiles.<hash>.config-N.content=/path/to/file
  2. These options reach GenerateHelmTemplate() but are completely ignored
  3. The Helm template kserve/templates/inferenceservice.yaml expects $configFileInfo.content to contain the file content
  4. Since no file content was read, the value is nil
  5. The indent function fails when receiving nil instead of a string

Affected Versions:

  • Arena v0.15.0 - v0.15.4 (all versions after commit 19b5133)

Proposed Fix:
Add a processSetFileOptions() function in pkg/util/helm/util.go that:

  1. Parses --set-file options from the options parameter
  2. Reads the file content from disk
  3. Merges the content into the values map at the appropriate nested keys
  4. Modifies GenerateHelmTemplate() to call this function when options are present

This fix maintains backward compatibility and restores the --config-file functionality without requiring the --helm-binary=true workaround.

What did you expect to happen?

The --config-file option should work as before, allowing users to mount configuration files into the container. The Helm template should receive the actual file content from the --set-file options and render successfully.

The expected behavior is that GenerateHelmTemplate():

  1. Receives the --set-file options
  2. Reads the content of the files referenced in those options
  3. Merges the file content into the Helm values map
  4. Passes the values to the template renderer with the content properly populated

Workaround:
Adding --helm-binary=true flag forces Arena to use the legacy mode (Helm CLI), which properly processes --set-file options:

arena serve kserve \
  --name=test-model \
  --image=kserve/sklearnserver:v0.11.0 \
  --model-name=iris \
  --model-uri=s3://model-bucket/iris \
  --config-file /host/path/config.yaml:/container/path/config.yaml \
  --helm-binary=true

Environment

Kubernetes version:

$ kubectl version
Client Version: v1.28.x
Server Version: v1.28.x

Arena version:

$ arena version
Version: v0.15.0 (or v0.15.1-v0.15.4)

Impacted by this bug?

Give it a 👍 We prioritize the issues with most 👍

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions