What happened?
When using arena serve kserve with the --config-file option, the deployment fails with a Helm template rendering error:
Error: template: kserve/templates/inferenceservice.yaml:423:36: executing "kserve/templates/inferenceservice.yaml" at <4>: invalid value; expected string
Command that failed:
arena serve kserve \
--name=test-model \
--image=kserve/sklearnserver:v0.11.0 \
--model-name=iris \
--model-uri=s3://model-bucket/iris \
--config-file /host/path/config.yaml:/container/path/config.yaml
Root Cause:
The bug was introduced in commit 19b5133e ("refactor: use helm lib instead of helm binary #1207") when Arena migrated from using Helm CLI to Helm Go SDK.
In pkg/util/helm/util.go, the GenerateHelmTemplate() function accepts an options parameter (which contains --set-file options generated from --config-file), but never uses these options. The function only reads values from the values file and ignores all --set-file parameters.
As a result:
- Config file paths are passed as
--set-file configFiles.<hash>.config-N.content=/path/to/file
- These options reach
GenerateHelmTemplate() but are completely ignored
- The Helm template
kserve/templates/inferenceservice.yaml expects $configFileInfo.content to contain the file content
- Since no file content was read, the value is
nil
- The
indent function fails when receiving nil instead of a string
Affected Versions:
- Arena v0.15.0 - v0.15.4 (all versions after commit 19b5133)
Proposed Fix:
Add a processSetFileOptions() function in pkg/util/helm/util.go that:
- Parses
--set-file options from the options parameter
- Reads the file content from disk
- Merges the content into the values map at the appropriate nested keys
- Modifies
GenerateHelmTemplate() to call this function when options are present
This fix maintains backward compatibility and restores the --config-file functionality without requiring the --helm-binary=true workaround.
What did you expect to happen?
The --config-file option should work as before, allowing users to mount configuration files into the container. The Helm template should receive the actual file content from the --set-file options and render successfully.
The expected behavior is that GenerateHelmTemplate():
- Receives the
--set-file options
- Reads the content of the files referenced in those options
- Merges the file content into the Helm values map
- Passes the values to the template renderer with the content properly populated
Workaround:
Adding --helm-binary=true flag forces Arena to use the legacy mode (Helm CLI), which properly processes --set-file options:
arena serve kserve \
--name=test-model \
--image=kserve/sklearnserver:v0.11.0 \
--model-name=iris \
--model-uri=s3://model-bucket/iris \
--config-file /host/path/config.yaml:/container/path/config.yaml \
--helm-binary=true
Environment
Kubernetes version:
$ kubectl version
Client Version: v1.28.x
Server Version: v1.28.x
Arena version:
$ arena version
Version: v0.15.0 (or v0.15.1-v0.15.4)
Impacted by this bug?
Give it a 👍 We prioritize the issues with most 👍
What happened?
When using
arena serve kservewith the--config-fileoption, the deployment fails with a Helm template rendering error:Command that failed:
Root Cause:
The bug was introduced in commit
19b5133e("refactor: use helm lib instead of helm binary #1207") when Arena migrated from using Helm CLI to Helm Go SDK.In
pkg/util/helm/util.go, theGenerateHelmTemplate()function accepts anoptionsparameter (which contains--set-fileoptions generated from--config-file), but never uses these options. The function only reads values from the values file and ignores all--set-fileparameters.As a result:
--set-file configFiles.<hash>.config-N.content=/path/to/fileGenerateHelmTemplate()but are completely ignoredkserve/templates/inferenceservice.yamlexpects$configFileInfo.contentto contain the file contentnilindentfunction fails when receivingnilinstead of a stringAffected Versions:
Proposed Fix:
Add a
processSetFileOptions()function inpkg/util/helm/util.gothat:--set-fileoptions from theoptionsparameterGenerateHelmTemplate()to call this function when options are presentThis fix maintains backward compatibility and restores the
--config-filefunctionality without requiring the--helm-binary=trueworkaround.What did you expect to happen?
The
--config-fileoption should work as before, allowing users to mount configuration files into the container. The Helm template should receive the actual file content from the--set-fileoptions and render successfully.The expected behavior is that
GenerateHelmTemplate():--set-fileoptionsWorkaround:
Adding
--helm-binary=trueflag forces Arena to use the legacy mode (Helm CLI), which properly processes--set-fileoptions:Environment
Kubernetes version:
Arena version:
Impacted by this bug?
Give it a 👍 We prioritize the issues with most 👍