Skip to content

Standardize inferencepool Helm templates and drop unnecessary tpl #1989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tsj-30 wants to merge 3 commits into
base: main
Choose a base branch
from
+4 −4
Open

Standardize inferencepool Helm templates and drop unnecessary tpl #1989

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7916e93
Standardize inferencepool Helm templates and drop unnecessary tpl
tsj-30 Dec 11, 2025
e99b808
Resolve comment
tsj-30 Dec 15, 2025
ce0372b
resolve comments
tsj-30 Dec 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions config/charts/inferencepool/templates/epp-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,32 +38,32 @@ spec:
{{- with .Values.inferenceExtension.sidecar.args }}
args:
{{- range . }}
- {{ tpl . $ | quote }}
- {{ . | quote }}
{{- end }}
{{- end }}
Comment on lines 38 to 43
Copy link
Contributor

@nirrozenbaum nirrozenbaum Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we use the same pattern of epp cmd-line args?
see here (there it is called flags, but you can keep in your PR the use of the term args, I'm trying to scope this PR to helm chart standartization):

gateway-api-inference-extension/config/charts/inferencepool/templates/epp-deployment.yaml

Lines 100 to 103 in 0f0dff6

{{- range $key, $value := .Values.inferenceExtension.flags }}
- --{{ $key }}
- "{{ $value }}"
{{- end }}

there are few more places in the yaml file that could be written using the same pattern

Copy link
Contributor

@nirrozenbaum nirrozenbaum Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it terms of readability, the above pattern makes the templates easier for understanding and keeping it very simple.

Copy link
Collaborator

@kfswain kfswain Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ to Nir's comments, so long as that works here. Not sure if this needs to support env vars also, but perhaps we decompose that to something else anyways, so we can keep consistency with EPP container config

{{- with .Values.inferenceExtension.sidecar.env }}
env:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- with .Values.inferenceExtension.sidecar.ports }}
ports:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- with .Values.inferenceExtension.sidecar.livenessProbe }}
livenessProbe:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- with .Values.inferenceExtension.sidecar.readinessProbe }}
readinessProbe:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- with .Values.inferenceExtension.sidecar.resources }}
resources:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- with .Values.inferenceExtension.sidecar.volumeMounts }}
volumeMounts:
{{- toYaml . | nindent 10 }}
{{- . | toYaml | nindent 10 }}
{{- end }}
{{- end }}
- name: epp
Expand Down Expand Up @@ -117,7 +117,7 @@ spec:
- name: metrics
containerPort: 9090
{{- if .Values.inferenceExtension.extraContainerPorts }}
{{- toYaml .Values.inferenceExtension.extraContainerPorts | nindent 8 }}
{{- .Values.inferenceExtension.extraContainerPorts | toYaml | nindent 8 }}
{{- end }}
livenessProbe:
{{- if gt (.Values.inferenceExtension.replicas | int) 1 }}
Expand Down Expand Up @@ -177,31 +177,31 @@ spec:
value: {{ .Values.inferenceExtension.tracing.sampling.samplerArg | quote }}
{{- end }}
{{- if .Values.inferenceExtension.env }}
{{- toYaml .Values.inferenceExtension.env | nindent 8 }}
{{- .Values.inferenceExtension.env | toYaml | nindent 8 }}
{{- end }}
volumeMounts:
- name: plugins-config-volume
mountPath: "/config"
{{- if .Values.inferenceExtension.volumeMounts }}
{{- toYaml .Values.inferenceExtension.volumeMounts | nindent 8 }}
{{- .Values.inferenceExtension.volumeMounts | toYaml | nindent 8 }}
{{- end }}
{{- include "gateway-api-inference-extension.latencyPredictor.containers" . | nindent 6 }}
volumes:
{{- if .Values.inferenceExtension.volumes }}
{{- toYaml .Values.inferenceExtension.volumes | nindent 6 }}
{{- .Values.inferenceExtension.volumes | toYaml | nindent 6 }}
{{- end }}
{{- if .Values.inferenceExtension.sidecar.volumes }}
{{- tpl (toYaml .Values.inferenceExtension.sidecar.volumes) $ | nindent 6 }}
{{- .Values.inferenceExtension.sidecar.volumes | toYaml | nindent 6 }}
{{- end }}
- name: plugins-config-volume
configMap:
name: {{ include "gateway-api-inference-extension.name" . }}
{{- include "gateway-api-inference-extension.latencyPredictor.volumes" . | nindent 6 }}
{{- if .Values.inferenceExtension.affinity }}
affinity:
{{- toYaml .Values.inferenceExtension.affinity | nindent 8 }}
{{- .Values.inferenceExtension.affinity | toYaml | nindent 8 }}
{{- end }}
{{- if .Values.inferenceExtension.tolerations }}
tolerations:
{{- toYaml .Values.inferenceExtension.tolerations | nindent 8 }}
{{- .Values.inferenceExtension.tolerations | toYaml | nindent 8 }}
{{- end }}
2 changes: 1 addition & 1 deletion config/charts/inferencepool/templates/epp-service.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ spec:
protocol: TCP
port: {{ .Values.inferenceExtension.metricsPort | default 9090 }}
{{- with .Values.inferenceExtension.extraServicePorts }}
{{- toYaml . | nindent 4 }}
{{- . | toYaml | nindent 4 }}
{{- end }}
type: ClusterIP
2 changes: 1 addition & 1 deletion config/charts/inferencepool/templates/epp-servicemonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ metadata:
labels:
{{- include "gateway-api-inference-extension.labels" . | nindent 4 }}
{{- with .Values.inferenceExtension.monitoring.prometheus.extraLabels }}
{{- toYaml . | nindent 4 }}
{{- . | toYaml | nindent 4 }}
{{- end }}
spec:
endpoints:
Expand Down
2 changes: 1 addition & 1 deletion config/charts/inferencepool/templates/istio.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,6 @@ spec:
insecureSkipVerify: true
{{- with (index (index $dr "trafficPolicy") "connectionPool") }}
connectionPool:
{{- toYaml . | nindent 6 }}
{{- . | toYaml | nindent 6 }}
{{- end }}
{{- end }}