-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Description
Is your feature request related to a problem ?
Since v65.6.0, kube-prometheus-stack supports the Gateway API and automatically creates an HTTPRoute resource if enabled. This generally works as expected. However, in the generated backendRefs, only name and port are specified:
- backendRefs:
- name: {{ $serviceName }}
port: {{ $servicePort }}
As described in the HTTPRoute CRD, the fields group, kind, and weight default to:
group: "", kind: Service, weight: 1
and are therefore added automatically by Kubernetes:
- backendRefs:
- group: ""
kind: Service
name: {{ $serviceName }}
port: {{ $servicePort }}
weight: 1
In GitOps setups (e.g. Argo CD), this causes reconciliation noise:
Argo CD removes group, kind, and weight because they are not present in the kube-prometheus-stack Helm template, while Kubernetes immediately re-adds them on apply, resulting in a perpetual diff loop.
Describe the solution you'd like.
Suggestion:
Explicitly set group, kind, and weight to the default values in the Helm template for all HTTPRoutes used in the chart
- backendRefs:
- group: ""
kind: Service
name: {{ $serviceName }}
port: {{ $servicePort }}
weight: 1
Or allow override using values.
values.yaml:
prometheus:
route:
main:
enabled: true
serviceWeight: 1
serviceKind: Service
serviceGroup: ""
templates/prometheus/route.yaml:
...
{{- $serviceKind := $route.serviceKind | default "Service" -}}
{{- $serviceGroup := $route.serviceGroup | default "" -}}
{{- $serviceWeight := $route.serviceWeight | default 1 -}}
...
- backendRefs:
- group: {{ $serviceGroup | quote }}
kind: {{ $serviceKind }}
name: {{ $serviceName }}
port: {{ $servicePort }}
weight: {{ $serviceWeight }}
Describe alternatives you've considered.
In Argo CD, this issue can be mitigated by ignoring changes to the defaulted fields on HTTPRoute resources, for example:
ignoreDifferences:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: monitoring
jqPathExpressions:
- .spec.rules[].backendRefs[].group
- .spec.rules[].backendRefs[].kind
- .spec.rules[].backendRefs[].weight
This resolves the reconciliation loop, but requires consumers of the chart to add tool-specific configuration. It would be cleaner if the Helm chart explicitly defined these fields so that the rendered manifests already match the normalized resources stored in Kubernetes, minimizing drift in GitOps workflows.
Additional context.
No response