Skip to content

[kube-prometheus-stack] Define HTTPRoute backendRef group/kind/weight explicitly #6441

@robasp

Description

@robasp

Is your feature request related to a problem ?

Since v65.6.0, kube-prometheus-stack supports the Gateway API and automatically creates an HTTPRoute resource if enabled. This generally works as expected. However, in the generated backendRefs, only name and port are specified:

    - backendRefs:
        - name: {{ $serviceName }}
          port: {{ $servicePort }}

As described in the HTTPRoute CRD, the fields group, kind, and weight default to:
group: "", kind: Service, weight: 1
and are therefore added automatically by Kubernetes:

    - backendRefs:
        - group: ""
          kind: Service
          name: {{ $serviceName }}
          port: {{ $servicePort }}
          weight: 1

In GitOps setups (e.g. Argo CD), this causes reconciliation noise:
Argo CD removes group, kind, and weight because they are not present in the kube-prometheus-stack Helm template, while Kubernetes immediately re-adds them on apply, resulting in a perpetual diff loop.

Describe the solution you'd like.

Suggestion:
Explicitly set group, kind, and weight to the default values in the Helm template for all HTTPRoutes used in the chart

    - backendRefs:
        - group: ""
          kind: Service
          name: {{ $serviceName }}
          port: {{ $servicePort }}
          weight: 1

Or allow override using values.

values.yaml:

prometheus:
  route:
    main:
      enabled: true
      serviceWeight: 1
      serviceKind: Service
      serviceGroup: ""

templates/prometheus/route.yaml:

...
{{- $serviceKind := $route.serviceKind | default "Service" -}}
{{- $serviceGroup := $route.serviceGroup | default "" -}}
{{- $serviceWeight := $route.serviceWeight | default 1 -}}

...

- backendRefs:
    - group: {{ $serviceGroup | quote }}
      kind: {{ $serviceKind }}
      name: {{ $serviceName }}
      port: {{ $servicePort }}
      weight: {{ $serviceWeight }}

Describe alternatives you've considered.

In Argo CD, this issue can be mitigated by ignoring changes to the defaulted fields on HTTPRoute resources, for example:

  ignoreDifferences:
  - group: gateway.networking.k8s.io
    kind: HTTPRoute
    namespace: monitoring
    jqPathExpressions:
      - .spec.rules[].backendRefs[].group
      - .spec.rules[].backendRefs[].kind
      - .spec.rules[].backendRefs[].weight

This resolves the reconciliation loop, but requires consumers of the chart to add tool-specific configuration. It would be cleaner if the Helm chart explicitly defined these fields so that the rendered manifests already match the normalized resources stored in Kubernetes, minimizing drift in GitOps workflows.

Additional context.

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions