Add global registry settings instead of per-chart ones

[pasha-gurkov]

Description:

Hello! We're running Envoy Gateway in an environment with a private container registry available and preferable to use over docker.io.

It is possible to achieve, but is a little bit cumbersome with the current chart setup.

First, instead of overriding the registry just once, we have to override everything twice, like so:

gateway-helm:
  deployment:
    envoyGateway:
      image:
        repository: private.registry.example.com/envoyproxy/gateway
global:
  images:
    ratelimit:
      image: private.registry.example.com/envoyproxy/ratelimit:123abc

Second, the common global overrides as used with bitnami charts (e.g. nginx https://github.com/bitnami/charts/blob/0f3c1ccdeefe6e244c9ff9577e06b6c7e9b4bb15/bitnami/nginx/values.yaml#L12) and some others (e.g. Grafana agent, https://github.com/grafana/agent/blob/18b357cd9cd0bbc5946b73e2ca82ad237c2d52d3/operations/helm/charts/grafana-agent/values.yaml#L10) don't really work: the globals defined right now are specific to Envoy charts. It kind of defeats the purpose of Helm globals, which is to be defined once at the top level, propagate and be applied to any charts down the road.

What I would like to propose is to change the charts so it's possible to define the registry just once at the top level as global.imageRegistry, and invert the current overriding logic: right now the locals take precedence over the globals, which means the global overriding won't apply if specified. With this change, it'd be possible to specify the registry only once at the very top level, and have an air-gapped Helm environment get the images from an internal registry. Whoever uses bitnami charts (and similar) along Envoy Gateway gets private registries working for free.

The override would look like this, somewhere at the top chart level:

global:
  imageRegistry: private.registry.example.com

I will be posting an MR addressing this shortly.

Relevant Links
Similar Grafana agent issue: grafana/agent#3929
Corresponding MR: grafana/agent#4127

[arkodg]

you probably need to only define it in one section

  global:
    images:
      envoyGateway:
        image: example.com/gateway:some-tag
        pullSecrets:
          - name: registry-creds
      ratelimit:
        image: example.com/ratelimit:some-tag
        pullSecrets:
          - name: registry-creds

[arkodg]

keeping this issue open to track only changing registry, w/o needing to change image and tag name

[pasha-gurkov]

@arkodg unfortunately this still has disadvantages: we need to specify everything fully twice, instead of just overriding registry and pullSecrets once; and whenever gateway chart is released, we have to perform an extra step of finding out current ratelimit tag and change that as well.