feat(vector): add emptyConfig and configSidecar parameter to enable dynamic vector configuration by label

charts/vector/README.md

@@ -17,6 +17,8 @@ helm repo update
 
 Kubernetes: `>=1.28.0-0`
 
+
+
 ## Quick start
 
 By default, Vector runs as a `StatefulSet` in the "Aggregator" role. It can alternatively run as a `Deployment` for the "Stateless-Aggregator" role or a `DaemonSet` for the "Agent" role.
@@ -108,7 +110,7 @@ customConfig:
 
 ### API exposure and health probes
 
-When using chart-managed default configuration (without `customConfig` and `existingConfigMaps`),
+When using chart-managed default configuration (without `customConfig`, `existingConfigMaps` and `emptyConfig`),
 Vector's API listens on `0.0.0.0:8686`, and the chart applies a default readiness probe
 (`grpc` on port `8686`).
 
@@ -147,6 +149,22 @@ helm install <RELEASE_NAME> \
 | autoscaling.targetMemoryUtilizationPercentage | int | `nil` | Target memory utilization for Vector's HPA. |
 | command | list | `[]` | Override Vector's default command. |
 | commonLabels | object | `{}` | Add additional labels to all created resources. |
+| configSidecar | object | `{"enabled":false,"ignoreAlreadyProcessed":false,"image":{"registry":"quay.io","repository":"kiwigrid/k8s-sidecar","sha":"","tag":"2.5.4"},"imagePullPolicy":"IfNotPresent","label":"vector-config","labelValue":"true","logLevel":"INFO","rbac":{"create":true},"uniqueFilenames":false,"watchMethod":"WATCH"}` | Sidecar container collects the configmaps with specified label and stores the included files into the respective folders. If existingConfigMaps parameter is used and configSidecar is enabled, ensure that the configmaps are marked with the appropriate label. |
+| configSidecar.enabled | bool | `false` | If true, create and use a sidecar container to manage vector configuration. |
+| configSidecar.ignoreAlreadyProcessed | bool | `false` | If true, already processed ConfigMaps are ignored on subsequent runs. |
+| configSidecar.image | object | `{"registry":"quay.io","repository":"kiwigrid/k8s-sidecar","sha":"","tag":"2.5.4"}` | Define the sidecar image to use. |
+| configSidecar.image.registry | string | `"quay.io"` | Override default registry for the sidecar image. |
+| configSidecar.image.repository | string | `"kiwigrid/k8s-sidecar"` | Override default repository and name for the sidecar image. |
+| configSidecar.image.sha | string | `""` | The SHA to use for the sidecar image. |
+| configSidecar.image.tag | string | `"2.5.4"` | The tag to use for the sidecar image. |
+| configSidecar.imagePullPolicy | string | `"IfNotPresent"` | sidecar image pull policy. |
+| configSidecar.label | string | `"vector-config"` | Label that the configmaps have to be marked with to be collected by the sidecar. |
+| configSidecar.labelValue | string | `"true"` | Value of the label that the configmaps are set to. |
+| configSidecar.logLevel | string | `"INFO"` | Log level for the sidecar container. Can be one of: DEBUG, INFO, WARN, ERROR, CRITICAL. |
+| configSidecar.rbac | object | `{"create":true}` | RBAC settings for config sidecar |
+| configSidecar.rbac.create | bool | `true` | Create Role and RoleBinding for config sidecar |
+| configSidecar.uniqueFilenames | bool | `false` | If true, the sidecar will ensure that filenames are unique where duplicate data keys exist. |
+| configSidecar.watchMethod | string | `"WATCH"` | Method to use to detect ConfigMap changes. With WATCH the sidecar will do a WATCH requests, with SLEEP it will list all ConfigMaps, then sleep for 60 seconds. |
 | containerPorts | list | `[]` | Manually define Vector's containerPorts, overriding automated generation of containerPorts. |
 | customConfig | object | `{}` | Override Vector's default configs, if used **all** options need to be specified. This section supports using helm templates to populate dynamic values. See Vector's [configuration documentation](https://vector.dev/docs/reference/configuration/) for all options. |
 | daemonSet.apiVersion | string | `""` | Override the DaemonSet apiVersion. Valid for the "Agent" role. |
@@ -155,9 +173,10 @@ helm install <RELEASE_NAME> \
 | defaultVolumes | list | See `values.yaml` | Default volumes that are mounted into pods. In most cases, these should not be changed. Use `extraVolumes`/`extraVolumeMounts` for additional custom volumes. |
 | dnsConfig | object | `{}` | Specify the [dnsConfig](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config) options for Vector Pods. |
 | dnsPolicy | string | `"ClusterFirst"` | Specify the [dnsPolicy](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy) for Vector Pods. |
+| emptyConfig | bool | `false` | Provide an empty config directory. Requires args to include flag to allow empty config (https://vector.dev/docs/reference/cli/). If set, this parameter takes precedence over existingConfigMaps, customConfig and the chart's default configs. |
 | env | list | `[]` | Set environment variables for Vector containers. |
 | envFrom | list | `[]` | Define environment variables from Secrets or ConfigMaps. |
-| existingConfigMaps | list | `[]` | List of existing ConfigMaps for Vector's configuration instead of creating a new one. Requires dataDir to be set. Additionally, containerPorts, service.ports, and serviceHeadless.ports should be specified based on your supplied configuration. If set, this parameter takes precedence over customConfig and the chart's default configs. |
+| existingConfigMaps | list | `[]` | List of existing ConfigMaps for Vector's configuration instead of creating a new one. Requires dataDir to be set. Additionally, containerPorts, service.ports, and serviceHeadless.ports should be specified based on your supplied configuration. If set, this parameter takes precedence over customConfig and the chart's default configs. This parameter is not considered with configSidecar enabled. Ensure the correct label key (configSidecar.label) and  value (configSidecar.labelValue) to load existing ConfigMaps with configSidecar enabled. |
 | extraContainers | list | `[]` | Extra Containers to be added to the Vector Pods. This also supports template content, which will eventually be converted to yaml. |
 | extraObjects | list | `[]` | Create extra manifests via values. Would be passed through `tpl` for templating. |
 | extraVolumeMounts | list | `[]` | Additional Volume to mount into Vector Containers. |

charts/vector/README.md.gotmpl

@@ -106,7 +106,7 @@ customConfig:
 
 ### API exposure and health probes
 
-When using chart-managed default configuration (without `customConfig` and `existingConfigMaps`),
+When using chart-managed default configuration (without `customConfig`, `existingConfigMaps` and `emptyConfig`),
 Vector's API listens on `0.0.0.0:8686`, and the chart applies a default readiness probe
 (`grpc` on port `8686`).
 

charts/vector/templates/_pod.tpl

@@ -36,6 +36,46 @@ initContainers:
 {{- tpl (toYaml .Values.initContainers) . | nindent 2 }}
 {{- end }}
 containers:
+{{- if .Values.configSidecar.enabled }}
+  - name: config-sidecar
+    {{- $sidecarSha := .Values.configSidecar.image.sha | trimPrefix "sha256:" | default "" -}}
+    {{- if and .Values.configSidecar.image.tag $sidecarSha }}
+    image: "{{ .Values.configSidecar.image.registry }}/{{ .Values.configSidecar.image.repository }}:{{ .Values.configSidecar.image.tag }}@sha256:{{ $sidecarSha }}"
+    {{- else if and (not .Values.configSidecar.image.tag) $sidecarSha }}
+    image: "{{ .Values.configSidecar.image.registry }}/{{ .Values.configSidecar.image.repository }}@sha256:{{ $sidecarSha }}"
+    {{- else }}
+    image: "{{ .Values.configSidecar.image.registry }}/{{ .Values.configSidecar.image.repository }}:{{ .Values.configSidecar.image.tag | default "latest" }}"
+    {{- end }}
+    imagePullPolicy: {{ .Values.configSidecar.imagePullPolicy }}
+    env:
+      {{- if .Values.configSidecar.ignoreAlreadyProcessed }}
+      - name: IGNORE_ALREADY_PROCESSED
+        value: "true"
+      {{- end }}
+      - name: METHOD
+        value: {{ .Values.configSidecar.watchMethod }}
+      - name: LABEL
+        value: "{{ .Values.configSidecar.label }}"
+      {{- with .Values.configSidecar.labelValue }}
+      - name: LABEL_VALUE
+        value: {{ quote . }}
+      {{- end }}
+      {{- with .Values.configSidecar.logLevel }}
+      - name: LOG_LEVEL
+        value: "{{ . }}"
+      {{- end }}
+      - name: FOLDER
+        value: "/etc/vector/"
+      - name: RESOURCE
+        value: "configmap"
+      {{- if .Values.configSidecar.uniqueFilenames }}
+      - name: UNIQUE_FILENAMES
+        value: "true"
+      {{- end }}
+    volumeMounts:
+      - name: config
+        mountPath: "/etc/vector/"
+{{- end }}
   - name: vector
 {{- with .Values.securityContext }}
     securityContext:
@@ -47,9 +87,26 @@ containers:
     command:
     {{- toYaml . | nindent 6 }}
 {{- end }}
-{{- with .Values.args }}
+{{- $args := list }}
+{{- if .Values.args }}
+{{- $args = .Values.args }}
+{{- end }}
+{{- if or $.Values.emptyConfig $.Values.configSidecar.enabled }}
+  {{- if not (has "--allow-empty-config" $args) }}
+    {{ $args = append $args "--allow-empty-config" }}
+  {{- end }}
+  {{- if not (has "--config-dir" $args) }}
+    {{ $args = concat $args (list "--config-dir" "/etc/vector/")  }}
+  {{- end }}
+{{- end }}
+{{- if $.Values.configSidecar.enabled }}
+  {{- if not (has "--watch-config" $args) }}
+    {{ $args = append $args "--watch-config"  }}
+  {{- end }}
+{{- end }}
+{{- if $args }}
     args:
-    {{- toYaml . | nindent 6 }}
+    {{- toYaml $args | nindent 6 }}
 {{- end }}
     env:
       - name: VECTOR_LOG
@@ -125,7 +182,7 @@ containers:
 {{- if .Values.readinessProbe }}
     readinessProbe:
       {{- toYaml .Values.readinessProbe | trim | nindent 6 }}
-{{- else if and (not .Values.existingConfigMaps) (not .Values.customConfig) }}
+{{- else if and (not .Values.existingConfigMaps) (not .Values.customConfig) (not .Values.emptyConfig) }}
     readinessProbe:
       httpGet:
         path: /health
@@ -145,14 +202,18 @@ containers:
 {{- end }}
     volumeMounts:
       - name: data
-        {{- if .Values.existingConfigMaps }}
+        {{- if or .Values.emptyConfig .Values.configSidecar.enabled }}
+        mountPath: "/var/lib/vector/"
+        {{- else if .Values.existingConfigMaps }}
         mountPath: "{{ if .Values.dataDir }}{{ .Values.dataDir }}{{ else }}{{ fail "Specify `dataDir` if you're using `existingConfigMaps`" }}{{ end }}"
         {{- else }}
         mountPath: "{{ .Values.customConfig.data_dir | default "/vector-data-dir" }}"
         {{- end }}
       - name: config
         mountPath: "/etc/vector/"
+{{- if not .Values.configSidecar.enabled }}
         readOnly: true
+{{- end }}
 {{- if (eq .Values.role "Agent") }}
 {{- with .Values.defaultVolumeMounts }}
 {{- toYaml . | nindent 6 }}
@@ -193,6 +254,9 @@ volumes:
     emptyDir: {}
 {{- end }}
   - name: config
+{{- if or .Values.emptyConfig .Values.configSidecar.enabled }}
+    emptyDir: {}
+{{- else }}
     projected:
       sources:
 {{- if .Values.existingConfigMaps }}
@@ -204,6 +268,7 @@ volumes:
         - configMap:
             name: {{ template "vector.fullname" . }}
 {{- end }}
+{{- end }}
 {{- if (eq .Values.role "Agent") }}
   - name: data
   {{- if .Values.persistence.hostPath.enabled }}

charts/vector/templates/configmap.yaml

@@ -1,18 +1,29 @@
-{{- if not .Values.existingConfigMaps }}
+{{- if not (or .Values.emptyConfig .Values.existingConfigMaps) }}
 apiVersion: v1
 kind: ConfigMap
 metadata:
   name: {{ include "vector.fullname" . }}
   namespace: {{ .Release.Namespace | quote }}
   labels:
     {{- include "vector.labels" . | nindent 4 }}
+    {{- if .Values.configSidecar.enabled}}
+    {{ .Values.configSidecar.label }}: {{ quote .Values.configSidecar.labelValue }}
+    {{- end }}
 data:
   {{- if .Values.customConfig }}
+  {{- if and .Values.configSidecar.enabled .Values.customConfig.data_dir }}
+  {{- $msg := "When configSidecar is enabled, customConfig.data_dir must not be set. " }}
+  {{- $msg = printf "%sVector uses /var/lib/vector before loading custom config and does not allow overrides. " $msg }}
+  {{- $msg = printf "%sRemove customConfig.data_dir or disable configSidecar." $msg }}
+  {{- fail $msg }}
+  {{- end }}
   vector.yaml: |
 {{ tpl (toYaml .Values.customConfig) . | indent 4 }}
   {{- else if or (eq .Values.role "Aggregator") (eq .Values.role "Stateless-Aggregator") }}
   aggregator.yaml: |
+    {{- if not .Values.configSidecar.enabled}}
     data_dir: /vector-data-dir
+    {{- end }}
     api:
       enabled: true
       address: 0.0.0.0:8686
@@ -55,7 +66,9 @@ data:
           codec: json
   {{- else if (eq .Values.role "Agent") }}
   agent.yaml: |
+    {{- if not .Values.configSidecar.enabled}}
     data_dir: /vector-data-dir
+    {{- end }}
     api:
       enabled: true
       address: 0.0.0.0:8686

charts/vector/templates/rbac.yaml

@@ -46,3 +46,39 @@ subjects:
     name: {{ include "vector.serviceAccountName" . }}
     namespace: {{ .Release.Namespace | quote }}
 {{- end }}
+{{- if and .Values.configSidecar.rbac.create .Values.configSidecar.enabled }}
+---
+# Permissions for the config sidecar to read ConfigMaps.
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: {{ include "vector.fullname" . }}-config-sidecar
+  namespace: {{ .Release.Namespace | quote }}
+  labels:
+    {{- include "vector.labels" . | nindent 4 }}
+rules:
+  - apiGroups:
+      - ""
+    resources:
+      - configmaps
+    verbs:
+      - get
+      - list
+      - watch
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: {{ include "vector.fullname" . }}-config-sidecar
+  namespace: {{ .Release.Namespace | quote }}
+  labels:
+    {{- include "vector.labels" . | nindent 4 }}
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: {{ include "vector.fullname" . }}-config-sidecar
+subjects:
+  - kind: ServiceAccount
+    name: {{ include "vector.serviceAccountName" . }}
+    namespace: {{ .Release.Namespace | quote }}
+{{- end }}

charts/vector/values.yaml

@@ -71,7 +71,6 @@ hostAliases: []
 #   - "foo.local"
 #   - "bar.local"
 
-
 # podManagementPolicy -- Specify the [podManagementPolicy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies)
 # for the StatefulSet. Valid for the "Aggregator" role.
 podManagementPolicy: OrderedReady
@@ -228,7 +227,6 @@ lifecycle: {}
   #     - /bin/sleep
   #     - "10"
 
-
 # minReadySeconds -- Specify the minimum number of seconds a newly spun up pod should wait to
 # pass healthchecks before it is considered available.
 minReadySeconds: 0
@@ -334,6 +332,8 @@ ingress:
 # existingConfigMaps -- List of existing ConfigMaps for Vector's configuration instead of creating a new one. Requires
 # dataDir to be set. Additionally, containerPorts, service.ports, and serviceHeadless.ports should be specified based on
 # your supplied configuration. If set, this parameter takes precedence over customConfig and the chart's default configs.
+# This parameter is not considered with configSidecar enabled. Ensure the correct label key (configSidecar.label) and 
+# value (configSidecar.labelValue) to load existing ConfigMaps with configSidecar enabled.
 existingConfigMaps: []
 
 # dataDir -- Specify the path for Vector's data, only used when existingConfigMaps are used.
@@ -359,6 +359,44 @@ customConfig: {}
   #     encoding:
   #       codec: json
 
+# emptyConfig -- Provide an empty config directory. Requires args to include flag to allow empty config (https://vector.dev/docs/reference/cli/).
+# If set, this parameter takes precedence over existingConfigMaps, customConfig and the chart's default configs.
+emptyConfig: false
+
+# configSidecar -- Sidecar container collects the configmaps with specified label and stores the included files into the respective folders.
+# If existingConfigMaps parameter is used and configSidecar is enabled, ensure that the configmaps are marked with the appropriate label.
+configSidecar:
+  # configSidecar.enabled -- If true, create and use a sidecar container to manage vector configuration.
+  enabled: false
+  # configSidecar.image -- Define the sidecar image to use.
+  image:
+    # configSidecar.image.registry -- Override default registry for the sidecar image.
+    registry: quay.io
+    # configSidecar.image.repository -- Override default repository and name for the sidecar image.
+    repository: kiwigrid/k8s-sidecar
+    # configSidecar.image.tag -- The tag to use for the sidecar image.
+    tag: "2.5.4"
+    # configSidecar.image.sha -- The SHA to use for the sidecar image.
+    sha: ""
+  # configSidecar.imagePullPolicy -- sidecar image pull policy.
+  imagePullPolicy: IfNotPresent
+  # configSidecar.label -- Label that the configmaps have to be marked with to be collected by the sidecar.
+  label: "vector-config"
+  # configSidecar.labelValue -- Value of the label that the configmaps are set to.
+  labelValue: "true"
+  # configSidecar.watchMethod -- Method to use to detect ConfigMap changes. With WATCH the sidecar will do a WATCH requests, with SLEEP it will list all ConfigMaps, then sleep for 60 seconds.
+  watchMethod: "WATCH"
+  # configSidecar.ignoreAlreadyProcessed -- If true, already processed ConfigMaps are ignored on subsequent runs.
+  ignoreAlreadyProcessed: false
+  # configSidecar.logLevel -- Log level for the sidecar container. Can be one of: DEBUG, INFO, WARN, ERROR, CRITICAL.
+  logLevel: "INFO"
+  # configSidecar.uniqueFilenames -- If true, the sidecar will ensure that filenames are unique where duplicate data keys exist.
+  uniqueFilenames: false
+  # configSidecar.rbac -- RBAC settings for config sidecar
+  rbac:
+    # configSidecar.rbac.create -- Create Role and RoleBinding for config sidecar
+    create: true
+
 # defaultVolumes -- Default volumes that are mounted into pods. In most cases, these should not be changed.
 # Use `extraVolumes`/`extraVolumeMounts` for additional custom volumes.
 # @default -- See `values.yaml`