[helm/elastic-agent] upgrade helm chart to collect rotated logs (#11129)

Updates the elastic-agent helm chart to allow the collection of
rotated logs, including the GZIP-compressed logs.

AI tools were used to generate the deploy/helm/elastic-agent/CONTRIBUTING.md

Author: AndersonQ

changelog/fragments/1762936912-upgrade-helm-chart-to-collect-rotated-logs.yaml

@@ -0,0 +1,32 @@
+# Kind can be one of:
+# - breaking-change: a change to previously-documented behavior
+# - deprecation: functionality that is being removed in a later release
+# - bug-fix: fixes a problem in a previous version
+# - enhancement: extends functionality but does not break or fix existing behavior
+# - feature: new functionality
+# - known-issue: problems that we are aware of in a given version
+# - security: impacts on the security of a product or a user’s deployment.
+# - upgrade: important information for someone upgrading from a prior version
+# - other: does not fit into any of the other categories
+kind: feature
+
+# Change summary; a 80ish characters long description of the change.
+summary: "BETA: upgrade helm chart to collect rotated logs"
+
+# Long description; in case the summary is not enough to describe the change
+# this field accommodate a description without length limits.
+# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
+#description:
+
+# Affected component; a word indicating the component this changeset affects.
+component: helm-chart
+
+# PR URL; optional; the PR number that added the changeset.
+# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
+# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
+# Please provide it if you are adding a fragment for a different PR.
+pr: https://github.com/elastic/elastic-agent/pull/11129
+
+# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
+# If not present is automatically filled by the tooling with the issue linked to the PR number.
+issue: https://github.com/elastic/elastic-agent/issues/10889

deploy/helm/elastic-agent/CONTRIBUTING.md

@@ -0,0 +1,105 @@
+<!--
+(NOTE: AI was used on this file)
+-->
+
+# Contributing to Elastic Agent Helm Chart
+
+This is an **INCOMPLETE** set of tips and explanations for developers who want
+to modify or contribute to the Elastic Agent Helm chart.
+
+## Testing Your Changes Locally
+
+By default, the agent is installed on the `kube-system` namespace. Thus you'll
+need to pass `-n kube-system` to the `helm` invocations that interact with the
+k8s cluster.
+
+### Rendering the Chart
+
+Use `helm template` to render the chart and verify the output without actually
+installing it:
+
+```bash
+# Using a values file
+helm template elastic-agent . -f values.yaml
+
+# Or setting values directly
+helm template elastic-agent . --set kubernetes.enabled=true --set outputs.default.type=ESPlainAuthAPI
+```
+
+This works just like `helm install` but only generates the Kubernetes manifests
+without applying them to the cluster.
+
+### Testing with Different Examples
+
+Test your changes against the example configurations in the `examples/` directory:
+
+```bash
+helm template elastic-agent . -f examples/kubernetes-default/agent-kubernetes-values.yaml
+```
+
+### Applying Changes to an Existing Deployment
+
+If you have already installed the chart and want to test modifications, use
+`helm upgrade`:
+
+```bash
+# Using a values file
+helm upgrade elastic-agent . -f your-values.yaml
+
+# Or using --set to override specific values
+helm upgrade elastic-agent . --set agent.version=9.3.0
+
+# Combine both approaches
+helm upgrade elastic-agent . -f your-values.yaml --set kubernetes.enabled=false
+```
+
+## Editing the README
+
+The README.md was originally generated using [helm-docs](https://github.com/norwoodj/helm-docs).
+However, it has diverged from the auto-generated version since.
+`helm-docs` is still usefull to can pick up changes to the `values.yaml` and
+format them as the others. However, running it will completely override the
+README.md.
+You might use `helm-docs` to render the changes you made to `values.yaml`. Then,
+find you changes, copy it, restore the README.md and manually add your changes.
+
+
+### Adding New Values Documentation
+
+When you add new values to `values.yaml`, make sure to:
+1. Add appropriate comments above the value (helm-docs uses these)
+2. Update the schema in `values.schema.json` accordingly
+
+Example format in `values.yaml`:
+```yaml
+# -- Enable the new feature
+# @section -- 6 - Elastic-Agent Configuration
+newFeature:
+  # -- Enable/disable the feature
+  enabled: false
+  # -- Configuration for the feature
+  config: {}
+```
+
+
+## Useful Commands Reference
+
+```bash
+# Install chart in a cluster with values file
+helm install elastic-agent . -f values.yaml -n kube-system
+
+# Install chart with --set flags
+helm install elastic-agent . --set outputs.default.type=ESPlainAuthAPI --set outputs.default.api_key=your_key -n kube-system
+
+# Combine values file and --set (--set takes precedence)
+helm install elastic-agent . -f values.yaml --set agent.version=9.3.0 -n kube-system
+
+# Upgrade existing installation
+helm upgrade elastic-agent . -f values.yaml -n kube-system
+
+# Dry-run to see what would change
+helm upgrade elastic-agent . -f values.yaml -n kube-system --dry-run --debug
+
+# Uninstall
+helm uninstall elastic-agent -n kube-system
+```

deploy/helm/elastic-agent/README.md

@@ -1,8 +1,3 @@
-<!--
-(NOTE: Do not edit README.md directly. It is a generated file!)
-(      To make changes, please modify README.md.gotmpl and run `helm-docs`)
--->
-
 # elastic-agent
 
 ![Version: 9.1.0-beta](https://img.shields.io/badge/Version-9.1.0--beta-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 9.1.0](https://img.shields.io/badge/AppVersion-9.1.0-informational?style=flat-square)
@@ -87,6 +82,7 @@ The chart built-in [kubernetes integration](https://docs.elastic.co/integrations
 | kubernetes.containers.state.enabled | bool | `true` | enable containers state stream (kube-state-metrics) [ref](https://www.elastic.co/guide/en/beats/metricbeat/8.11/metricbeat-metricset-kubernetes-state_container.html) |
 | kubernetes.containers.state.vars | object | `{}` | containers state stream vars |
 | kubernetes.containers.logs.enabled | bool | `true` | enable containers logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/container-logs) |
+| kubernetes.containers.logs.rotated_logs | bool | `false` | BETA: enable rotated logs ingestion. WARNING: enabling it might cause date re-ingestion on existing deployments. [ref](https://www.elastic.co/docs/reference/integrations/kubernetes/container-logs#ingesting-rotated-container-logs) |
 | kubernetes.containers.logs.vars | object | `{}` | containers logs stream vars |
 | kubernetes.containers.audit_logs.enabled | bool | `false` | enable containers audit logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/audit-logs) |
 | kubernetes.pods.metrics.enabled | bool | `true` | enable pods metric stream (kubelet) [ref](https://www.elastic.co/docs/current/integrations/kubernetes/kubelet#pod) |

deploy/helm/elastic-agent/templates/integrations/_kubernetes/_kubernetes_logs_containers.tpl

@@ -17,12 +17,19 @@ Config input for container logs
     namespace: {{ .Values.kubernetes.namespace }}
   use_output: {{ .Values.kubernetes.output }}
   streams:
+  {{- if $.Values.kubernetes.containers.logs.rotated_logs }}
+  - id: kubernetes-container-logs-${kubernetes.pod.uid}-${kubernetes.container.name}
+    gzip_experimental: true
+    paths:
+      - '/var/log/pods/${kubernetes.namespace}_${kubernetes.pod.name}_${kubernetes.pod.uid}/${kubernetes.container.name}/*.log*'
+  {{ else }}
   - id: kubernetes-container-logs-${kubernetes.pod.name}-${kubernetes.container.id}
+    paths:
+      - '/var/log/containers/*${kubernetes.container.id}.log'
+  {{ end }}
     data_stream:
       dataset: kubernetes.container_logs
       type: logs
-    paths:
-      - '/var/log/containers/*${kubernetes.container.id}.log'
     prospector.scanner.symlinks: {{ dig "vars" "symlinks" true .Values.kubernetes.containers.logs }}
     parsers:
       - container:

deploy/helm/elastic-agent/values.schema.json

@@ -229,6 +229,10 @@
                                     "type": "boolean",
                                     "description": "Enable containers logs stream."
                                 },
+                                "rotated_logs": {
+                                    "type": "boolean",
+                                    "description": "BETA: enable rotated logs ingestion. WARNING: enabling it might cause date re-ingestion on existing deployments. [ref](https://www.elastic.co/docs/reference/integrations/kubernetes/container-logs#ingesting-rotated-container-logs)."
+                                },
                                 "vars": {
                                     "type": "object",
                                     "description": "Stream variables.",

deploy/helm/elastic-agent/values.yaml

@@ -151,6 +151,8 @@ kubernetes:
       # -- enable containers logs stream [ref](https://www.elastic.co/docs/current/integrations/kubernetes/container-logs)
       # @section -- 2 - Kubernetes integration
       enabled: true
+      # -- BETA: enable rotated logs ingestion. WARNING: enabling it might cause date re-ingestion on existing deployments. [ref](https://www.elastic.co/docs/reference/integrations/kubernetes/container-logs#ingesting-rotated-container-logs)
+      rotated_logs: false
       # -- containers logs stream vars
       # @section -- 2 - Kubernetes integration
       vars: {}

pkg/testing/runner/runner.go

@@ -277,8 +277,10 @@ func (r *Runner) runK8sInstances(ctx context.Context, instances []StateInstance)
 		// run the actual tests on the host
 		result, runErr := batch.OS.Runner.Run(ctx, r.cfg.VerboseMode, nil, logger, r.cfg.AgentVersion, prefix, batch.Batch, env)
 		if runErr != nil {
-			logger.Logf("Failed to execute tests on instance: %s", err)
-			err = fmt.Errorf("failed to execute tests on instance %s: %w", instance.Name, err)
+			logger.Logf("Failed to execute tests on instance: %s",
+				runErr)
+			err = fmt.Errorf("failed to execute tests on instance %s: %w",
+				instance.Name, runErr)
 		}
 		resultsMx.Lock()
 		results[batch.ID] = result