CHANGELOG-DCA.rst

=============
Release Notes
=============

.. _Release Notes_7.82.0:

7.82.0
======

.. _Release Notes_7.82.0_Prelude:

Prelude
-------

Released on: 2026-08-05
Pinned to datadog-agent v7.82.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7820>`_.


.. _Release Notes_7.82.0_New Features:

New Features
------------

- The Cluster Agent's Prometheus HTTP Service Discovery provider supports
  an optional ``exclude_filter`` field per endpoint entry. The field
  accepts a CEL expression evaluated against each discovered target's
  ``host``, ``port``, and ``labels`` fields. Targets for which the
  expression returns ``true`` are skipped at collection time.


.. _Release Notes_7.82.0_Enhancement Notes:

Enhancement Notes
-----------------

- The Cluster Agent's leader election now uses a dedicated Kubernetes API server
  client with independently managed client-side rate limiting to be more resilient.

- Reduce scale-up stabilization windows for built-in DPA presets:
  Optimize Cost from 300s to 190s, Optimize Balance from 600s to 130s,
  and Optimize Performance from 900s to 70s. This allows faster
  scale-up response for workloads using autoscaling profiles.

- Cluster check stickiness is now enabled by default. The dispatcher biases
  check placement toward the runner where a check previously ran, reducing
  unnecessary check migrations. The behavior can be tuned or disabled via the
  following configuration options:
  
  - ``cluster_checks.stickiness_enabled`` — enable or disable stickiness (default: ``true``)
  - ``cluster_checks.stickiness_factor`` — multiplier applied to check cost when computing the bias (default: ``4.0``)
  - ``cluster_checks.stickiness_upper_limit`` — maximum bias applied regardless of check cost (default: ``1.0``)
  - ``cluster_checks.stickiness_lower_limit`` — minimum bias applied when stickiness is enabled (default: ``0.05``)


.. _Release Notes_7.82.0_Bug Fixes:

Bug Fixes
---------

- Fixed APM Single Step Instrumentation injecting the library twice when the
  admission webhook is reinvoked (for example on GKE Autopilot, where another
  mutating webhook triggers reinvocation). In CSI injection mode the pod has no
  init container, so the re-admission guard failed to detect that the pod was
  already instrumented and appended the injector to ``LD_PRELOAD`` a second time.
  The guard now also checks for the instrumentation volume, which is present in
  every injection mode.

- Fixed an issue where the Cluster Agent could associate a pod's detected
  languages with the wrong Deployment. The Cluster Agent now only attributes a pod's
  detected languages to a Deployment when the pod is owned by a ReplicaSet and
  the ReplicaSet derived from the pod name matches the owner ReplicaSet. Pods
  that are not owned by a ReplicaSet are no longer considered for
  Deployment-level language detection.

- Fix ``cluster_checks.nodes_reporting`` gauge drifting upward across
  leader elections. The metric is now correctly decremented when the
  cluster agent loses leadership and the node store is reset.

- Fix `agent` commands in DCA (listener should always be started)

- Fix permission in docker image when executing "/readsecret.sh" script with dd-agent user

- Fixed an issue in the KSM check where cluster-aggregate metrics
  (``kubernetes_state.container.<cpu|memory>_requested.total``,
  ``kubernetes_state.container.<cpu|memory|gpu|mig>_limit.total``,
  and the ``initcontainer`` equivalents) reported incorrect cluster totals
  when the check ran with ``pod_collection_mode: node_kubelet``. The
  aggregate metrics are now computed from a dedicated instance (running on
  the cluster-agent or a cluster-checks runner) in the new
  ``pod_collection_mode: cluster_aggregates_only`` mode, which watches all
  pods directly from the API server. To enable the fix, set the
  ``cluster_aggregates_enabled: true`` instance option on the
  ``node_kubelet`` and ``cluster_unassigned`` instances; those instances then
  suppress the affected accumulators, eliminating multi-source gauge collision
  at ingestion. Without that option the previous (colliding) behavior is
  unchanged, so it must be set alongside deploying the
  ``cluster_aggregates_only`` instance. The fix preserves the per-pod metric
  scaling benefit of ``node_kubelet`` mode while restoring correct cluster
  aggregates.

- Fix a bug in the orchestrator explorer check that led to trying to collect
  Kubernetes subresources under certain custom resource API groups.


.. _Release Notes_7.81.3:

7.81.3
======

.. _Release Notes_7.81.3_Prelude:

Prelude
-------

Released on: 2026-07-30
Pinned to datadog-agent v7.81.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7813>`_.


.. _Release Notes_7.81.2:

7.81.2
======

.. _Release Notes_7.81.2_Prelude:

Prelude
-------

Released on: 2026-07-22
Pinned to datadog-agent v7.81.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7812>`_.


.. _Release Notes_7.81.1:

7.81.1
======

.. _Release Notes_7.81.1_Prelude:

Prelude
-------

Released on: 2026-07-14
Pinned to datadog-agent v7.81.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7811>`_.


.. _Release Notes_7.81.0:

7.81.0
======

.. _Release Notes_7.81.0_Prelude:

Prelude
-------

Released on: 2026-07-08
Pinned to datadog-agent v7.81.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7810>`_.


.. _Release Notes_7.81.0_New Features:

New Features
------------

- The admission controller can now automatically pick the Datadog CSI
  driver as the library injection mechanism for APM single step
  instrumentation when the ``auto`` injection mode is selected, the
  Datadog CSI driver is installed in the cluster and APM support is
  advertised on its annotations. Otherwise, the admission controller
  falls back to the init container injection mechanism. This
  auto-detection is disabled by default.

- The admission webhook now writes a set of APM Single Step Instrumentation
  (SSI) observability annotations directly on mutated pods, making the full
  injection outcome inspectable via ``kubectl get pod -o yaml`` without
  requiring cluster-level access.
  
  New annotations written by the webhook:
  
  - ``internal.apm.datadoghq.com/injection-status``: overall outcome —
    ``injected``, ``partial``, ``skipped``, or ``error``.
  - ``internal.apm.datadoghq.com/injected-libraries``: JSON array listing
    every component the webhook attempted to inject (injector + per-language
    libraries), each with its name, image, and individual status.
  - ``internal.apm.datadoghq.com/effective-injection-mode``: the injection
    mode actually used (e.g. ``csi``, ``init_container``, ``csi (auto)``),
    set immediately after provider selection so it is present even when
    injection is subsequently skipped.
  - ``internal.apm.datadoghq.com/injection-error``: human-readable reason
    when injection was skipped or errored.
  - ``internal.apm.datadoghq.com/csi-driver-status``: observed state of the
    Datadog CSI driver at injection time — ``apm-enabled``, ``apm-disabled``
    (driver present but APM SSI not advertised), or ``not-installed``. Set
    independently of the configured injection mode.
  
  Per-library failures (unsupported language, library injection error) no
  longer prevent the webhook patch from being applied. The webhook now logs
  a warning and reflects the partial outcome in the annotations instead of
  discarding the entire mutation.

- Add support for ``CPURequestsRemoveLimitsMemoryRequestsAndLimits`` as a container
  ``controlledValues`` in ``DatadogPodAutoscaler`` and ``DatadogPodAutoscalerClusterProfile``.
  When set, CPU requests are controlled and any existing CPU limits are removed,
  allowing containers to burst freely. Memory requests and limits are controlled as usual.

- A ``DatadogInstrumentation`` custom resource can now target a Kubernetes
  ``Service`` to run checks against each of its endpoints.

- Add ``external_metrics_provider.autoscaler_autogen_label_selector`` configuration option to the Cluster Agent.
  When set, only HPAs and WPAs matching the label selector trigger autogeneration of ``DatadogMetric`` objects.
  Autoscalers with explicit ``datadogmetric@`` references are always tracked regardless of the selector.
  This allows filtering out autoscalers managed by other controllers (e.g. KEDA) to avoid creating unwanted ``DatadogMetric`` objects.


.. _Release Notes_7.81.0_Enhancement Notes:

Enhancement Notes
-----------------

- The Cluster Agent now reports its own pod name as ``pod_name`` in its
  inventory metadata payload (``datadog_cluster_agent_metadata``), providing
  a stable per-replica identifier for each Cluster Agent.

- Added ``kubernetes_apiserver_client_qps`` and ``kubernetes_apiserver_client_burst``
  configuration options to control the rate limiter for the Cluster Agent's Kubernetes
  API server client. Default QPS and burst values are increased.

- Add ``datadog-cluster-agent rotate-par-identity`` to rotate the Private
  Action Runner credentials. The new identity is written to the shared
  Kubernetes secret. Run a Kubernetes rollout restart of the Cluster Agent
  deployment to apply the new identity.

- Cluster checks now keep the same check ID across Cluster Agent restarts when
  their configuration is unchanged.


.. _Release Notes_7.81.0_Bug Fixes:

Bug Fixes
---------

- Fixed the Cluster Agent's cluster check rebalancing algorithm to operate
  on configuration digests rather than individual instance IDs. Previously,
  multi-instance configurations could be incorrectly split across different
  runners, causing inaccurate workload estimates and suboptimal rebalancing
  decisions.

- Fix APM auto-injection being blocked when a container has no CPU or memory
  limit and requests below the minimum threshold. The Admission Controller now correctly distinguishes between "no limit
  set" (unlimited resources) and "low limit", preventing the request value from
  being incorrectly used as the effective limit.

- Fixed an issue in the algorithm used to rebalance cluster checks
  that could cause unnecessary check moves between runners.

- Fix nginx AppSec init container image having the controller version tag
  appended even when ``admission_controller.appsec.nginx.init_image`` is
  set to a fully-qualified image reference that already includes a tag.


.. _Release Notes_7.80.4:

7.80.4
======

.. _Release Notes_7.80.4_Prelude:

Prelude
-------

Released on: 2026-07-01
Pinned to datadog-agent v7.80.4: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7804>`_.


.. _Release Notes_7.80.3:

7.80.3
======

.. _Release Notes_7.80.3_Prelude:

Prelude
-------

Released on: 2026-06-24
Pinned to datadog-agent v7.80.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7803>`_.


.. _Release Notes_7.80.2:

7.80.2
======

.. _Release Notes_7.80.2_Prelude:

Prelude
-------

Released on: 2026-06-17
Pinned to datadog-agent v7.80.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7802>`_.


.. _Release Notes_7.80.2_Bug Fixes:

Bug Fixes
---------

- Fixed an issue where the admission controller connectivity probe webhook
  did not include the AKS selector requirements when
  ``admission_controller.add_aks_selectors`` was enabled, which could cause
  repeated webhook reconciliation conflicts on AKS.


.. _Release Notes_7.80.1:

7.80.1
======

.. _Release Notes_7.80.1_Prelude:

Prelude
-------

Released on: 2026-06-12
Pinned to datadog-agent v7.80.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7801>`_.


.. _Release Notes_7.80.0:

7.80.0
======

.. _Release Notes_7.80.0_Prelude:

Prelude
-------

Released on: 2026-06-11
Pinned to datadog-agent v7.80.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7800>`_.


.. _Release Notes_7.80.0_Upgrade Notes:

Upgrade Notes
-------------

- Updated the bundled kube-state-metrics library from v2.13 to v2.18.
  The kube-state-metrics metric allow/deny list now uses ECMAScript
  regular expression syntax instead of Go ``regexp`` syntax. Most
  patterns are compatible, but users relying on Go-specific regex
  features (e.g. ``(?s)`` flag) in ``metric_allowlist`` or
  ``metric_denylist`` should update their patterns.


.. _Release Notes_7.80.0_New Features:

New Features
------------

- The Cluster Agent admission controller now reports connectivity probe
  failures to the Datadog Health Platform. When the admission webhook
  becomes unreachable, an ``admission-controller-connectivity-failure``
  health issue is raised with severity ``high`` and category
  ``availability``, including remediation steps. The issue is
  automatically resolved when connectivity is restored.

- Add a Prometheus HTTP Service Discovery (HTTP SD) provider for the Cluster
  Agent. The provider polls Prometheus-compatible HTTP SD endpoints and 
  generates check configurations for each discovered target. Configure
  endpoints under ``prometheus_http_sd.configs``, each providing its own 
  ``url`` and ``check_template``.

- Autoscaling profiles (``DatadogPodAutoscalerClusterProfile``) now support
  Argo Rollouts as a target workload type. The Cluster Agent automatically
  detects whether the Argo Rollouts CRD is installed at startup and, if
  present, watches Rollout resources for profile labels alongside
  Deployments and StatefulSets.

- The ``kubernetes_state`` core check now collects both ``endpoints`` and
  ``endpointslices`` resources by default, and emits new
  ``kubernetes_state.endpointslice.address_available`` and
  ``kubernetes_state.endpointslice.address_not_ready`` metrics for
  Kubernetes EndpointSlice objects, mirroring the existing
  ``kubernetes_state.endpoint.address_available`` and
  ``kubernetes_state.endpoint.address_not_ready`` metrics.


.. _Release Notes_7.80.0_Enhancement Notes:

Enhancement Notes
-----------------

- The orchestrator check now collects force-deleted pods by default. The
  ``orchestrator_explorer.terminated_pods_improved.enabled`` option will be
  removed in a future release.


.. _Release Notes_7.79.2:

7.79.2
======

.. _Release Notes_7.79.2_Prelude:

Prelude
-------

Released on: 2026-06-03
Pinned to datadog-agent v7.79.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7792>`_.


.. _Release Notes_7.79.2_Bug Fixes:

Bug Fixes
---------

- Cluster Agent: Evaluate AppSec sidecar admission webhook match conditions
  against the deleted object for pod deletion requests.

- Cluster Agent: Prevent disabled AppSec proxy injection cleanup from enabling
  the AppSec sidecar admission webhook.


.. _Release Notes_7.79.1:

7.79.1
======

.. _Release Notes_7.79.1_Prelude:

Prelude
-------

Released on: 2026-05-28
Pinned to datadog-agent v7.79.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7791>`_.


.. _Release Notes_7.79.0:

7.79.0
======

.. _Release Notes_7.79.0_Prelude:

Prelude
-------

Released on: 2026-05-20
Pinned to datadog-agent v7.79.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7790>`_.


.. _Release Notes_7.79.0_New Features:

New Features
------------

- Add AppSec injection support for ingress-nginx controllers. The Cluster Agent
  now automatically injects the Datadog nginx-datadog module into ingress-nginx
  controller pods, enabling AppSec protection without manual ``extraModules``
  configuration. Configurable via ``admission_controller.appsec.nginx.init_image``
  and ``admission_controller.appsec.nginx.module_mount_path``.

- Add spot scheduling. When enabled, the Cluster Agent assigns eligible workload pods to
  spot nodes and maintains configured percentage of spot replicas and minimum on-demand replica count.
  It automatically falls back to on-demand scheduling when spot pods remain pending longer than configured timeout.

- Add namespace-level batch onboarding for workload autoscaling profiles.
  The Cluster Agent now discovers all workloads in namespaces labeled with
  ``autoscaling.datadoghq.com/profile=<profile-name>`` and automatically
  manages ``DatadogPodAutoscaler`` entries for them. Individual workloads
  can opt out by setting ``autoscaling.datadoghq.com/profile=excluded``.


.. _Release Notes_7.79.0_Enhancement Notes:

Enhancement Notes
-----------------

- The ``datadog-cluster-agent clusterchecks`` CLI command now displays
  check execution status for checks running on Cluster Level Check (CLC)
  runners, matching the node agent ``agent status collector`` output format.
  This includes OK/ERROR status, total runs, metric samples, events,
  service checks, average execution time, last execution date, last
  successful execution date, and last error message.

- The cluster agent metadata payload now includes a
  ``clustercheck_integration_status`` field reporting check execution
  status (OK/ERROR) for cluster checks running on CLC runners. This enables
  the backend to populate ``datadog_agent_integration_status`` for cluster
  checks. The ``clustercheck_metadata`` field now also reports all instances
  for multi-instance checks and uses precomputed instance IDs for consistency.

- Add OOTB CRD collection for Gateway API, service mesh (Istio, Envoy Gateway,
  Traefik, Linkerd, Consul, Kuma), and ingress controller (NGINX, Traefik, Kong,
  HAProxy) custom resources. Three new per-family config flags allow operators to
  enable collection independently:
  ``orchestrator_explorer.custom_resources.ootb.gateway_api``,
  ``orchestrator_explorer.custom_resources.ootb.service_mesh``, and
  ``orchestrator_explorer.custom_resources.ootb.ingress_controllers``
  (all default to false).


.. _Release Notes_7.79.0_Security Notes:

Security Notes
--------------

- Upgrade the Docker SDK dependency from ``github.com/docker/docker`` v28.5.2
  to ``github.com/moby/moby`` v29 (``moby/moby/api`` v1.54.1,
  ``moby/moby/client`` v0.4.0) to fix CVE-2026-34040 (High, CVSS 7.8)
  and CVE-2026-33997 (Medium, CVSS 8.1).


.. _Release Notes_7.79.0_Bug Fixes:

Bug Fixes
---------

- Fix a deadlock in the orchestrator check that caused ``Cancel`` to hang
  indefinitely, leaking goroutines and preventing the check from being
  rescheduled. The issue occurred when ``TerminatedResourceBundle.Disable``
  tried to flush manifests through a channel whose consumer goroutine had
  already stopped.

- Honor label and annotation as tags configuration options for all Kubernetes resources.


.. _Release Notes_7.78.4:

7.78.4
======

.. _Release Notes_7.78.4_Prelude:

Prelude
-------

Released on: 2026-05-14
Pinned to datadog-agent v7.78.4: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7784>`_.


.. _Release Notes_7.78.3:

7.78.3
======

.. _Release Notes_7.78.3_Prelude:

Prelude
-------

Released on: 2026-05-07
Pinned to datadog-agent v7.78.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7783>`_.


.. _Release Notes_7.78.2:

7.78.2
======

.. _Release Notes_7.78.2_Prelude:

Prelude
-------

Released on: 2026-04-29
Pinned to datadog-agent v7.78.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7782>`_.


.. _Release Notes_7.78.1:

7.78.1
======

.. _Release Notes_7.78.1_Prelude:

Prelude
-------

Released on: 2026-04-23
Pinned to datadog-agent v7.78.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7781>`_.


.. _Release Notes_7.78.1_Bug Fixes:

Bug Fixes
---------

- Fixed a Cluster Agent issue where container-targeted APM library injection could mount a tracing library into all application containers in a pod instead of only the annotated container.


.. _Release Notes_7.78.0:

7.78.0
======

.. _Release Notes_7.78.0_Prelude:

Prelude
-------

Released on: 2026-04-15
Pinned to datadog-agent v7.78.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7780>`_.


.. _Release Notes_7.78.0_New Features:

New Features
------------

- Added an admission controller connectivity probe that periodically verifies
  the admission webhook is reachable from the Kubernetes API server. When a
  connectivity issue is detected, the probe logs environment-specific guidance
  for EKS, GKE, and AKS. Probe results are visible in the ``agent status``
  output under the Admission Controller section. The probe is disabled by
  default and can be enabled by setting ``admission_controller.probe.enabled``
  to ``true``. The probe uses dry-run ConfigMap creation requests in the
  cluster agent's namespace.

- Add Remote Configuration status section to ``datadog-cluster-agent status`` output and flares.
  This displays whether RC is enabled for the organization, whether the API key is authorized
  for Remote Configuration, and any last errors, matching the node agent's existing behavior.


.. _Release Notes_7.78.0_Enhancement Notes:

Enhancement Notes
-----------------

- Configurable support for TLS communication between the sidecar Agent 
  and the Cluster Agent via the agent-sidecar mutation webhook. Requires 
  elevated permissions for Cluster Agent to copy the certificate authority 
  to the target namespace as a secret.

- Single Step Instrumentation volumes are now mounted as read-only to prevent accidental writes to SSI artifacts.



.. _Release Notes_7.77.3:

7.77.3
======

.. _Release Notes_7.77.3_Prelude:

Prelude
-------

Released on: 2026-04-08
Pinned to datadog-agent v7.77.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7773>`_.


.. _Release Notes_7.77.2:

7.77.2
======

.. _Release Notes_7.77.2_Prelude:

Prelude
-------

Released on: 2026-04-01
Pinned to datadog-agent v7.77.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7772>`_.


.. _Release Notes_7.77.1:

7.77.1
======

.. _Release Notes_7.77.1_Prelude:

Prelude
-------

Released on: 2026-03-24
Pinned to datadog-agent v7.77.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7771>`_.


.. _Release Notes_7.77.0:

7.77.0
======

.. _Release Notes_7.77.0_Prelude:

Prelude
-------

Released on: 2026-03-18
Pinned to datadog-agent v7.77.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7770>`_.


.. _Release Notes_7.77.0_New Features:

New Features
------------

- Add APM tracing instrumentation to the Datadog Cluster Agent for improved observability
  and debugging in production environments. When enabled, the Cluster Agent emits APM traces
  for cluster check dispatching and rebalancing operations, surfacing patch failures and
  rebalancing decisions as span tags. 


.. _Release Notes_7.77.0_Enhancement Notes:

Enhancement Notes
-----------------

- Reduce admission controller downtime during certificate rotation.

- Add the ability to collect NodeClasses EKS Auto Mode custom resources (``eks.amazonaws.com`` API group) by default.

- Experimental: Adds support for collecting force-deleted pods in the orchestrator check using
  ``orchestrator_explorer.terminated_pods_improved.enabled``.


.. _Release Notes_7.76.3:

7.76.3
======

.. _Release Notes_7.76.3_Prelude:

Prelude
-------

Released on: 2026-03-09
Pinned to datadog-agent v7.76.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7763>`_.


.. _Release Notes_7.76.2:

7.76.2
======

.. _Release Notes_7.76.2_Prelude:

Prelude
-------

Released on: 2026-03-05
Pinned to datadog-agent v7.76.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7762>`_.


.. _Release Notes_7.76.1:

7.76.1
======

.. _Release Notes_7.76.1_Prelude:

Prelude
-------

Released on: 2026-02-26
Pinned to datadog-agent v7.76.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7761>`_.


.. _Release Notes_7.76.0:

7.76.0
======

.. _Release Notes_7.76.0_Prelude:

Prelude
-------

Released on: 2026-02-23
Pinned to datadog-agent v7.76.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7760>`_.


.. _Release Notes_7.76.0_New Features:

New Features
------------

- APM: Add ``apm_config.instrumentation.injection_mode`` configuration option to control APM library injection method.
  Possible values are ``auto`` (default), ``init_container``, and ``csi``.
  The ``auto`` mode automatically selects the best injection mode (currently uses init containers).
  The ``init_container`` mode is the legacy method that copies APM libraries into pods using init containers.
  The ``csi`` mode mounts APM libraries directly into pods using the Datadog CSI driver. It is experimental and requires Cluster Agent 7.76+ and the Datadog CSI driver.

- APM: Add CSI-based library injection as an alternative to init containers (experimental).
  This provides faster pod startup and reduced storage overhead.

- Reduced memory usage of compliance checks on large clusters


.. _Release Notes_7.76.0_Enhancement Notes:

Enhancement Notes
-----------------

- Reduced memory usage when pod collection is enabled in the Cluster Agent.


.. _Release Notes_7.76.0_Bug Fixes:

Bug Fixes
---------

- When injection fails for Single Step Instrumentation due to constrained resources, we add an annotation to the pod
  with a reason for the error. This annotation now matches all other annotations the webhook writes to a pod spec
  by prefixing the annotation with ``internal``. The full annotation is now: ``internal.apm.datadoghq.com/injection-error``


.. _Release Notes_7.76.0_Other Notes:

Other Notes
-----------

- Refactor the auto-instrumentation webhook's ``injectTracers`` function to use a modular, explicit mutation pattern.
  This improves code readability and maintainability. Edge case behavior may differ slightly, but overall functionality remains unchanged.


.. _Release Notes_7.75.4:

7.75.4
======

.. _Release Notes_7.75.4_Prelude:

Prelude
-------

Released on: 2026-02-17
Pinned to datadog-agent v7.75.4: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7754>`_.

.. _Release Notes_7.75.3:

7.75.3
======

.. _Release Notes_7.75.3_Prelude:

Prelude
-------

Released on: 2026-02-11
Pinned to datadog-agent v7.75.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7753>`_.


.. _Release Notes_7.75.2:

7.75.2
======

.. _Release Notes_7.75.2_Prelude:

Prelude
-------

Released on: 2026-02-04
Pinned to datadog-agent v7.75.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7752>`_.


.. _Release Notes_7.75.1:

7.75.1
======

.. _Release Notes_7.75.1_Prelude:

Prelude
-------

Released on: 2026-01-28
Pinned to datadog-agent v7.75.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7751>`_.

.. _Release Notes_7.75.0:

7.75.0
======

.. _Release Notes_7.75.0_Prelude:

Prelude
-------

Released on: 2026-01-21
Pinned to datadog-agent v7.75.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7750>`_.

.. _Release Notes_7.75.0_Upgrade Notes:

Upgrade Notes
-------------

- The Datadog Cluster Agent's mutating webhooks (part of the `Admission Controller <https://docs.datadoghq.com/containers/cluster_agent/admission_controller/?tab=datadogoperator>`_) previously included Single Step Instrumentation (SSI) settings in their default webhook label selectors. These SSI-specific settings, ``apm_config.instrumentation.enabled`` and ``apm_config.instrumentation.enabled_namespaces``, have been removed.

  For those using Single Step Instrumentation, no action is required and no behavior changes. For those using the ``config`` or ``tagsfromlabels`` webhooks for manually instrumented applications, behavior remains consistent with the `documented configuration <https://docs.datadoghq.com/containers/cluster_agent/admission_controller/?tab=datadogoperator#apm-and-dogstatsd>`_. Users that were unintentionally relying on the SSI settings without using SSI should add the appropriate pod label or enable ``mutate_unlabelled`` to preserve the previous behavior.


.. _Release Notes_7.75.0_Enhancement Notes:

Enhancement Notes
-----------------

- Single Step Instrumentation now uses the Python tracer major version 4 by default. Customers instrumenting Python
  applications through SSI should review the `4.0.0 <https://github.com/DataDog/dd-trace-py/releases/tag/v4.0.0>`_
  release notes and the [compatibility guide](https://docs.datadoghq.com/tracing/trace_collection/compatibility/python/)
  to ensure their Python applications are compatible.


.. _Release Notes_7.75.0_Bug Fixes:

Bug Fixes
---------

- We collect namespaces in Kubernetes for Single Step Instrumentation. We need this information to utilize namespace
  labels for workload selection. However, we also use this information to generate pod security polices for restricted
  namespaces. This change fixes an issue where we would only collect namespace information when target based workload
  selection was utilized instead of collecting namespaces for all Single Step Instrumentation configurations.

- For Single Step Instrumentation, pods that include a label disabling instrumentation no longer receive mutations from the instrumentation webhook, even if they are in an enabled namespace.

- When using Single Step Instrumentation with configuration-based targeting (enabled namespaces or targets) together with Local Lib Injection, the webhook no longer mutates pods in namespaces where instrumentation is disabled.


.. _Release Notes_7.74.1:

7.74.1
======

.. _Release Notes_7.74.1_Prelude:

Prelude
-------

Released on: 2026-01-12
Pinned to datadog-agent v7.74.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7741>`_.

.. _Release Notes_7.74.0:

7.74.0
======

.. _Release Notes_7.74.0_Prelude:

Prelude
-------

Released on: 2026-01-07
Pinned to datadog-agent v7.74.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7740>`_.

.. _Release Notes_7.74.0_New Features:

New Features
------------

- Add KSM Resource Type Sharding for improved performance in large Kubernetes clusters.
  This feature automatically splits the ``kubernetes_state_core`` check into multiple
  shards based on resource type groups (pods, nodes, others), enabling parallel execution
  across multiple Cluster Check Runners.


.. _Release Notes_7.74.0_Enhancement Notes:

Enhancement Notes
-----------------

- In the Helm check, the "helm_status" tag is now always set to "uninstalled"
  in delete events.


.. _Release Notes_7.74.0_Bug Fixes:

Bug Fixes
---------

- Fixed a deadlock in the Cluster Agent language detection handler that could cause
  event drops with the error "collector language-detection-follower dropped event(s)
  after 10s timeout". The fix releases the mutex before pushing events to workloadmeta
  to prevent blocking while holding the lock.


.. _Release Notes_7.73.3:

7.73.3
======

.. _Release Notes_7.73.3_Prelude:

Prelude
-------

Released on: 2025-12-31
Pinned to datadog-agent v7.73.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7733>`_.

.. _Release Notes_7.73.2:

7.73.2
======

.. _Release Notes_7.73.2_Prelude:

Prelude
-------

Released on: 2025-12-23
Pinned to datadog-agent v7.73.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7732>`_.

.. _Release Notes_7.73.1:

7.73.1
======

.. _Release Notes_7.73.1_Prelude:

Prelude
-------

Released on: 2025-12-17
Pinned to datadog-agent v7.73.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7731>`_.

.. _Release Notes_7.73.0:

7.73.0
======

.. _Release Notes_7.73.0_Prelude:

Prelude
-------

Released on: 2025-12-10
Pinned to datadog-agent v7.73.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7730>`_.

.. _Release Notes_7.73.0_Upgrade Notes:

Upgrade Notes
-------------

- This change removes support for v1 of the auto-instrumentation webhook used for Single Step Instrumentation. The v2
  implementation, which has been the default since Agent v7.57.0, is a drop-in replacement. This setting was never exposed
  in Helm or the Datadog Operator. If you previously set the DD_APM_INSTRUMENTATION_VERSION environment variable on
  the Cluster Agent, it is now ignored.

  If you use a private registry, add the ``apm-inject`` container to your registry before upgrading. No action is
  required for other users. For details on using private registries, see [Use a private container registry](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/single-step-apm/kubernetes/?tab=agentv764recommended#use-a-private-container-registry).


.. _Release Notes_7.73.0_New Features:

New Features
------------

- Customers using Single Step Instrumentation with target-based workload selection can now use language detection.
  Language detection greatly reduces startup time when all default libraries are configured for a target.

  A target is eligible for language detection if a target has no defined ``ddTraceVersions`` or if ``ddTraceVersions``
  matches the default set of SDKs. Once a language has been determined for a deployment, subsequent deploys
  only use the SDKs necessary for the detected language.


.. _Release Notes_7.73.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added namespace selectors excluding system namespace (``kube-system`` and the Datadog Agent's namespace)
  resources from Admission Controller mutation webhooks. This prevents mutation webhooks from unnecessarily
  intercepting system namespace resources, reducing misleading warnings or logs, and improving clarity
  about which resources are actually mutated.


.. _Release Notes_7.73.0_Bug Fixes:

Bug Fixes
---------

- The Cluster Agent Admission Controller now logs a warning instead of failing the webhook when the Admission Controller lacks permissions to access a pod’s owner.

- Fix default value of automountServiceAccountToken on ServiceAccounts when not set.

- There were several bugs for customers using Single Step Instrumentation with target-based workload selection, and
  also using local SDK injection. This change resolves an issue where if targets were defined, the Cluster Agent
  didn't respect the ``admission.datadoghq.com/enabled`` annotation or the ``admission_controller.mutate_unlabelled``
  configuration option, and only respected the language annotations.


.. _Release Notes_7.72.4:

7.72.4
======

.. _Release Notes_7.72.4_Prelude:

Prelude
-------

Released on: 2025-12-01
Pinned to datadog-agent v7.72.4: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7724>`_.


.. _Release Notes_7.72.4_Enhancement Notes:

Enhancement Notes
-----------------

- Update Cluster Agent to use latest schema for DatadogPodAutoscaler CRD.


.. _Release Notes_7.72.3:

7.72.3
======

.. _Release Notes_7.72.3_Prelude:

Prelude
-------

Released on: 2025-11-26
Pinned to datadog-agent v7.72.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7723>`_.


.. _Release Notes_7.72.3_Bug Fixes:

Bug Fixes
---------

- Fix a memory leak in the orchestrator check when the Cluster Agent is deployed with multiple replicas and leader-election enabled.


.. _Release Notes_7.72.2:

7.72.2
======

.. _Release Notes_7.72.2_Prelude:

Prelude
-------

Released on: 2025-11-12
Pinned to datadog-agent v7.72.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7722>`_.

.. _Release Notes_7.72.1:

7.72.1
======

.. _Release Notes_7.72.1_Prelude:

Prelude
-------

Released on: 2025-11-06
Pinned to datadog-agent v7.72.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7721>`_.

.. _Release Notes_7.72.0:

7.72.0
======

.. _Release Notes_7.72.0_Prelude:

Prelude
-------

Released on: 2025-11-03
Pinned to datadog-agent v7.72.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7720>`_.

.. _Release Notes_7.72.0_Bug Fixes:

Bug Fixes
---------

- Fixes an Admission Controller issue where the UDS socket host path was hardcoded, causing mutated pods to miss APM or DogStatsD sockets when custom hostSocketPath values were set via Helm or the Operator.

- Cluster Agent now scrubs secrets when generating flares and cluster check reports.


.. _Release Notes_7.71.2:

7.71.2
======

.. _Release Notes_7.71.2_Prelude:

Prelude
-------

Released on: 2025-10-15
Pinned to datadog-agent v7.71.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7712>`_.

.. _Release Notes_7.71.1:

7.71.1
======

.. _Release Notes_7.71.1_Prelude:

Prelude
-------

Released on: 2025-10-08
Pinned to datadog-agent v7.71.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7711>`_.

.. _Release Notes_7.71.0:

7.71.0
======

.. _Release Notes_7.71.0_Prelude:

Prelude
-------

Released on: 2025-10-01
Pinned to datadog-agent v7.71.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7710>`_.

.. _Release Notes_7.71.0_New Features:

New Features
------------

- Added support for gradual rollout in K8s SSI deployments.
  This feature is enabled if the following conditions are met:
  - ``DD_ADMISSION_CONTROLLER_AUTO_INSTRUMENTATION_ENABLED=true`` and/or ``DD_APM_INSTRUMENTATION_ENABLED=true``
  - Remote config is enabled in the organization
  - Using a Datadog registry (gcr.io/datadoghq, hub.docker.com/r/datadog, gallery.ecr.aws/datadog)
  This gradually rolls out access to new injector and tracer library releases to subsets of customers, with the ability to pause if issues are detected.


.. _Release Notes_7.71.0_Enhancement Notes:

Enhancement Notes
-----------------

- Collect Datadog custom resources in the orchestrator check.

- Add the ability to collect Argo Rollouts and Karpenter custom resources by default.


.. _Release Notes_7.70.2:

7.70.2
======

.. _Release Notes_7.70.2_Prelude:

Prelude
-------

Released on: 2025-09-15
Pinned to datadog-agent v7.70.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7702>`_.

.. _Release Notes_7.70.1:

7.70.1
======

.. _Release Notes_7.70.1_Prelude:

Prelude
-------

Released on: 2025-09-10
Pinned to datadog-agent v7.70.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7701>`_.

.. _Release Notes_7.70.0:

7.70.0
======

.. _Release Notes_7.70.0_Prelude:

Prelude
-------

Released on: 2025-09-03
Pinned to datadog-agent v7.70.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7700>`_.

.. _Release Notes_7.70.0_Enhancement Notes:

Enhancement Notes
-----------------

- Created a new configuration option `orchestrator_explorer.custom_resources.max_count` that will allow customers to increase the number of custom resources they can collect in the Cluster Agent.

- Time zones are now collected for CronJobs in the `orchestrator` check

- Collect EndpointSlices by default.


.. _Release Notes_7.70.0_Bug Fixes:

Bug Fixes
---------

- Correctly set the VerticalPodAutoscaler mode to "Auto" when not provided

- Fixed an issue that caused some Datadog Cluster Agent replicas to restart
  during the first install.

- When using leases for leader election with the
  ``leader_election_default_resource`` option, the Datadog Cluster Agent no
  longer creates an unused config map.

- Fix the ``cronjob.on_schedule_check`` service check of the ``kubernetes_state`` check for CronJobs that have a specific time zone.


.. _Release Notes_7.69.4:

7.69.4
======

.. _Release Notes_7.69.4_Prelude:

Prelude
-------

Released on: 2025-08-27
Pinned to datadog-agent v7.69.4: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7694>`_.

.. _Release Notes_7.69.3:

7.69.3
======

.. _Release Notes_7.69.3_Prelude:

Prelude
-------

Released on: 2025-08-26
Pinned to datadog-agent v7.69.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7693>`_.

.. _Release Notes_7.69.2:

7.69.2
======

.. _Release Notes_7.69.2_Prelude:

Prelude
-------

Released on: 2025-08-20
Pinned to datadog-agent v7.69.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7692>`_.

.. _Release Notes_7.69.1:

7.69.1
======

.. _Release Notes_7.69.1_Prelude:

Prelude
-------

Released on: 2025-08-18
Pinned to datadog-agent v7.69.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7691>`_.

.. _Release Notes_7.69.0:

7.69.0
======

.. _Release Notes_7.69.0_Prelude:

Prelude
-------

Released on: 2025-08-14
Pinned to datadog-agent v7.69.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7690>`_.

.. _Release Notes_7.69.0_Enhancement Notes:

Enhancement Notes
-----------------

- The auto-instrumentation webhook supports labels and annotations as tags configuration.
  If any of the label or annotation mappings for the incoming pod correspond to Universal
  Service Tags (``service``, ``env``, or ``version``), the webhook will also add the corresponding
  UST environment variable to the pod (``DD_SERVICE``, ``DD_ENV``, or ``DD_VERSION``).

- The autoinstrumentation webhook will now set a default security context for init containers
  if the pod is in a namespace with a restricted security context.  This can still be overridden by setting
  the environment variable ``DD_ADMISSION_CONTROLLER_AUTO_INSTRUMENTATION_INIT_SECURITY_CONTEXT``.

- Collect agent version in orchestrator check.


.. _Release Notes_7.68.3:

7.68.3
======

.. _Release Notes_7.68.3_Prelude:

Prelude
-------

Released on: 2025-07-28
Pinned to datadog-agent v7.68.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7683>`_.

.. _Release Notes_7.68.2:

7.68.2
======

.. _Release Notes_7.68.2_Prelude:

Prelude
-------

Released on: 2025-07-21
Pinned to datadog-agent v7.68.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7682>`_.

.. _Release Notes_7.68.1:

7.68.1
======

.. _Release Notes_7.68.1_Prelude:

Prelude
-------

Released on: 2025-07-17
Pinned to datadog-agent v7.68.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7681>`_.

.. _Release Notes_7.68.0:

7.68.0
======

.. _Release Notes_7.68.0_Prelude:

Prelude
-------

Released on: 2025-07-10
Pinned to datadog-agent v7.68.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7680>`_.

.. _Release Notes_7.68.0_New Features:

New Features
------------

- The admission controller can now enable kubelet API logging in the
  injected agent sidecar.


.. _Release Notes_7.68.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added a new metric to expose the ksm ``kube_cronjob_status_last_successful_time`` metric. The name of the metric is ``kubernetes_state.cronjob.duration_since_last_successful``.

- Single Step Instrumentation now uses the Python tracer major version 3 by default.


.. _Release Notes_7.68.0_Bug Fixes:

Bug Fixes
---------

- Stop sending telemetry associated with a DatadogMetric when the object is deleted.

- Fix a bug in the Kubernetes State Metrics (KSM) check where custom resource
  metrics were incorrectly named using the `kubernetes_state.customresource.<name>`
  pattern instead of the intended `kubernetes_state_customresource.<prefix>_<name>` format.

- Fixes a bug in the admission controller webhook that caused volume mounts to be skipped when other webhooks injected init containers after our own volume mounts had been added.

- Properly take into account the ``timeZone`` field of the ``CronJob`` objects in the ``kubernetes_state.cronjob.on_schedule_check`` service check.


.. _Release Notes_7.67.1:

7.67.1
======

.. _Release Notes_7.67.1_Prelude:

Prelude
-------

Released on: 2025-07-02
Pinned to datadog-agent v7.67.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7671>`_.

.. _Release Notes_7.67.0:

7.67.0
======

.. _Release Notes_7.67.0_Prelude:

Prelude
-------

Released on: 2025-06-18
Pinned to datadog-agent v7.67.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7670>`_.


.. _Release Notes_7.67.0_Deprecation Notes:

Deprecation Notes
-----------------

- The Kubernetes State check no longer supports VPA versions <0.7.0.


.. _Release Notes_7.67.0_Bug Fixes:

Bug Fixes
---------

- Fixes a bug where the Kubernetes State check could not generate VPA metrics for VPA versions 1.3.0+

- Fix data race in the orchestrator check for the sensitive data scrubber. Note: This
  would only occur when running the check on Datadog cluster check runners.


.. _Release Notes_7.66.1:

7.66.1
======

.. _Release Notes_7.66.1_Prelude:

Prelude
-------

Released on: 2025-06-03
Pinned to datadog-agent v7.66.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7661>`_.

.. _Release Notes_7.66.0:

7.66.0
======

.. _Release Notes_7.66.0_Prelude:

Prelude
-------

Released on: 2025-05-22
Pinned to datadog-agent v7.66.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7660>`_.

.. _Release Notes_7.66.0_New Features:

New Features
------------

- For workload selection in auto-instrumentation, users can now use the Kubernetes native ``valueFrom``
  as an alternative to ``value`` in ``ddTraceConfigs``. This enables dynamic, user-defined and label
  based value propagation to the tracing SDKs, like ``DD_SERVICE``.

- Collect `EndpointSlice` manifests in the orchestrator check.

- Tag resources from Cluster Agent Orchestrator check with all static tags.

.. _Release Notes_7.66.0_Bug Fixes:

Bug Fixes
---------

- Fix major data races in the orchestrator check for the Kubernetes resource collection.

- Fix data race in autodiscovery cluster checks provider.

- Fix data race in autodiscovery kube services provider.

- Fixes an issue with autoinstrumentation where sometimes a ``DD_SERVICE`` is
  not consistent between containers and init containers.

- The auto-instrumentation webhook no longer mutates the ``istio-proxy`` container.
  This fixes an issue with Kubernetes-native sidecars and the istio service mesh
  where a standard sidecar is moved to be the first init container by istio after it
  was mutated by auto-instrumentation.

- The cluster-agent kubernetes_metadata API now supports client specified annotations filtering.
  Clients can pass along filters as query parameters like '?filter=abc&filter=def'.


.. _Release Notes_7.65.2:

7.65.2
======

.. _Release Notes_7.65.2_Prelude:

Prelude
-------

Released on: 2025-05-13
Pinned to datadog-agent v7.65.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7652>`_.

.. _Release Notes_7.65.2_Bug Fixes:

Bug Fixes
---------

- Fix wrong computation of the init container resources in the autoinstrumentation webhook.


.. _Release Notes_7.65.1:

7.65.1
======

.. _Release Notes_7.65.1_Prelude:

Prelude
-------

Released on: 2025-05-08
Pinned to datadog-agent v7.65.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7651>`_.

.. _Release Notes_7.65.1_Bug Fixes:

Bug Fixes
---------

- Customers relying on the deprecated v1 implementation of the auto instrumentation webhook will no longer be forced
  to use the v2 implementation. This will provide additional time for customers to migrate from the v1 to the v2 implementation
  and ensure the v2 implementation adequately supports all existing use cases.


.. _Release Notes_7.65.0:

7.65.0
======

.. _Release Notes_7.65.0_Prelude:

Prelude
-------

Released on: 2025-05-06
Pinned to datadog-agent v7.65.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7650>`_.

.. _Release Notes_7.65.0_New Features:

New Features
------------

- [PREVIEW] Add support for mounting Datadog CSI volumes instead of hostpath
  volumes in the admission controller config webhook for sharing DogStatsD
  and APM UDS sockets with user applications. This requires the Datadog
  CSI driver to be installed and running on the cluster.


.. _Release Notes_7.65.0_Enhancement Notes:

Enhancement Notes
-----------------

- Collect terminated Kubernetes resources.


.. _Release Notes_7.64.3:

7.64.3
======

.. _Release Notes_7.64.3_Prelude:

Prelude
-------

Released on: 2025-04-10
Pinned to datadog-agent v7.64.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7643>`_.

.. _Release Notes_7.64.2:

7.64.2
======

.. _Release Notes_7.64.2_Prelude:

Prelude
-------

Released on: 2025-04-02
Pinned to datadog-agent v7.64.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7642>`_.

.. _Release Notes_7.64.1:

7.64.1
======

.. _Release Notes_7.64.1_Prelude:

Prelude
-------

Released on: 2025-03-20
Pinned to datadog-agent v7.64.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7641>`_.

.. _Release Notes_7.64.0:

7.64.0
======

.. _Release Notes_7.64.0_Prelude:

Prelude
-------

Released on: 2025-03-19
Pinned to datadog-agent v7.64.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7640>`_.

.. _Release Notes_7.64.0_Upgrade Notes:

Upgrade Notes
-------------

- Datadog Autoscaling is upgraded to use DatadogPodAutoscaler CRD v1alpha2 instead of v1alpha1. Remote (created in Datadog) autoscalers are automatically migrated. In-cluster (Local) autoscalers need to be migrated manually.


.. _Release Notes_7.64.0_New Features:

New Features
------------

- Enable collection of Pod Disruption Budgets by default in the orchestrator check.

- Target-based workload selection is now available for Single Step Instrumentation. This feature enables you to
  instrument specific workloads using pod and namespace label selectors. By applying user-defined labels, you can
  select workloads for instrumentation without modifying applications. For example, the following configuration
  injects the Python tracer with a default version for pods labeled with `language=python`:
  ```yaml
  instrumentation:
    enabled: true
    targets:
      - name: "Python Services"
        podSelector:
          matchLabels:
            language: "python"
        ddTraceVersions:
          python: "default"
  ```

  Targets can also be chained together, with the first matching rule taking precedence. For example, the following
  configuration installs the Python tracer for pods labeled `language=python` and the Java tracer for pods in a
  namespace labeled `language=java`. If a pod matches both rules, the first match takes precedence:
  ```
  instrumentation:
    enabled: true
    targets:
      - name: "Python Services"
        podSelector:
          matchLabels:
            language: "python"
        ddTraceVersions:
          python: "default"
      - name: "Java Namespaces"
        namespaceSelector:
          matchLabels:
            language: "java"
        ddTraceVersions:
          python: "default"
  ```

  Targets support tracer configuration options in the form of environment variables. All options must have the
  `DD_` prefix. The following example installs the Python tracer with profiling and data jobs enabled:
  ```
  instrumentation:
    enabled: true
    targets:
      - name: "Python Apps"
        podSelector:
          matchLabels:
            language: "python"
        ddTraceVersions:
          python: "v2"
        ddTraceConfigs:
          - name: "DD_PROFILING_ENABLED"
            value: "true"
          - name: "DD_DATA_JOBS_ENABLED"
            value: "true"
  ```


.. _Release Notes_7.64.0_Enhancement Notes:

Enhancement Notes
-----------------

- Enrich ``kubernetes_state`` metrics with resource labels or annotations
  as tags.

- The Datadog Cluster Agent admission controller agent sidecar injection now sets up
  Agent sidecars to run with securityContext of `readOnlyRootFilesystem:false` by default.
  Advanced users can customize the securityContext through clusterAgent.admissionController.agentSidecarInjection.profiles.

- When there are no pinned library versions in the autoinstrumentation webhook,
  use detected languages to omit unnecessary libraries.

- Error messages displayed in the DatadogMetric `Error` condition has been improved to reflect more accurately the source of the error.


.. _Release Notes_7.64.0_Deprecation Notes:

Deprecation Notes
-----------------

- ``DD_APM_INSTRUMENTATION_VERSION=v1`` has been deprecated and will default to ``v2``.


.. _Release Notes_7.64.0_Bug Fixes:

Bug Fixes
---------

- Include `gpu_vendor` pod tags on the Datadog Cluster Agent when
  enabling datadog.clusterTagger.collectKubernetesTags.

- When the Datadog Cluster Agent injects the Datadog Agent as a sidecar
  on a Job, the agent will now exit when the main Job completes.


.. _Release Notes_7.63.3:

7.63.3
======

.. _Release Notes_7.63.3_Prelude:

Prelude
-------

Released on: 2025-03-04
Pinned to datadog-agent v7.63.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7633>`_.

.. _Release Notes_7.63.2:

7.63.2
======

.. _Release Notes_7.63.2_Prelude:

Prelude
-------

Released on: 2025-02-28
Pinned to datadog-agent v7.63.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7632>`_.

.. _Release Notes_7.63.1:

7.63.1
======

.. _Release Notes_7.63.1_Prelude:

Prelude
-------

Released on: 2025-02-26
Pinned to datadog-agent v7.63.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7631>`_.

.. _Release Notes_7.63.0:

7.63.0
======

.. _Release Notes_7.63.0_Prelude:

Prelude
-------

Released on: 2025-02-19
Pinned to datadog-agent v7.63.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7630>`_.

.. _Release Notes_7.63.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added support for `kubernetesResourcesLabelsAsTags` and `kubernetesResourcesAnnotationsAsTags` in the
  orchestrator check. Kubernetes resources processed by the orchestrator check can now include labels
  and annotations as tags, improving consistency with existing tagging configurations.

- The Cluster Agent is now able to delete `ValidatingAdmissionWebhook` and `MutatingAdmissionWebhook`
  depending on the `admission_controller.validation.enabled` and `admission_controller.mutation.enabled` settings.
  Note that `admission_controller.enabled` must be set to `true` to allow the Cluster Agent to
  interact with the Kubernetes Admission Controller.


.. _Release Notes_7.63.0_Bug Fixes:

Bug Fixes
---------

- Fixes an issue with the
  ``datadog.cluster_agent.cluster_checks.configs_dispatched`` metric emitted
  by the Cluster Agent telemetry. The metric values could become inaccurate
  after the Cluster Agent loses and then regains leader status.


.. _Release Notes_7.62.3:

7.62.3
======

.. _Release Notes_7.62.3_Prelude:

Prelude
-------

Released on: 2025-02-14
Pinned to datadog-agent v7.62.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7623>`_.

.. _Release Notes_7.62.2:

7.62.2
======

.. _Release Notes_7.62.2_Prelude:

Prelude
-------

Released on: 2025-02-10
Pinned to datadog-agent v7.62.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7622>`_.


.. _Release Notes_7.62.2_Bug Fixes:

Bug Fixes
---------

- Fix issue where annotations as tags were not showing up properly
  when certain resource collectors were enabled.


.. _Release Notes_7.62.1:

7.62.1
======

.. _Release Notes_7.62.1_Prelude:

Prelude
-------

Released on: 2025-02-03
Pinned to datadog-agent v7.62.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7621>`_.

.. _Release Notes_7.62.0:

7.62.0
======

.. _Release Notes_7.62.0_Prelude:

Prelude
-------

Released on: 2025-01-29
Pinned to datadog-agent v7.62.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7620>`_.

.. _Release Notes_7.62.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added support for EndpointSlices in the metadata controller. This feature,
  enabled by setting `kubernetesUseEndpointSlices` to `true`, resolves limitations
  of Endpoints resources, particularly for services with more than 1000 addresses.
  Must add `endpointslices` to DCA cluster role.

- The `kubernetes_state_core` check now collects only metadata for configmaps,
  reducing memory, CPU, and network usage in the Cluster Agent while preserving
  full metric functionality.

- The Datadog Cluster Agent admission controller agent sidecar injection now sets up
  Agent sidecars to run with securityContext of `readOnlyRootFilesystem:false` by default.
  Advanced users can customize the securityContext via clusterAgent.admissionController.agentSidecarInjection.profiles.


.. _Release Notes_7.62.0_Bug Fixes:

Bug Fixes
---------

- Resolved a bug where `kube_service` tags were not properly deleted when a
  service's ties to pods on a specific node were removed but the service itself
  was not deleted.

- Add mapping for apiservices and customresourcedefinitions to KSM check to
  prevent errors on startup with discovering resources.

- Include `gpu_vendor` pod tags on the Datadog Cluster Agent when
  enabling datadog.clusterTagger.collectKubernetesTags.

- When the Datadog Cluster Agent injects the Datadog Agent as a sidecar
  on a Job, the agent will now exit when the main Job completes.


.. _Release Notes_7.61.0:

7.61.0
======

.. _Release Notes_7.61.0_Prelude:

Prelude
-------

Released on: 2025-01-13
Pinned to datadog-agent v7.61.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7610>`_.

.. _Release Notes_7.61.0_New Features:

New Features
------------

- Implements the Kubernetes Admission Events webhooks. This new webhooks will emit Datadog Events
  when receving Validation Admission requests. It will track deployments operations made by non-system
  users.
  The webhook is controlled by using the `admission_controller.kubernetes_admission_events.enabled` setting.

- The cluster-agent now can collect pod disruption budgets from the cluster.

- Add the capability in ``kubernetes_state_core`` check to collect metrics for Kubernetes custom resources

.. _Release Notes_7.61.0_Enhancement Notes:

Enhancement Notes
-----------------

- Cluster Agent: ``DatadogAgent`` custom resource, cluster Agent deployment, and node Agent daemonset manifests are now added to the flare archive when the Cluster Agent is deployed with the Datadog Operator (version 1.11.0+).

- Cluster Agent: Don't overwrite the LD_PRELOAD environment variable if it's already set, append the path to Datadog's injection library instead.

- Users can now enable a pod with SSI using a default language library version
  and no longer need to pin to a specific version.

.. _Release Notes_7.61.0_Bug Fixes:

Bug Fixes
---------

- The auto-instrumentation webhook no longer injects the default environment
  variables when disabled.


.. _Release Notes_7.60.1:

7.60.1
======

.. _Release Notes_7.60.1_Prelude:

Prelude
-------

Released on: 2024-12-19
Pinned to datadog-agent v7.60.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7601>`_.

.. _Release Notes_7.60.0:

7.60.0
======

.. _Release Notes_7.60.0_Prelude:

Prelude
-------

Released on: 2024-12-16
Pinned to datadog-agent v7.60.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7600>`_.


.. _Release Notes_7.60.0_Bug Fixes:

Bug Fixes
---------

- Fixes bug where incorrect timestamp would be used for unbundled Kubernetes events.

- Fixed an issue in the KSM check when it's configured with the option
  ``pod_collection_mode`` set to ``node_kubelet``. Previously, the check could
  fail to start if there was a timeout while contacting the API server. This
  issue has now been resolved.


.. _Release Notes_7.59.1:

7.59.1
======

.. _Release Notes_7.59.1_Prelude:

Prelude
-------

Released on: 2024-12-02
Pinned to datadog-agent v7.59.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7591>`_.

.. _Release Notes_7.59.0:

7.59.0
======

.. _Release Notes_7.59.0_Prelude:

Prelude
-------

Released on: 2024-11-07
Pinned to datadog-agent v7.59.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7590>`_.

.. _Release Notes_7.58.2:

7.58.2
======

.. _Release Notes_7.58.2_Prelude:

Prelude
-------

Released on: 2024-11-04
Pinned to datadog-agent v7.58.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7582>`_.

.. _Release Notes_7.58.1:

7.58.1
======

.. _Release Notes_7.58.1_Prelude:

Prelude
-------

Released on: 2024-10-24
Pinned to datadog-agent v7.58.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7581>`_.

.. _Release Notes_7.58.0:

7.58.0
======

.. _Release Notes_7.58.0_Prelude:

Prelude
-------

Released on: 2024-10-21
Pinned to datadog-agent v7.58.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7580>`_.

.. _Release Notes_7.58.0_New Features:

New Features
------------

- Added capability to tag any Kubernetes resource based on labels and annotations.
  This feature can be configured with `kubernetes_resources_annotations_as_tags` and `kubernetes_resources_labels_as_tags`.
  These feature configurations are associate group resources with annotations-to-tags (or labels-to-tags) map
  For example, `deployments.apps` can be associated with an annotations-to-tags map to configure annotations as tags for deployments.
  Example:
  {`deployments.apps`: {`annotationKey1`: `tag1`, `annotationKey2`: `tag2`}}

- The Kubernetes State Metrics (KSM) check can now be configured to collect
  pods from the Kubelet in node agents instead of collecting them from the API
  Server in the Cluster Agent or the Cluster check runners. This is useful in
  clusters with a large number of pods where emitting pod metrics from a
  single check instance can cause performance issues due to the large number
  of metrics emitted.


.. _Release Notes_7.58.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added a new option for the Cluster Agent
  ("admission_controller.inject_config.type_socket_volumes") to specify that
  injected volumes should be of type "Socket". This option is disabled by
  default. When set to true, injected pods will not start until the Agent
  creates the DogstatsD and trace-agent sockets. This ensures no traces or
  DogstatsD metrics are lost, but it can cause the pod to wait if the Agent
  has issues creating the sockets.


.. _Release Notes_7.58.0_Bug Fixes:

Bug Fixes
---------

- Fixed an issue that prevented the Kubernetes autoscaler from evicting pods
  injected by the Admission Controller.


.. _Release Notes_7.57.1:

7.57.1
======

.. _Release Notes_7.57.1_Prelude:

Prelude
-------

Released on: 2024-09-17
Pinned to datadog-agent v7.57.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7571>`_.

.. _Release Notes_7.57.0:


7.57.0
======

.. _Release Notes_7.57.0_Prelude:

Prelude
-------

Released on: 2024-09-09
Pinned to datadog-agent v7.57.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7570>`_.

.. _Release Notes_7.57.0_New Features:

New Features
------------

- The Cluster Agent now supports activating Continuous Profiling
  using Admission Controller.

- ``LimitRange`` and ``StorageClass`` resources are now collected by the orchestrator check.


.. _Release Notes_7.57.0_Enhancement Notes:

Enhancement Notes
-----------------

- The auto-instrumentation webhook (beta) uses a new injector library.


.. _Release Notes_7.57.0_Bug Fixes:

Bug Fixes
---------

- Fixes a rare bug where some Kubernetes events would be emitted
  without a timestamp and would be dropped upstream as a result.

- Library package versions for auto-instrumentation are now set to the latest major
  version of the library-package instead of `latest`.

  * java:v1
  * dotnet:v2
  * python:v2
  * ruby:v2
  * js:v5

- Fix APIServer error logs generated when external metrics endpoint is activated


.. _Release Notes_7.56.2:

7.56.2
======

.. _Release Notes_7.56.2_Prelude:

Prelude
-------

Released on: 2024-09-02
Pinned to datadog-agent v7.56.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7562>`_.

.. _Release Notes_7.56.1:

7.56.1
======

.. _Release Notes_7.56.1_Prelude:

Prelude
-------

Released on: 2024-08-29
Pinned to datadog-agent v7.56.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7561>`_.

.. _Release Notes_7.56.0:

7.56.0
======

.. _Release Notes_7.56.0_Prelude:

Prelude
-------

Released on: 2024-08-16
Pinned to datadog-agent v7.56.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7560>`_.

.. _Release Notes_7.56.0_Upgrade Notes:

Upgrade Notes
-------------

- Disables default injection of the .NET profiler dependency for Kubernetes auto_instrumentation.


.. _Release Notes_7.56.0_Enhancement Notes:

Enhancement Notes
-----------------

- Mark the NetworkPolicy collector as stable in the Cluster Agent

- Enabled language detection automatically in the injected agent sidecar on EKS Fargate when APM SSI is enabled.
  This is only available for users using the admission controller to automatically inject the agent sidecar.

- The orchestrator check can now scrub sensitive data from probes in pods specifications.


.. _Release Notes_7.56.0_Bug Fixes:

Bug Fixes
---------

- Fixes issue where the external metrics server would sometimes return metrics which had not
  been updated for longer than the configured `external_metrics_provider.max_age` as valid.
  In connection with this fix, a new config (`external_metrics_provider.query_validity_period`)
  has been added to account for the delay between when metrics are resolved and when they
  are queried by the various autoscaling controllers. It is set to 30 seconds by default.

.. _Release Notes_7.55.3:

7.55.3
================

.. _Release Notes_7.55.3_Prelude:

Prelude
-------

Released on: 2024-08-01
Pinned to datadog-agent v7.55.3: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7553>`_.

.. _Release Notes_7.55.2:

7.55.2
================

.. _Release Notes_7.55.2_Prelude:

Prelude
-------

Released on: 2024-07-25
Pinned to datadog-agent v7.55.2: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7552>`_.

.. _Release Notes_7.55.1:

7.55.1
================

.. _Release Notes_7.55.1_Prelude:

Prelude
-------

Released on: 2024-07-12
Pinned to datadog-agent v7.55.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7551>`_.

.. _Release Notes_7.55.0:

7.55.0
======

.. _Release Notes_7.55.0_Prelude:

Prelude
-------

Released on: 2024-07-11
Pinned to datadog-agent v7.55.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7550>`_.

.. _Release Notes_7.55.0_New Features:

New Features
------------

- Add support for `kubernetes_namespace_annotations_as_tags`.
  This new option is equivalent to the existing `kubernetes_namespace_labels_as_tags`,
  but it considers namespace annotations instead of namespace labels.
  With this new option, users can enrich tagging based on namespace annotations.

- Support namespace labels as tags on kubernetes events.


.. _Release Notes_7.55.0_Enhancement Notes:

Enhancement Notes
-----------------

- Add ``reason:backofflimitexceeded,deadlineexceeded`` to the
  ``kubernetes_state.job.failed`` metric to help users understand why a job failed.

- Reduced the memory used to store the tags.

- The Datadog cluster-agent container image is now using Ubuntu 24.04 noble
  as the base image.


.. _Release Notes_7.55.0_Bug Fixes:

Bug Fixes
---------

- Fixes an issue with large clusters where the Cluster
  Agent fails to collect all tags when
  `cluster_agent.collect_kubernetes_tags` is enabled.


.. _Release Notes_7.54.1:

7.54.1
================

.. _Release Notes_7.54.1_Prelude:

Prelude
-------

Released on: 2024-06-18
Pinned to datadog-agent v7.54.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7541>`_.

.. _Release Notes_7.54.0:

7.54.0
================

.. _Release Notes_7.54.0_Prelude:

Prelude
-------

Released on: 2024-05-29
Pinned to datadog-agent v7.54.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7540>`_.


.. _Release Notes_7.54.0_New Features:

New Features
------------

- Add `LimitRange` and `StorageClass` collection in the orchestrator check.


.. _Release Notes_7.54.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added retry mechanism to language detection patcher in order to retry failed patching operations.


.. _Release Notes_7.54.0_Bug Fixes:

Bug Fixes
---------

- Fix collection of numeric rolling update options in Kubernetes deployments
  and daemonsets.

- Fixed initialization of language expiration time for detected languages.


.. _Release Notes_7.53.0:

7.53.0 / 6.53.0
================

.. _Release Notes_7.53.0_Prelude:

Prelude
-------

Released on: 2024-04-30
Pinned to datadog-agent v7.53.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7530>`_.


.. _Release Notes_7.53.0_New Features:

New Features
------------

- APM library injection now works on EKS Fargate when the admission controller
  is configured to add an Agent sidecar in EKS Fargate.

- Cluster Agent now supports activating Application Security Management, Code Vulnerabilities, and
  Software Composition Analysis via Helm charts.


.. _Release Notes_7.53.0_Enhancement Notes:

Enhancement Notes
-----------------

- Add the `mutation_webhook` tag to `admission_webhooks.webhooks_received` and `admission_webhooks.response_duration` Cluster Agent telemetry.

- When using the admission controller to inject an Agent sidecar on EKS
  Fargate, `shareProcessNamespace` is now set to `true` automatically. This is
  to ensure that the process collection feature works.


.. _Release Notes_7.52.0:

7.52.0 / 6.52.0
================

.. _Release Notes_7.52.0_New Features:

New Features
------------

- Add agent sidecar injection webhook in `cluster-agent` Kubernetes admission controller. This new webhook adds the Agent as sidecar container in applicative Pods when it is required by the environment. For example with the EKS Fargate environment.


.. _Release Notes_7.52.0_Enhancement Notes:

Enhancement Notes
-----------------

- Introduces a new config option in the Cluster Agent to set the rebalance
  period when advanced dispatching is enabled:
  ``cluster_checks.rebalance_period``. The default value is 10 min.


.. _Release Notes_7.52.0_Bug Fixes:

Bug Fixes
---------

- Fix an issue where the admission controller would remove the field `restartPolicy`
  from native sidecar containers, preventing pod creation on Kubernetes 1.29+.

- Fix missing `kube_api_version` tag on HPA and VPA resources.


.. _Release Notes_7.51.0:

7.51.0 / 6.51.0
================

.. _Release Notes_7.51.0_Prelude:

Prelude
-------

Released on: 2024-02-19
Pinned to datadog-agent v7.51.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7510>`_.

.. _Release Notes_7.51.0_New Features:

New Features
------------

- Enable Horizontal Pod Autoscaler collection for the Orchestrator by default

- Add isolate command to clusterchecks to make it easier to pinpoint
  a check that that is causing high CPU/memory usage. Command can be
  run in the cluster agent with:
  `datadog-cluster-agent clusterchecks isolate --checkID=<checkID>`


.. _Release Notes_7.51.0_Enhancement Notes:

Enhancement Notes
-----------------

- Enable CRD collection by default in the orchestrator check.


.. _Release Notes_7.51.0_Bug Fixes:

Bug Fixes
---------

- Fixes a bug that would trigger unnecessary APIServer `List` requests from the Cluster Agent or Cluster Checks Runner.


.. _Release Notes_7.50.1:

7.50.1 / 6.50.1
================

.. _Release Notes_7.50.1_Bug Fixes:

Bug Fixes
---------

- Fixes a bug introduced in `7.50.0` preventing `DD_TAGS` to be added to `kubernetes_state.*` metrics.


.. _Release Notes_7.50.0:

7.50.0 / 6.50.0
================

.. _Release Notes_7.50.0_New Features:

New Features
------------

- Add language detection API handler to the cluster-agent.

- Report `rate_limit_queries_remaining_min` telemetry from `external-metrics` server.

- Added a new `--force` option to the `datadog-cluster-agent clusterchecks rebalance` command that allows you to force clustercheck rebalancing with utilization.

- [Beta] Enable `APM` library injection in `cluster-agent` admission controller based on automatic language detection annotations.


.. _Release Notes_7.50.0_Enhancement Notes:

Enhancement Notes
-----------------

- Show Autodiscovery information in the output of ``datadog-cluster-agent status``.

- Added CreateContainerConfigError wait reason to the `kubernetes_state.container.status_report.count.waiting` metric
  reported by the kubernetes_state_core check.

- Release the Leader Election Lock on shutdown to make the initialization of future cluster-agents faster.

- The Datadog cluster-agent container image is now using Ubuntu 23.10 mantic
  as the base image.


.. _Release Notes_7.50.0_Bug Fixes:

Bug Fixes
---------

- Fixed a bug in the ``kubernetes_state_core`` check that caused tag corruption when ``telemetry`` was set to ``true``.

- Fix stale metrics being reported by kubernetes_state_core check in some rare cases.

- Fixed a bug in the rebalancing of cluster checks. Checks that contained
  secrets were never rebalanced when the Cluster Agent was configured to not
  resolve check secrets (option ``secret_backend_skip_checks`` set to true).


.. _Release Notes_7.49.0:

7.49.0 / 6.49.0
================

.. _Release Notes_7.49.0_New Features:

New Features
------------

- Added option to attach profiling data to a flare.

- Increment cluster agent admission controller mutation attempts metric when library is auto-injected.


.. _Release Notes_7.49.0_Enhancement Notes:

Enhancement Notes
-----------------

- Added the ``check_name`` tag to the ``cluster_checks.configs_info`` metric emitted by the Cluster Agent telemetry.

- Sensitive information is now scrubbed from pod annotations.

- Skip collections for resources missing RBACs in orchestrator check


.. _Release Notes_7.49.0_Bug Fixes:

Bug Fixes
---------

- Remove openmetrics endpoint default value from `containerd` check default configuration.

- Resolved a conflict between the admission controller and the AKS admissions enforcer that previously led to a loop in reconciling the webhook.

- Fixes a panic in the Cluster Agent that happens when trying to unschedule a
  check that has not been dispatched to any runner.


.. _Release Notes_7.48.0:

7.48.0 / 6.48.0
================

.. _Release Notes_7.48.0_New Features:

New Features
------------

- Added the kubernetes_state.pod.tolerations metric to the KSM core check

- Add ``HorizontalPodAutoscaler`` collection in the orchestrator check.


.. _Release Notes_7.48.0_Enhancement Notes:

Enhancement Notes
-----------------

- Add safeguards for orchestrator CRD collection.

- The Datadog cluster-agent container image is now using Ubuntu 23.04 lunar
  as the base image.


.. _Release Notes_7.48.0_Bug Fixes:

Bug Fixes
---------

- Fixed an error in the calculations performed by the algorithm that rebalances cluster checks. Cluster checks are now more evenly distributed when advanced dispatching is enabled (``cluster_checks.advanced_dispatching_enabled`` is set to true).

- Service checks are no longer excluded from rebalancing decisions when advanced dispatching is enabled (``cluster_checks.advanced_dispatching_enabled`` is set to true).

- Fixes a rare bug in the Kubernetes State check that causes the Agent to incorrectly tag the ``kubernetes_state.job.complete`` service check.

- Removes an incorrect warning log message that mentions that the DD_POD_NAME
  env var is unknown.

- Fixes the KSM check to support HPA v2beta2 again. This stopped working in Agent v7.44.0.

- Adds the `kube_cluster_name` tag as a static global tag to the cluster agent when the `DD_CLUSTER_NAME` config option is set.
  This should fix an issue where the tag is not being attached to metrics in certain environments, such as EKS Fargate.

- Fixed a bug in the advanced dispatching of cluster checks. All the checks
  scheduled since the last rebalance were being scheduled in the same node.
  Now they should be distributed among the available nodes.


.. _Release Notes_7.47.0:

7.47.0 / 6.47.0
================

.. _Release Notes_7.47.0_Upgrade Notes:

Upgrade Notes
-------------

- Add support for leases in leader election which can be enabled by setting
  ``leader_election_default_resource`` to ``leases``, available since Kubernetes version 1.14.
  If this parameter is empty, leader election automatically detects if leases
  are available and uses them.
  Set ``leader_election_default_resource`` to ``configmap`` on clusters running
  Kubernetes versions previous to 1.14.


.. _Release Notes_7.47.0_New Features:

New Features
------------

- Auto-instrumentation admission controller now automatically activates crash tracking for Java applications


.. _Release Notes_7.47.0_Enhancement Notes:

Enhancement Notes
-----------------

- Expose to cluster-agent HistogramBuckets and Events check stats.
  It should help the cluster-agent to define a better cluster-checks
  dispatching.


.. _Release Notes_7.47.0_Bug Fixes:

Bug Fixes
---------

- The Cluster Agent Admission Controller now injects DD_DOGSTATSD_URL when used in `socket` mode (default), allowing DogStatsD clients to work without configuration.

- Fix persistent volume type for local volumes.


.. _Release Notes_7.46.0:

7.46.0 / 6.46.0
================

.. _Release Notes_7.46.0_New Features:

New Features
------------

- Enable collection of Vertical Pod Autoscalers by default in the orchestrator check.


.. _Release Notes_7.46.0_Enhancement Notes:

Enhancement Notes
-----------------

- Collect conditions for a variety of Kubernetes resources.

- Collect persistent volume source in the orchestrator check.


.. _Release Notes_7.46.0_Bug Fixes:

Bug Fixes
---------

- Fix the timeout for idle HTTP connections.

- When the cluster-agent is started with ``hostNetwork: true``, the leader election mechanism was using a node name instead of the pod name. This was breaking the “follower to leader” forwarding mechanism.
  This change introduce the ``DD_POD_NAME`` environment variable as a more reliable way to set the cluster-agent pod name. It is supposed to be filled by the Kubernetes downward API.


.. _Release Notes_7.45.0:

7.45.0 / 6.45.0
================

.. _Release Notes_7.45.0_Enhancement Notes:

Enhancement Notes
-----------------

- Add "active" tag on the telemetry datadog.cluster_agent.external_metrics.datadog_metrics tag.
  The label active is true if DatadogMetrics CR is used, false otherwise.

- Library injection via Admission Controller: Allow configuring the CPU and Memory requests/limits for library init containers.

- Validate the orchestration config provided by the user.


.. _Release Notes_7.45.0_Bug Fixes:

Bug Fixes
---------

- Fix the admission controller in socket mode for pods with init containers.

- Fix resource requirements detection for containers without any request and
  limit set.

- The KSM core check now correctly handles labels and annotations with
  uppercase letters defined in the "labels_as_tags" and "annotations_as_tags"
  config attributes.


.. _Release Notes_7.44.0:

7.44.0 / 6.44.0
================

.. _Release Notes_7.44.0_New Features:

New Features
------------

- Add conditions to Vertical Pod Autoscalers

- Experimental: Support Ruby library injection through the Admission Controller on Kubernetes.


.. _Release Notes_7.44.0_Enhancement Notes:

Enhancement Notes
-----------------

- Add new metrics for the KSM Core check for extended resources:
  - Pod requests and limits of the network bandwidth extended resource: `kubernetes_state.container.network_bandwidth_limit`, `kubernetes_state.container.network_bandwidth_requested`
  - The capacity and allocatable network bandwidth extended resource of a node: `kubernetes_state.node.network_bandwidth_allocatable`, `kubernetes_state.node.network_bandwidth_capacity`

- Admission Controller: Add telemetry around auto-instrumentation via remote config.

- The UDS socket volume when using the Admission Controller is now mounted in readOnly mode.


.. _Release Notes_7.43.0:

7.43.0 / 6.43.0
================

.. _Release Notes_7.43.0_New Features:

New Features
------------

- Starts the collecting of Vertical Pod Autoscalers within Kubernetes clusters.

- Enable orchestrator manifest collection by default


.. _Release Notes_7.43.0_Bug Fixes:

Bug Fixes
---------

- Make the cluster-agent admission controller able to inject libraries for several languages in a single pod.


.. _Release Notes_7.42.0:

7.42.0 / 6.42.0
================

.. _Release Notes_7.42.0_New Features:

New Features
------------

- Supports the collection of custom resource definition and custom resource manifests for the orchestrator explorer.


.. _Release Notes_7.42.0_Enhancement Notes:

Enhancement Notes
-----------------

- Collects Unified Service Tags for the orchestrator explorer product.


.. _Release Notes_7.41.0:

7.41.0 / 6.41.0
================

.. _Release Notes_7.41.0_New Features:

New Features
------------

- Add ``Namespace`` collection in the orchestrator check and enable it by default.


.. _Release Notes_7.41.0_Enhancement Notes:

Enhancement Notes
-----------------

- Improves performance of the Cluster Agent admission controller on large pods.


.. _Release Notes_7.40.0:

7.40.0 / 6.40.0
================

.. _Release Notes_7.40.0_New Features:

New Features
------------

- Experimental: The Datadog Admission Controller can inject the Python APM library into Kubernetes containers for auto-instrumentation.

- The orchestrator check is now able to discover resources to collect based
  on API groups available in the Kubernetes cluster.


.. _Release Notes_7.40.0_Enhancement Notes:

Enhancement Notes
-----------------

- The admission controller now injects variables and volume mounts to init containers in addition to regular containers.

- Chunk orchestrator payloads by size and weight

- KSM Core check: Add the ``helm_chart`` tag automatically from the standard helm label ``helm.sh/chart``.

- Helm check: Add a ``helm_chart`` tag, equivalent to the standard helm label ``helm.sh/chart`` (see https://helm.sh/docs/chart_best_practices/labels/).


.. _Release Notes_7.40.0_Bug Fixes:

Bug Fixes
---------

- Fixed an edge case in the Admission Controller when ``mutateUnlabelled`` is enabled and ``configMode`` is set to ``socket``.
  This combination could prevent the creation of new DaemonSet Agent pods.

- Fixed a resource leak in the helm check.


.. _Release Notes_7.39.0:

7.39.0 / 6.39.0
================

.. _Release Notes_7.39.0_New Features:

New Features
------------

- Experimental: The Datadog Admission Controller can inject the Node and Java APM libraries into Kubernetes containers for auto-instrumentation.


.. _Release Notes_7.39.0_Enhancement Notes:

Enhancement Notes
-----------------

- When injecting env vars with the admission controller, env
  vars are now prepended instead of appended, meaning that
  Kubernetes [dependent environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-interdependent-environment-variables/)
  can now depend on these injected vars.

- The ``helm`` check has new configuration parameters:
  - ``extra_sync_timeout_seconds`` (default 120)
  - ``informers_resync_interval_minutes`` (default 10)

- Improves the `labelsAsTags` feature of the Kubernetes State Metrics core check by performing the transformations of characters ['/' , '-' , '.']
  to underscores ['_'] within the Datadog agent.
  Previously users had to perform these conversions manually in order to discover the labels on their resources.


.. _Release Notes_7.39.0_Bug Fixes:

Bug Fixes
---------

- Fix the DCA ``leader_election_is_leader`` metric that could sometimes report ``is_leader="false"`` on the leader instance

- Fixed an error when running ``datadog-cluster-agent status`` with
  ``DD_EXTERNAL_METRICS_PROVIDER_ENABLED=true`` and no app key set.

- The KSM Core check now handles cron job schedules with time zones.


.. _Release Notes_7.39.0_Other Notes:

Other Notes
-----------

- Align Cluster Agent version to Agent version. Cluster Agent will now be released with 7.x.y tags


.. _Release Notes_dca-1.22.0_dca-1.22.X:

dca-1.22.0
==========

.. _Release Notes_dca-1.22.0_dca-1.22.X_Prelude:

Prelude
-------

Released on: 2022-07-26
Pinned to datadog-agent v7.38.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7380--6380>`_.

.. _Release Notes_dca-1.22.0_dca-1.22.X_New Features:

New Features
------------

- Enable collection of Ingresses by default in the orchestrator check.

.. _Release Notes_dca-1.21.0_dca-1.21.X:

dca-1.21.0
==========

.. _Release Notes_dca-1.21.0_dca-1.21.X_Prelude:

Prelude
-------

Released on: 2022-06-28
Pinned to datadog-agent v7.37.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7370--6370>`_.

.. _Release Notes_dca-1.21.0_dca-1.21.X_Enhancement Notes:

Enhancement Notes
-----------------

- The Cluster Agent followers now forward queries to the Cluster Agent leaders themselves. This allows a reduction in the overall number of connections to the Cluster Agent and better spreads the load between leader and forwarders.

- Make the name of the ConfigMap used by the Cluster Agent for its leader election configurable.

- The Datadog Cluster Agent exposes a new metric ``endpoint_checks_configs_dispatched``.


.. _Release Notes_dca-1.21.0_dca-1.21.X_Bug Fixes:

Bug Fixes
---------

- Fix a panic occuring during the invocation of the `check` command on the
  Cluster Agent if the Orchestrator Explorer feature is enabled.

- Fix the node count reported for Kubernetes clusters.


.. _Release Notes_dca-1.20.0_dca-1.20.X:

dca-1.20.0
==========

.. _Release Notes_dca-1.20.0_dca-1.20.X_Prelude:

Prelude
-------

Released on: 2022-05-22
Pinned to datadog-agent v7.36.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7360--6360>`_.

.. _Release Notes_dca-1.20.0_dca-1.20.X_New Features:

New Features
------------

- The Datadog Admission Controller supports multiple configuration injection
  modes through the ``admission_controller.inject_config.mode`` parameter
  or the ``DD_ADMISSION_CONTROLLER_INJECT_CONFIG_MODE`` environment variable:
  - ``hostip``: Inject the host IP. (default)
  - ``service``: Inject Datadog's local-service DNS name.
  - ``socket``: Inject the Datadog socket path.

- Collect ResourceRequirements for jobs and cronjobs for kubernetes live containers.


.. _Release Notes_dca-1.20.0_dca-1.20.X_Enhancement Notes:

Enhancement Notes
-----------------

- Added a configuration option to admission controller to allow
  configuration of the failure policy. Defaults to Ignore which
  was the previous default. The default of Ignore means that pods
  will still be admitted even if the webhook is unavailable to
  inject them. Setting to Fail will require the admission controller
  to be present and pods to be injected before they are allowed to run.

- The admission controller's reinvocation policy is now set to ``IfNeeded`` by default.
  It can be changed using the ``admission_controller.reinvocation_policy`` parameter.

- The Datadog Cluster Agent now supports internal profiling.

- KSM core check: add a new ``kubernetes_state.cronjob.complete``
  service check that returns the status of the most recent job for
  a cronjob.


.. _Release Notes_dca-1.20.0_dca-1.20.X_Security Notes:

Security Notes
--------------

- Cluster Agent API (only used by Node Agents) is now only server with TLS >= 1.3 by default. Setting "cluster_agent.allow_legacy_tls" to true allows to fallback to TLS 1.0.


.. _Release Notes_dca-1.20.0_dca-1.20.X_Bug Fixes:

Bug Fixes
---------

- Fix the node count reported for Kubernetes clusters.

- Fixed an issue that created lots of log messages when the DCA admission controller was enabled on AKS.

- Time-based metrics (for example, `kubernetes_state.pod.age`, `kubernetes_state.pod.uptime`) are now comparable in the Kubernetes state core check.

- Fix a risk of panic when multiple KSM Core check instances run concurrently.

- Remove noisy Kubernetes API deprecation warnings in the Cluster Agent logs.


.. _Release Notes_dca-1.20.0_dca-1.20.X_Other Notes:

Other Notes
-----------

- Change the default value of the external metrics provider port from 443 to 8443.
  This will allow to run the cluster agent with a non-root user for better security.
  This was already the default value in the Helm chart and in the datadog operator.


.. _Release Notes_dca-1.19.0_dca-1.19.X:

dca-1.19.0
==========

.. _Release Notes_dca-1.19.0_dca-1.19.X_Prelude:

Prelude
-------

Released on: 2022-04-12
Pinned to datadog-agent v7.35.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7350--6350>`_.

.. _Release Notes_dca-1.19.0_dca-1.19.X_New Features:

New Features
------------

- Collect ResourceRequirements on other K8s workloads as well for live containers (Deployment, StatefulSet, ReplicaSet, DaemonSet)
- Enable collection of Roles/RoleBindings/ClusterRoles/ClusterRoleBindings/ServiceAccounts by default in the orchestrator check.
- Add ``Ingress`` collection in the orchestrator check.

.. _Release Notes_dca-1.19.0_dca-1.19.X_Bug Fixes:

Bug Fixes
---------

- Fix a bug that prevents scrubbing sensitive content on the DaemonSet resource.
- Fix a bug that prevents scrubbing sensitive content on the StatefulSet resource.

.. _Release Notes_dca-1.19.0_dca-1.19.X_Enhancement Notes:

Enhancement Notes
-----------------

- Adds a new histogram metric `admission_webhooks_response_duration` to monitor the admission-webhook's response time. The existing metric `admission_webhooks_webhooks_received` is now a counter.
- The cluster agent has an external metrics provider feature to allow using Datadog queries in Kubernetes HorizontalPodAutoscalers.
    It sometimes faces issues like:
    2022-01-01 01:01:01 UTC | CLUSTER | ERROR | (pkg/util/kubernetes/autoscalers/datadogexternal.go:79 in queryDatadogExternal) | Error while executing metric query ... truncated... API returned error: Query timed out
    To mitigate this problem, use the new ``external_metrics_provider.chunk_size`` parameter to reduce the number of queries that are batched by the Agent and sent together to Datadog.

.. _Release Notes_dca-1.18.0_dca-1.18.X:

dca-1.18.0
==========

.. _Release Notes_dca-1.18.0_dca-1.18.X_Prelude:

Prelude
-------

Released on: 2022-03-01
Pinned to datadog-agent v7.34.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7340--6340>`_.

.. _Release Notes_dca-1.18.0_dca-1.18.X_New Features:

New Features
------------

- Add an ``external_metrics_provider.endpoints`` parameter that allows to specify a list of external metrics provider endpoints.
If the first one fails, the DCA will query the next ones.
- Support file-based endpoint checks.
- Enable collection of PV/PVCs by default in the orchestrator check
- File-based cluster checks support Autodiscovery.

.. _Release Notes_dca-1.18.0_dca-1.18.X_Bug Fixes:

Bug Fixes
---------

- Fix the ``Admission Controller``/``Webhooks info`` section of the cluster agent ``agent status`` output on Kubernetes 1.22+.
Although the cluster agent was able to register its webhook with both the ``v1beta1`` and the ``v1`` version of the Administrationregistration API, the ``agent status`` command was always using the ``v1beta1``, which has been removed in Kubernetes 1.22.
- Improve error handling of deleted HPA objects.
- Fix an issue where scrubbing custom sensitive words would not work as intended for the orchestrator check.
- Fixed a bug that could prevent the Admission Controller from starting when the External Metrics Provider is enabled.
- Fix the caculation of orchestrator cache hits.


.. _Release Notes_dca-1.17.0_dca-1.17.X:

dca-1.17.0
==========

.. _Release Notes_dca-1.17.0_dca-1.17.X_Prelude:

Prelude
-------

Released on: 2022-01-26
Pinned to datadog-agent v7.33.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7330>`_.

.. _Release Notes_dca-1.17.0_dca-1.17.X_New Features:

New Features
------------

- Collect PVC tag on pending pods
- Add the ability to filter for check names in the cluster checks output.


.. _Release Notes_dca-1.17.0_dca-1.17.X_Bug Fixes:

Bug Fixes
---------

- Add reworked status output for orchestrator section on CLC setups.

.. _Release Notes_dca-1.17.0_dca-1.17.X_Security:

Security
--------

- Fix the removal of the "kubectl.kubernetes.io/last-applied-configuration" annotation on new collected resources

.. _Release Notes_dca-1.17.0_dca-1.17.X_Enhancement Notes:

Enhancement Notes
-----------------

- Add autoscaler resource kind (hpa,wpa) inside the DatadogMetrics status references.

.. _Release Notes_dca-1.16.0_dca-1.16.X:

dca-1.16.0
==========

.. _Release Notes_dca-1.16.0_dca-1.16.X_Prelude:

Prelude
-------

Released on: 2021-11-10
Pinned to datadog-agent v7.32.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7320>`_.

.. _Release Notes_dca-1.16.0_dca-1.16.X_New Features:

New Features
------------

- Introduce the collection of the following resources: ClusterRole, ClusterRoleBinding, Role, RoleBinding, ServiceAccount.

.. _Release Notes_dca-1.16.0_dca-1.16.X_Bug Fixes:

Bug Fixes
---------

- Fix tags for PV resources in the Orchestrator Explorer (type and phase).
- Fix an edge case in which the Cluster Agent's Admission Controller doesn't update the Webhook object according to specified configuration.

.. _Release Notes_dca-1.15.0_dca-1.15.X:

dca-1.15.0
==========

.. _Release Notes_dca-1.15.0_dca-1.15.X_Prelude:

Prelude
-------

Released on: 2021-09-13
Pinned to datadog-agent v7.31.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7310>`_.

.. _Release Notes_dca-1.15.0_dca-1.15.X_New Features:

New Features
------------

- Enable ``StatefulSet`` collection by default in the orchestrator check.
- Add ``PV`` and ``PVC`` collection in the orchestrator check.
- Added possibility to use the `maxAge` attribute defined in the datadogMetric CRD overriding the global `maxAge`.


.. _Release Notes_dca-1.14.0_dca-1.14.X:

dca-1.14.0
==========

.. _Release Notes_dca-1.14.0_dca-1.14.X_Prelude:

Prelude
-------

Released on: 2021-08-12
Pinned to datadog-agent v7.30.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7300>`_.

.. _Release Notes_dca-1.14.0_dca-1.14.X_New Features:

New Features
------------

- Enable ``DaemonSet`` collection by default in the orchestrator check. Add ``StatefulSet`` collection in the orchestrator check.

.. _Release Notes_dca-1.14.0_dca-1.14.X_Enhancement Notes:

Enhancement Notes
-----------------

- The Cluster Agent's Admission Controller now uses the ``admissionregistration.k8s.io/v1`` kubernetes API when available.
- The Cluster Agent can be instructed to dispatch cluster checks without decrypting secrets. The node Agent or the cluster check runner will fetch the secrets after receiving the configurations from the Cluster Agent. This can be enabled by setting ``DD_SECRET_BACKEND_SKIP_CHECKS`` to ``true`` in the Cluster Agent config.
- The Cluster Agent's external metrics provider now serves an OpenAPI endpoint.
- Add the ability to change log_level at runtime. To set the log_level to ``debug`` the following command should be used: ``agent config set log_level debug``.
- Improve status and flare for the Cluster Check Runners.

.. _Release Notes_dca-1.14.0_dca-1.14.X_Bug Fixes:

Bug Fixes
---------

- Show different orchestrator status collection information between follower and leader.
- Fix an edge case where the Admission Controller doesn't update the certificate according to the Cluster Agent configuration.

.. _Release Notes_dca-1.13.1_dca-1.13.X:

dca-1.13.1
==========

.. _Release Notes_dca-1.13.1_dca-1.13.X_Prelude:

Prelude
-------

Released on: 2021-07-05
Pinned to datadog-agent v7.29.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7290>`_.

Bug Fixes
---------

- Fix the embedded security policy version to match the one from the agent.


.. _Release Notes_dca-1.13.0_dca-1.13.X:

dca-1.13.0
==========

.. _Release Notes_dca-1.13.0_dca-1.13.X_Prelude:

Prelude
-------

Released on: 2021-06-22
Pinned to datadog-agent v7.29.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/master/CHANGELOG.rst#7290>`_.


.. _Release Notes_dca-1.13.0_dca-1.13.X_New Features:

New Features
------------

- Collect the DaemonSet resources for the orchestrator explorer.


.. _Release Notes_dca-1.13.0_dca-1.13.X_Enhancement Notes:

Enhancement Notes
-----------------

- The Cluster Agent exposes a new metric `external_metrics.datadog_metrics` to track the validity of DatadogMetric objects.

- Add additional status information in orchestrator section output. Whether collection works and whether cluster name is set.


.. _Release Notes_dca-1.13.0_dca-1.13.X_Bug Fixes:

Bug Fixes
---------

- Autodetect EC2 cluster name

- Decrease the Admission Controller timeout to avoid edge cases where high timeouts can cause ignoring the ``failurePolicy`` (see kubernetes/kubernetes#71508).

- The Cluster Agent's admission controller now requires the pod label ``admission.datadoghq.com/enabled=true`` to inject standard labels. This optimizes the number of mutation webhook requests.


.. _Release Notes_dca-1.12.0_dca-1.12.X:

dca-1.12.0
==========

.. _Release Notes_dca-1.12.0_dca-1.12.X_Prelude:

Prelude
-------

  Pinned to datadog-agent v7.28.0-rc.5

.. _Release Notes_dca-1.12.0_dca-1.12.X_New Features:

New Features
------------

- The cluster-agent container now tries to remove any folder beginning by ``..`` in paths of
  files mounted in ``/conf.d`` while copying them to the cluster-agent config folder

- collect cluster resource for orchestrator explorer.

- It's now possible to template the kube_cluster_name tag in DatadogMetric queries
  Example: avg:nginx.net.request_per_s{kube_container_name:nginx,kube_cluster_name:%%tag_kube_cluster_name%%}

- It's now possible to template any environment variable (as seen by the Datadog Cluster Agent) as tag in DatadogMetric queries
  Example: avg:nginx.net.request_per_s{kube_container_name:nginx,kube_cluster_name:%%env_DD_CLUSTER_NAME%%}


.. _Release Notes_dca-1.12.0_dca-1.12.X_Enhancement Notes:

Enhancement Notes
-----------------

- It is now possible to configure a custom timeout for the MutatingWebhookConfigurations
  objects controlled by the Cluster Agent via DD_ADMISSION_CONTROLLER_TIMEOUT_SECONDS. (Default: 30 seconds)

- The Datadog Cluster Agent's Admission Controller now uses a namespaced secrets informer.
  It no longer needs permissions to watch secrets at the cluster scope.

- The cluster agent now uses the same configuration than the security agent for
  the logs endpoints configuration. The parameters (such as `logs_dd_url` can be
  either be specified in the `compliance_config.endpoints` section or through
  environment variables (such as DD_COMPLIANCE_CONFIG_ENDPOINTS_LOGS_DD_URL).

- Improve the resilience of the connection of controllers to the External Metrics Server by moving to a dynamic client for the WPA controller.


.. _Release Notes_dca-1.12.0_dca-1.12.X_Upgrade Notes:

Upgrade Notes
-------------

- Change base Docker image used to build the Cluster Agent imges, moving from debian:bullseye to ubuntu:20.10.
  In the future the Cluster Agent will follow Ubuntu stable versions.


.. _Release Notes_dca-1.12.0_dca-1.12.X_Bug Fixes:

Bug Fixes
---------

- Fix a potential file descriptors leak.

- The Cluster Agent can now be configured to use tls 1.2 via DD_FORCE_TLS_12=true

- Fix "Error creating expvar server" error log when running the Datadog Cluster Agent CLI commands.

- Fix a bug preventing the
  "DD_ORCHESTRATOR_EXPLORER_ORCHESTRATOR_ADDITIONAL_ENDPOINTS" environment
  variable to be read.


.. _Release Notes_dca-1.11.0_dca-1.11.X:

dca-1.11.0
==========

.. _Release Notes_dca-1.11.0_dca-1.11.X_Prelude:

Prelude
-------

Released on: 2021-03-02
Pinned to datadog-agent v7.26.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7260--6260>`_.


.. _Release Notes_dca-1.11.0_dca-1.11.X_New Features:

New Features
------------

- Support Prometheus Autodiscovery for Kubernetes Services.


.. _Release Notes_dca-1.11.0_dca-1.11.X_Enhancement Notes:

Enhancement Notes
-----------------

- Add `external_metrics_provider.api_key` and `external_metrics_provider.app_key` parameters overriding default `api_key` and `app_key` if set.

- Add a new external_metrics_provider.endpoint config in datadog-cluster.yaml
  and a DD_EXTERNAL_METRICS_PROVIDER_ENDPOINT environment variable to
  override the default Datadog API endpoint to query external metrics from,
  in place of the global DATADOG_HOST. It also makes the external metrics
  provider respect DD_SITE if DD_EXTERNAL_METRICS_PROVIDER_ENDPOINT is not
  set.

- Node schedulability is now a dedicated tag on kubernetes node resources.


.. _Release Notes_dca-1.11.0_dca-1.11.X_Bug Fixes:

Bug Fixes
---------

- Fix dual shipping for orchestrator resources in the cluster agent.


.. _Release Notes_dca-1.10.0_dca-1.10.X:

1.10.0
==========

Prelude
-------

Released on: 2021-03-02
    Pinned to datadog-agent v7.24.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7240--6240>`_..

.. _Release Notes_dca-1.10.0_dca-1.10.X_New Features:

New Features
------------

- Add a new command 'datadog-cluster-agent health' to show the cluster
  agent's health, similar to the already existing `agent health`.

- collect node information for the orchestrator explorer

- Fill DatadogMetric `AutoscalerReferences` field to ease usage/investigation of DatadogMetrics

- The Cluster Agent can now collect stats from Cluster Level Check runners
  to optimize its dispatching logic and rebalance the scheduled checks.

- Allow providing custom tags to orchestrator resources.


.. _Release Notes_dca-1.10.0_dca-1.10.X_Enhancement Notes:

Enhancement Notes
-----------------

- Add new configuration parameter to allow 'GroupExec' permission on the secret-backend command.
  The new parameter ('secret_backend_command_allow_group_exec_perm') is now enabled by default in the cluster-agent image.

- Add resolve option to endpoint checks through new annotation `ad.datadoghq.com/endpoints.resolve`. With `ip` value, it allows endpoint checks to target static pods

- Expose metrics for the cluster level checks advanced dispatching.


.. _Release Notes_dca-1.10.0_dca-1.10.X_Bug Fixes:

Bug Fixes
---------

- Fix 'readsecret.sh' permission in Cluster-Agent dockerfiles that removes `other` permission.

- Fix issue in Cluster Agent when using external metrics without DatadogMetrics where multiple HPAs using the same metricName + Labels would prevent all HPAs (except 1st one) to get values from Datadog

- Ensure that leader election runs if orchestrator_explorer and leader_election are enabled.

- Rename node role tag from "node_role" to "kube_node_role" in orchestrator_explorer collection.


.. _Release Notes_dca-1.9.1_dca-1.9.x:

1.9.1
=====

.. _Release Notes_dca-1.9.1_dca-1.9.x_Prelude:

Prelude
-------

Released on: 2020-10-21
Pinned to datadog-agent v7.23.1: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7231>`_..

.. _Release Notes_dca-1.9.1_dca-1.9.x_Bug Fixes:

Bug Fixes
---------

- Support of secrets in JSON environment variables, added in `7.23.0`, is
  reverted due to a side effect (e.g. a string value of `"-"` would be loaded as a list). This
  feature will be fixed and added again in a future release.


.. _Release Notes_1.9.0:

1.9.0
=====

.. _Release Notes_1.9.0_Prelude:

Prelude
-------

Released on: 2020-10-13
Pinned to datadog-agent v7.23.0: `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7230--6230>`_..

New Features
------------

- Collect the node and cluster resource in Kubernetes for the Orchestrator Explorer (#6297).
- Add `resolve` option to the endpoint checks (#5918).
- Add `health` command (#6144).
- Add options to configure the External Metrics Server (#6406).

Enhancement Notes
-----------------

- Fill DatadogMetric `AutoscalerReferences` field to ease usage/investigation of DatadogMetrics (#6367).
- Only run compliance checks on the Cluster Agent leader (#6311).
- Add `orchestrator_explorer` configuration to enable the cluster-id ConfigMap creation and Orchestrator Explorer instanciation (#6189).

Bug Fixes
---------

- Fix transformer for gibiBytes and gigaBytes (#6437).
- Fix `cluster-agent` commands to allow executing the `readsecret.sh` script for the secret backend feature (#6445).
- Fix issue with External Metrics when several HPAs use the same query (#6412).

.. _Release Notes_1.8.0:

1.8.0
=====

.. _Release Notes_1.8.0_Prelude:

Prelude
-------

Released on: 2020-08-07

New Features
------------

- Add compliance check command to the DCA CLI (#5930)
- Add `clusterchecks rebalance` command (#5839)
- Add collection of additional Kubernetes resource types (deployments, replicaSets and services) for Live Containers (#6082, #5999)


Enhancement Notes
-----------------

- Support "ignore AD tags" parameter for cluster/endpoint checks (#6115)
- Use APIserver connection retrier (#6106)

.. _Release Notes_1.7.0:

1.7.0
=====

.. _Release Notes_1.7.0_Prelude:

Prelude
-------

Released on: 2020-07-20

This version contains the changes released with version 7.21.0 of the core agent.
Please refer to the `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7210--6210>`_.

New Features
------------

- Add support of DatadogMetric CRD to allow autoscaling based on arbitrary queries (#5384)
- Add Admission Controller to inject Entity ID, standard tags and agent host (useful in serverless environments)

Enhancement Notes
-----------------

- Add `leader_election_is_leader` metric to allow label joins (#5819)


.. _Release Notes_1.6.0:

1.6.0
=====

.. _Release Notes_1.6.0_Prelude:

Prelude
-------

Released on: 2020-06-11

This version contains the changes released with version 7.20.0 of the core agent.
Please refer to the `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7200--6200>`_.

Bug Fixes
---------

- Wait for client-go cache to sync for endpoints/services (#5291)
- Consider check failure in advanced rebalancing (#5441)

New Features
------------

- Autodiscover standard tags for Cluster and Endpoint Checks (#5241)

Enhancement Notes
-----------------

- Adds a metric to monitor the advanced dispatching algorithm (#4970)

.. _Release Notes_1.5.2:

1.5.2
=====

.. _Release Notes_1.5.2_Prelude:

Prelude
-------

Released on: 2020-02-11

Minor release on 1.5 branch

Bug Fixed
------------

- Fix agent commands in DCA (always start listener) (#4870)

.. _Release Notes_1.5.1:

1.5.1
=====

.. _Release Notes_1.5.1_Prelude:

Prelude
-------

Released on: 2020-02-06

Minor release on 1.5 branch

Bug Fixed
------------

- [DCA] fix cluster-agent flare panic (#4838)
- Remove setcap NET_BIND_SERVICE as we cannot make it work with user namespaces used in the CI (#4846)
- Add service listener in endpoints to watch for newly annotated services (#4816)
- Fix typo (#4831)

.. _Release Notes_1.5.0:

1.5.0
=====

.. _Release Notes_1.5.0_Prelude:

Prelude
-------

Released on: 2020-01-28

This version contains the changes released with version 7.17.0 of the core agent.
Please refer to the `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#7170>`_.

New Features
------------

- Adding logic to show DCA status for clc (#4738)
- Introduce Rate Limiting Stats in the /metrics of the Cluster Agent (#4669)
- MetricServer generates k8s event on HPA

Enhancement Notes
-----------------

- Add cluster-name tag in host tags (#4558)
- Add read-secret command in cluster-agent to use as secrets backend (#4639)
- Adding logic to show DCA status for clc (#4738)
- Allow dots in cluster names (#4611)
- Check if CheckMetadata exist before iterating over it in cluster agent status page (#4728)
- Grant CAP_NET_BIND_SERVICE capability to the cluster_agent (#4439)
- Ignore invalid cluster names instead of panicking (#4549)
- Fix eventrecorder init (#4732)
- Handle NewHandler failure better in setupClusterCheck (#4447)
- Adding User-Agent to the DCA client
- Filter non-cluster-checks (#4566)

.. _Release Notes_1.4.0:

1.4.0
=====

.. _Release Notes_1.4.0_Prelude:

Prelude
-------

Released on: 2019-11-06

This version contains the changes released with version 6.15.0 of the core agent.
Please refer to the `CHANGELOG <https://github.com/DataDog/datadog-agent/blob/main/CHANGELOG.rst#6150>`_.

New Features
------------

- Introducing the Advanced dispatching logic to rebalancing Cluster Level Checks [#4068, #4226, #4344]
- Enable the Endpoint check logic [#3853, #3704]
- HTTP proxy support for the external metrics provider #4191
- Improve External Metrics Provider resiliency [#4285, #3727]
- Revamp the Kubernetes event collection check [#4259, #4346, #4342, #4337, #4314]

Enhancement Notes
-----------------

- Update Gopkg.lock with new import #3837
- Fix kubernetes_apiserver default config file #3854
- Fix registration of the External Metrics Server's API #4233
- Fixing status of the Cluster Agent if the External Metrics Provider is not enabled #4277
- Fix how the endpoints check source is displayed in agent command outputs #4357
- Fix how we invalidate changed Endpoints config #4363
- Get Cluster Level Checks runner IPs from headers #4386
- Fixing output of `agent status` #4352

1.3.2
=====
2019-07-09

- Fix Cluster-agent failure with `cluster-agent flare` command.

1.3.1
=====
2019-06-19

- Fix "Kube Services" service: `kube service` tags attached to pod are not consistent.

.. _Release Notes_1.3.0:

1.3.0
=====

.. _Release Notes_1.3.0_Prelude:

Prelude
-------

Released on: 2019-05-07

The Datadog Cluster Agent can now auto-discover config templates for kubernetes endpoints checks and expose them to node Agents via its API. This feature is compatible with the version 6.12.0 and up of the Datadog Agent.

Refer to `the official documentation <https://docs.datadoghq.com/agent/autodiscovery/endpointschecks/>`_ to read more about this feature.


1.3.0-rc.3
==========
2019-05-03

Bug Fixes
---------
- Fix race condition: immutable MetaBundle stored in DCA cache.

1.3.0-rc.2
==========
2019-04-30

Bug Fixes
---------
- Fix race condition in Cluster Agent's API handler.

1.3.0-rc.1
==========
2019-04-24

New Features
------------
- The Cluster Agent can now auto-discover config templates for kubernetes endpoints checks and expose them to node Agents via its API
- Add the ``config`` and ``configcheck`` command to the cluster agent CLI
- Add the ``diagnose`` command to the cluster agent CLI and flare
- Add cluster_checks.extra_tags option to allow users to add tags globally to the cluster level checks.

Enhancement Notes
-----------------
- Improving Lifecycle of the External Metrics Provider
- Support milliquantities for the External Metrics Provider
- Move some logs from info to debug, in order to generates fewer noisy logs when running correctly.

.. _Release Notes_1.2.0:

1.2.0
=====

.. _Release Notes_1.2.0_Prelude:

Prelude
-------

Released on: 2019-02-25

The Datadog Agent now supports distributing Cluster Level Checks. This feature is compatible with the version 6.9.0 and up of the Datadog Agent.

Refer to `the official documentation <https://docs.datadoghq.com/agent/autodiscovery/clusterchecks/>`_ to read more about this feature.

1.2.0-rc.5
==========
2019-02-14

Bug Fixes
---------
- Ensure dangling cluster checks can be re-scheduled

1.2.0-rc.4
==========
2019-02-12

Bug Fixes
---------
- Fix re-scheduling of the same clusterchecks config on the same node

1.2.0-rc.3
==========
2019-02-11

Enhancement Notes
-----------------
- Sign docker images when pushing to Docker Hub

Bug Fixes
---------
- Fix configcheck verbose output
- Fix AutoDiscovery rescheduling issue when no template variables
- Remove resolved configs when template are removed
- Support adding/removing the AD annotation to an existing kube service
- Only expose cluster-check prometheus metrics when leading
- Fix support for custom metrics case sensitivity

1.2.0-rc.2
==========
2019-02-05

Enhancement Notes
-----------------
The External Metrics Provider is now agnostic of the case, both on the metric name and the labels extracted from HPAs.

Bug Fixes
---------
- Cluster Agent HPA metrics case support

New Features
------------
- Add GetLeaderIP method to LeaderEngine
- Add kube_service config provider
- Allow to set additional Autodiscovery sources by envvars
- Add dispatching metrics in clusterchecks module
- Add a health probe in the ccheck dispatching logic
- Add kube-services AD listener
- Cluster-checks: handle leader election and follower->leader redirection
- Enable clusterchecks in DCA master
- Support /conf.d in cluster-agent image
- Fix clustercheck leader not starting its dispatching logic
- Use the appropriate port when redirecting node-agents to leader
- Cluster-checks: patch configurations on schedule
- Add configcheck/config cmd on the cluster agent
- Add clustercheck info to the cluster-agent's status and flare
- Make error in clusterchecks cmd clear when feature is disabled

1.2.0-rc.1
==========
2019-01-31

Note
----
The release of the RC1 was dismissed to embed a fix for the CI runners used to build the image.
- Go 1.11.5 compliancy + 1.11.5 for every CI
The official release of the Datadog Cluster Agent 1.2.0 starts with the RC2.

.. _Release Notes_1.1.0:

1.1.0
=====

.. _Release Notes_1.1.0_Prelude:

Prelude
-------

The version 1.1.0 of the Cluster Agent introduces new features and enhancements around the External Metrics Provider.

1.1.0-rc.2
==========
2018-11-21

Bug Fixes
---------
- Get goautoneg from github
- Fix datadog external metric query when no label is set

1.1.0-rc.1
==========
2018-11-20

Enhancement Notes
-----------------
- Migrating back to official custom metrics lib
- Change test to remove flakiness

New Features
------------
- Disable cluster checks in cluster-agent 1.1.x
- Allow users to change the custom metric provider port, to run as non-root
- Adding rollup and fix to circumvent time aggregation
- clusterchecks: simple dispatching logic
- Honor external metrics provider settings in cluster-agent status
- Run cluster-agent as non-root, support read-only rootfs
- Only push cluster-agent-dev:master from master

Bug Fixes
---------
- Fix folder permissions on containerd
- Adding fix for edge case in external metrics
- Fix flare if can't access APIServer
- DCA: fix custom metrics server
- Avoid panicking for missing fields in HPA

.. _Release Notes_1.0.0:

1.0.0
=====

.. _Release Notes_1.0.0_Prelude:

Prelude
-------

Released on: 2018-10-18

The Datadog Cluster Agent is compatible with versions 6.5.1 and up of the Datadog Agent.

- Please refer to the `6.5.0 tag on datadog-agent  <https://github.com/DataDog/datadog-agent/releases/tag/6.5.0>`_ for the list of changes on the Datadog Agent.

It is only supported in containerized environments.

- Please find the image on `our Docker Hub <https://hub.docker.com/r/datadog/cluster-agent/tags/>`_.

1.0.0-rc.4
==========
2018-10-17

Enhancement Notes
-----------------
- Expose telemetry metrics with the Open Metrics format instead of expvar

Bug Fixes
---------
- add mutex logic and safe guards to avoid race condition in the Autoscalers Controller.

1.0.0-rc.3
==========
2018-10-15

Enhancement Notes
-----------------
- Leverage diff logic to only update the internal custom metrics store and Config Map with relevant changes.
- Better logging on the Autoscalers Controller

Bug Fixes
---------
- Make sure only the leader sync Autoscalers.
- Forget keys from the informer's queue to avoid borking the Autoscalers Controller.

1.0.0-rc.2
==========
2018-10-11

Enhancement Notes
-----------------

- Support `agent` and `datadog-cluster-agent` for the CLI of the Datadog Cluster Agent
- Retrieve hostname in GCE

1.0.0-rc.1
==========
2018-10-04

New Features
------------

- Implement the External Metrics Interface to allow for the Horizontal Pod Autoscalers to be based off of Datadog metrics.
- Use informers to be up to date with the Horizontal Pod Autoscalers object in the cluster.
- Implement the metadata mapper.
- Use informers to be up to date with the Endpoints and Nodes objects in the cluster.
- Serve cluster level metadata on an external endpoint, `kube_service` tag is available.
- Serve node labels as tags.
- Run the kube_apiserver check to collect events and run a service check against each component of the Control Plane.
- Implements the `flare`, `status` and `version` commands similar to the node agent.