Merge branch 'main' into pedro.cordeiro/macos-shutdown-notable-event

Author: PedroCordeiroDataDog

.adms/bazel/adms.mirror.cfg

@@ -113,6 +113,7 @@ rewrite (ftp.osuosl.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (gnupg.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (kerberos.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (packages.microsoft.com)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
+rewrite (download.visualstudio.microsoft.com)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (www.freetds.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (www.lua.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2
 rewrite (www.sqlite.org)/(.*) depot-read-api-bzl.us1.ddbuild.io/$1/$2

.agents/skills/write-e2e/SKILL.md

@@ -0,0 +1,48 @@
+# Stale APIs and where they are written
+
+Load when working from an internal Confluence E2E page, an older branch, or any snippet whose form you cannot find in use under `test/new-e2e/tests/` — several otherwise-trustworthy sources teach forms that do not compile.
+
+The third column is the point of this file. Knowing that a form is wrong helps only if you can recognise what taught it to you, and these sources are otherwise reliable, so a snippet from them reads as trustworthy.
+
+## Will not compile
+
+| Wrong | Right | Why it is reached for |
+|---|---|---|
+| `awshost.WithAgentOptions(agentparams...)` | `awshost.WithRunOptions(ec2.WithAgentOptions(agentparams...))` | It is the correct form on Azure and GCP, so it transfers wrongly to AWS |
+| `ec2docker.WithExtraComposeManifest(...)` | it is a `dockeragentparams` option: `scenariodocker.WithAgentOptions(dockeragentparams.WithExtraComposeManifest(...))` | The option configures compose, so the scenario package looks like its home |
+| `winazurehost` as the default Windows provisioner | `winawshost`; Azure's Windows provisioner has no test or CI precedent | Confluence recommends Azure for faster Windows boot |
+
+## Compiles, behaves wrong
+
+| Wrong | Right | Why it is easy to get wrong |
+|---|---|---|
+| `dockeragentparams.WithEnvironmentVariables` to configure the agent | `dockeragentparams.WithAgentServiceEnvVariable(key, value)` | The name reads like the general case, but it only reaches the docker-compose command and its interpolation, not the agent process |
+| `os.Getenv("E2E_FAKEINTAKE_IMAGE_OVERRIDE")` and other `E2E_*` reads | the runner parameter store, e.g. `runner.GetProfile().ParamStore().GetBoolWithDefault(parameters.SkipWindows, false)` | Direct reads bypass the profile and resolve differently in CI than locally |
+
+## Confirming a form before you use it
+
+AWS provisioners nest their options inside `WithRunOptions` while Azure, GCP, and local ones are flat, so a snippet moved between clouds needs its shape adjusted. Both shapes are current and neither is deprecated; `references/environments.md` § "Two option shapes" has the rule and the per-provisioner naming that goes with it.
+
+```bash
+grep -n '^func With' test/e2e-framework/testing/provisioners/<path>/*.go
+grep -rn '<OptionName>' test/new-e2e/tests/ test/new-e2e/examples/ | head
+```
+
+A name with no usage anywhere in `tests/` or `examples/` is worth double-checking; the widely used forms are widely used because they work.
+
+## Confluence pages
+
+The internal E2E pages carry useful material — the cloud-selection rule, the credentials runbook, per-suite runtime measurements — alongside instructions that have since moved:
+
+| Page says | Current |
+|---|---|
+| `inv …` | `dda inv …` |
+| `.gitlab/e2e/e2e.yml` | `.gitlab/test/e2e/e2e.yml` |
+| `test/new-e2e/pkg/utils/e2e` | `test/e2e-framework/testing/e2e` |
+| framework lives in the `test-infra-definitions` repository | it lives in this repository under `test/e2e-framework/` |
+| `compare_to: main` in a rule | `compare_to: $COMPARE_TO_BRANCH` |
+| `deploy_deb_testing-a7_x64` as an artifact job | `agent_deb-x64-a7` |
+
+## Fixing what you find
+
+When a repository document turns out to be wrong, correct it in the same change. The root `AGENTS.md` asks for this, and it is the only thing that keeps this list from growing: a trap fixed at the source stops reaching the next agent, while a trap recorded here only helps whoever loads this file.

.agents/skills/write-e2e/references/api-traps.md

@@ -0,0 +1,146 @@
+# CI wiring
+
+Load when adding a new test package or changing which build artifacts a test consumes. A test in a package no job references never runs.
+
+Where the wiring lives:
+
+| File | Holds |
+|---|---|
+| `.gitlab-ci.yml` | The rule template that decides when the job runs |
+| `.gitlab/test/e2e/e2e.yml` | The job itself (Linux and cross-platform) |
+| `.gitlab/windows/test/e2e/windows.yml` and `.gitlab/windows/test/e2e_install_packages/windows.yml` | Windows jobs |
+| `.gitlab/JOBOWNERS` | Who is notified when the job fails |
+
+## The rule template
+
+Rule template names are abbreviated and do not track directory names — `tests/agent-runtimes` is gated by `.on_arun_or_e2e_changes`. Find the existing one rather than guessing:
+
+```bash
+grep -n '_or_e2e_changes:' .gitlab-ci.yml
+```
+
+A new one references the shared branch rule and adds the paths that should trigger it:
+
+```yaml
+.on_myarea_or_e2e_changes:
+  - !reference [.on_e2e_main_release_or_rc]
+  - changes:
+      paths:
+        - comp/myarea/**/*
+        - pkg/myarea/**/*
+        - test/new-e2e/tests/myarea/**/*
+      compare_to: $COMPARE_TO_BRANCH
+```
+
+`.on_e2e_main_release_or_rc` is what makes the job run on `main`, release branches, and release candidates regardless of the diff. The `changes` block is what additionally runs it on a pull request that touches those paths. List the implementation paths, not only the test path — otherwise a change to the feature will not exercise its own test.
+
+## The job
+
+Extend a template that already declares the right artifact dependencies rather than hand-writing `needs`:
+
+| Template | Brings in | Use for |
+|---|---|---|
+| `.new_e2e_template` | test binaries, tooling, fakeintake | Tests needing no agent package |
+| `.new_e2e_template_needs_deb_x64` | `agent_deb-x64-a7`, `agent_deb-x64-a7-fips` | Host tests on Ubuntu or Debian |
+| `.new_e2e_template_needs_container_deploy_linux` | `qa_agent_linux`, `qa_agent_linux_jmx`, `qa_dca`, `qa_dogstatsd` | Docker and Kubernetes on Linux |
+| `.new_e2e_template_needs_container_deploy` | the above plus the Windows agent images | Container tests covering Windows |
+| `.new_e2e_template_needs_windows_x64` | `windows_msi_and_bosh_zip_x64-a7` and its FIPS variant | Windows host tests (defined in `.gitlab/windows/test/e2e/windows.yml`) |
+
+```yaml
+new-e2e-myarea:
+  extends: .new_e2e_template_needs_deb_x64
+  rules:
+    - !reference [.on_myarea_or_e2e_changes]
+    - !reference [.manual]
+  variables:
+    TARGETS: ./tests/myarea
+    TEAM: myteam
+    EXTRA_PARAMS: --skip "Windows"
+    ON_NIGHTLY_FIPS: "true"
+```
+
+`TARGETS` is relative to `test/new-e2e/`. `TEAM` routes test results. `EXTRA_PARAMS` passes `--run` and `--skip` regexes, which is how a suite split across `_nix_test.go` and `_win_test.go` gets divided between jobs. `ON_NIGHTLY_FIPS` also runs the job in the nightly FIPS pipeline.
+
+When none of the templates fits, compose from the shared reference so you inherit the base dependencies:
+
+```yaml
+  needs:
+    - !reference [.needs_new_e2e_template]
+    - agent_rpm-x64-a7
+```
+
+Other artifact jobs: `agent_rpm-x64-a7` (RPM distributions), `deploy_windows_testing-a7` and `deploy_windows_testing-a7-fips` (Windows MSI), `deploy_installer_oci` (Fleet Automation packages).
+
+Ask for only the artifacts the test consumes. Without `needs`, GitLab waits for every earlier stage; with too many, the job blocks on builds it never uses and lengthens the pipeline for everyone. A test that hangs waiting for an image usually has a missing `needs`, not a broken test.
+
+## Windows jobs
+
+Each Windows installer test function provisions its own VM, so those jobs fan out with `parallel: matrix`, one entry per test function, and select with an anchored regex:
+
+```yaml
+  parallel:
+    matrix:
+      - E2E_MSI_TEST: TestInstall
+      - E2E_MSI_TEST: TestUpgrade
+  variables:
+    TARGETS: ./tests/windows/install-test
+    EXTRA_PARAMS: --run "$E2E_MSI_TEST$"
+```
+
+Adding a test function to one of those packages means adding a matrix entry, otherwise it never runs.
+
+## Pre-initialising expensive infrastructure
+
+A cluster that takes five to ten minutes to create can be built once by a separate job:
+
+```yaml
+new-e2e-myarea-init:
+  extends: .new_e2e_template
+  stage: e2e_init
+  variables:
+    TARGETS: ./tests/myarea
+    E2E_INIT_ONLY: "true"
+
+new-e2e-myarea:
+  extends: .new_e2e_template
+  needs:
+    # extends replaces needs rather than merging, and the inherited before_script
+    # unpacks these artifacts — dropping them fails the job on a missing tarball.
+    - !reference [.needs_new_e2e_template]
+    - new-e2e-myarea-init
+  variables:
+    TARGETS: ./tests/myarea
+    E2E_PRE_INITIALIZED: "true"
+```
+
+`new-e2e-containers-eks` in `.gitlab/test/e2e/e2e.yml` is the in-tree version; it re-references `.new_e2e_template_needs_container_deploy` alongside its init job for the same reason.
+
+Worth it for EKS and similar; unnecessary for a single VM.
+
+## Ownership
+
+Add the job to `.gitlab/JOBOWNERS`, which is what dispatches a failure notification:
+
+```
+new-e2e-myarea*                   @DataDog/myteam
+```
+
+`new-e2e*` defaults to `@DataDog/agent-devx`, so a job without its own entry pages the framework team instead of the team that owns the behavior. Add the test directory to `.github/CODEOWNERS` as well — that governs review, not notifications, and the two are separate on purpose.
+
+## Budgets and branch coverage
+
+`test/new-e2e/codereview_guideline.md` § "Keeping tests fast" sets the wall-time budgets. A job gated on every pull request regardless of paths is rare by design and needs justifying.
+
+Most E2E jobs run only on `main`, release branches, and release candidates. A change whose only coverage is such a job gets no pull-request signal — say so in the report, since a reviewer cannot tell from a green pipeline. That is a disclosure, not a label: a test-only pull request still takes `qa/no-code-change`, and `qa/rc-required` is reserved for changes that genuinely can only be validated on a release candidate.
+
+## Dynamic test skipping
+
+Some e2e jobs prune themselves from the inside. A job whose `rules` reference `.dynamic_tests` (`.gitlab-ci.yml`) is created on any change under `pkg/`, `cmd/`, or `comp/` — deliberately broad — and then the `--impacted` flag on `new-e2e-tests.run` consults a coverage index and skips the tests in that job the diff does not touch. It selects tests within a job, never between jobs, so it neither replaces nor relaxes the `changes` rule above.
+
+Three consequences for a test author:
+
+- A test the index does not know about is never skipped. The skip list is `indexed tests − impacted tests` (`tasks/libs/dynamic_test/index.py`), so a newly added test always runs.
+- Pruning happens on dev branches only. `main`, release branches, tagged commits, and triggered pipelines run everything, as does setting `RUN_E2E_TESTS=on` or the breakglass secret.
+- A failure to load the index is logged and the run falls back to the full suite, so a missing index costs time rather than coverage.
+
+Ask in `#agent-devx-help` when a job needs an artifact or a cloud capability that does not exist yet.