From b6c8bae2c8f0aa7cfb3401bbfed2267614360f31 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Wed, 5 Aug 2026 17:39:18 +0200 Subject: [PATCH 1/9] feat: add `/follow-pr` skill --- .agents/skills/follow-pr/SKILL.md | 98 +++++++++++++++++++++++ .agents/skills/follow-pr/create_devenv.sh | 17 ++++ 2 files changed, 115 insertions(+) create mode 100644 .agents/skills/follow-pr/SKILL.md create mode 100755 .agents/skills/follow-pr/create_devenv.sh diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md new file mode 100644 index 000000000000..939766cc79cc --- /dev/null +++ b/.agents/skills/follow-pr/SKILL.md @@ -0,0 +1,98 @@ +--- +name: babysit-pr +description: >- + Monitor the current PR's GitLab pipeline to completion, then report success or investigate a failure. + Use when the user asks to follow, babysit, watch, or wait on a PR/pipeline, or just after pushing to / creating a PR. +model: sonnet +--- + +# Babysit PR + +Watch the latest Gitlab CI pipeline for the current PR to a terminal state and report the outcome. + +## Step 0: Ensure correct environment +The appropriate tool for this usecase is `ddgl`, and more specifically `ddgl attach`. +Check if `ddgl` is available - `which ddgl`. If so, move to [Step 1](#step-1-determine-the-target). Otherwise, use a dev env as specified below. + +### Ensuring a dev env +Check for the existence of a previous dev env by using `dda env dev show`. +**If there are existing dev envs**: +- Check if the current repo is properly mounted into that env (`repos` and `extra_(mount|volume)_specs` fields) +- Check the current state of that dev env. + +If the environment is already started and contains the right repo, move to [the next step](#using-a-dev-env). +Otherwise, create one by using `./create_devenv.sh`, the ID will be automatically exported as `ATTACH_DEVENV_ID` + +### Using a dev env +To run commands inside a dev env, use the following template: +```bash +dda env dev run --id ${ATTACH_DEVENV_ID} -- [command] +``` +Watch out for space-splitting. For example: +```bash +dda env dev run --id babysit-pr-attach-7C2C42F6 -- ddgl attach --detail=normal --follow --plain +``` + +## Step 1: Determine the target + +If the user gave a ref, branch, or pipeline ID, pass it through (`--ref ` or `--pipeline `). +Otherwise omit both — `ddgl attach` resolves the pipeline for the current branch on its own. + +## Step 2: Start monitoring + +All pipeline discovery, polling, follow/rebind, and timeout handling is covered by the internals of `ddgl attach`. +Do not implement a second polling loop or persist monitoring state of your own. + +Check whether you have a long-lived monitoring tool available, one that can run a command in the background and forward each stdout line as it arrives, without a timeout of its own (e.g. Claude Code's `Monitor` tool). + +**With such a tool:** start it on + +```bash +ddgl attach --plain --follow --detail=full [--ref | --pipeline ] +``` + +and wait for a `[FINAL]` line — no `--timeout` needed. + +**Without one:** run it in the foreground, bounded so the invocation cannot +outlive your own harness timeout: + +```bash +ddgl attach --plain --follow --detail=full --timeout 600 [--ref | --pipeline ] +``` + +If the `[FINAL]` line reports a timeout (not a pipeline outcome), start an identical invocation again. +This is safe: `attach` is stateless and each invocation begins with a fresh snapshot of the pipeline. + +> NOTE: If the pipeline is already terminal or does not exist when you start monitoring, the user might have just pushed and the pipeline is still waiting to be created. +> In this case, wait for a minute or two and then re-attempt monitoring. The `--follow` argument will make sure `ddgl attach` always monitors the latest pipeline for the ref. + +## Step 3: Interpret the output + +You may see: + +- `[POLL]` - rollup summary after a changed poll tick (jobs done/total, stage, + failure count). Informational only. +- `[INFO]` - an informational log from `ddgl` itself. +- `[PIPE]` - a change in the pipeline status. +- `[JOB]` - a job finished running and changed state. +- `[FINAL]` - the terminal, authoritative outcome. Treat this line as the + source of truth regardless of the command's exit code — it names the pipeline id, terminal status, and, on failure, the failed job names. + +## Step 4: Act on the outcome + +- **Pipeline Success:** stop monitoring and report the pipeline succeeded. +- **Some job failed, but the pipeline is still running**: Ask the user whether to continue monitoring, or if this job failure is already a problem. If it is, move to [Step 5](#step-5-follow-up-on-failures) +- **Pipeline failed or canceled:** Stop monitoring, report the status, and move to [Step 5](#step-5-follow-up-on-failures). +- **Timeout `[FINAL]`:** re-invoke `ddgl attach` as in Step 2; this is not a true terminal outcome. +- **Unexpected error** (from `ddgl` itself, or from the monitoring tool): report what happened. Do not attempt a recovery action. + +## Step 5: follow-up on failures +Use other `ddgl` features to investigate failures on the pipeline that failed +Using the pipeline id from the `[FINAL]` line: +1. `ddgl jobs get --pipeline --failed --json` — failed-job metadata. +2. `mkdir -p failures && ddgl logs --pipeline --failed --output failures/` — export failed-job log output to `failures/` +3. Compare the failure evidence against the current PR's diff. Decide + whether the failure is likely caused by this PR, needs more evidence, or + is unrelated (e.g. flaky infra, an unrelated pre-existing failure). +4. If PR-caused, propose the smallest concrete fix — do not apply it. If + not, or inconclusive, report the evidence and your reasoning. diff --git a/.agents/skills/follow-pr/create_devenv.sh b/.agents/skills/follow-pr/create_devenv.sh new file mode 100755 index 000000000000..0fbafe12c0c5 --- /dev/null +++ b/.agents/skills/follow-pr/create_devenv.sh @@ -0,0 +1,17 @@ +#!/bin/sh + +set -e pipefail + +# Get the repo path +REPO=$(git rev-parse --show-toplevel) + +# Create a unique ID +ID=babysit-pr-attach-$(uuidgen | cut -d'-' -f1) + +# Use --no-pull to make sure it starts quickly +# Explicitly pass the repo to avoid cloning or missing the automatic bind-mount. Note it will be mounted as `/repos/the/full/absolute/path` inside the container +# But the CWD is changed appropriately when doing this way, so it's fine +dda env dev start --no-pull --repo "${REPO}" --id "${ID}" + +echo "Created env: ${ID}" +export ATTACH_DEVENV_ID="${ID}" From cff0ba3f08466ede037d397728b19704bb83c487 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Wed, 5 Aug 2026 17:41:08 +0200 Subject: [PATCH 2/9] feat: add a hint to `/create-pr` to invoke `/follow-pr` --- .agents/skills/create-pr/SKILL.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 85530e828eb7..9725e5e70339 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -39,6 +39,7 @@ Create a pull request for the current branch following the Datadog Agent contrib - **Motivation**: A reason why the change is made. Point to an issue if applicable. Include drawbacks or tradeoffs if any. - **Describe how you validated your changes**: How you validated the change (tests added/run, benchmarks, manual testing). Only needed when testing included work not covered by test suites. - **Additional Notes**: Any extra context, links to predecessor PRs if part of a chain, notes that make code understanding easier. **Only include this section if there is genuinely useful context to add** — omit it entirely rather than filling it with filler. +11. Once the PR is pushed, ask the user if they want to follow CI status for this PR. If yes, invoke the `/follow-pr` skill. ## PR Description Guidelines (from CONTRIBUTING.md) @@ -86,4 +87,5 @@ EOF ## Output -Return the PR URL when done. +If you are not following the PR status (step 11): Return the PR URL when done. +Otherwise, defer to `/follow-pr`. From c2a2d4a66639a04bc74fc464c3c28818b5fa4148 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Wed, 5 Aug 2026 18:06:19 +0200 Subject: [PATCH 3/9] fix: properly `babysit`->`follow` --- .agents/skills/follow-pr/SKILL.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md index 939766cc79cc..f27f529b8667 100644 --- a/.agents/skills/follow-pr/SKILL.md +++ b/.agents/skills/follow-pr/SKILL.md @@ -1,12 +1,12 @@ --- -name: babysit-pr +name: follow-pr description: >- Monitor the current PR's GitLab pipeline to completion, then report success or investigate a failure. Use when the user asks to follow, babysit, watch, or wait on a PR/pipeline, or just after pushing to / creating a PR. model: sonnet --- -# Babysit PR +# Follow PR Watch the latest Gitlab CI pipeline for the current PR to a terminal state and report the outcome. @@ -30,7 +30,7 @@ dda env dev run --id ${ATTACH_DEVENV_ID} -- [command] ``` Watch out for space-splitting. For example: ```bash -dda env dev run --id babysit-pr-attach-7C2C42F6 -- ddgl attach --detail=normal --follow --plain +dda env dev run --id follow-pr-attach-7C2C42F6 -- ddgl attach --detail=normal --follow --plain ``` ## Step 1: Determine the target From e1368a6e1521807cc34de1b4b4bb55c30838aa22 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 14:19:36 +0200 Subject: [PATCH 4/9] fix: mention appropriate behavior when running insde a dev env --- .agents/skills/follow-pr/SKILL.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md index f27f529b8667..c4015605617e 100644 --- a/.agents/skills/follow-pr/SKILL.md +++ b/.agents/skills/follow-pr/SKILL.md @@ -15,7 +15,9 @@ The appropriate tool for this usecase is `ddgl`, and more specifically `ddgl att Check if `ddgl` is available - `which ddgl`. If so, move to [Step 1](#step-1-determine-the-target). Otherwise, use a dev env as specified below. ### Ensuring a dev env -Check for the existence of a previous dev env by using `dda env dev show`. +First, check if you are running in a dev env: `test -f /.started` will exit 0 if so. If you are in an outdated devenv without `ddgl`, stop and notify the user to recreate his dev env. +Otherwise, check for the existence of a dev env by using `dda env dev show`. + **If there are existing dev envs**: - Check if the current repo is properly mounted into that env (`repos` and `extra_(mount|volume)_specs` fields) - Check the current state of that dev env. From 8626ff99575a215d07f6261b0b6fa8bb2becfb5b Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 14:26:01 +0200 Subject: [PATCH 5/9] fix: remove variable export from `create-devenv.sh` Would be useless anyway as the env vars are not exported back in the calling shell or across Bash tool calls --- .agents/skills/follow-pr/SKILL.md | 4 ++-- .agents/skills/follow-pr/create_devenv.sh | 1 - 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md index c4015605617e..b94b83ee66ed 100644 --- a/.agents/skills/follow-pr/SKILL.md +++ b/.agents/skills/follow-pr/SKILL.md @@ -23,12 +23,12 @@ Otherwise, check for the existence of a dev env by using `dda env dev show`. - Check the current state of that dev env. If the environment is already started and contains the right repo, move to [the next step](#using-a-dev-env). -Otherwise, create one by using `./create_devenv.sh`, the ID will be automatically exported as `ATTACH_DEVENV_ID` +Otherwise, create one by using `./create_devenv.sh`, then use the environment ID printed by the script in subsequent commands. ### Using a dev env To run commands inside a dev env, use the following template: ```bash -dda env dev run --id ${ATTACH_DEVENV_ID} -- [command] +dda env dev run --id -- [command] ``` Watch out for space-splitting. For example: ```bash diff --git a/.agents/skills/follow-pr/create_devenv.sh b/.agents/skills/follow-pr/create_devenv.sh index 0fbafe12c0c5..d4cdd2610cac 100755 --- a/.agents/skills/follow-pr/create_devenv.sh +++ b/.agents/skills/follow-pr/create_devenv.sh @@ -14,4 +14,3 @@ ID=babysit-pr-attach-$(uuidgen | cut -d'-' -f1) dda env dev start --no-pull --repo "${REPO}" --id "${ID}" echo "Created env: ${ID}" -export ATTACH_DEVENV_ID="${ID}" From 72c09e62c32d8893cf6eae2d01fe432b9a76bec9 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 14:28:41 +0200 Subject: [PATCH 6/9] fix: final missing `babysit->follow` rename --- .agents/skills/follow-pr/create_devenv.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/follow-pr/create_devenv.sh b/.agents/skills/follow-pr/create_devenv.sh index d4cdd2610cac..a9a25417d166 100755 --- a/.agents/skills/follow-pr/create_devenv.sh +++ b/.agents/skills/follow-pr/create_devenv.sh @@ -6,7 +6,7 @@ set -e pipefail REPO=$(git rev-parse --show-toplevel) # Create a unique ID -ID=babysit-pr-attach-$(uuidgen | cut -d'-' -f1) +ID=follow-pr-attach-$(uuidgen | cut -d'-' -f1) # Use --no-pull to make sure it starts quickly # Explicitly pass the repo to avoid cloning or missing the automatic bind-mount. Note it will be mounted as `/repos/the/full/absolute/path` inside the container From 4ac126416fbeeecfb3f6bad14a34f45bbde3f833 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 14:29:02 +0200 Subject: [PATCH 7/9] fix: explicitely specify retry time --- .agents/skills/follow-pr/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md index b94b83ee66ed..603519845273 100644 --- a/.agents/skills/follow-pr/SKILL.md +++ b/.agents/skills/follow-pr/SKILL.md @@ -66,7 +66,7 @@ If the `[FINAL]` line reports a timeout (not a pipeline outcome), start an ident This is safe: `attach` is stateless and each invocation begins with a fresh snapshot of the pipeline. > NOTE: If the pipeline is already terminal or does not exist when you start monitoring, the user might have just pushed and the pipeline is still waiting to be created. -> In this case, wait for a minute or two and then re-attempt monitoring. The `--follow` argument will make sure `ddgl attach` always monitors the latest pipeline for the ref. +> In this case, wait for 60 seconds and then re-attempt monitoring. The `--follow` argument will make sure `ddgl attach` always monitors the latest pipeline for the ref. ## Step 3: Interpret the output From fd4e8132983b07a83375d8010a75f1bf6a88ed20 Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 14:33:10 +0200 Subject: [PATCH 8/9] feat: give a bit more detail to `create-devenv.sh`'s final `echo` --- .agents/skills/follow-pr/create_devenv.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.agents/skills/follow-pr/create_devenv.sh b/.agents/skills/follow-pr/create_devenv.sh index a9a25417d166..3d84113717b5 100755 --- a/.agents/skills/follow-pr/create_devenv.sh +++ b/.agents/skills/follow-pr/create_devenv.sh @@ -13,4 +13,4 @@ ID=follow-pr-attach-$(uuidgen | cut -d'-' -f1) # But the CWD is changed appropriately when doing this way, so it's fine dda env dev start --no-pull --repo "${REPO}" --id "${ID}" -echo "Created env: ${ID}" +echo "Created env to use for the PR attach task: ${ID}" From 8d01ea83bd70687640daa9b6002b7feb6760035a Mon Sep 17 00:00:00 2001 From: Pierre-Louis Veyrenc Date: Fri, 7 Aug 2026 17:01:09 +0200 Subject: [PATCH 9/9] feat: add guidance for dealing with flaky failures --- .agents/skills/follow-pr/SKILL.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.agents/skills/follow-pr/SKILL.md b/.agents/skills/follow-pr/SKILL.md index 603519845273..a240ac27768a 100644 --- a/.agents/skills/follow-pr/SKILL.md +++ b/.agents/skills/follow-pr/SKILL.md @@ -83,7 +83,10 @@ You may see: ## Step 4: Act on the outcome - **Pipeline Success:** stop monitoring and report the pipeline succeeded. -- **Some job failed, but the pipeline is still running**: Ask the user whether to continue monitoring, or if this job failure is already a problem. If it is, move to [Step 5](#step-5-follow-up-on-failures) +- **Some job failed, but the pipeline is still running**: + Note that most jobs on `datadog-agent` CI have at least one retry for combating flakiness - especially jobs running e2e tests, kmt etc. + Unit test, linter or build failures are less likely to be flakes. If you think it is likely the job's failure is just a flake, continue monitoring - gitlab will retry the job once automatically. + Otherwise, ask the user whether to continue monitoring, or if this job failure is already a problem. In the latter case, move to [Step 5](#step-5-follow-up-on-failures) - **Pipeline failed or canceled:** Stop monitoring, report the status, and move to [Step 5](#step-5-follow-up-on-failures). - **Timeout `[FINAL]`:** re-invoke `ddgl attach` as in Step 2; this is not a true terminal outcome. - **Unexpected error** (from `ddgl` itself, or from the monitoring tool): report what happened. Do not attempt a recovery action.