[Backport 7.78.x] Use dd-pkg for public image publication (BARX-1697) - #51945
Merged
Conversation
Stop installing `dd-pkg` at runtime in Agent CI. - `.log_dd_pkg_version` logs the preinstalled `dd-pkg` version before use, so jobs fail fast if the build image does not provide the CLI. - Removes `DD_PKG_VERSION` and the per-job `DD_PKG_ARCH` variables that only existed to download the runtime tarball. - Keeps existing `dd-pkg` upload, lint, sign, and promote usage unchanged. ```mermaid flowchart LR Old[CI job starts] --> Download[curl dd-pkg tarball from S3] Download --> Run[dd-pkg upload/sign/promote] New[CI job starts with updated build image] --> Version[dd-pkg version] Version --> Run ``` `dd-pkg` should be part of the build images instead of being downloaded during every CI job. The required buildimage changes are now merged: - Buildimages change: DataDog/datadog-agent-buildimages#1176 - Agent buildimage bump: #51283 Jira: https://datadoghq.atlassian.net/browse/BARX-1747 - `git diff --check` - `dda inv linter.gitlab-ci --test=all` - `dda inv invoke-unit-tests.run --tests gitlab_api --directory tasks/unit_tests` - Pre-push `gitlab-configuration` hook passed while pushing the branch. This intentionally keeps `DD_PKG_GITLAB_URL` and all actual `dd-pkg` command usage. Only the runtime installation path was removed. Co-authored-by: ali.benabdallah <ali.benabdallah@datadoghq.com> (cherry picked from commit 2e067e5)
DataDog/datadog-agent-buildimages#1184 published build images at `v116490909-fc35a1a5`. Agent needs to consume that tag before the stacked dd-pkg public-images rollout. - Replace `v114725131-b7fd2a07` with `v116490909-fc35a1a5` for existing CI image pins in `.gitlab-ci.yml`. - This updates `CI_IMAGE_BTF_GEN`, `CI_IMAGE_DOCKER_X64`, `CI_IMAGE_DOCKER_ARM64`, `CI_IMAGE_GITLAB_AGENT_DEPLOY`, `CI_IMAGE_LINUX`, `CI_IMAGE_RPM_X64`, `CI_IMAGE_RPM_ARM64`, and `CI_IMAGE_RPM_ARMHF`. - Keep suffixes empty. - Leave Windows pins unchanged because they were on `v114471858-795a98da`, not `v114725131-b7fd2a07`. Buildimages changelog: DataDog/datadog-agent-buildimages@b7fd2a0...fc35a1a Jira: BARX-1697 Not rerun per request. Co-authored-by: ali.benabdallah <ali.benabdallah@datadoghq.com> (cherry picked from commit 710221c)
Agent public-image publish jobs should move through `dd-pkg publish-image` so artifact-gateway can own policy and signature handling. During rollout, release engineers need the current direct `public-images` trigger to remain the default until individual paths are enabled.
- Add a `dd-pkg publish-image` path to the shared `.docker_publish_job_definition`.
- Use the Datadog public feature flag `ci-public-images-dd-pkg` to choose between dd-pkg and the legacy direct trigger.
- Default to the legacy direct trigger when the flag is disabled, missing, or lookup fails.
- Rely on `dda self feature` built-in CI scopes, and add `ci.project.name`, `registries`, and `destination_repo` scopes for rollout targeting.
- Log the explicit extra scopes and feature flag result before selecting the publish path.
- Preserve the previous no-signing behavior with `--signing=false` and `IMG_SIGNING=false`.
- Keep the existing 30-minute publish timeout and release suffix handling.
```mermaid
flowchart LR
Job[Agent publish job] --> Flag{ci-public-images-dd-pkg}
Flag -- true/targeted opt-in --> DDPkg[dd-pkg publish-image]
DDPkg --> Gateway[artifact-gateway]
Gateway --> PublicImages[public-images]
Flag -- false/default --> Legacy[direct public-images trigger]
Legacy --> PublicImages
```
Jira: BARX-1697
Use `ci-public-images-dd-pkg` as a Datadog public boolean feature flag. The code default is `false`, so the legacy trigger remains the fallback when the flag is disabled or unavailable.
Feature flag: https://app.datadoghq.com/feature-flags/6bfc0383-6dc0-44e9-8a93-37496ad9d2d4?environmentId=a0e7e064-fec8-42fa-b8e1-728a342d8f2a&envs=a0e7e064-fec8-42fa-b8e1-728a342d8f2a&fromUser=false&refresh_mode=sliding&showSuggestionSuccess=false&start=1780670636049&end=1780674236049&paused=false
Useful targeting scopes:
- `ci.project.name=datadog-agent`
- `ci.job.name=<CI_JOB_NAME>`
- `ci.stage.name=<CI_JOB_STAGE>`
- `git.branch=<CI_COMMIT_BRANCH>`
- `registries=<IMG_REGISTRIES>`
- `destination_repo=agent-dev`
- `git diff --check`
- `dda inv linter.gitlab-ci --input-file .gitlab/distribution.yml --test release`
- `dda inv linter.gitlab-ci --input-file .gitlab-ci.yml --test release`
- pre-commit hooks passed
- pre-push hooks passed, including `gitlab-configuration`
- GitLab deploy test: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/pipelines/116757190
Co-authored-by: ali.benabdallah <ali.benabdallah@datadoghq.com>
(cherry picked from commit 419dbf9)
AliDatadog
added
changelog/no-changelog
Contributor
|
Contributor
Gitlab CI Configuration Changes
|
| Removed | Modified | Added | Renamed |
|---|---|---|---|
| 1 | 202 | 1 | 0 |
Updated: .gitlab/distribution.yml
Changes Summary
| Removed | Modified | Added | Renamed |
|---|---|---|---|
| 1 | 69 | 1 | 0 |
ℹ️ Diff available in the job log.
AliDatadog
marked this pull request as ready for review
June 9, 2026 08:37
🐑 PR Shepherd is maintaining this PRI watch your PR and automatically fix CI failures, rebase your branch, handle flaky tests, and push it to the merge queue when it's ready. More about what I do → Guide To pause me on this PR, add the |
FlorentClarret
approved these changes
Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport #51599 to
7.78.x.This PR also includes prerequisite backports:
Jira:
Conflict notes:
.gitlab/common/...and.gitlab/deploy_containers/...layout.IMG_REGISTRIES: defaulthunk because there is no target job on those branches.Validation:
git diff --checkon every backport branch.YAML.parse_filesyntax parse for every changed YAML file on every backport branch.git log --format='%h %G? %s' -3on every branch.401 Unauthorizedin that older linter/auth path.