Skip to content

[CI] Add preferCached for docker-startup commands#244766

Merged
delanni merged 3 commits into
elastic:mainfrom
delanni:skip-epr-pull-if-preferCached-is-true
Dec 4, 2025
Merged

[CI] Add preferCached for docker-startup commands#244766
delanni merged 3 commits into
elastic:mainfrom
delanni:skip-epr-pull-if-preferCached-is-true

Conversation

@delanni
Copy link
Copy Markdown
Member

@delanni delanni commented Dec 1, 2025

Summary

Add preferCached as an option in the test util for starting docker images, turn it on for all EPR image pulls, in order to prevent costly docker pulls in FTRs/integration tests.

After #244517 we still have a small temporal zone in between an agent being provisioned and the tests being ran, where a promotion would trigger a docker pull, that depletes disk space, it also would make pulls slow, and potentially time out tests.

This PR suggests, we should always use the cached variant (for EPR at least). With this, the above risk zone is no longer causing pulls, it's just that we are still testing the previously promoted and cached image, probably not risky.

Closes: #244476

@delanni delanni added Team:Operations Kibana-Operations Team release_note:skip Skip the PR/issue when compiling release notes backport:all-open Backport to all branches that could still receive a release labels Dec 1, 2025
@elastic elastic deleted a comment from elasticmachine Dec 2, 2025
@delanni delanni marked this pull request as ready for review December 2, 2025 15:10
@delanni delanni requested review from a team as code owners December 2, 2025 15:10
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@delanni delanni requested review from joeypoon and tomsonpl December 2, 2025 15:10
@botelastic botelastic Bot added ci:project-deploy-observability Create an Observability project Team:Fleet Team label for Observability Data Collection Fleet team Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. labels Dec 2, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Dec 2, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@botelastic botelastic Bot added the Team:obs-presentation Focus: APM UI, Infra UI, Hosts UI, Universal Profiling, Obs Overview and left Navigation label Dec 3, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/obs-presentation-team (Team:obs-presentation)

@delanni delanni removed the ci:project-deploy-observability Create an Observability project label Dec 3, 2025
@delanni
Copy link
Copy Markdown
Member Author

delanni commented Dec 3, 2025

/ci

@elasticmachine
Copy link
Copy Markdown
Contributor

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/test 313 314 +1
Unknown metric groups

API count

id before after diff
@kbn/test 370 371 +1

History

@botelastic botelastic Bot added the ci:project-deploy-observability Create an Observability project label Dec 4, 2025
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Dec 4, 2025

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@rmyz rmyz removed the Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. label Dec 4, 2025
@delanni delanni merged commit 5336f0d into elastic:main Dec 4, 2025
18 checks passed
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.19, 9.1, 9.2

https://github.com/elastic/kibana/actions/runs/19924800677

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 4, 2025
## Summary
Add `preferCached` as an option in the test util for starting docker
images, turn it on for all EPR image pulls, in order to prevent costly
docker pulls in FTRs/integration tests.

After elastic#244517 we still have a small temporal zone in between an agent
being provisioned and the tests being ran, where a promotion would
trigger a docker pull, that depletes disk space, it also would make
pulls slow, and potentially time out tests.

This PR suggests, we should always use the cached variant (for EPR at
least). With this, the above risk zone is no longer causing pulls, it's
just that we are still testing the previously promoted and cached image,
probably not risky.

(cherry picked from commit 5336f0d)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 4, 2025
## Summary
Add `preferCached` as an option in the test util for starting docker
images, turn it on for all EPR image pulls, in order to prevent costly
docker pulls in FTRs/integration tests.

After elastic#244517 we still have a small temporal zone in between an agent
being provisioned and the tests being ran, where a promotion would
trigger a docker pull, that depletes disk space, it also would make
pulls slow, and potentially time out tests.

This PR suggests, we should always use the cached variant (for EPR at
least). With this, the above risk zone is no longer causing pulls, it's
just that we are still testing the previously promoted and cached image,
probably not risky.

(cherry picked from commit 5336f0d)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Dec 4, 2025
## Summary
Add `preferCached` as an option in the test util for starting docker
images, turn it on for all EPR image pulls, in order to prevent costly
docker pulls in FTRs/integration tests.

After elastic#244517 we still have a small temporal zone in between an agent
being provisioned and the tests being ran, where a promotion would
trigger a docker pull, that depletes disk space, it also would make
pulls slow, and potentially time out tests.

This PR suggests, we should always use the cached variant (for EPR at
least). With this, the above risk zone is no longer causing pulls, it's
just that we are still testing the previously promoted and cached image,
probably not risky.

(cherry picked from commit 5336f0d)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.19
9.1
9.2

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Dec 4, 2025
…245213)

# Backport

This will backport the following commits from `main` to `9.2`:
- [[CI] Add `preferCached` for docker-startup commands
(#244766)](#244766)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"alex.szabo@elastic.co"},"sourceCommit":{"committedDate":"2025-12-04T09:54:11Z","message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","Team:Fleet","backport:all-open","ci:project-deploy-observability","v9.3.0","Team:obs-presentation"],"title":"[CI]
Add `preferCached` for docker-startup
commands","number":244766,"url":"https://github.com/elastic/kibana/pull/244766","mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/244766","number":244766,"mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}}]}]
BACKPORT-->

Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
kibanamachine added a commit that referenced this pull request Dec 4, 2025
…245212)

# Backport

This will backport the following commits from `main` to `9.1`:
- [[CI] Add `preferCached` for docker-startup commands
(#244766)](#244766)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"alex.szabo@elastic.co"},"sourceCommit":{"committedDate":"2025-12-04T09:54:11Z","message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","Team:Fleet","backport:all-open","ci:project-deploy-observability","v9.3.0","Team:obs-presentation"],"title":"[CI]
Add `preferCached` for docker-startup
commands","number":244766,"url":"https://github.com/elastic/kibana/pull/244766","mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/244766","number":244766,"mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}}]}]
BACKPORT-->

Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
kibanamachine added a commit that referenced this pull request Dec 4, 2025
…245211)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[CI] Add `preferCached` for docker-startup commands
(#244766)](#244766)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Alex
Szabo","email":"alex.szabo@elastic.co"},"sourceCommit":{"committedDate":"2025-12-04T09:54:11Z","message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507","branchLabelMapping":{"^v9.3.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","Team:Fleet","backport:all-open","ci:project-deploy-observability","v9.3.0","Team:obs-presentation"],"title":"[CI]
Add `preferCached` for docker-startup
commands","number":244766,"url":"https://github.com/elastic/kibana/pull/244766","mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.3.0","branchLabelMappingKey":"^v9.3.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/244766","number":244766,"mergeCommit":{"message":"[CI]
Add `preferCached` for docker-startup commands (#244766)\n\n##
Summary\nAdd `preferCached` as an option in the test util for starting
docker\nimages, turn it on for all EPR image pulls, in order to prevent
costly\ndocker pulls in FTRs/integration tests.\n\nAfter #244517 we
still have a small temporal zone in between an agent\nbeing provisioned
and the tests being ran, where a promotion would\ntrigger a docker pull,
that depletes disk space, it also would make\npulls slow, and
potentially time out tests.\n\nThis PR suggests, we should always use
the cached variant (for EPR at\nleast). With this, the above risk zone
is no longer causing pulls, it's\njust that we are still testing the
previously promoted and cached image,\nprobably not
risky.","sha":"5336f0d72e870adfed881b0720cfbeb84e261507"}}]}]
BACKPORT-->

Co-authored-by: Alex Szabo <alex.szabo@elastic.co>
@delanni delanni deleted the skip-epr-pull-if-preferCached-is-true branch December 8, 2025 13:24
JordanSh pushed a commit to JordanSh/kibana that referenced this pull request Dec 9, 2025
## Summary
Add `preferCached` as an option in the test util for starting docker
images, turn it on for all EPR image pulls, in order to prevent costly
docker pulls in FTRs/integration tests.

After elastic#244517 we still have a small temporal zone in between an agent
being provisioned and the tests being ran, where a promotion would
trigger a docker pull, that depletes disk space, it also would make
pulls slow, and potentially time out tests.

This PR suggests, we should always use the cached variant (for EPR at
least). With this, the above risk zone is no longer causing pulls, it's
just that we are still testing the previously promoted and cached image,
probably not risky.
delanni added a commit that referenced this pull request Dec 10, 2025
## Summary
Edit on #244766 / #245211

There were a few configs that didn't exist on main where this config was
introduced, so they were pulling the docker image every time, thus
exhausting disk space.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:all-open Backport to all branches that could still receive a release ci:project-deploy-observability Create an Observability project release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team Team:obs-presentation Focus: APM UI, Infra UI, Hosts UI, Universal Profiling, Obs Overview and left Navigation Team:Operations Kibana-Operations Team v8.19.9 v9.1.9 v9.2.3 v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Two-step EPR package promotion to prevent image pulls

9 participants