Skip to content

chore: util to clean cached images#259335

Merged
TamerlanG merged 4 commits intoelastic:mainfrom
TamerlanG:fix/clean-cache-docker-images
Mar 24, 2026
Merged

chore: util to clean cached images#259335
TamerlanG merged 4 commits intoelastic:mainfrom
TamerlanG:fix/clean-cache-docker-images

Conversation

@TamerlanG
Copy link
Copy Markdown
Contributor

@TamerlanG TamerlanG commented Mar 24, 2026

Summary

Our VMs ship with a bunch of cached Docker images that waste disk space on jobs that don't need them. Added clean_cached_images to clear them out before those jobs run.

@TamerlanG TamerlanG self-assigned this Mar 24, 2026
@TamerlanG TamerlanG requested a review from a team as a code owner March 24, 2026 12:22
@TamerlanG TamerlanG added Team:Operations Kibana-Operations Team release_note:skip Skip the PR/issue when compiling release notes backport:all-open Backport to all branches that could still receive a release labels Mar 24, 2026
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-operations (Team:Operations)

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 24, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 43b55bab-b2a5-448e-9cc8-90b4f336e555

📥 Commits

Reviewing files that changed from the base of the PR and between d4f2e6c and 05db282.

📒 Files selected for processing (1)
  • .buildkite/scripts/common/util.sh

📝 Walkthrough

Walkthrough

A new clean_cached_images() Bash function was added to .buildkite/scripts/common/util.sh; it collects unique Docker image IDs and performs best-effort removals (docker rmi -f via xargs -r, ignoring failures) and then runs docker image prune -af (also ignoring failures). Inline cached-image cleanup blocks in .buildkite/scripts/steps/artifacts/docker_image.sh and .buildkite/scripts/steps/package_testing/test.sh were replaced with calls to this new function.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • 🛠️ Update Documentation: Commit on current branch
  • 🛠️ Update Documentation: Create PR

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.buildkite/scripts/common/util.sh:
- Around line 216-218: The clean_cached_images function uses an invalid grep
invocation and therefore never removes images; update it to safely pass image
IDs to docker rmi (use command substitution piped to xargs with a no-op-if-empty
flag such as xargs -r or xargs --no-run-if-empty and run docker rmi -f) and
change docker image prune -f to docker image prune -af to include all unused
images; modify the body of clean_cached_images to use the image-ID pipeline with
xargs and the -a prune flag so the function reliably deletes cached images.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: aa418f80-f106-4110-a9f6-e7a98f75e702

📥 Commits

Reviewing files that changed from the base of the PR and between 62a71dc and bf4ca48.

📒 Files selected for processing (3)
  • .buildkite/scripts/common/util.sh
  • .buildkite/scripts/steps/artifacts/docker_image.sh
  • .buildkite/scripts/steps/package_testing/test.sh

@delanni delanni self-requested a review March 24, 2026 12:43
@TamerlanG TamerlanG enabled auto-merge (squash) March 24, 2026 12:49
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
.buildkite/scripts/common/util.sh (1)

220-220: ⚠️ Potential issue | 🔴 Critical

Remove stray closing brace — syntax error.

This unmatched } breaks shell parsing; any script sourcing util.sh will fail to load.

,

Proposed fix
 clean_cached_images() {
   docker images -q | sort -u | xargs -r docker rmi || true
   docker image prune -af || true
 }
-}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.buildkite/scripts/common/util.sh at line 220, There is an unmatched closing
brace ('}') at the end of the script that causes a syntax error; remove this
stray '}' so all function/block braces are balanced (verify functions referenced
in the file are closed correctly), then re-run shellcheck or source the script
to confirm no parsing errors remain.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.buildkite/scripts/common/util.sh:
- Line 220: There is an unmatched closing brace ('}') at the end of the script
that causes a syntax error; remove this stray '}' so all function/block braces
are balanced (verify functions referenced in the file are closed correctly),
then re-run shellcheck or source the script to confirm no parsing errors remain.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 96883517-45f2-4173-afc9-58ca83447a9c

📥 Commits

Reviewing files that changed from the base of the PR and between bf4ca48 and d4f2e6c.

📒 Files selected for processing (1)
  • .buildkite/scripts/common/util.sh

TamerlanG and others added 2 commits March 24, 2026 14:17
Co-authored-by: Alex Szabo <delanni.alex@gmail.com>
@delanni
Copy link
Copy Markdown
Member

delanni commented Mar 24, 2026

/ci

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Mar 24, 2026

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: 05db282
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-259335-05db282afedd

Failed CI Steps

Metrics [docs]

✅ unchanged

History

cc @TamerlanG

@TamerlanG TamerlanG merged commit 5dac63e into elastic:main Mar 24, 2026
17 checks passed
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.19, 9.2, 9.3

https://github.com/elastic/kibana/actions/runs/23495342064

@kibanamachine
Copy link
Copy Markdown
Contributor

💔 All backports failed

Status Branch Result
8.19 Backport failed because of merge conflicts
9.2 Backport failed because of merge conflicts
9.3 Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 259335

Questions ?

Please refer to the Backport tool documentation

TamerlanG added a commit to TamerlanG/kibana that referenced this pull request Mar 24, 2026
(cherry picked from commit 5dac63e)

# Conflicts:
#	.buildkite/scripts/common/util.sh
#	.buildkite/scripts/steps/artifacts/docker_image.sh
TamerlanG added a commit to TamerlanG/kibana that referenced this pull request Mar 24, 2026
(cherry picked from commit 5dac63e)

# Conflicts:
#	.buildkite/scripts/common/util.sh
#	.buildkite/scripts/steps/artifacts/docker_image.sh
TamerlanG added a commit to TamerlanG/kibana that referenced this pull request Mar 24, 2026
(cherry picked from commit 5dac63e)

# Conflicts:
#	.buildkite/scripts/common/util.sh
#	.buildkite/scripts/steps/artifacts/docker_image.sh
@TamerlanG
Copy link
Copy Markdown
Contributor Author

💚 All backports created successfully

Status Branch Result
9.3
9.2
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

jbudz added a commit that referenced this pull request Mar 24, 2026
mbondyra added a commit to mbondyra/kibana that referenced this pull request Mar 24, 2026
…ra/kibana into dashboard_align_attachment_to_api

* 'dashboard_align_attachment_to_api' of github.com:mbondyra/kibana: (45 commits)
  [OTel Tracing] HTTP instrumentation (elastic#258663)
  Replace deprecated EUI icons in files owned by @elastic/ml-ui (elastic#255624)
  [Codeowners] add missing codeowners for security_solution_api_integration tests (elastic#259223)
  [CI] fix bad imports that came from a merge-race (elastic#259383)
  Add `.claude/worktrees/` to `.gitignore` (elastic#259192)
  Improve unknown-key validation error message in @kbn/config-schema (elastic#258633)
  [ML] Update Security ML jobs to use entity analytics fields for host and user fields (elastic#255339)
  [Table sweep] Update table columns responsiveness in Index Management and Dashboards (elastic#259340)
  skip failing test suite (elastic#258790)
  skip failing test suite (elastic#259261)
  chore: util to clean cached images (elastic#259335)
  [Entity Store] Use last_seen for automated resolution watermark (elastic#258574)
  [One Workflow] Fix flaky alert trigger Scout test by removing order-dependent assertions (elastic#259299)
  Skip serverless Discover request counts tests for MKI (elastic#259333)
  [Security Solution] render header title in new document flyout in Security Solution and Discover (elastic#258166)
  [Agent Builder] register inference endpoint feature (elastic#259259)
  [Agent Builder] Skills Command Menu - Add descriptions and scope options to agent (elastic#258964)
  [Streams][Streamlang][API] Fully use meta({id}) to reuse schema partials in OAS output (elastic#259275)
  fix(files_example): add tableCaption to EuiInMemoryTable for a11y (elastic#258289)
  [Entity Store] Adding list endpoint with query filter (elastic#258320)
  ...
TamerlanG added a commit that referenced this pull request Mar 25, 2026
# Backport

This will backport the following commits from `main` to `8.19`:
- [chore: util to clean cached images
(#259335)](#259335)

<!--- Backport version: 11.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Tamerlan
Gudabayev","email":"37669316+TamerlanG@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-03-24T14:40:24Z","message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:all-open","ci:build-serverless-image","v9.4.0"],"title":"chore:
util to clean cached
images","number":259335,"url":"https://github.com/elastic/kibana/pull/259335","mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/259335","number":259335,"mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}}]}]
BACKPORT-->
TamerlanG added a commit that referenced this pull request Mar 25, 2026
# Backport

This will backport the following commits from `main` to `9.3`:
- [chore: util to clean cached images
(#259335)](#259335)

<!--- Backport version: 11.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Tamerlan
Gudabayev","email":"37669316+TamerlanG@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-03-24T14:40:24Z","message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:all-open","ci:build-serverless-image","v9.4.0"],"title":"chore:
util to clean cached
images","number":259335,"url":"https://github.com/elastic/kibana/pull/259335","mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/259335","number":259335,"mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}}]}]
BACKPORT-->
TamerlanG added a commit that referenced this pull request Mar 25, 2026
# Backport

This will backport the following commits from `main` to `9.2`:
- [chore: util to clean cached images
(#259335)](#259335)

<!--- Backport version: 11.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Tamerlan
Gudabayev","email":"37669316+TamerlanG@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-03-24T14:40:24Z","message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Operations","release_note:skip","backport:all-open","ci:build-serverless-image","v9.4.0"],"title":"chore:
util to clean cached
images","number":259335,"url":"https://github.com/elastic/kibana/pull/259335","mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/259335","number":259335,"mergeCommit":{"message":"chore:
util to clean cached images
(#259335)","sha":"5dac63ee0cf693cf397755da249ff08f8d28cae9"}}]}]
BACKPORT-->
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Mar 26, 2026
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:all-open Backport to all branches that could still receive a release ci:build-serverless-image release_note:skip Skip the PR/issue when compiling release notes Team:Operations Kibana-Operations Team v8.19.14 v9.2.8 v9.3.3 v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants