Skip to content

Add cleanup pipeline to remove unusable images#51

Merged
bthomee merged 16 commits intomainfrom
bthomee/cleanup
Sep 30, 2025
Merged

Add cleanup pipeline to remove unusable images#51
bthomee merged 16 commits intomainfrom
bthomee/cleanup

Conversation

@bthomee
Copy link
Contributor

@bthomee bthomee commented Sep 24, 2025

This PR adds a weekly cleanup job to remove untagged, orphaned, ghost, and partial images.

I compared a few actions, and this one has seen the most recent development and offers many options to get rid of images.

Per the description at https://github.com/dataaxiom/ghcr-cleanup-action:

  • delete-untagged = Delete all untagged images
  • delete-ghost-images = Delete multi-architecture images where all underlying platform images are missing
  • delete-partial-images = Delete multi-architecture images where some (but not all) underlying platform images are missing
  • delete-orphaned-images = Delete tagged images which have no parent (e.g. referrers and cosign tags missing their parent)

I tested this workflow first by adding an on: push and enabling the dry-run option in the action, to confirm it was finding the correct images to delete.

@bthomee
Copy link
Contributor Author

bthomee commented Sep 24, 2025

See a dry-run here: https://github.com/XRPLF/ci/actions/runs/17984493390.

Copy link
Collaborator

@mathbunnyru mathbunnyru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a few questions:

  1. Is there a reason why we need to do this at all?

  2. Maybe it would be better to eliminate the creation of something we don't need, instead of trying to cleanup?

@bthomee
Copy link
Contributor Author

bthomee commented Sep 25, 2025

I have a few questions:

  1. Is there a reason why we need to do this at all?
  2. Maybe it would be better to eliminate the creation of something we don't need, instead of trying to cleanup?
  1. If you look at an arbitrary image, e.g. ci/debian-bookworm, you'll see there are ~250 tagged versions and ~3000 untagged versions. I suppose some of the latter are layers that are useful, but from the dry-run it became clear many can be safely deleted.
  2. I don't know how these are created by the pipeline and how to prevent them. Since we're using @Bronek's logic, maybe he knows?

@Bronek
Copy link
Collaborator

Bronek commented Sep 25, 2025

I have a few questions:

  1. Is there a reason why we need to do this at all?
  2. Maybe it would be better to eliminate the creation of something we don't need, instead of trying to cleanup?
  1. If you look at an arbitrary image, e.g. ci/debian-bookworm, you'll see there are ~250 tagged versions and ~3000 untagged versions. I suppose some of the latter are layers that are useful, but from the dry-run it became clear many can be safely deleted.
  2. I don't know how these are created by the pipeline and how to prevent them. Since we're using @Bronek's logic, maybe he knows?

I think there are two sources of untagged images:

  1. old stuff which can be safely removed, e.g. pushed from on: pull_request etc.
  2. temporary gcc build image with debian-gcc which also can be removed after the pipeline which created it has completed

Re. 2 I have a TODO item to replace temporary images with tagged, they take way too much time & CPU to be considered temporary. But that will only work if we have a separate pipeline workflow to create & push them first, and I did not write one yet.

@Bronek Bronek requested a review from mathbunnyru September 29, 2025 09:54
@bthomee
Copy link
Contributor Author

bthomee commented Sep 29, 2025

@mathbunnyru any outstanding concerns or are we good to go ahead for now? We can always revisit if no longer rebuilding the GCC images reduces or stops the growth of untagged images being published.

Copy link
Collaborator

@mathbunnyru mathbunnyru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@bthomee bthomee merged commit 07cfb8d into main Sep 30, 2025
12 checks passed
@bthomee bthomee deleted the bthomee/cleanup branch September 30, 2025 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants