Skip to content

Implement Asset garbage collection #2368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft

Implement Asset garbage collection #2368

wants to merge 2 commits into from

Conversation

mvandenburgh
Copy link
Member

@mvandenburgh mvandenburgh commented Apr 30, 2025

This PR implements asset garbage collection. See #2367 for the design doc/further explanation.

Leaving this as draft for now as it is contingent on consensus for #2367.

TODO:

  • Add tests

@mvandenburgh mvandenburgh added the minor Increment the minor version when merged label Apr 30, 2025
Comment on lines +31 to +42
return Asset.objects.alias(
has_version=Exists(
Version.objects.filter(
assets=OuterRef('id'),
),
)
).filter(
has_version=False,
published=False,
blob__isnull=False, # only delete assets with blobs; zarrs are not supported yet
modified__lt=timezone.now() - ASSET_EXPIRATION_TIME,
)
Copy link
Member

@jjnesbitt jjnesbitt Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any downside to doing the following instead? This achieves the same effect using a JOIN, which is simpler if nothing else.

Suggested change
return Asset.objects.alias(
has_version=Exists(
Version.objects.filter(
assets=OuterRef('id'),
),
)
).filter(
has_version=False,
published=False,
blob__isnull=False, # only delete assets with blobs; zarrs are not supported yet
modified__lt=timezone.now() - ASSET_EXPIRATION_TIME,
)
return Asset.objects.filter(
versions__isnull=True,
published=False,
blob__isnull=False, # only delete assets with blobs; zarrs are not supported yet
modified__lt=timezone.now() - ASSET_EXPIRATION_TIME,
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
minor Increment the minor version when merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants