Remove content app redirect for manifest fetching#2358
Draft
gerrod3 wants to merge 1 commit into
Draft
Conversation
a173a87 to
848ee41
Compare
fixes: pulp#1974 Assisted by: claude-sonnet-4.6
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1974
https://redhat.atlassian.net/browse/PULP-486
To properly remove the redirect for manifest fetching I had to remove the PullThroughCaching redirect logic as well. Now the entirety of PullThroughCaching lives on the registry_api side, no more content app logic needed. These are the three scenarios we were handling for pull-through caching:
Upon first fetch you need to create the tag and download the manifest and initiate a task to download the rest of the image in the background (
tag=None, local_manifest=None). If the upstream manifest already exists on Pulp you simply need to create the tag and add it and the image's content to the repo (tag=None, local_manifest is not None). On the second fetch of a tag with pull-through you need to check to see if the tag has been updated on the remote. There are three scenarios for this: one - new upstream manifest that needs to be downloaded to Pulp (tag is not None, local_manifest=None), two - new upstream manifest already in Pulp needs to be added to repo (tag is not None, local_manifest is not None), and three - no changes on upstream (tag is not None, tag.manifest.digest == local_manifest.digest). If the tag doesn't exist on the remote, but we have the tag locally then we should serve our local copy (tag is not None, fetch_manifest raised ApiException).When fetching a manifest by digest we either have it or we don't. If we don't, we check the upstream if it's there. If yes, then we check if the manifest is in Pulp locally and just needs to be added to the repo, or if we need to download the whole image in the background.
When fetching a blob and we don't have it in the repository we check to see if the remote has the blob. If so, we create the blob as an on-demand content and redirect to the content-app for it to finish streaming the blob from the upstream. We also add it to the
pending_blobsto speed up future requests.📜 Checklist
See: Pull Request Walkthrough