Paginate Docker tag listing and classify registry error responses

[robaiken]

Summary

Docker/docker_compose update jobs fail with unknown_error for images whose registry can't return the full tag list in one response. The clearest case is images with very large tag counts — e.g. hexpm/elixir has ~993k tags, so Docker Hub's GET /v2/hexpm/elixir/tags/list (requested unpaginated) times out with an HTTP 504 after 30s on every run.

Root cause: Dependabot::Docker::UpdateChecker#tags_from_registry requests the tag list without a page size, and the docker_registry2 gem wraps the resulting 504 in DockerRegistry2::RegistryHTTPException, which wasn't rescued — so it surfaced as unknown_error. Separately, a 403 raises RegistryAuthorizationException (sibling of the already-handled RegistryAuthenticationException), which was also unclassified.

Changes

• Paginate on demand. tags_from_registry makes a single unpaginated request for the common case, and falls back to a paginated request (?n=100) when the registry returns an HTTP error trying to serve the full list, following the registry's pagination links. This keeps the efficient single call for normal images while letting large-but-servable images succeed via pagination.
• Classify registry HTTP errors. RegistryHTTPException (incl. 504) now raises RegistryError carrying the HTTP status, instead of unknown_error. RegistryError is registry-neutral, so a public registry like Docker Hub is not mislabelled as a private source.
• Classify 403. RegistryAuthorizationException now raises PrivateSourceAuthenticationFailure alongside the existing 401 handling, at every registry-touching rescue site.

This also fixes the docker_compose ecosystem, which reuses Dependabot::Docker::UpdateChecker.

Tests

• 504 → paginated fallback resolves the latest version (asserts the ?n=100 request is made).
• Persistent 504 → RegistryError with status == 504.
• 403 → PrivateSourceAuthenticationFailure.

Full docker suite: 573 examples, 0 failures; RuboCop clean; Sorbet No errors!.

Verified with a dry-run against the real failing repo

docker_compose in /docker-compose/hexpm (image hexpm/elixir):

Before

=> checking for updates 1/1
An error occurred: DockerRegistry2::RegistryHTTPException, Registry request failed with status 504

After

=> checking for updates 1/1
=> handled error whilst updating hexpm/elixir: registry_error {status: 504, msg: "Registry request failed with status 504"}
Dry-run completed successfully.

Note on hexpm/elixir specifically

Pagination works for normal images (verified: library/alpine returns a bounded page fine). For hexpm/elixir (~1M tags), Docker Hub returns 504 even for a bounded ?n=100 page, so pagination can't rescue this specific image — but the failure is now a graceful, classified registry_error (with the HTTP status) instead of an opaque unknown_error. Pagination still benefits the broad middle ground of large-but-servable images that currently 504 only on the unpaginated request.

[Copilot]

Pull request overview

Adds resilient Docker registry tag retrieval and clearer registry error classification.

Changes:

• Falls back to paginated tag retrieval after registry HTTP failures.
• Maps 403 and persistent HTTP failures to appropriate Dependabot errors.
• Adds coverage for pagination fallback and error classification.

Show a summary per file

File	Description
docker/lib/dependabot/docker/update_checker.rb	Implements pagination fallback and registry exception handling.
docker/spec/dependabot/docker/update_checker_spec.rb	Tests 403 and 504 behavior.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (1)

docker/lib/dependabot/docker/update_checker.rb:765

• In docker_registry2 1.19, HTTP 429 is also wrapped as RegistryHTTPException, so this rescue immediately sends another request with ?n=100 while the registry is rate-limiting us. Pagination cannot resolve throttling and the extra request can prolong it; re-raise 429 so the outer handler classifies it without making another call.

      rescue DockerRegistry2::RegistryHTTPException
        raise if page_size

        fetch_tags_from_registry(page_size: TAGS_PAGE_SIZE)

• Files reviewed: 3/5 changed files
• Comments generated: 2
• Review effort level: Medium

[robaiken]

Addressed the suppressed 429 concern in 3579b14 as well: tag-list HTTP 429 responses are re-raised for immediate RegistryError classification without issuing a paginated follow-up request, and the spec verifies only one request is made.

[Copilot]

Review details

Comments suppressed due to low confidence (3)

docker/lib/dependabot/docker/update_checker.rb:502

• With docker_registry2 1.19, digest/dohead wrap non-special HTTP failures such as 500, 502, 503, and 504 in RegistryHTTPException. This best-effort cooldown path does not rescue that class, so those responses now escape as unknown_error instead of being logged and skipped as they were when the transient RestClient exceptions were caught. Include the wrapper here; status-aware retrying for transient 5xx responses can be handled separately.

             DockerRegistry2::RegistryAuthorizationException,

docker/lib/dependabot/docker/update_checker.rb:1180

• The 1.19 client also raises RegistryHTTPException from the manifest and config-blob requests used by this timestamp fallback. Because this rescue omits it, a registry 5xx now aborts version/date validation instead of following this method's documented fail-open behavior and returning nil. Rescue the wrapper alongside the other registry failures.

             DockerRegistry2::RegistryAuthorizationException,

docker/lib/dependabot/docker/update_checker.rb:1587

• Per-platform manifest/config requests can now raise RegistryHTTPException for HTTP failures, but that wrapper is not caught here. A 5xx on any one platform therefore escapes and fails the update instead of using the intended empty-hash fallback. Rescue the HTTP wrapper with the other registry failures.

             DockerRegistry2::RegistryAuthorizationException,

• Files reviewed: 3/5 changed files
• Comments generated: 1
• Review effort level: Medium

[Copilot]

Review details

Comments suppressed due to low confidence (2)

docker/lib/dependabot/docker/update_checker.rb:504

• docker_registry2 1.19 converts server responses such as 500/502/503/504 to RegistryHTTPException, which is not in transient_docker_errors. Either the manifest GET or publication HEAD can therefore escape this fail-open cooldown path as unknown_error instead of logging and skipping cooldown. Include the new HTTP exception here (and add a server-response regression case) to preserve this method's existing failure behavior.

      rescue *transient_docker_errors,
             DockerRegistry2::RegistryAuthenticationException,
             DockerRegistry2::RegistryAuthorizationException,
             RestClient::Forbidden,
             RestClient::TooManyRequests => e

docker/lib/dependabot/docker/update_checker.rb:1588

• A 1.19 server error while fetching any per-platform manifest/blob is now RegistryHTTPException, so it bypasses this fail-open rescue and aborts timestamp validation as unknown_error. This path previously handled 500/502/503 through the RestClient classes in transient_docker_errors; include the new exception so failed platform metadata still degrades to {}.

      rescue *transient_docker_errors, DockerRegistry2::RegistryAuthenticationException,
             DockerRegistry2::RegistryAuthorizationException,
             RestClient::Forbidden, JSON::ParserError => e

• Files reviewed: 3/5 changed files
• Comments generated: 0 new
• Review effort level: Medium