Skip to content

fix(EC-1778): use go-gather WithTransport options for HTTP transports#3327

Merged
robnester-rh merged 10 commits into
conforma:mainfrom
robnester-rh:EC-1778
Jun 3, 2026
Merged

fix(EC-1778): use go-gather WithTransport options for HTTP transports#3327
robnester-rh merged 10 commits into
conforma:mainfrom
robnester-rh:EC-1778

Conversation

@robnester-rh

Copy link
Copy Markdown
Contributor

Summary

  • Update go-gather dependency from v1.1.0 to v1.2.0
  • Replace removed goci.Transport/ghttp.Transport global mutations with WithTransport functional options on constructed gatherer instances
  • gatherFunc now dispatches OCI/HTTP sources via Matcher before falling back to the registry for git/file sources
  • Update all downloader tests for the new API surface

Test plan

  • go test -race -tags=unit ./internal/downloader/ — all 6 tests pass
  • make test — all unit, integration, and generative tests pass across the CLI
  • Local code review — no critical or important issues
  • CodeRabbit review — 0 findings

resolves: EC-1778

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 1, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: d3a801b5-8dbd-461d-8d57-eee592a6aa78

📥 Commits

Reviewing files that changed from the base of the PR and between 4f665de and e6200bf.

📒 Files selected for processing (1)
  • internal/downloader/downloader.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/downloader/downloader.go

📝 Walkthrough

Walkthrough

The PR refactors the downloader to initialize OCI and HTTP gatherers once as singletons and route downloads by URL scheme instead of always querying the registry. It also refactors transport initialization to build a shared base with optional tracing before creating separate retry-enabled transports. Dependencies are bumped to match.

Changes

Gatherer Singleton Architecture and Transport Routing

Layer / File(s) Summary
Dependency updates
go.mod
github.com/conforma/go-gather bumped to v1.2.0 and github.com/go-git/go-git/v5 bumped to v5.18.0.
Singleton gatherers and scheme-based routing
internal/downloader/downloader.go
net/http alias import added; package-level ociGatherer and httpGatherer singletons introduced; gatherFunc rewritten to initialize once and dispatch by URL scheme (oci:///oci:: to ociGatherer, http:///https:// to httpGatherer if matcher succeeds, else registry fallback); _initialize refactored to build base RoundTripper with optional tracing, then create separate retry-enabled transports for OCI and HTTP gatherers.
Test adaptations for singleton pattern
internal/downloader/downloader_test.go
TestOCITracing and TestHTTPTracing now nil global ociGatherer and httpGatherer on cleanup; TestOCITracing calls _initialize() directly and invokes ociGatherer.Gather(); TestOCIClientConfiguration simplified to verify only _initialize() output; TestHTTPClientConfiguration retains retry assertions while updating cleanup.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: replacing global transport mutations with WithTransport functional options in go-gather, which is the core refactoring across all modified files.
Description check ✅ Passed The description clearly relates to the changeset, detailing dependency updates, API migration, dispatch logic changes, and test updates that correspond to the file-level modifications.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@qodo-for-conforma

Copy link
Copy Markdown

Review Summary by Qodo

Migrate downloader to go-gather v1.2.0 WithTransport API

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• Migrate from removed goci.Transport/ghttp.Transport globals to WithTransport functional
  options
• Update go-gather dependency from v1.1.0 to v1.2.0
• Implement Matcher-based dispatch in gatherFunc for OCI/HTTP sources
• Refactor _initialize to construct gatherer instances with transport stack
• Update all downloader tests to work with new gatherer variable pattern
Diagram
flowchart LR
  A["go-gather v1.2.0"] -->|"WithTransport options"| B["_initialize"]
  B -->|"constructs"| C["ociGatherer"]
  B -->|"constructs"| D["httpGatherer"]
  E["gatherFunc"] -->|"Matcher dispatch"| C
  E -->|"Matcher dispatch"| D
  E -->|"fallback"| F["registry"]
  G["Transport Stack"] -->|"tracing + retry"| B

Loading

Grey Divider

File Changes

1. internal/downloader/downloader.go ✨ Enhancement +28/-10

Replace transport globals with WithTransport gatherers

• Add net_http import alias for stdlib net/http to avoid conflict with internal/http
• Introduce package-level ociGatherer and httpGatherer variables initialized via sync.OnceFunc
• Rewrite _initialize to build shared base transport (with optional tracing), create independent
 retry transports, and construct gatherer instances via WithTransport functional options
• Replace gatherFunc single registry lookup with Matcher-based dispatch: OCI first, HTTP second,
 registry fallback for git/file sources
• Remove all references to removed goci.Transport and ghttp.Transport globals

internal/downloader/downloader.go


2. internal/downloader/downloader_test.go 🧪 Tests +23/-15

Update tests for new gatherer variable pattern

• Update TestOCITracing cleanup to reset ociGatherer and httpGatherer vars instead of removed
 transport globals
• Update TestHTTPTracing cleanup similarly; add direct _initialize() call and
 httpGatherer.Gather() invocation to test HTTP path specifically (avoiding OCI Matcher dispatch on
 localhost)
• Simplify TestOCIClientConfiguration to assert gatherer type only, since OCIGatherer.transport
 is private; transport wiring verified end-to-end by TestOCITracing
• Rewrite TestHTTPClientConfiguration to inspect httpGatherer.Client.Transport instead of
 removed ghttp.Transport global; add cleanup for gatherer vars

internal/downloader/downloader_test.go


3. go.mod Dependencies +2/-2

Update go-gather and go-git dependencies

• Update github.com/conforma/go-gather from v1.1.0 to v1.2.0
• Update github.com/go-git/go-git/v5 from v5.17.1 to v5.18.0

go.mod


View more (3)
4. go.sum Dependencies +4/-4

Update dependency checksums

• Update checksums for github.com/conforma/go-gather v1.2.0
• Update checksums for github.com/go-git/go-git/v5 v5.18.0

go.sum


5. docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md 📝 Documentation +158/-0

Add design specification for WithTransport migration

• Add comprehensive design specification for the WithTransport migration
• Document architecture: package-level gatherer vars with sync.OnceFunc, Matcher-based dispatch,
 registry fallback
• Specify production changes to _initialize and gatherFunc with code examples
• Detail test strategy for each test function, including rationale for TestOCIClientConfiguration
 simplification (private transport field)
• Document import alias conventions and acceptance criteria

docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md


6. docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md 📝 Documentation +532/-0

Add implementation plan with task-by-task guidance

• Add detailed 7-task implementation plan with step-by-step instructions
• Task 1: Update go-gather to v1.2.0 with verification steps
• Tasks 2-6: Rewrite production code and update each test function with exact code replacements and
 test commands
• Task 7: Run full test suite and verify compilation
• Include commit message templates and expected test outcomes for each task

docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md


Grey Divider

Qodo Logo

@qodo-for-conforma

qodo-for-conforma Bot commented Jun 1, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0)

Grey Divider


Action required

1. HTTP scheme routed to OCI ✓ Resolved 🐞 Bug ≡ Correctness
Description
gatherFunc checks ociGatherer.Matcher(source) before httpGatherer.Matcher(source) without
first respecting an explicit http:///https:// scheme, so URLs like localhost/127.0.0.1 can be
routed to the OCI gatherer. This can prevent intended HTTPS file downloads from
localhost/registry-like hosts because they will be handled by OCIGatherer rather than the HTTP
gatherer.
Code

internal/downloader/downloader.go[R55-60]

Relevance

⭐⭐ Medium

No historical review evidence about URL scheme vs OCI/HTTP matcher ordering in downloader routing.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new dispatch in gatherFunc checks OCI before HTTP. The PR’s own tests document that
127.0.0.1 matches the OCI registry pattern and that gatherFunc would therefore select OCI
instead of HTTP, and the design doc explicitly states OCI matcher includes localhost. Since
Download allows https://... sources, this routing affects real CLI downloads for HTTPS
localhost/registry-like hosts.

internal/downloader/downloader.go[52-66]
internal/downloader/downloader_test.go[231-238]
docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md[91-94]
internal/downloader/downloader.go[139-141]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`gatherFunc` dispatches using OCI matcher first, which can match `localhost`/`127.0.0.1` even when the input is an explicit `http(s)://...` URL. This causes explicit HTTP(S) URLs to be handled by the OCI gatherer.

### Issue Context
The design doc claims “no overlap” between matchers, but the test suite had to bypass `gatherFunc` for `127.0.0.1` because it would route to OCI.

### Fix Focus Areas
- internal/downloader/downloader.go[52-66]

### Suggested fix
Add an explicit scheme-based dispatch before the Matcher-based switch, e.g.:
- If `source` starts with `"oci://"` or `"oci::"` -> OCI gatherer
- Else if `source` starts with `"http://"` or `"https://"` -> HTTP gatherer
- Else fall back to current Matcher ordering (OCI first, then HTTP, then registry)

Optionally, update/extend unit coverage so `TestHTTPTracing` can call `gatherFunc(...)` directly once explicit scheme routing is in place.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/downloader/downloader.go`:
- Around line 55-59: The current routing checks ociGatherer.Matcher(source)
before httpGatherer.Matcher(source), which can misroute HTTP URLs (e.g.,
127.0.0.1) to the OCI gatherer; modify the gather routing so HTTP(s) URLs are
matched first—either by placing the httpGatherer.Matcher(source) case before
ociGatherer.Matcher(source) in the switch or by adding an explicit scheme check
(strings.HasPrefix(source,"http://") || strings.HasPrefix(source,"https://"))
that routes to httpGatherer.Gather(ctx, source, destination) before falling back
to ociGatherer.Gather(ctx, source, destination); update the switch in the gather
function where ociGatherer.Matcher and httpGatherer.Matcher are referenced to
implement this change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 99df4c82-897b-45b9-b2b8-021d7cf07f26

📥 Commits

Reviewing files that changed from the base of the PR and between 1026da7 and 357d4c3.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md
  • docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md
  • go.mod
  • internal/downloader/downloader.go
  • internal/downloader/downloader_test.go

Comment thread internal/downloader/downloader.go Outdated
Comment thread internal/downloader/downloader.go Outdated
@codecov

codecov Bot commented Jun 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag Coverage Δ
acceptance 53.49% <78.57%> (-2.12%) ⬇️
generative 17.00% <0.00%> (-0.82%) ⬇️
integration 28.00% <0.00%> (+1.44%) ⬆️
unit 69.16% <85.71%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
internal/downloader/downloader.go 95.74% <100.00%> (+0.87%) ⬆️

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

robnester-rh and others added 8 commits June 2, 2026 12:59
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Resolve three gaps: explicit sync.OnceFunc wiring, concrete
TestOCIClientConfiguration strategy, import alias conventions.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7 tasks: go.mod update, production rewrite, 4 test updates, final
verification. TDD-sequenced with exact code and commands.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace removed goci.Transport/ghttp.Transport global mutations with
constructed gatherer instances using WithTransport functional options.
gatherFunc now dispatches via Matcher (OCI, HTTP) before falling back
to the registry for git/file sources.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace references to removed goci.Transport/ghttp.Transport globals
with gatherer var cleanup. TestOCIClientConfiguration simplified to
type assertion (transport is private). TestHTTPClientConfiguration
inspects httpGatherer.Client.Transport instead of removed global.
TestHTTPTracing calls httpGatherer.Gather directly since 127.0.0.1
now matches the OCI registry pattern in the new dispatch order.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…names

The OCI Matcher matches localhost/127.0.0.1 as known registries, which
misroutes git-over-HTTP sources hosted on localhost in acceptance tests.
Dispatch to custom gatherers only for explicit oci:// or http(s)://
scheme prefixes; bare hostnames fall through to the registry where
git matchers are checked before OCI matchers.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md`:
- Around line 190-208: Update the documented gatherFunc to match the real
implementation in downloader.go: replace the Matcher-based dispatch
(ociGatherer.Matcher / httpGatherer.Matcher) with explicit scheme prefix checks
using strings.HasPrefix for OCI ("oci://" or "oci::") and HTTP ("https://" or
"http://"), and preserve the additional conditional call to httpGatherer.Matcher
inside the HTTP branch before delegating to httpGatherer.Gather; keep the
registry fallback using registry.GetGatherer and its error handling unchanged.

In `@docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md`:
- Around line 72-88: The spec's dispatch snippet and "Matcher ordering
rationale" must be updated to match the implemented gatherFunc logic: describe
that dispatch uses explicit scheme-prefix checks (e.g., "oci://" routed to
ociGatherer) plus a conditional httpGatherer.Matcher guard for "http(s)://"
while leaving bare hostnames (like "quay.io/repo:tag") to fall through to
registry.GetGatherer; replace the Matcher-only examples (ociGatherer.Matcher,
httpGatherer.Matcher) with the actual scheme-check logic, explain why this
prevents bare hostnames from being captured by the OCI gatherer, and update any
ordering rationale to reflect this implementation detail.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 67b02627-4777-4cbf-9d91-0426a8aba259

📥 Commits

Reviewing files that changed from the base of the PR and between a3ec50f and 4f665de.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (5)
  • docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md
  • docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md
  • go.mod
  • internal/downloader/downloader.go
  • internal/downloader/downloader_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • go.mod
  • internal/downloader/downloader_test.go

Comment thread docs/superpowers/plans/2026-05-29-ec1778-downloader-withtransport.md Outdated
Comment thread docs/superpowers/specs/2026-05-28-ec1778-downloader-withtransport-design.md Outdated
These were working artifacts for the implementation process,
not deliverables.

reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added size: M and removed size: XXL labels Jun 2, 2026
reference: EC-1778

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@fullsend-ai-review

Copy link
Copy Markdown

Review

Findings

Medium

  • [test-inadequate] internal/downloader/downloader_test.go:274TestOCIClientConfiguration coverage was significantly reduced. The previous version verified both the transport type (*retry.Transport) and that MaxRetry was correctly wired through the retry policy. The new version only asserts that ociGatherer is of type *goci.OCIGatherer, which would pass even if WithTransport were omitted entirely. The comment notes the transport field is unexported, but TestHTTPClientConfiguration accesses httpGatherer.Client.Transport — suggesting an asymmetry in the go-gather API. If no public accessor exists for OCIGatherer, consider an integration-level test (e.g., a registry handler that fails then succeeds) to verify retry behavior is actually wired.

Low

  • [edge-case] internal/downloader/downloader.go:56 — The http:// case in the gatherFunc switch is dead code in production. The Download function's isSecure check rejects any source starting with http: before gatherFunc is called. This branch can only be exercised by tests calling gatherFunc directly (e.g., TestHTTPTracing). Not a bug, but could mislead future maintainers into thinking plain HTTP downloads are supported through the normal flow.

Info

  • [sub-agent-failure] N/A — The intent-coherence, style-conventions, and docs-currency sub-agents did not return findings: model unavailable on deployment. These are sonnet-tier dimensions and do not block the review.

@fullsend-ai-review fullsend-ai-review Bot added the requires-manual-review Review requires human judgment label Jun 3, 2026

@joejstuart joejstuart left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@robnester-rh robnester-rh merged commit 7c9c60f into conforma:main Jun 3, 2026
18 checks passed
@robnester-rh robnester-rh deleted the EC-1778 branch June 3, 2026 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-manual-review Review requires human judgment size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants