Skip to content

fix: fall back to direct provider registry when init via Hermitcrab mirror fails - #939

Open
mvanhorn wants to merge 2 commits into
padok-team:mainfrom
mvanhorn:fix/642-hermitcrab-stale-provider-fallback
Open

fix: fall back to direct provider registry when init via Hermitcrab mirror fails#939
mvanhorn wants to merge 2 commits into
padok-team:mainfrom
mvanhorn:fix/642-hermitcrab-stale-provider-fallback

Conversation

@mvanhorn

Copy link
Copy Markdown

Summary

Burrito cannot fix Hermitcrab's cache, but it can stop a stale mirror from failing runs: make the runner fall back to the direct registry when init through the network mirror fails. In internal/utils/runner/network_mirror.go, add a RemoveNetworkMirrorConfig (delete the generated config.tfrc and unset TF_CLI_CONFIG_FILE) alongside CreateNetworkMirrorConfig. In internal/runner/actions.go, change ExecInit so that when r.config.Hermitcrab.Enabled is true and r.exec.Init(r.workingDir) returns an error, it logs a warning that the Hermitcrab mirror may be stale or unreachable, removes the mirror config, and retries init once directly against the upstream registry (returning the retry's error if that also fails).

Why this matters

On burrito 0.7.0 with the default Hermitcrab provider-cache setup, runners fail terraform init with "Failed to query available provider packages ... the previously-selected version X is no longer available" when a provider version released after the Hermitcrab pod started is required. The reporter confirmed Hermitcrab's cached version index for hashicorp/consul was missing 2.22.0 and that restarting Hermitcrab repopulated it. A maintainer (corrieriluca) traced the staleness to Hermitcrab's version-list caching (it refreshes via an If-Modified-Since request that never invalidates the cache), so the mirror can serve an index that lacks recently released versions indefinitely.

See #642.

Testing

  • Happy path: Hermitcrab enabled, first init succeeds - no fallback triggered, mirror config left in place. - Fallback path: Hermitcrab enabled, first init fails - config.tfrc is removed, TF_CLI_CONFIG_FILE is unset, init is retried once and its result returned. - Both fail: first init and direct-registry retry fail - error is propagated (run still fails, with a clear log about the mirror). - Hermitcrab disabled: init failure is returned immediately with no retry.

Fixes #642

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: 📋 Backlog

Development

Successfully merging this pull request may close these issues.

Hermitcrab not refreshing providers

2 participants