[Bug] Pushing hangs when no upstream connection

[andaaron]

Discussed in #3520

^{Originally posted by mottetm November 7, 2025}
Hi everyone,

We have a registry set up with on-demand sync to our main upstream registry, this works very well for us when we are able to access the upstream registry but our issues start when we loose that connectivity. We need to support delivery of images to these offline registries and would have like to do this without disabling the on-demand sync. However, during the push process, the initial HEAD request hangs when the registry tries to sync with the upstream manifest which is not available, or sometimes, it resolves within a reasonable amout of time, it's a bit random. We were hoping that the request would timeout predictably and that the local data would be used, if available, but that doesn't seem to happen consistently. Is that the expected behavior? There doesn't seem to be any timeout set on the image sync, adding one could be a good solution to the issue. I'm happy to contribute some PR but would love to hear your opinion first.

Best,
Matthieu

[andaaron]

Created an issue, as it seems to be a bug.

[rchincha]

Do we need a predictable timeout for the HEAD request and then error handling for that?

[mottetm]

What about making the timeout configurable? The users know best what their use-case is and if it makes sense to time out quickly or to wait for the upstream response.

[andaaron]

Is this the same issue as in #2505? There is a PR in progress at #3537

[mottetm]

Kind of the opposite. In our case, the call to the upstream registry will never return. But push are still waiting for the call to complete. Which happens after sometime more than 15 mins.

Nevermind, after looking at the PR, this would be most likely be covered since we will be able to set a predictable timeout on the upstream operation. Will have a try and see if it works.

[andaaron]

Hi @mottetm, please let us know how this works for you.

Note I added more timeouts in #3574, which is not released yet, but it is merged on main,