Microsoft.NET.Build.Containers.IntegrationTests are pulling from Docker Hub

[akoeplinger]

Build Information

Build: https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=733130
Build error leg or test failing: Microsoft.NET.Build.Containers.IntegrationTests.DockerRegistryTests.WriteToPrivateBasicRegistry
Pull request: https://github.com/dotnet/sdk.git/pull/42019

We need to stop pulling from upstream Docker Hub since we're hitting rate limits.

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "You have reached your pull rate limit",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=733130
Error message validated: [You have reached your pull rate limit]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/8/2024 7:20:41 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[akoeplinger]

@marcpopMSFT who owns Microsoft.NET.Build.Containers?

[baronfel]

@dotnet/sdk-container-builds-maintainers

[baronfel]

@akoeplinger the image in question is registry:2, which we need/use to validate our logic for communicating with container registries via the container registry APIs. Do you know if this image is mirrored anywhere that we can pull from? It's not currently on mcr.microsoft.com.

[nagilson]

Bumping this thread since it's affecting more PRs

[nagilson]

We don't have a path forward to fix this yet - Aspire moved to using their own ACR for storing the images they use, but we need an engineering-systems-managed solution for our entire stack. There's an issue tracking this request but I can't find it at the moment.

[baronfel]

Could be fixed at a root cause level by dotnet/dnceng#3389

[MichaelSimons]

@akoeplinger the image in question is registry:2, which we need/use to validate our logic for communicating with container registries via the container registry APIs. Do you know if this image is mirrored anywhere that we can pull from? It's not currently on mcr.microsoft.com.

@baronfel - It sounds like the test has this dependency and is what is pulling the image. One possible option to workaround scenarios where tests are pulling images directly from DockerHub would be to create a wrapper buildtools image. The Dockerfile would just be a FROM statement. The buildtools images are stored on MCR and therefore you would be able to get around this rate limiting issue.

[nagilson]

@dotnet/sdk-container-builds-maintainers Do you agree with @MichaelSimons suggestion? This has a pretty large impact on PR failure rates. If this is still an issue going forward a month or two down the line, I will sadly suggest that we just disable these tests

[marcpopMSFT]

@donJoseLuis @MichalPavlik I'm told by baronfel that there may be a solution to this of setting up our own ACR mirror of these so we don't get blocked by docker from pulling too often. He said to talk to @eerhardt about that. Is that something you can potentially drive getting set up?

As you can see above, this fails a couple of dozen PRs every week that have to be rerun. It's not a crazy blocker but that's high enough to be something we want eyes on to get fixed eventually.