Auto detection of base image's architecture behaves different with docker-container than with docker driver

[Shaked]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

Hi

I am trying to build the following Dockerfile on a x86 machine:

FROM nvcr.io/nvidia/l4t-base:r32.4.3 as build-detector
RUN echo "test"

The image above is only available for arm64 as you can see on Nvidia's website.

I have created a builder:

docker buildx create  --platform "linux/arm64,linux/amd64" --name mirrored --use

And then I ran a build and inspect its architecture, it seems to be correct:

docker buildx build -t  mystuff/tests:random-test . --load
docker images insepct mystuff/tests:random-test | grep Arch
  "Architecture": "arm64",

When I use --push instead of --load, I see that the pushed image's architecture is now amd64 and the arm64 is not available:

docker pull mystuff/tests:random-test --platform linux/arm64
no matching manifest for linux/arm64 in the manifest list entries

This can be seen in the registry's manifest:

{
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "schemaVersion": 2,
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:..",
      "size": 4089,
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:...",
      "size": 566,
      "annotations": {
        "vnd.docker.reference.digest": "sha256:...",
        "vnd.docker.reference.type": "attestation-manifest"
      },
      "platform": {
        "architecture": "unknown",
        "os": "unknown"
     ...

When I run the same with the default docker driver, it seems to work as expected and I end up with an arm64 image pushed to the registry.

Only when I use a --platform linux/arm64 the docker-container driver pushes the correct image to the registry.

Is this a bug? Am I missing something or maybe it's intentional?

Thank you

Expected behaviour

The docker-container driver should push the correct platform automatically even when --platform is not stated explicitly. The same has with the docker driver.

Actual behaviour

Pushing an arm64-based image to a remote registry using the docker-container driver on a x86 machine ends up as amd64 instead of arm64 like when using the docker driver.

Buildx version

github.com/docker/buildx v0.11.0 687feca

Docker info

Client:
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 1677a17964311325ed1c31e2c0a3589ce6d5c30d
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.15.0-1039-azure
 Operating System: Alpine Linux v3.18 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 6.781GiB
 Name: pmwwhs-w7wph-4tt9g
 ID: 423511ef-3e6f-467f-a746-3cb9a12491be
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 31
  Goroutines: 45
  System Time: 2023-07-01T18:22:22.52115758Z
  EventsListeners: 0
 Username: datalorecustomer
 Experimental: false
 Insecure Registries:
  acr.agents.svc.cluster.local:5000
  gcr.agents.svc.cluster.local:5001
  local.agents.svc.cluster.local:5000
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/

Builders list

mirrored *  docker-container
  mirrored0 tcp://localhost:2375 running v0.11.6                               linux/amd64, linux/arm64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/riscv64, linux/ppc64le, linux/s390x, linux/mips64le, linux/mips64
default     docker
  default   default              running v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Configuration

FROM nvcr.io/nvidia/l4t-base:r32.4.3 as build-detector
RUN echo "test"

Build logs

[+] Building 23.3s (9/9) FINISHED                                                                                                 docker-container:mirrored
 => [internal] booting buildkit                                                                                                                        1.5s
 => => pulling image moby/buildkit:buildx-stable-1                                                                                                     1.0s
 => => creating container buildx_buildkit_mirrored0                                                                                                    0.5s
 => [internal] load build definition from Dockerfile                                                                                                   0.1s
 => => transferring dockerfile: 108B                                                                                                                   0.0s
 => [internal] load .dockerignore                                                                                                                      0.1s
 => => transferring context: 2B                                                                                                                        0.0s
 => [internal] load metadata for nvcr.io/nvidia/l4t-base:r32.4.3                                                                                       2.3s
 => [auth] nvidia/l4t-base:pull,push token for nvcr.io                                                                                                 0.0s
 => [1/2] FROM nvcr.io/nvidia/l4t-base:r32.4.3@sha256:547dc36b81eddb7ca8eadd956c61bd96bf432486830701b3dbb019be7f6c9ce2                                13.6s
 => => resolve nvcr.io/nvidia/l4t-base:r32.4.3@sha256:547dc36b81eddb7ca8eadd956c61bd96bf432486830701b3dbb019be7f6c9ce2                                 0.0s
 => => sha256:7b3abe05cce9a64176cafb4f56d699540ccc12063462de5353e62ac5334dbd67 17.62kB / 17.62kB                                                       0.5s
 => => sha256:c3cf45f768ae5699ac472387cac259bf614f117e7a1fa98560280bfa15c39ea0 278B / 278B                                                             0.5s
 => => sha256:8f6eafd35194e6bb690bdd43dffdd5e5a0ab662dd8ad822748193b8fdc160768 209.78kB / 209.78kB                                                     0.5s
 => => sha256:998db3ceb21fc91aef9d96c6c85f03b5310579c2f5dd0d1018370ca887dff152 497.47kB / 497.47kB                                                     0.6s
 => => sha256:20f4f4b58bf17448f5bbdf8003c7d66b8ccf5553e5b5daed7a292d0903160e19 249.94kB / 249.94kB                                                     0.2s
 => => sha256:801ec5982390ea3fb0f4aa88bba9f4e40964a344697b83108d802d71c3330014 2.86MB / 2.86MB                                                         0.3s
 => => sha256:bc1f730938663ba413550ea0aa048f35f4fdc231016f01fd5355a3b3d5eb6ec8 290.99kB / 290.99kB                                                     0.2s
 => => sha256:da342f04dbe2005756875ee6ddf47770be87d67af6c4bc17444f74e603306d47 15.93MB / 15.93MB                                                       0.4s
 => => sha256:6c31b14f8325dfb9ff739785fbff43cc912475b7252748c93423033d4611bf5c 38.09MB / 38.09MB                                                       0.9s
 => => sha256:d6e9ae6c556ed24e47ebf35ed5b5d0d56c733c5db49d50d8816490996f2774ea 230B / 230B                                                             0.2s
 => => sha256:77c272c815a43c64f4e89e080d3ae067ff27c100ad9fbf6afd4e7c255e595a61 325B / 325B                                                             0.2s
 => => sha256:607a871f53a8ccb1159ba280ffe2b6b8409e87b4c01508798f4a07d12a938f89 321B / 321B                                                             0.2s
 => => sha256:01067c06de7159c4094cbe7113255cb07f85adc9ed2504fedbe1aad844e7db0f 248B / 248B                                                             0.2s
 => => sha256:f3cef679c5581ecdf52875c3f1c5ae538880b58236368f9c45cd28669489dfe5 227B / 227B                                                             0.2s
 => => sha256:2e07510b3b6f2c875e01f961e425165be82b540345e081580f99daf4d3a78682 171.07MB / 171.07MB                                                     3.1s
 => => sha256:7c6fea64666ec0db02138a26412f10da997a841803190484fb621f2114ecf8a1 221B / 221B                                                             0.2s
 => => sha256:692bb77a7fc0e48cc022472f0e39cc5d95e449765435d41d249a3abf005eec4f 971B / 971B                                                             0.3s
 => => sha256:9f1f9f625de331f1915fc1792851fb8608c944254442cb7e4c25f30e3b2c0383 38.80kB / 38.80kB                                                       0.2s
 => => sha256:c796196a519464022981d69da0876d4aa59b9868f1f6e0f1c2eebfe3221173cd 24.56MB / 24.56MB                                                       0.5s
 => => extracting sha256:c796196a519464022981d69da0876d4aa59b9868f1f6e0f1c2eebfe3221173cd                                                              1.2s
 => => extracting sha256:9f1f9f625de331f1915fc1792851fb8608c944254442cb7e4c25f30e3b2c0383                                                              0.9s
 => => extracting sha256:692bb77a7fc0e48cc022472f0e39cc5d95e449765435d41d249a3abf005eec4f                                                              0.1s
 => => extracting sha256:7c6fea64666ec0db02138a26412f10da997a841803190484fb621f2114ecf8a1                                                              0.0s
 => => extracting sha256:2e07510b3b6f2c875e01f961e425165be82b540345e081580f99daf4d3a78682                                                              5.8s
 => => extracting sha256:f3cef679c5581ecdf52875c3f1c5ae538880b58236368f9c45cd28669489dfe5                                                              0.0s
 => => extracting sha256:01067c06de7159c4094cbe7113255cb07f85adc9ed2504fedbe1aad844e7db0f                                                              0.0s
 => => extracting sha256:607a871f53a8ccb1159ba280ffe2b6b8409e87b4c01508798f4a07d12a938f89                                                              0.0s
 => => extracting sha256:77c272c815a43c64f4e89e080d3ae067ff27c100ad9fbf6afd4e7c255e595a61                                                              0.0s
 => => extracting sha256:d6e9ae6c556ed24e47ebf35ed5b5d0d56c733c5db49d50d8816490996f2774ea                                                              0.1s
 => => extracting sha256:6c31b14f8325dfb9ff739785fbff43cc912475b7252748c93423033d4611bf5c                                                              0.8s
 => => extracting sha256:da342f04dbe2005756875ee6ddf47770be87d67af6c4bc17444f74e603306d47                                                              0.4s
 => => extracting sha256:bc1f730938663ba413550ea0aa048f35f4fdc231016f01fd5355a3b3d5eb6ec8                                                              0.1s
 => => extracting sha256:801ec5982390ea3fb0f4aa88bba9f4e40964a344697b83108d802d71c3330014                                                              0.4s
 => => extracting sha256:20f4f4b58bf17448f5bbdf8003c7d66b8ccf5553e5b5daed7a292d0903160e19                                                              0.1s
 => => extracting sha256:998db3ceb21fc91aef9d96c6c85f03b5310579c2f5dd0d1018370ca887dff152                                                              0.0s
 => => extracting sha256:8f6eafd35194e6bb690bdd43dffdd5e5a0ab662dd8ad822748193b8fdc160768                                                              0.1s
 => => extracting sha256:c3cf45f768ae5699ac472387cac259bf614f117e7a1fa98560280bfa15c39ea0                                                              0.0s
 => => extracting sha256:7b3abe05cce9a64176cafb4f56d699540ccc12063462de5353e62ac5334dbd67                                                              0.0s
 => [2/2] RUN echo "test"                                                                                                                              2.7s
 => exporting to image                                                                                                                                 2.7s
 => => exporting layers                                                                                                                                0.2s
 => => exporting manifest sha256:9be744eef62787ede0b2c71ad33c359dace3922ae20f8aede39ae10556d033dd                                                      0.0s
 => => exporting config sha256:9af36a3466aacf8cc8c82b0f55b33631040b0f8f7ffbf3b7450be4d63ee82cf3                                                        0.0s
 => => exporting attestation manifest sha256:82e3c9fd517d63bfbd735f8cbc57e805d5764f7d9c63e577a46fa275b06fa179                                          0.1s
 => => exporting manifest list sha256:9023962316fe49bd2bc353319ecd03157e554a9039db80069ac69c768a3ad58b                                                 0.0s
 => => pushing layers                                                                                                                                  0.6s
 => => pushing manifest for abcdef.azurecr.io/abcdefrealtime/abcdef-detector-tests:random-test@sha256:9023962316fe49bd2bc353319ecd03157e554a9039db800  1.6s
 => [auth] abcdefrealtime/abcdef-detector-tests:pull,push token for abcdef.azurecr.io                                                                  0.0s

Additional info

No response

[crazy-max]

I'm not able to repro:

docker buildx ls
NAME/NODE            DRIVER/ENDPOINT                                       STATUS   BUILDKIT                              PLATFORMS
builder              docker-container
  builder0           unix:///var/run/docker.sock                           running  v0.12.0                               linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
default *            docker
  default            default                                               running  v0.11.7-0.20230525183624-798ad6b0ce9f linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

$ docker buildx --builder builder build -t crazymax/buildx:1924-container --push .
...
$ docker buildx imagetools inspect crazymax/buildx:1924-container --format "{{json .Image.Architecture}}"
"arm64"

$ docker buildx --builder default build -t crazymax/buildx:1924-docker --push .
...
$ docker buildx imagetools inspect crazymax/buildx:1924-docker --format "{{json .Image.Architecture}}"
"arm64"

Even when specifying preferred platforms like you do:

$ docker buildx create  --platform "linux/arm64,linux/amd64" --name mirrored --use
$ docker buildx --builder mirrored build -t crazymax/buildx:1924-mirrored --push .
...
$ docker buildx imagetools inspect crazymax/buildx:1924-mirrored --format "{{json .Image.Architecture}}"
"arm64"

The endpoint tcp://localhost:2375 of your mirrored builder looks a bit odd to me. Can you show the output of docker context ls command?