Unable to produce consistent digest id even with cache provided

[aggo15]

I'm simulating a scenario where the builder instance and it's respective volume was removed (the worst case scenario where the build server was down and had to do reprovision), then rebuild docker image with same source code and push it to a private registry. Finally I observed if digest id produced in private registry is the same or not.

Here is the command I use to rebuild the image.

docker buildx build --builder mybuilder \
  --cache-from=type=local,src=/buildx-cache --cache-to=type=local,dest=/buildx-cache,mode=max \
  --platform linux/arm/v6,linux/arm/v7,linux/arm64 . -t <repo_name>:<tag_name> --push

Here is how I create my builder instance

docker buildx create --use --name mybuilder --driver-opt network=host --buildkitd-flags '--allow-insecure-entitlement network.host'

My docker version and buildx version

~ $ docker buildx version
github.com/docker/buildx v0.8.2-docker 6224def4dd2c3d347eee19db595348c50d7cb491

~ $ docker version
Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:34 2022
 OS/Arch:           linux/arm64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:01:01 2022
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Here is the list of things I did for the simulation

1. First build - repo_name:first, digest id=280783b7369a
2. Execute docker container stop <moby_container_name> && docker container prune && docker volume prune
3. Second build - repo_name:second, digest id=815cdf6e7658, notice the digest id is different from first
4. Execute docker container stop <moby_container_name> && docker container prune && docker volume prune
5. Third build - repo_name:third, digest id=815cdf6e7658, notice the digest id is the same as second build.

My expectation is all three build should produce the same digest id. The actual result is only 2nd build onwards only the digest id will be the same, the first and second build digest id is always different.

Is this issue known and is there anything I can do to ensure the digest id produced is same?

Thanks in advance.

[tonistiigi]

Please make a runnable reproducer for your case and post it to moby/buildkit issue tracker.