tilt-prepare: "kind load" fails for docker >= version 29

[bnallapeta]

What steps did you take and what happened?

When running tilt up locally on an Apple Silicon Mac (M1/M2/M3), the environment preparation fails during the tilt-prepare step. Specifically, it fails when attempting to pre-load cert-manager images into the kind cluster:

Image: "quay.io/jetstack/cert-manager-cainjector:v1.19.1" with ID "sha256:c7898aece8fb08102fca0b37683e37cb94e0a77c0d15b8e3c9128f6c04c868e0" not yet present on node "capi-test-control-plane", loading...
ERROR: failed to load image: command "docker exec --privileged -i capi-test-control-plane ctr --namespace=k8s.io images import --all-platforms --digests --snapshotter=overlayfs -" failed with error: exit status 1
Command Output: ctr: content digest sha256:d5578165a84137e9228e4573bb1697b6a3ca7de09b5b20f53a2c97137d57fcef: not found

What did you expect to happen?

tilt up should complete successfully and the development environment should start without errors.

Cluster API version

main branch

Kubernetes version

$ k version
Client Version: v1.34.2
Kustomize Version: v5.7.1
Server Version: v1.31.2
Warning: version difference between client (1.34) and server (1.31) exceeds the supported minor version skew of +/-1

Anything else you would like to add?

No response

Label(s) to be applied

/kind bug
/area devtools

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[bnallapeta]

/assign

[chrischdi]

Cross-posting:

Note: this issue is also relevant when using docker community edition >= version 29 for any OS.

I think the workaround used in CI should work here too: using crane pull "$image" "image.tar" to pull the image and docker load -i "image.tar" to load it.

Afterwards preload work.

[chrischdi]

/retitle tilt-prepare: "kind load" fails for docker >= version 29

[chrischdi]

xref, related kind issue:

kubernetes-sigs/kind#3795

xref, related containerd issue:

containerd/containerd#11344