Local images not pulled - misunderstood of k3d image imprt conecpt?

[exocode]

I am not getting my local images pulled, I am sure that there is a short way to achieve this, but I didn't found out.

Please help me to get my local docker image into the k3d-cluster

Is this done it with k3d image import or with

docker push k3d-metashop-cluster-registry:12345

May I misunderstood the docs or they are outdated, or I am simpy too stupid. (You may make a noob-tutorial :-) for guys like me)

This is the error I get

Failed to pull image "docker.io/library/metashop-backend-api:local": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/metashop-backend-api:local": failed to resolve reference "docker.io/library/metashop-backend-api:local": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

I create my clusters like so:

k3d cluster create (without any config and without an explicit registry)

as well I tried to

k3d cluster create --file k3d-config.yml

k3d-config.yml (mostly copied from the docs)

# k3d configuration file, saved as e.g. /home/me/myk3dcluster.yaml
apiVersion: k3d.io/v1alpha2 # this will change in the future as we make everything more stable
kind: Simple # internally, we also have a Cluster config, which is not yet available externally
name: metashop-cluster # name that you want to give to your cluster (will still be prefixed with `k3d-`)
servers: 1 # same as `--servers 1`
agents: 2 # same as `--agents 2`
# kubeAPI: # same as `--api-port myhost.my.domain:6445` (where the name would resolve to 127.0.0.1)
  # host: "metashop.localhost" # important for the `server` setting in the kubeconfig
  # hostIP: "127.0.0.1" # where the Kubernetes API will be listening on
  # hostPort: "6445" # where the Kubernetes API listening port will be mapped to on your host system
image: rancher/k3s:v1.20.4-k3s1 # same as `--image rancher/k3s:v1.20.4-k3s1`
# network: metashop-net # same as `--network my-custom-net`
# token: superSecretToken # same as `--token superSecretToken`
volumes: # repeatable flags are represented as YAML lists
  # we create a persistentVolume named data
  - volume: data:/data # same as `--volume '/my/host/path:/path/in/node@server[0];agent[*]'`
    nodeFilters:
      - server[0]
      - agent[*]
  - volume: /Users/jan/kubernetes-volumes/:/data-local # same as `--volume '/my/host/path:/path/in/node@server[0];agent[*]'`
    nodeFilters:
      - server[0]
      - agent[*]
ports:
  - port: 8080:80 # same as `--port '8080:80@loadbalancer'`
    nodeFilters:
      - loadbalancer
# labels:
#   - label: foo=bar # same as `--label 'foo=bar@agent[1]'`
#     nodeFilters:
#       - agent[1]
# env:
#   - envVar: bar=baz # same as `--env 'bar=baz@server[0]'`
#     nodeFilters:
#       - server[0]
#######
#######
# I ALSO TRIED TO COMMENT IN / OUT 
# registries: 
#    create: true
#######
#######
# ALSO TRIED 
# registries: 
#    create: true
#    use:
#     - k3d-registry.localhost:5000 # 
registries: # define how registries should be created or used
  create: true # creates a default registry to be used with the cluster; same as `--registry-create`
  # use:
  #   - k3d-registry.localhost:5000 # some other k3d-managed registry; same as `--registry-use 'k3d-myotherregistry:5000'`
options:
  k3d: # k3d runtime settings
    wait: true # wait for cluster to be usable before returining; same as `--wait` (default: true)
    timeout: "60s" # wait timeout before aborting; same as `--timeout 60s`
    disableLoadbalancer: false # same as `--no-lb`
    disableImageVolume: false # same as `--no-image-volume`
    disableRollback: false # same as `--no-Rollback`
    disableHostIPInjection: false # same as `--no-hostip`
  # k3s: # options passed on to K3s itself
  #   extraServerArgs: # additional arguments passed to the `k3s server` command; same as `--k3s-server-arg`
  #     - --tls-san=my.host.domain
    # extraAgentArgs: [] # addditional arguments passed to the `k3s agent` command; same as `--k3s-agent-arg`
  kubeconfig:
    updateDefaultKubeconfig: true # add new cluster to your default Kubeconfig; same as `--kubeconfig-update-default` (default: true)
    switchCurrentContext: true # also set current-context to the new cluster's context; same as `--kubeconfig-switch-context` (default: true)

I added my image like so:

❯ k3d image import metashop-backend-api:local --cluster metashop-cluster --trace
DEBU[0000] Runtime Info:
&{Name:docker Endpoint:/var/run/docker.sock Version:20.10.7 OSType:linux OS:Docker Desktop Arch:x86_64 CgroupVersion:1 CgroupDriver:cgroupfs Filesystem:extfs}
DEBU[0000] Importing image(s) [[metashop-backend-api:local]] from runtime [{}] into cluster(s) [[{Name:metashop-cluster Network:{Name: ID: External:false IPAM:{IPPrefix:invalid IP prefix IPsUsed:[] Managed:false}} Token: Nodes:[] InitNode:<nil> ExternalDatastore:<nil> KubeAPI:<nil> ServerLoadBalancer:<nil> ImageVolume:}]]...
INFO[0000] Importing image(s) into cluster 'metashop-cluster'
DEBU[0000] Selected image 'metashop-backend-api:local' found in runtime
TRAC[0000] TranslateContainerDetailsToNode: Checking for default object label app=k3d on container /k3d-metashop-cluster-serverlb
TRAC[0000] TranslateContainerDetailsToNode: Checking for default object label app=k3d on container /k3d-metashop-cluster-agent-1
TRAC[0000] TranslateContainerDetailsToNode: Checking for default object label app=k3d on container /k3d-metashop-cluster-agent-0
TRAC[0000] TranslateContainerDetailsToNode: Checking for default object label app=k3d on container /k3d-metashop-cluster-server-0
TRAC[0000] TranslateContainerDetailsToNode: Checking for default object label app=k3d on container /k3d-metashop-cluster-registry
DEBU[0000] Attaching to cluster's image volume 'k3d-metashop-cluster-images'
INFO[0000] Starting k3d-tools node...
TRAC[0000] Creating node from spec
&{Name:k3d-metashop-cluster-tools Role:noRole Image:docker.io/rancher/k3d-tools:v4.4.6 Volumes:[k3d-metashop-cluster-images:/k3d/images /var/run/docker.sock:/var/run/docker.sock] Env:[] Cmd:[] Args:[noop] Ports:map[] Restart:false Created: Labels:map[app:k3d k3d.cluster:metashop-cluster] Networks:[k3d-metashop-cluster] ExtraHosts:[] ServerOpts:{IsInit:false KubeAPI:<nil>} AgentOpts:{} GPURequest: Memory: State:{Running:false Status: Started:} IP:{IP:invalid IP Static:false}}
TRAC[0000] Creating docker container with translated config
&{ContainerConfig:{Hostname:k3d-metashop-cluster-tools Domainname: User: AttachStdin:false AttachStdout:false AttachStderr:false ExposedPorts:map[] Tty:false OpenStdin:false StdinOnce:false Env:[K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml] Cmd:[noop] Healthcheck:<nil> ArgsEscaped:false Image:docker.io/rancher/k3d-tools:v4.4.6 Volumes:map[] WorkingDir: Entrypoint:[] NetworkDisabled:false MacAddress: OnBuild:[] Labels:map[app:k3d k3d.cluster:metashop-cluster k3d.role:noRole k3d.version:v4.4.6] StopSignal: StopTimeout:<nil> Shell:[]} HostConfig:{Binds:[k3d-metashop-cluster-images:/k3d/images /var/run/docker.sock:/var/run/docker.sock] ContainerIDFile: LogConfig:{Type: Config:map[]} NetworkMode: PortBindings:map[] RestartPolicy:{Name: MaximumRetryCount:0} AutoRemove:false VolumeDriver: VolumesFrom:[] CapAdd:[] CapDrop:[] CgroupnsMode: DNS:[] DNSOptions:[] DNSSearch:[] ExtraHosts:[] GroupAdd:[] IpcMode: Cgroup: Links:[] OomScoreAdj:0 PidMode: Privileged:true PublishAllPorts:false ReadonlyRootfs:false SecurityOpt:[] StorageOpt:map[] Tmpfs:map[/run: /var/run:] UTSMode: UsernsMode: ShmSize:0 Sysctls:map[] Runtime: ConsoleSize:[0 0] Isolation: Resources:{CPUShares:0 Memory:0 NanoCPUs:0 CgroupParent: BlkioWeight:0 BlkioWeightDevice:[] BlkioDeviceReadBps:[] BlkioDeviceWriteBps:[] BlkioDeviceReadIOps:[] BlkioDeviceWriteIOps:[] CPUPeriod:0 CPUQuota:0 CPURealtimePeriod:0 CPURealtimeRuntime:0 CpusetCpus: CpusetMems: Devices:[] DeviceCgroupRules:[] DeviceRequests:[] KernelMemory:0 KernelMemoryTCP:0 MemoryReservation:0 MemorySwap:0 MemorySwappiness:<nil> OomKillDisable:<nil> PidsLimit:<nil> Ulimits:[] CPUCount:0 CPUPercent:0 IOMaximumIOps:0 IOMaximumBandwidth:0} Mounts:[] MaskedPaths:[] ReadonlyPaths:[] Init:0xc0001cb85f} NetworkingConfig:{EndpointsConfig:map[k3d-metashop-cluster:0xc0004e4180]}}
DEBU[0000] Created container k3d-metashop-cluster-tools (ID: 51040ae6eb4770f02048568cdfc3dfd84d2bafc751ffcf535a87b46343dd0c7f)
DEBU[0000] Node k3d-metashop-cluster-tools Start Time: 2021-06-23 11:09:19.571849 +0200 CEST m=+0.407748351
TRAC[0000] Starting node 'k3d-metashop-cluster-tools'
INFO[0000] Starting Node 'k3d-metashop-cluster-tools'
DEBU[0003] Truncated 2021-06-23 09:09:23.0358274 +0000 UTC to 2021-06-23 09:09:23 +0000 UTC
INFO[0003] Saving 1 image(s) from runtime...
DEBU[0003] Executing command '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' in node 'k3d-metashop-cluster-tools'
TRAC[0004] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0005] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0006] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0007] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0008] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0009] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0010] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0059] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0060] Exec process '[./k3d-tools save-image -d /k3d/images/k3d-metashop-cluster-images-20210623110923.tar metashop-backend-api:local]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
DEBU[0061] Exec process in node 'k3d-metashop-cluster-tools' exited with '0'
INFO[0061] Importing images into nodes...
INFO[0061] Importing images from tarball '/k3d/images/k3d-metashop-cluster-images-20210623110923.tar' into node 'k3d-metashop-cluster-agent-0'...
INFO[0061] Importing images from tarball '/k3d/images/k3d-metashop-cluster-images-20210623110923.tar' into node 'k3d-metashop-cluster-agent-1'...
INFO[0061] Importing images from tarball '/k3d/images/k3d-metashop-cluster-images-20210623110923.tar' into node 'k3d-metashop-cluster-server-0'...
DEBU[0062] Executing command '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' in node 'k3d-metashop-cluster-server-0'
DEBU[0062] Executing command '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' in node 'k3d-metashop-cluster-agent-1'
DEBU[0062] Executing command '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' in node 'k3d-metashop-cluster-agent-0'
TRAC[0062] Exec process '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-agent-0'.. sleeping for 1 second...

TRAC[0152] Exec process '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-server-0'.. sleeping for 1 second...
TRAC[0153] Exec process '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-agent-1'.. sleeping for 1 second...
TRAC[0153] Exec process '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-server-0'.. sleeping for 1 second...
TRAC[0153] Exec process '[ctr image import /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-agent-0'.. sleeping for 1 second...
DEBU[0154] Exec process in node 'k3d-metashop-cluster-agent-0' exited with '0'
DEBU[0154] Exec process in node 'k3d-metashop-cluster-agent-1' exited with '0'
DEBU[0154] Exec process in node 'k3d-metashop-cluster-server-0' exited with '0'
INFO[0154] Removing the tarball(s) from image volume...
DEBU[0154] Executing command '[rm -f /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' in node 'k3d-metashop-cluster-tools'
TRAC[0154] Exec process '[rm -f /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0155] Exec process '[rm -f /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
TRAC[0156] Exec process '[rm -f /k3d/images/k3d-metashop-cluster-images-20210623110923.tar]' still running in node 'k3d-metashop-cluster-tools'.. sleeping for 1 second...
DEBU[0157] Exec process in node 'k3d-metashop-cluster-tools' exited with '0'
INFO[0157] Removing k3d-tools node...
DEBU[0157] Deleting node k3d-metashop-cluster-tools ...
INFO[0159] Deleted k3d-metashop-cluster-tools
INFO[0159] Successfully imported image(s)
INFO[0159] Successfully imported 1 image(s) into 1 cluster(s)

I tried in my deployment the following image paths (tried that on each cluster I created above):

myregistry.localhost
myregistry.localhost:5000
k3d-metashop-cluster-registry:5000/metashop-backend-api:local
k3d-metashop-cluster-registry/metashop-backend-api:local
k3d-registry.localhost:5000/metashop-backend-api:local
k3d-registry.localhost/metashop-backend-api:local
k3d-registry.localhost:5000/metashop-backend-api:local
docker.io/library/metashop-backend-api:local

When I run

❯ docker exec k3d-metashop-cluster-agent-0 ctr image ls

REF                                                                                                       TYPE                                                      DIGEST                                                                  SIZE      PLATFORMS                                                         LABELS
docker.io/library/elasticsearch:7.12.1                                                                    application/vnd.docker.distribution.manifest.list.v2+json sha256:622f854572780281bc85b5fde33be27e99670941ed8b7eea5ba4aaf533fa64ec 413.3 MiB linux/amd64,linux/arm64/v8                                        io.cri-containerd.image=managed
docker.io/library/elasticsearch@sha256:622f854572780281bc85b5fde33be27e99670941ed8b7eea5ba4aaf533fa64ec   application/vnd.docker.distribution.manifest.list.v2+json sha256:622f854572780281bc85b5fde33be27e99670941ed8b7eea5ba4aaf533fa64ec 413.3 MiB linux/amd64,linux/arm64/v8                                        io.cri-containerd.image=managed
docker.io/library/metashop-backend-api:local                                                              application/vnd.docker.distribution.manifest.v2+json      sha256:067bf53596aeaf6392060bd7a30428d6377c8ed39b56dd46945f3368398c54b1 1.0 GiB   linux/amd64                                                       io.cri-containerd.image=managed
......
.....
....
...
..
.

What did you expect to happen

Deploying my local images in k3d cluster.

Screenshots or terminal output

pod log Failed to load logs: container "api" in pod "api-58454769-b8wwh" is waiting to start: trying and failing to pull image

Which OS & Architecture

❯ uname -a
Darwin Jans-MacBook-Pro.local 20.5.0 Darwin Kernel Version 20.5.0: Sat May  8 05:10:33 PDT 2021; root:xnu-7195.121.3~9/RELEASE_X86_64 x86_64

Which version of k3d

❯ k3d --version
k3d version v4.4.6
k3s version latest (default)

Which version of docker

❯ docker version
dClient:
 Cloud integration: 1.0.17
 Version:           20.10.7
 API version:       1.41
 Go version:        go1.16.4
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:22 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.7
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       b0f5bc3
  Built:            Wed Jun  2 11:54:58 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

    ~/Coding/RubymineProjects/metashop    master !3 ?7 ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
❯ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  compose: Docker Compose (Docker Inc., 2.0.0-beta.3)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 15
 Server Version: 20.10.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.775GiB
 Name: docker-desktop
 ID: QNIV:Y7XP:E3DC:4XNQ:OM3S:37PJ:5BMI:VLPO:PCZV:CQAN:PCE5:MBUZ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

many thanks in advance

[iwilltry42]

Hi @exocode , thanks for creating this issue!
Since you imported the image and didn't push it to a registry, the image "is just there" in the nodes, so you cannot pull it.
Make sure that your imagePullPolicy to ifNotPresent, and not Always, so it doesn't try to pull.
You can use the image name just like you used it for the import command.