Skip to content

[BUG] k3d-managed registries with tld .localhost unreachable from inside the cluster #1608

@AntonOfTheWoods

Description

@AntonOfTheWoods

What did you do

  • How was the cluster created?
    • k3d cluster create -x A -y B

You can create an environment where you'll see the issue with multipass:

# multipass launch --cloud-init ~/conf/multipass/config.yaml --cpus 2 --disk 25G --memory 8G --bridged --name k3dbug 24.04

. /etc/os-release

sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $VERSION_CODENAME stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update && sudo apt -y dist-upgrade && sudo apt -y autoremove
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin libnss3-tools libnss-myhostname
sudo snap install kubectl --classic

# make sure this is less than wireguard or much badness will ensue!!!
sudo mkdir -p /etc/docker/
echo '{"mtu":1380}' | sudo tee /etc/docker/daemon.json

getent group docker || sudo groupadd docker
sudo usermod -aG docker $USER

# docker network create --opt com.docker.network.driver.mtu=1380 --driver bridge k3d-network
# curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
# k3d cluster create -c k3d-config.yml
# k3d configuration file
apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: badaddress
servers: 1

registries: # define how registries should be created or used
  create: # creates a default registry to be used with the cluster; same as `--registry-create registry.localhost`
    # name: registry.k3d.internal # works
    name: k3d-reg.localhost
    host: "0.0.0.0"
    hostPort: "5000"

network: k3d-network
ports:
  - port: 80:80
    nodeFilters:
      - loadbalancer
  - port: 443:443
    nodeFilters:
      - loadbalancer
  • What did you do afterwards?
# from the "host"
curl -i http://k3d-reg.localhost:5000/v2/ # -> works

kubectl run wget-pod --image=busybox --restart=Never --rm -it -- wget -S -O - http://k3d-reg.localhost:5000/v2/ # -> works

kubectl run curl-test --image=curlimages/curl -it --rm --restart=Never -- curl k3d-reg.localhost:5000/v2/
If you don't see a command prompt, try pressing enter.
warning: couldn't attach to pod/curl-test, falling back to streaming logs: Internal error occurred: unable to upgrade connection: container curl-test not found in pod curl-test_default
curl: (7) Failed to connect to k3d-reg.localhost port 5000 after 0 ms: Could not connect to server
pod "curl-test" deleted
pod default/curl-test terminated (Error)

What did you expect to happen

All utilities should be able to access the k3d-managed registries.

Screenshots or terminal output

see above

Which OS & Architecture

  • output of k3d runtime-info
arch: x86_64
cgroupdriver: systemd
cgroupversion: "2"
endpoint: /var/run/docker.sock
filesystem: extfs
infoname: k3dbug
name: docker
os: Ubuntu 24.04.3 LTS
ostype: linux
version: 28.3.3

Which version of k3d

  • output of k3d version
k3d version v5.8.3
k3s version v1.31.5-k3s1 (default)

Which version of docker

Client: Docker Engine - Community
 Version:           28.3.3
 API version:       1.51
 Go version:        go1.24.5
 Git commit:        980b856
 Built:             Fri Jul 25 11:34:09 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.3.3
  API version:      1.51 (minimum version 1.24)
  Go version:       go1.24.5
  Git commit:       bea959c
  Built:            Fri Jul 25 11:34:09 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.27
  GitCommit:        05044ec0a9a75232cad458027ca83437aae3f4da
 runc:
  Version:          1.2.5
  GitCommit:        v1.2.5-0-g59923ef
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
Client: Docker Engine - Community
 Version:    28.3.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.26.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.39.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 3
  Running: 3
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 28.3.3
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-79-generic
 Operating System: Ubuntu 24.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.75GiB
 Name: k3dbug
 ID: 63ae0892-e5a5-4765-b214-287041b60521
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

If I use a non- .localhost domain (putting the domain in my hosts' /etc/hosts) then everything works as expected - I can call it from my host and the correct alias and access is available to all tools directly from the workloads (including curl and everything else).

This literally wasted an entire day for me, so at the very least it would be good to add a note to the docs. The docs proudly claim that libnss-myhostname will allow you to use .localhost domains but THAT IS NOT TRUE, at least not for Ubuntu LTS 24.04.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions