Distroless container images unable to resolve *.local names #2469

Preflight Checklist

• I agree to follow the Code of Conduct that this project adheres to.
• I have searched the issue tracker for an issue that matches the one I want to file, without success.
• I am not looking for support or already pursued the available support channels without success.

Version

2.31.0

Storage Type

Kubernetes

Installation Type

Official container image

Expected Behavior

I want to use a distroless-based Dex build with an upstream LDAP IDP with address ending in .local.

More specifically, I want to run Dex and OpenLDAP on a single Kubernetes cluster for testing purposes, and I want to use a .svc.cluster.local address for OpenLDAP.

Actual Behavior

When I try to login to Dex, I see an error: LDAP Result Code 200 "Network Error": dial tcp: lookup ldap.openldap.svc.cluster.local: device or resource busy

Steps To Reproduce

1. Get a Kubernetes cluster
2. Build a distroless Dex container image (e.g., docker build --build-arg BASEIMAGE=gcr.io/distroless/static:latest -t dex .)
3. Deploy OpenLDAP to Kubernetes
4. Deploy distroless Dex to Kubernetes (ensuring to use .local LDAP host)
5. Login to Dex
6. See error above

Additional Information

• Discussed in Slack with @sagikazarmark: https://kubernetes.slack.com/archives/C011URMR41W/p1649691359149169
• Easiest solution for now is probably to document it
• The reason for this is as follows:
  • Go uses a CGO resolver to resolve .local DNS names when CGO is enabled: https://github.com/golang/go/blob/a6f6932b3ee87d9607ce246228e23f9a08dacc31/src/net/conf.go#L190-L193
  • CGO is currently enabled in Dex builds because we need it for SQL storage
  • A distroless container does not contain the dynamic libraries needed to support a CGO DNS implementation (it calls getaddrinfo(3) which actually calls dlopen)
• There are 2 known workarounds:
  • Build Dex binary with -tags netgo
  • Run Dex container with GODEBUG=netdns=go
• Using the pure Go DNS resolver has at least one downside: Go doesn’t support some features in /etc/resolv.conf and /etc/nsswitch.conf (per https://pkg.go.dev/net#hdr-Name_Resolution)

Configuration

No response

Logs

No response