Description
If an iptables version is used that does not support or expose (in case a wrapper script is used arround iptables) the --wait flag issues occur if multiple containers are started at the same time. As only one iptables instance can run at any time, parallel calls to iptables fail and the rules for the dns resolver are not added to the container (in case a docker network is used).
Symptom:
Containers can not access other containers or the internet due to the DNS resolution failing.
Steps to reproduce:
- Use an iptables wrapper or version without --wait
- Create a docker network
- Add multiple containers to this network that are restarted on system start
- Restart docker or the system to start all containers at once
Cause for this issue:
Docker will detect that the xtables lock (iptables/iptables.go:121) is not supported and use its internal locking mechanism (bestEffortLock) instead (iptables/iptables.go:537).
The issue is that this lock is a mutex and can only provide locking within the same process. This becomes an issue if docker networks are used. In this case a new resolver process for each container is spawned via "reexec" to add the required iptables rules into the container network namespace (resolver_unix.go:29). If multiple containers start in parallel, multiple of these resolver setup processes are spawned which each have their own independent bestEffortLock and thus try to execute iptables in parallel to each other - and fail while attempting to do this.