Improve the documentation examples and advice for native IPv6 Docker networking

[daryll-swer]

Is this a docs issue?

• My issue is about the documentation content or website

Type of issue

Other

Description

Few issues:

1. The bare minimum “example” should generally be a /64 (whereby we route a /64 or larger to the Docker host) instead of /112 or potentially some other outside nibble-bit boundary prefix length.
2. The IPv6 notation used in the documentation is not correct, in direct violation of RFC5952
3. ULAs shouldn't be promoted or advised, sources below:

• https://datatracker.ietf.org/doc/draft-ietf-v6ops-ula/
• https://datatracker.ietf.org/doc/html/draft-ietf-6man-rfc6724-update-06

4. An additional problem with ULA is enforcing the need for NAT66 which in turn breaks native IPv6 networking and the benefits of end-to-end principle, the usage of NAT66 also introduces the need for ALG (in Linux world, called NAT Helpers) which renders the benefits of IPv6 to zero, on par with traditional NAT44 or NAT444 environments.
5. Overall lacks network engineering point of view and operational insights

Location

https://docs.docker.com/config/daemon/ipv6/

Suggestion

For point 1:

We simply replace the “2001:0DB8::/112” string with “2001:db8::/64”
The idea for a minimum /64 came from the fact that IPv6 networking was to be based on prefix-length, and not “number of addresses” as the address space is 128-Bits. This was reflected in the original SLAAC specifications, in addition to additional operational information on BCOP-690. We should not be promoting archaic IPv4-centric mentality in native IPv6 networking.

For point 2:

Firstly, the alphabets in an IPv6 address is lower-case always, secondly we always remove all leading zeros in the compressed IPv6 notation format, meaning in effect:

• “2001:0DB8::/64” is not recommended
• “2001:db8::/64” is recommended

Please refer to section 4 of RFC5952

For point 3, 4 & 5:

I am willing to help improve this aspect of the Docker IPv6 documentation by integrating network engineering perspective and operational insights directly into the Docker docs.

The basic idea of native IPv6 networking is: No NAT66/NPTv6 or ULA.

I'm of course aware of poorly implemented IPv6 in popular cloud providers/IaaS companies, whereby the user is forced to rely on ULA/NAT66 or some hacks with NDP Proxy or MACVLAN, but of course, this is not a valid reason to push for ULA/NAT66/NPTv6 from the official documentation of Docker.

I authored an extensive native IPv6 best practices guide below, that folks may want to give a read on to get fully thorough information that simply cannot be reproduced in a tiny GitHub issue:
https://blog.apnic.net/2023/04/04/ipv6-architecture-and-subnetting-guide-for-network-engineers-and-operators/

I've written extensively on various topics of network engineering, in particular IPv6. I'm personally willing to help improve the Docker docs to push for native IPv6 networking using some realistic examples. I'm not sure how the Docker docs writing/improvement process is handled, but if I could get in a direct discussion with the relevant folks, it would be much appreciated.

On a side note, I'm happily using Docker for a few years now, with native IPv6 networking (no ULA/NAT66/NPTv6) 🙂

Tasks

Preview Give feedback

No tasks being tracked yet.

[dvdksn]

Thank you @daryll-swer for raising this issue, and for providing so much detail and offering to help out!

Let me loop in @robmry and @akerouanton, maintainers of libnetwork in moby. I think describing some of these concept from a network engineer's perspective sounds like a great enhancement, if we could pull that off!

[NiKiZe]

I can only agree to what @daryll-swer has brought up.
And that current Docker documentation has caused conflict more than once when users/developers has said that Docker documentation is to be followed instead of RFC and actuall best practice.

Thanks!

[robmry]

Yes, these sound like good changes!

@dvdksn - what's the best approach, I guess Daryll can create PRs for us to review and discuss?

[daryll-swer]

I guess Daryll can create PRs for us to review and discuss?

While a PR would make sense for the actual documentation change. I think we should probably get on a call or something first, to discuss some ideas and concepts I have in mind about native IPv6 networking on Docker, and see if we can then, from there, make a plan of action to update the Docker IPv6 docs (through the PRs?).

[robmry]

Yes, sure - let's do that. I'll try to sync with Albin, and we'll get something set up ...

[caliban511]

So how can the IPv6 network of the docker host communicate with the IPv6 subnet in docker? I haven't seen any documentation on this.

[daryll-swer]

So how can the IPv6 network of the docker host communicate with the IPv6 subnet in docker?

I'm not sure what you mean @caliban511.

[caliban511]

I'm not sure what you mean @caliban511.

What I mean is that the IPv6 subnet inside docker cannot communicate with the outside.
Although the containers inside docker have obtained IPv6 subnet addresses by adding IPv6 subnets, they still cannot access pure IPv6 websites on the public network.
I have not found a way to let the IPv6 subnet communicate with the public network.

[NiKiZe]

By using DHCPv6-PD every subnet gets its own /64
Traffic is routed thru the host.

The host should be able to request at least a /60 for such usage. And yes that requires that the network is setup correctly, with v6 you need to get more than a /64.

[daryll-swer]

What I mean is that the IPv6 subnet inside docker cannot communicate with the outside. Although the containers inside docker have obtained IPv6 subnet addresses by adding IPv6 subnets, they still cannot access pure IPv6 websites on the public network. I have not found a way to let the IPv6 subnet communicate with the public network.

What you are referring to is just routing. Docker's job is not network architecture and design, Docker's job is container abstraction tooling and deployment. So it's not a surprise that they didn't document how to design an IPv6-native network and how to route a prefix to the Docker host.

That being said, I am talking to Docker net dev team via emails and I did suggest they add some docs on routing basic for IPv6. This is my copy/paste from the email I sent them:

As I explained in the GitHub issue, the minimum (smallest, longest) prefix length should be a /64. This /64 is to be routed to the Docker host; There are 3 methods for routing a prefix to the host:

1. By using DHCPv6 ia_pd

   • Somewhat, fairly simple to configure and deploy. For both the DHCPv6 server upstream and the DHCPv6 client daemon running on the host.
   • Downside? The network operator (whoever is providing the network underlay to the Docker host owner) now needs to maintain and operate a DHCPv6 server and on top that, need to maintain AAA to ensure the “host” gets the correct prefix, statically, and survive reboots etc.

2. By using BGP (eBGP with private ASNs) with FRR or BIRD (FRR seems to be more popular)

   • More scalable method, allows for advanced traffic engineering with BGP features such as BGP communities, path manipulation etc.
   • The Docker host (in a properly architected network) could actually influence the path a packet from host, takes towards the upstream network (or internet) through BGP communities. Think of a large data centre network that spans from Amsterdam all the way to Spain. BGP communities will allow the app developer to influence (not 100% hard control) the path a packet takes based on network analytics (bandwidth, latency, packet loss, customer patterns etc)
   • Also my most favourite method.

3. By using static routes

   • Doesn't scale, prone to errors/failures when a next-hop dies/flaps. Not recommended for production.
   • I would go as far to say, not recommended even for a home office network, bare minimum a person should opt for OSPF (if they aren't capable of BGP operations).

This problem is more generalised and not limited to Docker situation, I don't know why but network engineering knowledge seems to be considered obsolete/unnecessary by many people in this industry to the point that everyone just knows “NAT” [as they don't know how to route from edge router all the way down to the host using eBGP driven design (for example)] and overlapping RFC1918 ranges (as they do not even know how to subnet).

The host should be able to request at least a /60 for such usage. And yes that requires that the network is setup correctly, with v6 you need to get more than a /64.

@NiKiZe for Docker-specific-only host, you don't really need more than a single /64 routed to that host, because you will anyways, just create a Linux bridge and slap a /64 on that bridge and all containers then reside behind that bridge and get a /128 out of the /64 routed address space. I've never found a use-case for Docker networking to go beyond one bridge or beyond a single cleanly routed /64.

Docker Swarm is something I've not used, but I suppose there may be a use case for more than one /64 there.