Binding to a freshly-assigned IPv6 address fails with an opaque `Address not available` (tentative address / DAD), no retry or hint

[dukedorje]

Summary

Calling Endpoint::builder(...).bind_addr("[<ipv6>]:<port>") with an IPv6 address that was assigned to the interface moments earlier fails with:

Error: failed to bind iroh endpoint
  0: Failed to bind sockets
  1: Address not available (os error 99)   // EADDRNOTAVAIL

The address is present on the interface (scope global), but a freshly-added IPv6 address is tentative until Duplicate Address Detection (DAD) completes (~1–2 s on Linux), and bind(2) to a tentative address returns EADDRNOTAVAIL. Once DAD finishes, the same bind succeeds.

This is partly just how IPv6 works, so it may not be iroh's bug to "fix" outright — but the developer experience is rough: the failure is a bare OS error with no indication that the address is tentative or that retrying shortly would succeed, and iroh makes no attempt to tolerate a recoverable, common startup race.

Environment

• iroh 1.0.0
• Reproduced on Linux (armv7 container); the tentative-address behavior is standard Linux IPv6.

Repro

ip -6 addr add fd00:1234::5/64 dev eth0     # starts tentative; DAD pending
# bind immediately, before DAD completes:

let ep = Endpoint::builder(presets::Minimal)
    .relay_mode(RelayMode::Disabled)
    .bind_addr("[fd00:1234::5]:45888".parse().unwrap())?
    .bind()
    .await?;   // <-- Err: Failed to bind sockets / Address not available (os error 99)

Bind a second or two later (after DAD) and it succeeds. Assigning the address nodad, or enabling optimistic DAD (net.ipv6.conf.<if>.optimistic_dad=1), also makes the immediate bind succeed.

Expected / nicer behaviors (any one would help)

1. Clearer error — when a user-specified bind address fails with EADDRNOTAVAIL, mention that the address may be tentative (IPv6 DAD) and a brief retry may succeed, instead of a bare "Address not available". This alone would save a lot of head-scratching.
2. Optional short retry — for an explicitly user-supplied bind_addr, retry the bind a few times over ~1–2 s before giving up, so a node that self-assigns its bind address at startup doesn't lose a race with DAD.
3. Docs — note that an explicit IPv6 bind_addr must already have passed DAD (or be added nodad / optimistic).

Context

Hit while bringing up a relay-less LAN mesh where each node self-assigns its own address at startup and then binds the endpoint to it. The address assignment and the bind happen back-to-back in the same process, so the DAD window is almost always still open. (We've since moved to IPv4 for unrelated reasons, so this isn't blocking us — filing because the opaque error seems likely to trip up anyone self-assigning an IPv6 bind address.)

[matheus23]

When we designed the bind_addr API, one of our explicit goals was to surface binding errors, in case the .bind_addr API was used.
In a way you should be able to think of iroh::Endpoint as very similar to "just a QUIC-capable socket". Doing the same thing with something like quinn::Endpoint or noq::Endpoint would result in the same behavior. In this framing, I don't think retrying on the iroh side would be the correct thing to do.

With respect to a better error message, we already surface all of the information we get from the socket2 API. I suspect we could special-case catch the EADDRNOTAVAIL and provide a hint for the error message, I think that'd be neat (but really ideally socket2 or the OS would give better error messages 🙃). Providing such hints might either work by tacking on some fake error "context" when we detect the EADDRNOTAVAIL, or we add a hint feature to n0-error (which I think would actually be really cool, too!).

Adding docs for this seems reasonable to me, happy to review a PR for that :)

[dukedorje]

yeah sure would be nice if the OS would tell us anything when something fails.. tracing a failure and auto-healing would go a long way for finally making networking get out of the way.

[flub]

I think I'd be open to detecting that a socket is still doing DAD and than having a short backoff. It would be good if could check more than just the error code though? Can netdev's Interface::oper_state help with this perhaps? Or something else in netdev?

[matheus23]

I'm realizing that we're internally rebinding sockets to fix broken sockets during connections...

In this framing, I don't think retrying on the iroh side would be the correct thing to do.

So I have to retract this statement. We should totally do the retry internally.