IPv6 is inconsistent with getaddrinfo

[axelkar]

Sometimes addresses *.local only return IPv6 addresses.

This is a major issue when using .local mDNS, since you apparently have to specify the interface when using link-local addresses.

But to be fair it's happens pretty randomly.

Sometimes running dog myothermachine.local (requesting the A record for myothermachine.local from 127.0.0.53) helps, but doesn't currently.

Here's a Python script to reproduce it:

# set lan_device_1 and lan_device_2

import socket
import subprocess
import time

def check(comment, hostname):
    print(f'  {comment}')
    for tup in socket.getaddrinfo(hostname, 443, type=socket.SOCK_STREAM):
        (family, type, proto, canonname, sockaddr) = tup
        type_str = 'IPv6' if family == socket.AF_INET6 else 'IPv4'
        print('   ', type_str)

def check_all():
    check('localhost', 'localhost')
    check('own hostname', socket.gethostname() + '.local')
    check('google.com', 'google.com')
    check('LAN device 1', lan_device_1)
    check('LAN device 2', lan_device_2)

check_all()

import sys
if len(sys.argv) > 1 and sys.argv[1] == 'immutable':
    sys.exit(0)

print('stopping nsncd')
subprocess.run(['sudo', 'systemctl', 'stop', 'nscd'], check=True)

check_all()

print('starting nsncd')
subprocess.run(['sudo', 'systemctl', 'start', 'nscd'], check=True)

print('restarting the process')
# even sleep doesn't help here. I'm guessing glibc doesn't reconnect to nsncd
subprocess.run([sys.executable, sys.argv[0], 'immutable'], check=True)

$ python main.py
  localhost
    IPv6
    IPv4
  own hostname
    IPv4
    IPv6
  google.com
    IPv4
    IPv6
  LAN device 1
    IPv4
    IPv6
  LAN device 2
    IPv6
stopping nsncd
  localhost
    IPv6
    IPv4
  own hostname
    IPv4
    IPv6
  google.com
    IPv4
    IPv6
  LAN device 1
    IPv4
    IPv6
  LAN device 2
    IPv4
    IPv6
starting nsncd
restarting the process
  localhost
    IPv6
    IPv4
  own hostname
    IPv4
    IPv6
  google.com
    IPv4
    IPv6
  LAN device 1
    IPv6
  LAN device 2
    IPv6

Example of ssh not working

$ ssh myothermachine.local
ssh: connect to host myothermachine.local port 22: Invalid argument
$ ssh myothermachine.local -v  
[...]
debug1: Connecting to myothermachine.local [fe80::1234] port 22.
debug1: connect to address fe80::1234 port 22: Invalid argument
ssh: connect to host myothermachine.local port 22: Invalid argument

System info

x86_64-linux
Linux 6.14.0, NixOS, 25.05 (Warbler), 25.05.20250402.2c8d3f4
systemd 257.3
GNU libc 2.40
nsncd 1.5.1

[axelkar]

I'm not sure if nsncd should also return the link ID in the IPv6 (see "Now 'ssh' gets IPv6 link-local addresses" in the below link).
https://bbs.archlinux.org/viewtopic.php?id=239604

But the main issue here is that it doesn't return an IPv4 address.

[axelkar]

Maybe the part about not returning a scope ID is a resolved issue, because glibc seems to support it, as seen here.

[picnoir]

As of February 2024, the scopeID was not getting sent in the hostent. The address was just a couple addrfamily (AF_INET or AF_INET6) + actual addr (4 or 16 bytes depending on the addr family).

This was currently a major implementation limitation as I was mentioning there: https://mail.gnu.org/archive/html/guix-devel/2024-02/msg00309.html .

I just had a quick glance at the codebase, but I couldn't pinpoint where the marshalling is actually happening anymore. The Nscd client is not the easiest part of glibc to navigate :/

So maybe this has been indeed fixed by upstream. But I think it's unlikely, this codebase is not super maintained.

So, overall, for a good mDNS support (and other stuff relying on link-local adresses resolutions), we'd need to extend the glibc nscd protocol first.

[axelkar]

I couldn't find the IPv6 scope ID getting returned even without nsncd on.
Maybe the function which is able to return scope IDs (IIRC nss_getaddrinfo4_r) function isn't getting called (I saw somewhere that apparently distros were disabling it)? I'll have to try and test with ltrace.

I think the issue with nsncd is that the scope id is in SocketAddrV6, and not IpAddrV6, and you're skipping everything other than the canon name and the IP address from dns-lookup. Not sure about the nscd protocol.

[axelkar]

I forgot to mention this in the above comment, but the most important part for me and most other people running networks which support both IPv4 and IPv6 is that the IPv4 results are skipped. It works with glibc without nsncd, but not with nsncd. Why?

I guess I'll have to modify nsncd to add debug logging, but I'm not sure if the Nixpkgs build already has debug symbols, so I could use gdb.

[picnoir]

I couldn't find the IPv6 scope ID getting returned even without nsncd on.

Yeah, as I was saying, it's not supported by the nscd (without the n, the glibc one) protocol¹. It's expected to be missing: we're not sending that to the wire.

I just compared a nscd(glibc 2.40)/nsncd getai query², the scopeid is still not there, the protocol looks unchanged.

I think the issue with nsncd is that the scope id is in SocketAddrV6, and not IpAddrV6, and you're skipping everything other than the canon name and the IP address from dns-lookup. Not sure about the nscd protocol.

Yes, that's how you would implement that in nsncd. But I think you'll first need to convince to the glibc maintainers to modify the nscd wire format to include the scopeid, and likely implement that in glibc/nscd.

Footnotes

1. edit. To make this more explicit: the nsncd project do not control the nscd wire format, the glibc project does. Nsncd is communicating with the nscd glibc client stubs. If we're up to change the format, the clients will have to support the new wire format as well. ↩

2. You can do that by making https://github.com/flokli/sockburp listening on /run/nscd/socket and send the queries to the nscd and nsncd sockets. ↩

[axelkar]

I couldn't find the IPv6 scope ID getting returned even without nsncd on.

I was talking about glibc without nsncd

[axelkar]

Thanks for the detailed comment. Do you have any idea what happens to the IPv4 addresses?

[axelkar]

Returning a scope ID has been supported since nss-resolve's inception, so I'm not sure why they don't get added when nsncd isn't running. Edit: at least according to my Python script. See below how resolvectl query and the dns-lookup crate are returning them.
systemd/systemd@4d1cf1e
https://github.com/systemd/systemd/blob/52990feb2bdf62f977e8cea44633d2e037912a28/src/nss-resolve/nss-resolve.c#L323

---

resolvectl query seems to be returning the IPv6 addresses with scope IDs from the cache. Not returning IPv4 addresses (when not specifying the record types explicitly) is likely because it knows they have scope IDs and will work.

$ resolvectl query mymachine.local
mymachine.local: fe80::xxxx%2       -- link: enp4s0
             (MyMachine.local)

-- Information acquired via protocol mDNS/IPv6 in 735us.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: cache
$ resolvectl query mymachine.local
mymachine.local: 192.168.1.10                      -- link: enp4s0
             fe80::xxxx%2       -- link: enp4s0
             (MyMachine.local)

-- Information acquired via protocol mDNS/IPv4 in 1.5ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: network
$ resolvectl query mymachine.local
mymachine.local: fe80::xxxx%2       -- link: enp4s0
             (MyMachine.local)

-- Information acquired via protocol mDNS/IPv6 in 767us.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: cache

With nsncd disabled, and resolvectl's D-Bus API returning IPv6 addresses, getaddrinfo still returns both IPv4 and IPv6 addresses.

Flushing resolved's caches and immediately running getaddrinfo (with nsncd enabled) does sometimes return the IPv4 address for some devices.

---

May 18 21:03:36.921 DEBG GETAI, log_resp: [AddrInfo { flags: 0, address: 2, socktype: 1, protocol: 6, sockaddr: 192.168.1.10:0, canonname: Some("MyMachine.local") }], hostname: "mymachine.local", request_type: GETAI, thread: worker_3
[...]
May 18 21:03:43.517 DEBG GETAI, log_resp: [AddrInfo { flags: 0, address: 10, socktype: 1, protocol: 6, sockaddr: [fe80::xxxx%2]:0, canonname: Some("MyMachine.local") }], hostname: "mymachine.local", request_type: GETAI, thread: worker_3

I added some logging to nsncd, and it now seems that the issue is in resolved or in something between resolved and nsncd. No idea why resolved's D-Bus API (and the NSS module using something other than D-Bus) returns IPv6-only from the cache, when explicit A record requests work. It could return IPv4 and IPv6 in that order, like without nsncd.

Do you think the __nss_disable_nscd could be breaking something? Why does getaddrinfo work properly outside of nsncd (without nscd connection), but not inside of it (also without nscd connection).

[picnoir]

I feel like I was not super clear at #157 (comment)

The scopeID is missing in the nscd wire protocol. That's a design shortcoming from the Glibc side. That's why you won't get any scope ID as soon as you start using nscd or nscd.

[axelkar]

I feel like I was not super clear at #157 (comment)

The scopeID is missing in the nscd wire protocol. That's a design shortcoming from the Glibc side. That's why you won't get any scope ID as soon as you start using nscd or nscd.

I understand that, and the issue I have is that the IPv4 address isn't being returned. I don't care about the scope ID for now. I fear it stems from resolved, but I published my findings here.