Proposal Details

Linux 6.12 introduced AT_HANDLE_MNT_ID_UNIQUE which allows name_to_handle_at(2) to return a 64-bit unique mount ID (akin to STATX_MNT_ID_UNIQUE). This was mostly added because when using open_by_handle_at(2), knowing that you are operating on the correct mount is quite important and unique mount IDs solve that problem.

Unfortunately, trying to use this flag with unix.NameToHandleAt leads to two issues:

1. This flag causes the kernel to interpret the pointer argument passed to be treated as a uint64_t* and attempt to write a 64-bit value to the pointer. In principle this will cause 4 bytes of stack corruption with unix.NameToHandleAt (because it allocates a _C_int for this operation which is 4 bytes on most architectures).
   Note that I haven't gone through with gdb to see if this actually causes corruption. I wouldn't be surprised if alignment with Go's 8-byte int (at least on 64-bit systems) makes this a non-issue -- nothing crashes if you try it FWIW.
2. In order to support it in Go, we would need to return a uint64 so that 32-bit machines can still use unique mount IDs. Obviously changing it is not allowed in general, but a quick GitHub code search shows quite a few examples of unix.NameToHandleAt usage that do use the returned mount ID as argument to strconv.Atoi and similar usages that mean changing the existing types will definitely break users.

My proposal would be to create the following new API:

func NameToHandleAt2(dirfd int, path string, flags int) (handle FileHandle, mountID uint64, err error)

and have the classic unix.NameToHandleAt be marked as deprecated and have it silently mask out AT_HANDLE_MNT_ID_UNIQUE (to avoid programs using AT_HANDLE_MNT_ID_UNIQUE but silently ignoring the part of the mount ID that actually guarantees its uniqueness).

The main downside with this proposal is that the kernel might choose to create a name_to_handle_at2(2) in the future which would conflict with our naming and cause user confusion. My first idea was to call it NameToHandleAtUnique but that makes it feel less general purposes -- I'm open to alternative name ideas (maybe NameToHandleAt64?).