detect nix store on unsupported filesystems

[mschwaig]

Is your feature request related to a problem? Please describe.

A colleague of mine tried to install Nix with /nix on an NTFS mount. He had no idea that this would not work out well for him.

The way that this goes wrong right now is that the install itself fails with "error: suspicious ownership or permission on /nix/store/<some storepath>". If you migrate your Nix store to NTFS later on the same kind of error starts showing up.

Describe the solution you'd like

I wish this was error condition that gets caught and named more clearly, with a dedicated error message, by the installer before it starts the actual install as well as by the Nix daemon on startup.

Describe alternatives you've considered

I guess the alternative is keeping things as they are.

Additional context

Some steps to reproduce this with a fresh install:

0. start out on a some Linux distro like Ubuntu with no Nix installed
1. mount some NTFS volume at /nix

• sudo mkdir /nix
• sudo mount -t ntfs /dev/vdb /nix

2. run the nix installer

• sh <(curl -L https://nixos.org/nix/install) --daemon
• answer install questions until install starts running

3. install fails

~~> Setting up shell profiles for Fish with conf.d/nix.fish inside /etc/fish /usr/local/etc/fish /opt/homebrew/etc/fish /opt/local/etc/fish

~~> Setting up the default profile

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo HOME=/root /nix/store/vi8vjs4m4dnj5lwgm9p6071m0s7v4p26-nix-2.20.3/bin/nix-env -i /nix/store/vi8vjs4m4dnj5lwgm9p6071m0s7v4p26-nix-2.20.3

to install a bootstrapping Nix in to the default profile

installing 'nix-2.20.3'
building '/nix/store/ykahjqw485vhanx3g5wz2q1nk9rrby04-user-environment.drv'...
error: suspicious ownership or permission on '/nix/store/ykahjqw485vhanx3g5wz2q1nk9rrby04-user-environment.drv.chroot/nix/store/xi786yj9a2wc7mcf0ngybskxz4rl8vrx-user-environment' for output 'out'; rejecting this build output

---- oh no! --------------------------------------------------------------------
Oh no, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.

:(

We'd love to help if you need it.

You can open an issue at
https://github.com/NixOS/nix/issues/new?labels=installer&template=installer.md

Or get in touch with the community: https://nixos.org/community

Priorities

Add 👍 to issues you find important.

[mschwaig]

I have briefly looked into how this condition could be detected.

What came to my mind was some good detection method for the underlying file system at /nix/store and a deny list of file systems that are known to cause issues.

Using df and lsblk as follows might work:

mschwaig@box:~$ df -P /nix/store # to get the block device
Filesystem     1024-blocks   Used Available Capacity Mounted on
/dev/vdb          20971516 174436  20797080       1% /nix
mschwaig@box:~$ lsblk --fs /dev/vdb # to get the FSTYPE for teh given block device
NAME FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINT
vdb  ntfs         ntfs  4CF7D1352E12543B                       19,8G     1% /nix

[thufschmitt]

@mschwaig : Could you try instead to call the thing that breaks on NFS (not sure what it is exactly) to see whether the filesystem behaves properly for Nix?

[mschwaig]

Sorry, could you rephrase that? I don't know what you mean, but I will try to clarify.
I'm talking about NTFS (New Technology File System) - Microsoft's file system.

I would guess that the things that Nix stumbles over are

• having a different permissions system than Unix,
• having timestamps with a different granularity and starting time, and
• having different semantics for symlinks.

I think they did a bunch of work on interop with the Unix semantics of those things for WSL, but what I'm talking about is a Linux system with an NTFS driver, and I have no idea if any of that work could be accessed in native Linux.

[thufschmitt]

Sorry, could you rephrase that?

Sorry it was late, my message wasn't really clear indeed :D

What I was suggesting was: NTFS doesn't work for Nix because of its non-Unixy semantics. Rather than detecting NTFS, the installer could detect that the store directory doesn't have the right behavior¹. Something like:

touch /nix/store/.test
chmod 700 /nix/store/.test
if [[ $(stat -c %s /nix/store/.test) != "700" ]]; then
  echo "Error: '/nix/store' isn't a valid POSIX filesystem, I can't install Nix on it"
  exit 1
fi
rm /nix/store/.test

Footnotes

1. Especially since according to https://askubuntu.com/a/74851, NTFS can be POSIX-compatible with the right mount flags ↩