[CA-drvs] nix-store --repair-path creates files with invalid atime and mtime

[fogti]

Describe the bug

discovered while compiling dosfstools:

dosfstools> make: Makefile.am: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
dosfstools> make: Warning: File 'Makefile.am' has modification time 15558496220 s in the future                                                                                                                                                                                                                                             
dosfstools> make: configure.ac: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999                                                                    
dosfstools> CDPATH="${ZSH_VERSION+.}:" && cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash '/build/source/missing' aclocal-1.16
dosfstools>  cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash /build/source/missing automake-1.16 --foreign
dosfstools> CDPATH="${ZSH_VERSION+.}:" && cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash '/build/source/missing' autoconf
dosfstools>  cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash ./config.status Makefile
dosfstools> config.status: creating Makefile                   
dosfstools> make: Makefile.am: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
dosfstools> make: Warning: File 'Makefile.am' has modification time 15558496215 s in the future
dosfstools> make: configure.ac: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
dosfstools> CDPATH="${ZSH_VERSION+.}:" && cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash '/build/source/missing' aclocal-1.16
dosfstools>  cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash /build/source/missing automake-1.16 --foreign
dosfstools> CDPATH="${ZSH_VERSION+.}:" && cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash '/build/source/missing' autoconf
dosfstools>  cd . && /nix/store/z65p5clqbxdjmix6kpml8965l20ybka8-bash-4.4-p23/bin/bash ./config.status Makefile

e.g.

# stat /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/{Makefile.am,configure.ac}
  File: /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/Makefile.am
  Size: 256             Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 10230746    Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-18 23:22:21.134854825 +0200
Modify: 1970-01-01 00:00:00.000000000 +0100
Change: 2021-05-18 23:24:10.630524636 +0200
 Birth: 2021-05-10 03:07:03.638628770 +0200
  File: /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/configure.ac
  Size: 2445            Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 10230750    Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-18 23:22:21.134854825 +0200
Modify: 1970-01-01 00:00:00.000000000 +0100
Change: 2021-05-18 23:24:10.626524648 +0200
 Birth: 2021-05-10 03:07:03.642628821 +0200

# unshare -m

## mount -o remount,rw /nix/store

## rm -rf /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source

## stat /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/{Makefile.am,configure.ac}
stat: cannot stat '/nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/Makefile.am': No such file or directory
stat: cannot stat '/nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/configure.ac': No such file or directory

## exit

# nix store repair /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source

# stat /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/{Makefile.am,configure.ac}
  File: /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/Makefile.am
  Size: 256             Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 10230746    Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-18 23:22:21.134854825 +0200
Modify: 1970-01-01 00:00:00.000000000 +0100
Change: 2021-05-18 23:26:47.270091788 +0200
 Birth: 2021-05-10 03:07:03.638628770 +0200
  File: /nix/store/fcwfh7ym57709z3ja46p624zwcq1fnyd-source/configure.ac
  Size: 2445            Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 10230750    Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-18 23:22:21.134854825 +0200
Modify: 1970-01-01 00:00:00.000000000 +0100
Change: 2021-05-18 23:26:47.266091798 +0200
 Birth: 2021-05-10 03:07:03.642628821 +0200

Steps To Reproduce

unknown

Expected behavior

the timestamps should be proper normalized.

nix-env --version output

# nix-env --version
nixenv (Nix) 2.4pre20210514_de9e43c

Additional context

[fogti]

ok, I was able to create a Makefile skeleton which repros the make behavoir partially:

t: /nix/store/<random existing store path>-source/<random existing file>
	echo 1 $^ > $@

log when broken:

# stat /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4
  File: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4
  Size: 2988            Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 9447691     Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-27 19:48:01.371440743 +0000
Modify: 1969-12-31 23:00:00.000000000 +0000
Change: 2021-05-27 19:46:20.583110366 +0000
 Birth: 2021-05-10 01:01:43.290612068 +0000

# nix-shell -p gnumake

[nix-shell:~/tmp]# make t
make: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
make: Warning: File '/nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4' has modification time 15557715546 s in the future
echo 1 /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4 > t
make: warning:  Clock skew detected.  Your build may be incomplete.

[nix-shell:~/tmp]# realpath $(which make)
/nix/store/q9fgn95mrli8a8dl1h04pjcsf7fwry3w-gnumake-4.3/bin/make

# for i in /nix/store/*-gnumake*/bin/make; do rm -f t; echo "$i"; "$i" --version; "$i" t; done
/nix/store/7fqqkrz6m11793cw7by8wbn2q87za8ny-gnumake-4.3/bin/make
bash: /nix/store/7fqqkrz6m11793cw7by8wbn2q87za8ny-gnumake-4.3/bin/make: cannot execute binary file: Exec format error
bash: /nix/store/7fqqkrz6m11793cw7by8wbn2q87za8ny-gnumake-4.3/bin/make: cannot execute binary file: Exec format error
/nix/store/czmfg4jagyd970aknr7wwrfgp58bkdrz-gnumake-4.3/bin/make
GNU Make 4.3
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
make: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
make: Warning: File '/nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4' has modification time 15557714690 s in the future
echo 1 /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4 > t
make: warning:  Clock skew detected.  Your build may be incomplete.
/nix/store/jn3856jac75m9i9a2cfci7mji7a5pp48-gnumake-4.3/bin/make
bash: /nix/store/jn3856jac75m9i9a2cfci7mji7a5pp48-gnumake-4.3/bin/make: cannot execute binary file: Exec format error
bash: /nix/store/jn3856jac75m9i9a2cfci7mji7a5pp48-gnumake-4.3/bin/make: cannot execute binary file: Exec format error
/nix/store/q9fgn95mrli8a8dl1h04pjcsf7fwry3w-gnumake-4.3/bin/make
GNU Make 4.3
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
make: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
make: Warning: File '/nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4' has modification time 15557714690 s in the future
echo 1 /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4 > t
make: warning:  Clock skew detected.  Your build may be incomplete.
/nix/store/rk892zsagrsd5r3484j959cjym0hp12d-gnumake-4.3/bin/make
GNU Make 4.3
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
make: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4: Timestamp out of range; substituting 2514-05-30 01:53:03.999999999
make: Warning: File '/nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4' has modification time 15557714690 s in the future
echo 1 /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4 > t
make: warning:  Clock skew detected.  Your build may be incomplete.

log when working:

# stat /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile
  File: /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile
  Size: 2527      	Blocks: 8          IO Block: 4096   regular file
Device: 803h/2051d	Inode: 5902304     Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-11 19:52:32.351851850 +0200
Modify: 1970-01-01 01:00:01.000000000 +0100
Change: 2021-05-11 11:01:23.562483073 +0200
 Birth: 2021-05-11 11:01:23.557483061 +0200

# nix-shell -p gnumake
[nix-shell:~/tmp]# make t
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t

[nix-shell:~/tmp]# make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

# cd tmp; for i in /nix/store/*-gnumake*/bin/make; do rm -f t; echo "$i"; "$i" --version; "$i" t; done
/nix/store/2hy69k93agkzk9jbfh75jb1f94gfdylb-gnumake-4.2.1/bin/make
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/3pl1nfvldwpblhj68939k3wzd8z4ww86-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/7fqqkrz6m11793cw7by8wbn2q87za8ny-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/8psvlkcgbzlwvazx7w8vnxpf16big6gh-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/dk7s6714az7paaqwrca488z7xryh6wf3-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/dp6y0n9cba79wwc54n1brg7xbjsq5hka-gnumake-4.2.1/bin/make
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/jn3856jac75m9i9a2cfci7mji7a5pp48-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/n6pcva97lbir5fi50hqa6wpq5m493482-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/q4pqixkp09yaqg98nlw3nr0clvhz2b53-gnumake-4.2.1/bin/make
GNU Make 4.2.1
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/sah8k7hycg5wwb4l3knxdvpl373iy65i-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t
/nix/store/vdcwnh52j109yzbqpfn7x8y78fvja1vq-gnumake-4.3/bin/make
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
echo 1 /nix/store/395052nvd1nac4hk783xqaqg5svrvaqn-source/Makefile > t

This probably means that this is not a simple bug in Nix, but something else.... but I don't know what's going wrong there...

[fogti]

related: https://mail.gnu.org/archive/html/bug-guix/2020-09/msg00036.html

[fogti]

interesting: Modify: 1969-12-31 23:00:00.000000000 +0000 = Modify: 1970-01-01 00:00:00.000000000 +0100... that isn't right (= too early)... should be 00:00:01 at least.

[thufschmitt]

I can’t reproduce this (neither using my main store, or with --store $PWD/nixstore to get something more self-contained), all the files on my store do have their timestamp properly set at epoch. Do you have a more standalone repro (ideally starting from a blank store to make sure that there’s no hidden state hidden)

[fogti]

wget https://ytrizja.de/misc/libarchive-source.nar
mkdir -p /root/tmp-store
nix-store --store /root/tmp-store --restore /root/tmp-store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source < ./libarchive-source.nar
stat /root/tmp-store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/CMakeLists.txt

result with Nix 2.4pre20210506_db6ab75 and 2.4pre20210514_de9e43c

  File: /root/tmp-store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/CMakeLists.txt
  Size: 78993           Blocks: 160        IO Block: 4096   regular file
Device: 802h/2050d      Inode: 9962520     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-29 21:44:40.046277991 +0000
Modify: 2021-05-29 21:44:40.046277991 +0000
Change: 2021-05-29 21:44:40.046277991 +0000
 Birth: 2021-05-29 21:44:40.046277991 +0000

Sadly, this does not reproduce the problem at all...

try to repair using nix 2.4pre20210506_db6ab75

but nix 2.4pre20210506_db6ab75 does indeed restore it correctly:

# nix-env --version
nix-env (Nix) 2.4pre20210506_db6ab75
# nix-store --repair-path /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source
copying path '/nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source' from 'https://cache.nixos.org'...
# stat /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/CMakeLists.txt 
  File: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/CMakeLists.txt
  Size: 78993           Blocks: 160        IO Block: 4096   regular file
Device: 802h/2050d      Inode: 9451901     Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-27 19:48:01.339440649 +0000
Modify: 1970-01-01 00:00:01.000000000 +0000 <-- correct
Change: 2021-05-29 21:53:31.249778467 +0000
 Birth: 2021-05-15 12:24:11.172764758 +0000
# stat /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4
  File: /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build/autoconf/ax_append_compile_flags.m4
  Size: 2988            Blocks: 8          IO Block: 4096   regular file
Device: 802h/2050d      Inode: 9447691     Links: 2
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-05-29 21:58:05.213333971 +0000
Modify: 1969-12-31 23:00:00.000000000 +0000 <-- WTF, why?!
Change: 2021-05-29 21:53:30.517779540 +0000
 Birth: 2021-05-10 01:01:43.290612068 +0000
# # (urgh)

(although I have now two buggy nix versions on that machine, I'll try to continue from here (e.g. trying to install a more recent nix version, which has fewer bugs))

[fogti]

P.S: tried to fix it for now using:

unshare -m 
# in the newly opened shell:
mount -o remount,rw /nix/store
find /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/build -print -exec touch -r /nix/store/vv2ksh443zynl57zvw3vihq9v0znzlbv-source/CMakeLists.txt '{}' +
exit

TZ=UTC touch -at 197001010000.01 -mt 197001010000.01 $file would also work.

[fogti]

with the newest nix version Nix 2.4pre20210529_1f39092, this also seems to happen with fixed-output derivations.