`nix-build --check` does not detect nondeterministic builds in CA mode

[trofi]

Noticed on real non-deterministic R package.

Minimal reproducer:

$ nix-build -E 'with import <nixpkgs> {}; builtins.derivation { name = "flaky"; builder = "${bash}/bin/bash"; args = [ "-c" "${coreutils}/bin/date +%N > $out" ]; system = builtins.currentSystem; __contentAddressed = true; outputHashAlgo = "sha256"; outputHashMode = "recursive"; }' # --check
/nix/store/yx1fa0gbdcp6rxw6snjqz9scalxy73sg-flaky

$ nix-build -E 'with import <nixpkgs> {}; builtins.derivation { name = "flaky"; builder = "${bash}/bin/bash"; args = [ "-c" "${coreutils}/bin/date +%N > $out" ]; system = builtins.currentSystem; __contentAddressed = true; outputHashAlgo = "sha256"; outputHashMode = "recursive"; }' --check
Resolved derivation: '/nix/store/yb82n19qzyxiii3nyb6zh7rfg3jlm94j-flaky.drv' -> '/nix/store/f3iqgsn2n1nvhwf7c80x29ha39hl0g5q-flaky.drv'...
checking outputs of '/nix/store/f3iqgsn2n1nvhwf7c80x29ha39hl0g5q-flaky.drv'...
/nix/store/yx1fa0gbdcp6rxw6snjqz9scalxy73sg-flaky

It should fail on check. The same without __contentAddressed fails as expected:

$ nix-build -E 'with import <nixpkgs> {}; builtins.derivation { name = "flaky"; builder = "${bash}/bin/bash"; args = [ "-c" "${coreutils}/bin/date +%N > $out" ]; system = builtins.currentSystem; }' # --check
/nix/store/mv7ng4zx4bsaq0rscn9583qah8fdf3kw-flaky

$ nix-build -E 'with import <nixpkgs> {}; builtins.derivation { name = "flaky"; builder = "${bash}/bin/bash"; args = [ "-c" "${coreutils}/bin/date +%N > $out" ]; system = builtins.currentSystem; }' --check
checking outputs of '/nix/store/82ccy5x3xzgz8974l8yh25jfi1jvd68l-flaky.drv'...
error: derivation '/nix/store/82ccy5x3xzgz8974l8yh25jfi1jvd68l-flaky.drv' may not be deterministic: output '/nix/store/mv7ng4zx4bsaq0rscn9583qah8fdf3kw-flaky' differs

CC @regnat

[stale]

I marked this as stale due to inactivity. → More info

[roberth]

Fun fact: glibc does not survive CA-derivation transformation: ld-linux.so has the code that does memcmp("/nix/store/path", p, N) and gcc transforms that into a sequence of movabs $part1, %rax; xorq %rax, %rsi; jne ...; movabs $part2, %rax; ... dilutin the original store path into 8-byte chunks. CA-transformation can't rewrite that and loses paths at https://sourceware.org/git/?p=glibc.git;a=blob;f=elf/dl-load.c;h=8a89b71016d426796e602b60555696649871c6ae;hb=HEAD#l162

We can disable CA derivation on glibc if really needed. But it would be even nicer if nix was able to detect unstable derivations at least in --check mode: #5336

-- @trofi on Matrix

--check on a CA derivation should be able to detect impurities that arise from the placeholder output path.
To put it simply, $out must be different on the second run (or in --check), so that the impurity manifests itself.
This way we can detect chopped up references that fail to be rewritten.