git clean follows directory junctions rather than removing them

[jeremyd2019]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

git version 2.47.1.windows.1
cpu: x86_64
built from commit: 2cd22437f64229935dc564db969cbcbfed5e9045
sizeof-long: 4
sizeof-size_t: 8
shell-path: D:/git-sdk-64-build-installers/usr/bin/sh
feature: fsmonitor--daemon
libcurl: 8.11.0
OpenSSL: OpenSSL 3.2.3 3 Sep 2024
zlib: 1.3.1

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.22631.4602]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
> type "$env:USERPROFILE\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path:
Default Branch Option:
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: WinSSL
CRLF Option: CRLFAlways
Bash Terminal Option: ConHost
Git Pull Behavior Option: FFOnly
Use Credential Manager: Enabled
Performance Tweaks FSCache: Enabled
Enable Symlinks: Disabled
Enable FSMonitor: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

ARM64 Windows

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Run git from python.

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

git clean -xfdf

• What did you expect to occur after running these commands?

Successfully removing non-tracked files.

• What actually happened instead?

BUG: compat/win32/fscache.c:128: Trying to allocate fsentry for long path 'B/src/rustc-1.83.0-src/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnull

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

Obtained during msys2-autobuild run building rust. I kind of suspect some type of circular link that Git isn't detecting/handling properly. Unfortunately I rmdir /s'ed the thing to get the runner back, but I could probably reproduce it again if you'd like me to try some commands to figure out what the link is and why Git for Windows followed it instead of removing it.

[dscho]

Oh wow, is this path longer than 4095 characters? https://github.com/git-for-windows/git/blob/v2.47.1.windows.1/compat/win32/fscache.c#L128

[jeremyd2019]

I think the actual bug is that it's following a symlink (reparse point).

I ran the build and I see:

> dir
 Directory of C:\_\B\src\rustc-1.83.0-src\build-CLANGARM64\aarch64-pc-windows-gnullvm\stage0-sysroot\lib\rustlib\rustc-src

12/17/2024  07:46 PM    <DIR>          .
12/17/2024  07:46 PM    <DIR>          ..
12/17/2024  07:46 PM    <JUNCTION>     rust [\??\C:\_\B\src\rustc-1.83.0-src]

> fsutil reparsepoint query rust
Reparse Tag Value : 0xa0000003
Tag value: Microsoft
Tag value: Name Surrogate
Tag value: Mount Point
Substitue Name offset: 0
Substitue Name length: 62
Print Name offset:     64
Print Name Length:     0
Substitute Name:       \??\C:\_\B\src\rustc-1.83.0-src

Reparse Data Length: 0x4a
Reparse Data:
0000:  00 00 3e 00 40 00 00 00  5c 00 3f 00 3f 00 5c 00  ..>.@...\.?.?.\.
0010:  43 00 3a 00 5c 00 5f 00  5c 00 42 00 5c 00 73 00  C.:.\._.\.B.\.s.
0020:  72 00 63 00 5c 00 72 00  75 00 73 00 74 00 63 00  r.c.\.r.u.s.t.c.
0030:  2d 00 31 00 2e 00 38 00  33 00 2e 00 30 00 2d 00  -.1...8.3...0.-.
0040:  73 00 72 00 63 00 00 00  00 00                    s.r.c.....

Hopefully that helps you figure out why git clean followed rather than removed the reparse point.

[jeremyd2019]

It looks like this is a Directory Junction reparse point (a la mklink /j), so that should make it easier to reproduce:

mkdir test
cd test
git init
mkdir -p 'B/src/rustc-1.83.0-src/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/'
cmd //s //c 'mklink /j B\src\rustc-1.83.0-src\build-CLANGARM64\aarch64-pc-windows-gnullvm\stage0-sysroot\lib\rustlib\rustc-src\rust B\src\rustc-1.83.0-src'
git clean -xfdf

Unfortunately, I'm not getting the BUG output on my x86_64 machine, I'm seeing:

warning: could not open directory 'B/src/rustc-1.83.0-src/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src/rust/build-CLANGARM64/aarch64-pc-windows-gnullvm/': Filename too long
Deletion of directory 'B/src/rustc-1.83.0-src/build-CLANGARM64/aarch64-pc-windows-gnullvm/stage0-sysroot/lib/rustlib/rustc-src' failed. Should I try again? (y/n)

But either way, git clean really shouldn't be trying to follow directory junction reparse points, it could easily be wiping out files outside the repository.

Since it's not hitting the BUG, the reproducer could just as easily be:

mkdir test
cd test
git init
cmd //s //c 'mklink /j foo .'
git clean -xfdf

or if you want to be evil, make the link target c:\windows or something 😈

[jeremyd2019]

Cygwin/MSYS2 seems to know those are symlinks:

> msys2.cmd -here -c "stat rust"

  File: rust -> /c/_/B/src/rustc-1.83.0-src
  Size: 27              Blocks: 0          IO Block: 65536  symbolic link
Device: f6731b96h/4134738838d   Inode: 562949954086902  Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (197108/Administrator)   Gid: (197121/    None)
Access: 2024-12-17 19:46:25.373475100 +0000
Modify: 2024-12-17 19:46:25.373475100 +0000
Change: 2024-12-17 19:46:25.373475100 +0000
 Birth: 2024-12-17 19:46:25.373475100 +0000

[jeremyd2019]

Has anyone gotten a chance to look into this? At a guess, I'd say file_attr_to_st_mode and finddata2dirent should handle IO_REPARSE_TAG_MOUNT_POINT, much like readlink_1 does.

[jeremyd2019]

Relevant comment on python/cpython#67596 (comment)

I am almost thinking this could be considered a security issue, that git clean could cause destruction outside of a working copy. I know various symlink-traversal issues have been considered security vulnerabilities.

[jeremyd2019]

• I was not able to find an open or closed issue matching what I'm seeing

Oops, I see now #3358 and #4383