Open
Description
Verification
- I searched for recent similar issues at https://github.com/rust-lang/rustup/issues?q=is%3Aissue+is%3Aopen%2Cclosed and found no duplicates.
- I am on the latest version of Rustup according to https://github.com/rust-lang/rustup/tags and am still able to reproduce my issue.
Problem
Since 1.28.0, we have been finding subtle yet impossible to debug issues in our CI, where we get nonsensical results like this when trying to call Cargo:
Traceback (most recent call last):
File "<string>", line 24, in <module>
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\main.py", line 224, in main
Main(sys.argv[1:])
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\main.py", line 52, in __init__
self.run_command()
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\main.py", line 192, in run_command
res = commands.run(command, self.config, self.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\commands\__init__.py", line 79, in run
return _commands[command].run(config, args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\commands\bootstrap.py", line 179, in run
run_until_complete(build_tools_task)
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\utils\__init__.py", line 763, in run_until_complete
result = loop.run_until_complete(tasks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\bootstrap\build_tools.py", line 145, in fetch_recipes
await Fetch.fetch(self.cookbook, self.recipes, False, False, False, False, jobs)
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\commands\fetch.py", line 112, in fetch
await fetch_print_wrapper(recipe.name, stepfunc)
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\commands\fetch.py", line 100, in fetch_print_wrapper
await stepfunc()
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\build\recipe.py", line 102, in async_wrapped
ret = await stepfunc(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\build\source.py", line 456, in fetch
await self.extract_impl(fetching=True)
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\build\source.py", line 487, in extract_impl
await self.cargo_vendor(not fetching or self.offline)
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\build\source.py", line 137, in cargo_vendor
ct = await shell.async_call_output(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\utils\shell.py", line 345, in async_call_output
raise e
File "C:\Users\Administrator\runner\builds\amyspark\cerbero\cerbero\utils\shell.py", line 328, in async_call_output
proc = await asyncio.create_subprocess_exec(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\subprocess.py", line 218, in create_subprocess_exec
transport, protocol = await loop.subprocess_exec(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\base_events.py", line 1694, in subprocess_exec
transport = await self._make_subprocess_transport(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\windows_events.py", line 399, in _make_subprocess_transport
transp = _WindowsSubprocessTransport(self, protocol, args, shell,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\base_subprocess.py", line 36, in __init__
self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
File "C:\Python311\Lib\asyncio\windows_events.py", line 921, in _start
self._proc = windows_utils.Popen(
^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\asyncio\windows_utils.py", line 153, in __init__
super().__init__(args, stdin=stdin_rfd, stdout=stdout_wfd,
File "C:\Python311\Lib\subprocess.py", line 1024, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Python311\Lib\subprocess.py", line 1493, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [WinError 3] The system cannot find the path specified
C:/Users/Administrator/runner/builds/amyspark/cerbero/cb/rust/cargo/bin/cargo.exe vendor C:/Users/Administrator/runner/builds/amyspark/cerbero/cerbero-sources/bindgen-cli-0.71.1/cargo-vendor C:/Users/Administrator/runner/builds/amyspark/cerbero/cb/sources/build-tools/bindgen-cli-0.71.1
C:/Users/Administrator/runner/builds/amyspark/cerbero/cb/rust/cargo/bin/cargo.exe True
['cargo-clippy.exe', 'cargo-fmt.exe', 'cargo-miri.exe', 'cargo.exe', 'clippy-driver.exe', 'rls.exe', 'rust-analyzer.exe', 'rust-gdb.exe', 'rust-gdbgui.exe', 'rust-lldb.exe', 'rustc.exe', 'rustdoc.exe', 'rustfmt.exe', 'rustup.exe']
Where Cargo clearly exists, yet is impossible to be executed. It's simple to notice why, once the files are captured:
I tracked down the regression to this: rustup uses symlinks on Windows: #3677
Steps
- Install Rust on a Dockerized Windows environment, via rustup-init. Use multiple targets (in our case i386 and x86-64, both windows-msvc and windows-gnu)
- Call any Cargo command
Possible Solution(s)
Revert the commit 7e644c6.
Notes
Build log here: https://gitlab.freedesktop.org/amyspark/cerbero/-/jobs/74755751
Rustup version
(We don't log `rustup --version` but I can verify it's 1.28.1 for x86_64-pc-windows-msvc)Installed toolchains
(Same as above: stable-x86_64-pc-windows-msvc)OS version
Windows Server LTSC 2022