[build.ps1] Set the default architecture using an environment variable #104474
Conversation
|
Tagging subscribers to this area: @dotnet/area-infrastructure-libraries |
|
The usual way to deal with this problem is to have a small local wrapper script(s) that are optimized for your dev workflows. For example, I like release configuration as the default for my dev workflow and only override the specific components to debug/checked as necessary. Should we have a environment variable for that too? |
eng/build.ps1
Outdated
| function Get-Help() { | ||
| Write-Host "Common settings:" | ||
| Write-Host " -arch (-a) Target platform: x86, x64, arm, arm64, or wasm." | ||
| Write-Host " Pass a comma-separated list to build for multiple architectures." | ||
| Write-Host (" [Default: {0} (Depends on your console's architecture.)]" -f [System.Runtime.InteropServices.RuntimeInformation]::ProcessArchitecture.ToString().ToLowerInvariant()) | ||
| Write-Host (" [Default: {0} (Depends on your console's architecture or the DOTNET_RUNTIME_DEFAULT_ARCH environment variable.)]" -f @(Get-Default-Arch)) |
There was a problem hiding this comment.
This covers for the top level build.sh1, but it does not cover other entrypoints into the dev workflow. For example, it does not cover running dotnet build .... directly that is part of number of documented dev workflows. If we want to go with the environment variables as the solution for overriding defaults, should it be respected for all entrypoints into the dev workflow?
There was a problem hiding this comment.
The peculiarity of build.sh1 is that it resolves the default architecture from the process architecture of the calling shell. The architecture of dotnet build ... is controlled by what SDKs you have installed and the PATH environment variable. I've never had issues running the correct sdk from a 32-bit shell.
There was a problem hiding this comment.
The peculiarity of build.sh1 is that it resolves the default architecture from the process architecture of the calling shell.
build.ps1 resolves the default architecture from the process architecture of the calling powershell.exe. It sounds like that you have the x64 powershell.exe on the PATH in your environment, but you actually want to have the arm64 powershell.exe. Is there arm64 powershell on Windows Arm64?
There was a problem hiding this comment.
Windows Arm does have native builds for Powershell, however I'm using mingw which only has x64 builds at the moment. A similar issue with SSH connections to Windows arm devices.
There was a problem hiding this comment.
Would it be better to workaround the problem by changing the path in your mingw environment to point to native powershell? It would cover this and all other powershell.exe invocations.
There was a problem hiding this comment.
Even if you run it as %windir%\sysnative\cmd.exe ?
That folder doesn't exist on ARM builds. Similar to powershell.exe it's the same executable that adapts depending on the parent architecture:
The non-native shells keep you on non-native architecture. It has been always the case.
I understand that this is the case, however like I said this does create problems in the workflows for those that do rely on tools that aren't arm64 native yet. While creating a wrapper script does technically resolve the issue, the dev loop workflow largely relies on muscle memory: I work on x64 and arm64 machines interchangeably so even with the workaround I have to explicitly remember that the current machine is arm64 to make sure the right script/arch parameter is being used. If I don't, I will have lost > 5 minutes waiting on a build that was targeting the wrong architecture. This has been happening frequently enough to me for it to be a problem.
There was a problem hiding this comment.
That folder doesn't exist on ARM builds.
That's surprising. It has been a virtual folder that only exists on non-native contexts. It has been the escape hatch to get back to native context. For example, this is how you can relaunch native x64 cmd from the emulated x86 cmd on Windows x64:
C:\Windows>set PROCESSOR_ARCHITECTURE
PROCESSOR_ARCHITECTURE=x86
C:\Windows>%windir%\sysnative\cmd.exe
Microsoft Windows [Version 10.0.22631.3810]
(c) Microsoft Corporation. All rights reserved.
C:\Windows>set PROCESSOR_ARCHITECTURE
PROCESSOR_ARCHITECTURE=AMD64
Is it really the case that these same steps do not work in emulated x64 cmd on Windows arm64?
There was a problem hiding this comment.
Assuming there existed a build of cmd.exe or powershell.exe that consistently launched an arm64 process, then it would solve my issue. The only copy of cmd.exe I could find other than the System32 one is the one at SysWOW64 which launches as an x86 process.
There was a problem hiding this comment.
You are right. I have done some reading on this topic. The system .exes are multi-arch on Windows Arm, and so there is no way to control the system process architecture by selecting different .exes. Instead, the process launching the .exe controls the process architecture.
For example, start command allows you to set the desired process architecture.start /B /WAIT /MACHINE arm64 %windir%\System32\cmd.exe should always launch native cmd prompt on Windows arm64.
There was a problem hiding this comment.
Thanks, the /machine parameter seems to work for me. By installing the following script called powershell in my PATH:
#!/usr/bin/env bash
start //b //wait //machine arm64 powershell.exe "$@"I'm able to force native powershell process from the mingw environment. This solves the problem using build.sh as the entrypoint, but not for the build.cmd entrypoint.
It's common practice in CLI tools in general, so I don't see why we couldn't do this for other parameters as well. |
The downside is that it results into harder to debug systems and problems like "this build command works on one machine, but not on the other machine". |
True, but we have embraced that approach in the product itself. Maybe it makes to be cautious about what knobs we do expose via environment variables, but in this case it seems warranted to me. |
|
If you set this, does your build environment use arm64 .NET runtime and tools to build, or does it use the slow emulated x64 .NET runtime and tools to build? |
It should be equivalent to passing the |
Verify that
The primary use of the |
Would it make sense to have a more general environment variable that allows you to set defaults for any option? For example, if you set it to |
|
That seems somewhat brittle to me, there's also the UX issue of dealing with space escaping in all different shell types. |
I've verified that it's running all arm64 processes when the arch parameter is specified explicitly. |
We use this pattern in other places today, e.g.: Line 112 in 9efc798 My thinking behind this suggestion was about making this feature as powerful as possible while keeping it as simple as possible. Having a new environment variable for every build.cmd/sh option that somebody may want to have a custom default for does not look pretty. |
It should be ok as long as the order of parameters isn't significant and there is no existing trailing parameters scheme that is being passed to nested processes. I defer to @ViktorHofer on whether it's appropriate to use. |
I agree that it would be better to have one env var to rule them all instead of multiple separate for every potential input. In general, I'm not a fan of introducing a new env var as the current input configuration system is already complex enough but if this is just meant to be set for an enhanced dev workflow (and gets documented) then I'm OK with that. As long as it doesn't further complicate our CI / local dev workflow configuration matrix. |
|
One issue with the general purpose environment variable is that it doesn't seem to have a good override mechanism other than unsetting it. If I had set my |
|
Closing since we have a workaround via #104033 (comment) |
The actual You are right that it would not work for parameters that can be specified more than once. |
I'm creating a PR with the original prototype since my dev workflow is currently blocked by this.
Fix #104033.