[fix] Fix dotnet muxer resolution when the current process runs under proot (e.g. Termux/Android)

[github-actions]

🤖 This is an automated fix generated by the Issue Triage agent.

Fixes #16446

Root cause

DotnetHostHelper.TryGetDotnetPathByArchitecture has a fast path that returns the current process as the dotnet muxer when its architecture matches the target and its file name is dotnet/dotnet.exe:

if (_processHelper.GetCurrentProcessArchitecture() == targetArchitecture)
{
    string currentProcessFileName = _processHelper.GetCurrentProcessFileName()!;
    if (Path.GetFileName(currentProcessFileName) == _muxerName)
    {
        muxerPath = currentProcessFileName;
        return true;
    }
    ...
}

GetCurrentProcessFileName() was implemented as Process.MainModule?.FileName. On Unix, .NET derives MainModule.FileName from the first mapping in /proc/self/maps. Under proot (used by Termux on Android to sandbox aarch64 Linux rootfs), an unrelated placeholder loader binary is injected at the lowest address of every process, so MainModule.FileName reports that loader path instead of the real dotnet executable — even though the process genuinely is running as the dotnet muxer.

Because the reported file name isn't dotnet, the fast path is skipped and resolution falls through to DOTNET_ROOT* / global registration / default installation search branches. All of those branches call IsValidArchitectureMuxer, which on Linux never determines an architecture (no ELF header inspection), so every candidate — including a perfectly valid, explicitly configured muxer — is rejected as "incompatible architecture", and dotnet test fails with "Could not find a 'dotnet' host for the '(ARCH)' architecture."

Fix

ProcessHelper.GetCurrentProcessFileName() (the common/.NET-targeted implementation used by both net462 and netcoreapp builds) now prefers Environment.ProcessPath on .NET (netcoreapp) builds, falling back to Process.MainModule.FileName if it's unexpectedly null. Environment.ProcessPath resolves the executable path directly (via /proc/self/exe on Linux) and is unaffected by the extra loader mapping injected by proot, so the fast path correctly recognizes the current process as the dotnet muxer.

This only changes behavior on .NET (netcoreapp) targets; the net462 target is unaffected (it doesn't have Environment.ProcessPath).

Testing

• Added ProcessHelperTests.GetCurrentProcessFileNameShouldPreferEnvironmentProcessPathOverMainModuleFileName, which asserts ProcessHelper.GetCurrentProcessFileName() returns Environment.ProcessPath on .NET builds.
• Ran the existing DotnetHostHelperTest suite (test/Microsoft.TestPlatform.CoreUtilities.UnitTests) — no regressions; the 2 pre-existing failures in that suite (GetDotnetPathByArchitecture_DefaultInstallation_Win) are Windows-registry-only tests that already fail on Linux before this change (verified via git stash).
• Verified the new test and the full vstest.console.UnitTests ProcessHelperTests suite pass on Linux (net11.0).

An end-to-end acceptance test isn't feasible here since the bug only reproduces inside a proot sandbox (Termux/Android), which isn't available in the standard CI/acceptance test environment; the fix is instead validated with a focused unit test against the corrected ProcessHelper behavior plus the existing DotnetHostHelperTest coverage of the consuming code path.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• southcentralus0.in.applicationinsights.azure.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "southcentralus0.in.applicationinsights.azure.com"

See Network Configuration for more information.

🔍 Triaged by Issue Repro Triage & Auto-Fix 🔍

---

Note

This was originally intended as a pull request, but GitHub Actions is not permitted to create or approve pull requests in this repository.
The changes have been pushed to branch fix/issue-16446-503a024bd16ea886.

Click here to create the pull request

To fix the permissions issue, go to Settings → Actions → General and enable Allow GitHub Actions to create and approve pull requests. See also: gh-aw FAQ

Show patch preview (88 of 88 lines)

From e9666b4e266cb8656f9ca39c72565b1e5a7102d0 Mon Sep 17 00:00:00 2001
X-GH-AW-Base-Commit: 03153101e469b0e7341fe91f174231e95bdd28fd
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Wed, 9 Sep 2026 12:53:07 +0000
Subject: [PATCH] Fix dotnet muxer resolution when process is sandboxed (e.g.
 proot)

Under environments like proot (used by Termux on Android), the
current process's executable file name reported by
Process.MainModule.FileName does not match the real executable,
because .NET derives it from the first mapping in /proc/self/maps,
and proot injects an unrelated loader binary at the lowest address
of every process.

This breaks DotnetHostHelper.TryGetDotnetPathByArchitecture's fast
path, which checks whether the current process is the dotnet muxer
by comparing Path.GetFileName(GetCurrentProcessFileName()) against
"dotnet"/"dotnet.exe". Under proot the check fails even though the
process really is running as the dotnet muxer, so resolution falls
through to the search branches, all of which are rejected on Linux
because IsValidArchitectureMuxer never determines an architecture
for the ELF format.

Fix ProcessHelper.GetCurrentProcessFileName (on .NET builds) to
prefer Environment.ProcessPath, which resolves the executable path
directly (e.g. via /proc/self/exe on Linux) and is not affected by
the extra loader mapping.

Fixes #16446

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 .../common/System/ProcessHelper.cs             |  9 +++++++++
 .../ProcessHelperTests.cs                      | 18 ++++++++++++++++++
 2 files changed, 27 insertions(+)

diff --git a/src/Microsoft.TestPlatform.PlatformAbstractions/common/System/ProcessHelper.cs b/src/Microsoft.TestPlatform.PlatformAbstractions/common/System/ProcessHelper.cs
index cb40d81..40e2704 100644
--- a/src/Microsoft.TestPlatform.PlatformAbstractions/common/System/ProcessHelper.cs
+++ b/src/Microsoft.TestPlatform.PlatformAbstractions/common/System/ProcessHel
... (truncated)

[github-actions]

Thanks for the analysis! This is a duplicate of #16446, which already has an open fix PR: #16462. Closing the loop here to avoid duplicate parallel PRs — please follow #16462 for review/merge status.

🔍 Triaged by Issue Repro Triage & Auto-Fix 🔍