MSBuild <Exec>/ToolTask resets the console output code page, prohibiting unicode output

[tristanlabelle]

Steps to reproduce

Repro project (save as repro.proj):

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <Target Name="Print">
        <Exec Command="chcp" />
    </Target>       
</Project>

Repro commands in cmd.exe:

>chcp
Active code page: 850

>msbuild repro.proj
Microsoft (R) Build Engine version 16.4.0-preview-19517-01+31c846a51 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/1/2019 8:35:23 AM.
Project "repro.proj" on node 1 (default targets).
Print:
  chcp
  Active code page: 850
Done Building Project "repro.proj" (default targets).


Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.18

>chcp 65001
Active code page: 65001

>msbuild repro.proj
Microsoft (R) Build Engine version 16.4.0-preview-19517-01+31c846a51 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 11/1/2019 8:35:29 AM.
Project "repro.proj" on node 1 (default targets).
Print:
  chcp
  Active code page: 850
Done Building Project "repro.proj" (default targets).


Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.23

Note: 850 is the Western Europe/US code page, representing most latin-based characters. 65001 is UTF-8, which can represent all Unicode characters.

Expected behavior

When run inside MSBuild, chcp should output the same code page as its ancestor cmd.exe.

Programs should inherit the console code page from their ancestors.

Actual behavior

When run inside MSBuild, chcp outputs 850 rather than 65001 as set on the ancestor cmd.exe.

MSBuild's <Exec> task executes its commands in such a way that the child processes do not inherit the console code page from MSBuild (which itself inherited it from its parent). The console code page for children processes is rather reset to the OEM code page, which cannot encode Unicode characters in any Windows OS language in its default configuration. This means that it is impossible to use MSBuild to run a program that will output Japanese characters on an English OS without it coming out as a bunch of ?????.

Specifically, the use of ProcessStartInfo.CreateNoWindow = true in ToolTask will reset the console code page of the child process to the OEM code page. If this is fixed, the batch file should also be written in the console code page since that is what cmd.exe will be using to read it. Pull request coming soon.

Environment data

msbuild /version output: Microsoft (R) Build Engine version 16.4.0-preview-19517-01+31c846a51 for .NET Framework
OS info: Windows 10

[tristanlabelle]

If ProcessStartInfo.CreateNoWindow cannot be set to false for other reasons, there is an alternative, contrived solution that would work:

MSBuild could prepend $"@chcp {Console.OutputEncoding.CodePage}\r\n" to the batch file and write the batch with the encoding from Console.OutputEncoding. cmd.exe would be started with OEM as its console code page due to CreateNoWindow = true. However, after executing the first line of the batch file, it would switch to the same console output code page as MSBuild, essentially reimplementing inheritance. The following lines of the batch file would then also be decoded using the Console.OutputEncoding code page (yes, cmd.exe is that horrible), hence why it is necessary to encode the batch file itself in that encoding. MSBuild would have to set ProcessStartInfo.StandardOutputEncoding = Console.OutputEncoding in order to decode the commands' output using the console output code page that they should be honoring.

[tristanlabelle]

Note that <PostBuildEvent> is implemented in terms of <Exec>, so it is also subject to this issue.