[Discussion] System.Console re-design

[adamsitnik]

The initial design of System.Console was mainly driven by Windows OS capabilities and available APIs.

When .NET became cross platform, a number of issues arose (click on the details button below to see the full list) as there was no good way of mapping some Windows-specific concepts to Unix. A good example are all Console.Window* APIs: on Unix the Window is owned by the Terminal app, the .NET Console App has no control over it.

The current design makes no distinction between Console and Terminal, which is also a source of plenty of issues (mostly for apps that have redirected output).

Moreover, System.Console is a static class and there is no common abstraction that would allow not only for testing but also for integrating projects like spectre.console and System.CommandLine.Rendering.

By creating this issue I would like to start a public and transparent discussion about what is wrong with the current design of System.Console and how we could fix that. Based on the results of this discussion, I would like to create a proposal for .NET 7.

cc @jonsequitur @KathleenDollard @patriksvensson @colombod @Keboo @HowardvanRooijen @daxian-dbw

• Console.ReadKey:
  • Console.ReadKey returns incomplete information for some key sequences on Linux Console.ReadKey returns incomplete information for some key sequences on Linux #802 (.NET 6)
  • System.ConsoleKeyInfo can not handle Unicode surrogate pair and Emoji Sequences System.ConsoleKeyInfo can not handle Unicode surrogate pair and Emoji Sequences #27828
  • Console.ReadKey() doesn't work for keys from the numeric keypad in the PuTTY ssh client Console.ReadKey() doesn't work for keys from the numeric keypad in the PuTTY ssh client #25735
  • [Windows Terminal] Console.KeyAvailable causes the next Unicode character input 'EN DASH' to be skipped in console [Windows Terminal] Console.KeyAvailable causes the next Unicode character input 'EN DASH' to be skipped in console #38966
  • .NET 5 fails to read the HOME key on ubuntu16.04 .NET 5 fails to read the HOME key on ubuntu16.04 #44621
  • Console.ReadKey and pressing SHIFT+END returns invalid escape sequence on WSL/Ubuntu Console.ReadKey and pressing SHIFT+END returns invalid escape sequence on WSL/Ubuntu #45597
  • System.Console.ReadKey fails in MSYS' console mintty System.Console.ReadKey fails in MSYS' console mintty #49210
  • Let System.ConsoleKeyInfo able to represent keypress in the Unicode world Let System.ConsoleKeyInfo able to represent keypress in the Unicode world #51085
  • Console.ReadKey throws in unexpected circumstances Console.ReadKey throws in unexpected circumstances #59059
  • Support language input keys Support language input keys #63034
• Redirection:
  • console redirection is broken console redirection is broken #22314
  • Don't dup stdin & stderr as this can break some std io redicection code Don't dup stdin & stderr as this can break some std io redicection code #25394
  • Console.CursorLeft, Console.CursorTop, Console.WindowWidth, Console.WindowHeight and Console.WindowTopLeft doesn't throw an IOException for invalid values on Windows 7 x86 when Input is redirected Console.CursorLeft, Console.CursorTop, Console.WindowWidth, Console.WindowHeight and Console.WindowTopLeft doesn't throw an IOException for invalid values on Windows 7 x86 when Input is redirected #43138
• Colors:
  • Bright console colors don't work correctly on Linux Bright console colors don't work correctly on Linux #23241
  • Implement a way to enable color in console even when the output is redirected Implement a way to enable color in console even when the output is redirected #33980
  • Overload Console.Write and Console.WriteLine to take ConsoleColor as a parameter Overload Console.Write and Console.WriteLine to take ConsoleColor as a parameter #39746
  • Developers using Console can employ VT/ANSI APIs (eg to use embedded color codes) Developers using Console can employ VT/ANSI APIs (eg to use embedded color codes) #44513 (.NET 6)
  • Add overloads to Console.Write that also take a ConsoleColor parameter ([API Proposal]: Add overloads to Console.Write that also take a ConsoleColor parameter #61731)
  • Console.Foreground does not correctly set the color in every Terminal (Console.Foreground does not correctly set the color in every Terminal #62186)
• Terminal:
  • MacOS 10.13.6 Console.SetWindowSize() exception MacOS 10.13.6 Console.SetWindowSize() exception #27216
  • Terminal hardware is always left in application mode (keypad_xmit) after running a console program Terminal hardware is always left in application mode (keypad_xmit) after running a console program #27626
  • Console.Clear() doesn't clear the scrollback buffer on Unix-like platforms Console.Clear() doesn't clear the scrollback buffer on Unix-like platforms #28355
  • Setting Console.CursorVisible property does not change visibility in Linux Setting Console.CursorVisible property does not change visibility in Linux #31063
  • Cannot use left/right arrow keys to edit text in Console.ReadLine on Unix Cannot use left/right arrow keys to edit text in Console.ReadLine on Unix #38051
  • System.Console.Read() needlessly changes terminal settings on linux System.Console.Read() needlessly changes terminal settings on linux #49129
  • Console issues with multithreaded c# app only when run on Linux Console issues with multithreaded c# app only when run on Linux #49301
• Encoding:
  • Changing console's OutputEncoding on linux to unicode generates garbage Changing console's OutputEncoding on linux to unicode generates garbage #29735
  • Consider changing the default console encoding on Windows to UTF-16 Consider changing the default console encoding on Windows to UTF-16 #31466
  • Console.OutputEncoding by default returns a wrong value on Windows Nano Server Console.OutputEncoding by default returns a wrong value on Windows Nano Server #42959\
  • Console UTF-8 input is misbehaving on Windows Console UTF-8 input is misbehaving on Windows #43295
• Console.CancelKeyPress:
  • Third party code may remove cancel keypress handler Third party code may remove cancel keypress handler #30464
  • Setting Console.TreatControlCAsInput to true causes text to be removable Setting Console.TreatControlCAsInput to true causes text to be removable #44667
  • .NET 5 apps can no longer intercept SIGINT signals (receive CancelKeyPress events) when running under Docker .NET 5 apps can no longer intercept SIGINT signals (receive CancelKeyPress events) when running under Docker #51221
• Other:
  • FreeBSD: System.Console is not working right FreeBSD: System.Console is not working right #23653
  • Console.Write prints a new line on iOS Console.Write prints a new line on iOS #36440
  • Console.In.Peek() always returning EOF after the first read line on Windows Console.In.Peek() always returning EOF after the first read line on Windows #40735
  • Console.KeyAvailable throws when input isn't redirected on Windows 7 x86 Console.KeyAvailable throws when input isn't redirected on Windows 7 x86  #43015
  • "System.UnauthorizedAccessException: Access to the path is denied." from Console.WriteLine on Ubuntu 20.10 "System.UnauthorizedAccessException: Access to the path is denied." from Console.WriteLine on Ubuntu 20.10 #48654
• Performance:
  • Console.OpenStandard* few times slower on Linux Console.OpenStandard* few times slower on Linux #31396
• New APIs:
  • We need a cross-platform way to detect key-down state We need a cross-platform way to detect key-down state #24314
  • Enhance Console.ReadLine() to return at Tab Enhance Console.ReadLine() to return at Tab #800
  • Add API for detecting broken pipe when write to stdout(or stderr) Add API for detecting broken pipe when write to stdout(or stderr) #30540
  • Proposal: Async Console I/O APIs Proposal: Async Console I/O APIs #299
• Test issues:
  • [mono] Test failed on windows: WindowAndCursorProps.Title_Set_Windows [mono] Test failed on windows: WindowAndCursorProps.Title_Set_Windows #34454
  • System.Console tests failing on iOS System.Console tests failing on iOS #36878
  • System.Console.Tests fail on Android System.Console.Tests fail on Android #37465
  • CancelKeyPressTests.HandlerInvokedForSigQuit test failing on OSX in CI CancelKeyPressTests.HandlerInvokedForSigQuit test failing on OSX in CI #38998
• Docs:
  • System.Console: Backport MS Docs documentation to triple slash System.Console: Backport MS Docs documentation to triple slash #48959

Tagging subscribers to this area: @carlossanlop
See info in area-owners.md if you want to be subscribed.

Issue Details

---

The initial design of System.Console was mainly driven by Windows OS capabilities and available APIs.

When .NET became cross platform, a number of issues arose (click on the details button below to see the full list) as there was no good way of mapping some Windows-specific concepts to Unix. A good example are all Console.Window* APIs: on Unix the Window is owned by the Terminal app, the .NET Console App has no control over it.

The current design makes no distinction between Console and Terminal, which is also a source of plenty of issues (mostly for apps that have redirected output).

Moreover, System.Console is a static class and there is no common abstraction that would allow not only for testing but also for integrating projects like spectre.console and System.CommandLine.Rendering.

By creating this issue I would like to start a public and transparent discussion about what is wrong with the current design of System.Console and how we could fix that. Based on the results of this discussion, I would like to create a proposal for .NET 7.

cc @jonsequitur @KathleenDollard @patriksvensson @colombod @Keboo @HowardvanRooijen @daxian-dbw

• Console.ReadKey:
  • Console.ReadKey returns incomplete information for some key sequences on Linux Console.ReadKey returns incomplete information for some key sequences on Linux #802 (.NET 6)
  • System.ConsoleKeyInfo can not handle Unicode surrogate pair and Emoji Sequences System.ConsoleKeyInfo can not handle Unicode surrogate pair and Emoji Sequences #27828
  • Console.ReadKey() doesn't work for keys from the numeric keypad in the PuTTY ssh client Console.ReadKey() doesn't work for keys from the numeric keypad in the PuTTY ssh client #25735
  • [Windows Terminal] Console.KeyAvailable causes the next Unicode character input 'EN DASH' to be skipped in console [Windows Terminal] Console.KeyAvailable causes the next Unicode character input 'EN DASH' to be skipped in console #38966
  • .NET 5 fails to read the HOME key on ubuntu16.04 .NET 5 fails to read the HOME key on ubuntu16.04 #44621
  • Console.ReadKey and pressing SHIFT+END returns invalid escape sequence on WSL/Ubuntu Console.ReadKey and pressing SHIFT+END returns invalid escape sequence on WSL/Ubuntu #45597
  • System.Console.ReadKey fails in MSYS' console mintty System.Console.ReadKey fails in MSYS' console mintty #49210
  • Let System.ConsoleKeyInfo able to represent keypress in the Unicode world Let System.ConsoleKeyInfo able to represent keypress in the Unicode world #51085
• Redirection:
  • console redirection is broken console redirection is broken #22314
  • Don't dup stdin & stderr as this can break some std io redicection code Don't dup stdin & stderr as this can break some std io redicection code #25394
  • Console.CursorLeft, Console.CursorTop, Console.WindowWidth, Console.WindowHeight and Console.WindowTopLeft doesn't throw an IOException for invalid values on Windows 7 x86 when Input is redirected Console.CursorLeft, Console.CursorTop, Console.WindowWidth, Console.WindowHeight and Console.WindowTopLeft doesn't throw an IOException for invalid values on Windows 7 x86 when Input is redirected #43138
• Colors:
  • Bright console colors don't work correctly on Linux Bright console colors don't work correctly on Linux #23241
  • Implement a way to enable color in console even when the output is redirected Implement a way to enable color in console even when the output is redirected #33980
  • Overload Console.Write and Console.WriteLine to take ConsoleColor as a parameter Overload Console.Write and Console.WriteLine to take ConsoleColor as a parameter #39746
  • Developers using Console can employ VT/ANSI APIs (eg to use embedded color codes) Developers using Console can employ VT/ANSI APIs (eg to use embedded color codes) #44513 (.NET 6)
• Terminal:
  • MacOS 10.13.6 Console.SetWindowSize() exception MacOS 10.13.6 Console.SetWindowSize() exception #27216
  • Terminal hardware is always left in application mode (keypad_xmit) after running a console program Terminal hardware is always left in application mode (keypad_xmit) after running a console program #27626
  • Console.Clear() doesn't clear the scrollback buffer on Unix-like platforms Console.Clear() doesn't clear the scrollback buffer on Unix-like platforms #28355
  • Setting Console.CursorVisible property does not change visibility in Linux Setting Console.CursorVisible property does not change visibility in Linux #31063
  • Cannot use left/right arrow keys to edit text in Console.ReadLine on Unix Cannot use left/right arrow keys to edit text in Console.ReadLine on Unix #38051
  • System.Console.Read() needlessly changes terminal settings on linux System.Console.Read() needlessly changes terminal settings on linux #49129
  • Console issues with multithreaded c# app only when run on Linux Console issues with multithreaded c# app only when run on Linux #49301
• Encoding:
  • Changing console's OutputEncoding on linux to unicode generates garbage Changing console's OutputEncoding on linux to unicode generates garbage #29735
  • Consider changing the default console encoding on Windows to UTF-16 Consider changing the default console encoding on Windows to UTF-16 #31466
  • Console.OutputEncoding by default returns a wrong value on Windows Nano Server Console.OutputEncoding by default returns a wrong value on Windows Nano Server #42959\
  • Console UTF-8 input is misbehaving on Windows Console UTF-8 input is misbehaving on Windows #43295
• Console.CancelKeyPress:
  • Third party code may remove cancel keypress handler Third party code may remove cancel keypress handler #30464
  • Setting Console.TreatControlCAsInput to true causes text to be removable Setting Console.TreatControlCAsInput to true causes text to be removable #44667
  • .NET 5 apps can no longer intercept SIGINT signals (receive CancelKeyPress events) when running under Docker .NET 5 apps can no longer intercept SIGINT signals (receive CancelKeyPress events) when running under Docker #51221
• Other:
  • FreeBSD: System.Console is not working right FreeBSD: System.Console is not working right #23653
  • Console.Write prints a new line on iOS Console.Write prints a new line on iOS #36440
  • Console.In.Peek() always returning EOF after the first read line on Windows Console.In.Peek() always returning EOF after the first read line on Windows #40735
  • Console.KeyAvailable throws when input isn't redirected on Windows 7 x86 Console.KeyAvailable throws when input isn't redirected on Windows 7 x86  #43015
  • "System.UnauthorizedAccessException: Access to the path is denied." from Console.WriteLine on Ubuntu 20.10 "System.UnauthorizedAccessException: Access to the path is denied." from Console.WriteLine on Ubuntu 20.10 #48654
• Performance:
  • Console.OpenStandard* few times slower on Linux Console.OpenStandard* few times slower on Linux #31396
• New APIs:
  • We need a cross-platform way to detect key-down state We need a cross-platform way to detect key-down state #24314
  • Enhance Console.ReadLine() to return at Tab Enhance Console.ReadLine() to return at Tab #800
  • Add API for detecting broken pipe when write to stdout(or stderr) Add API for detecting broken pipe when write to stdout(or stderr) #30540
  • Proposal: Async Console I/O APIs Proposal: Async Console I/O APIs #299
• Test issues:
  • [mono] Test failed on windows: WindowAndCursorProps.Title_Set_Windows [mono] Test failed on windows: WindowAndCursorProps.Title_Set_Windows #34454
  • System.Console tests failing on iOS System.Console tests failing on iOS #36878
  • System.Console.Tests fail on Android System.Console.Tests fail on Android #37465
  • CancelKeyPressTests.HandlerInvokedForSigQuit test failing on OSX in CI CancelKeyPressTests.HandlerInvokedForSigQuit test failing on OSX in CI #38998
• Docs:
  • System.Console: Backport MS Docs documentation to triple slash System.Console: Backport MS Docs documentation to triple slash #48959

Author:	adamsitnik
Assignees:	adamsitnik
Labels:	Design Discussion, area-System.Console
Milestone:	Future

[adamsitnik]

@mdh1418 @MaximLipnin is there any chance that you could share your experience from #41184 #50931 ?

[mdh1418]

@adamsitnik The work in #41184 and #50931 was primarily done to leverage the platform compatibility analyzer and warn users when a particular API was unsupported on the specific platforms. I believe we had taken note of which APIs failed with PlatformNotSupportedException when running library tests on Browser-wasm Android and also iOS +tvOS (which also had PlatformNotSupportedException for System.Console APIs, and we proceeded to mark those APIs with the Unsupported attribute

[phillip-haydon]

Microsoft needs to give @patriksvensson a ton of money and replace with spectre.

[HowardvanRooijen]

@phillip-haydon If you notice, Patrik is CC'd on the first message in the thread, because we've been having fortnightly calls for many months on the subject! The point of this thread is to mobilise "an army of the willing" who want to see these improvements invested in, and delivered! My original request was "please can we interop between System.CommandLine and Spectre?" 😁

[phillip-haydon]

I saw, I'm just totally blown away at spectre, been following it on twitter, we need to give Patrik more recognition that he's built something amazing :D (and the contributors to the project too)

[HowardvanRooijen]

100% I've said that Spectre delivers the "high level productivity API", that enables me to build impressive things quickly. It's like having the WebControls / WinForm Controls / WPF Components for the CLI, but there are also the lower-level APIs that are needed to deliver that experience consistently across all the platforms that .NET Supports. Hence my tagline of:

Spectre.Console + System.CommandLine = ❤

[maloo]

Please make sure it works great with https://github.com/migueldeicaza/gui.cs

[adamsitnik]

@migueldeicaza would you like to provide some feedback based on your experience with building https://github.com/migueldeicaza/gui.cs?