Reported incompatibility with GNU Octave

[DHowett]

The GNU Octave is a cross platform project built to use the default OS terminal. Currently we would love to be able to identify if that is the new Windows Terminal. There is some incompatibility between conhost and the new Windows Terminal that is causing strange artifacts and program crashes for users of the new Windows Terminal that became very apparent when win11 made it the default. Octave is primarily developed on linux and crossbuilt for windows users, and we have few windows developers working that part of the program.

For now we wanted to create a script to try to identify if Octave was started in the Windows Terminal to prompt the user to switch, but so far have just settled on checking if the the registry key was set for having Windows Console Host be system default, and prompt the Windows users to switch the system default to that if it's not. Yes, it is a rather poor workaround. but it's effective. Until we have the developer resources to better troubleshoot the conhost / windows terminal incompatibility (or finish developing a separate, cross-platform compatible terminal widget to bundle with Octave), that change solves the problem for the users. But it does come with some false positives for the prompt because it gets set off when the windows setting is "let windows decide" even if that wouldn't use the new terminal, causing some other user confusion. So, it would at least be cleaner if at startup we could positively identify the Windows terminal.

Originally posted by @NRJank in #7434 (comment)

[DHowett]

This as well - thanks for the comment! I hate for us to be incompatible with anything unless we've decreed it so (and mostly that's just "we decree it so when we think 35-year-old Windows-specific apps have had it too good")

[NRJank]

hah, understood, and thanks again. had looked at creating an issue when we had a better idea of what to report was the issue, but useful troubleshooting has been limited to date. If i can find some useful snippets of that I'll link and/or summarize. I think at the moment that it's related to different terminal buffer behavior, which I have read is a known issue between old and new terminals.

[vefatica]

I have been using Octave as a WT profile for a few weeks and I have had no problems. Are particular incompatibilities noted anywhere?

[lhecker]

Out of the links above, this one seems to have the most information: https://octave.discourse.group/t/octave-crashing-with-weird-characters-and-not-loading/4951

[vefatica]

Thanks! What's shown there is bizarre. As I said I have had no problems. I have seen no evidence that Octave uses VT control sequences (but coming from UNIX, it might).

[NRJank]

apologies for letting this sit, seems i didn't catch the later comment notifications.

a small amount of debugging was done and described in this Octave developer comment/thread

Summary description:

The garbled output in the command window probably happens because escape sequence processing gets turned on even when we don’t want it and the scrolling problem happens because the new console doesn’t respect the buffer size that we want to set.

Whether it was reasonable for us to expect the previous behavior or we were just “lucky” that it worked for us, I don’t know (see comment #45 for some more about that).