GraalPython on Windows consistently reports that only a "dumb" terminal is available

[mwsealey]

I am sure this is a well known behavior but these days to not expect a terminal to support modern VT features feels bad. On Windows this is provided by a kernel32.dll call SetConsoleMode with two flags, ENABLE_VIRTUAL_TERMINAL_PROCESSING and possibly improved with ENABLE_VIRTUAL_TERMINAL_INPUT. This has to be enabled on a per-process basis (e.g. it cannot be set by an application which quits, and be inherited by the next application to run) so every GraalPython instance needs to do so.

This is usually handled for CPython in packages like 'colorama' and even the CPython 3.13 REPL by directly accessing the Windows API through ctypes but for a Java environment this is probably not a suitable method. Luckily the Java library doing terminal processing for Graal and the source of the "dumb terminal" warning message - JLine - supports this just fine through all of the available JANSI, JNI, JNA and FFM TerminalProviders and would provide the same support on Windows.

JNA support is explicitly disabled by GraalPy's frontend, which is curious decision and writes off support via that TerminalProvider. JANSI and JNI support are potentially less desirable still than JNA, and in any case require stub libraries, which may explain why they are not in use. In any case the latest JLine release deprecates them. But Graal lives in a Java 23+ world and the JLine FFM (Panama) TerminalProvider should be available for use.

Reasons why Windows-based Graal languages should support decent terminal processing:

• Windows 11 comes with the improved ConHost underlying an enhanced Windows Terminal by default, a terminal layer which has been expressly updated to support modern VT processing and may have been doing so for near a decade.
• Windows 10 would be the last version of Windows not to come with the updated Terminal app, but still runs the updated ConHost albeit perhaps not as feature-rich. It is about to go out of Support this year (2025) some time after the release of Java 25
• If users do not wish to use ConHost based terminals they'll be using something else - all the good terminal providers (e.g. ConEmu) have supported enhanced VT processing for years.
• CPython 3.13.x supports it as above and moving to 3.13 compatibility without the REPL would be embarrassing

Reasons why it might not be as easy as updating/massaging the JLine usage:

• People who like e.g. the Java-based POSIX emulation might not be happy about FFM usage
• FFM TerminalProvider expressly adds a requirement for --enable-native-access=ALL-UNNAMED or similar passed to the JVM and this will get stricter over time with future Java (without it, it'll throw a warning today, but Java 26 may throw an exception if missing)
• GraalPython would need a reliable way to detect environments where FFM is not desirable and then enforce the dumb terminal.

[steve-s]

JNA support is explicitly disabled by GraalPy's frontend, which is curious decision and writes off support via that TerminalProvider. JANSI and JNI support are potentially less desirable still than JNA, and in any case require stub libraries, which may explain why they are not in use. In any case the latest JLine release deprecates them. But Graal lives in a Java 23+ world and the JLine FFM (Panama) TerminalProvider should be available for use.

Yes, exactly. We shade the JLine library and the JNA and JNI terminal providers are not included in that artifact.

JNA provider is disabled because it is not supported on Native Image and would also require another dependency.

For JNI provider: we would have to shade accordingly the stub C library for JNI, build it on all supported platforms and bundle all the native libraries into the resulting jar. We can do that, but it would be relatively complex and until now we did not have any feedback regarding JLine on Windows - so thank you for bringing this to our attention!

Since in the next release Native Image will add support for FFM on all supported platforms, the FFM terminal provider is what we plan to go ahead with. There is already PR in progress that will add and enable by default the FFM terminal provider in our shaded JLine for JDK22+ (using multi release jar). If native access is not enabled, then JLine will automatically fall back to dumb terminal on Windows and to the exec terminal on Linux/MacOS. In our launchers in standalone distributions we will enable native access, and for the embedding use case it will be up to the user who controls the java options to decide whether to enable it or not.

I will post updates regarding the FFM based terminal provider in this thread.