Forcing a Docked On-Screen Keyboard in Windows 11 for Headless Setups

[emilwojcik93]

Hi Apollo & Artemis Community and Developers,

Firstly, a huge thank you for creating such powerful tools. Apollo and Artemis have been pivotal in enabling a high-performance headless gaming and remote desktop experience.

I'm writing to share an in-depth journey into a persistent UI challenge and to seek collective wisdom or developer insight: achieving a reliably docked Windows 11 on-screen keyboard (OSK) that behaves like it does on native tablet/handheld hardware.

My Setup

• Host OS: Windows 11 Pro (fully headless – no physical monitor, keyboard, or mouse).
• Streaming Server: Apollo (latest version, via Sunshine).
• Client: Artemis on an Android tablet/phone.

The Goal: Seamless, Integrated On-Screen Keyboard

The objective is to replicate the fluid OSK experience found on devices like the Lenovo Legion Go or ASUS ROG Ally. When the OSK is invoked, it should:

1. Dock cleanly to the bottom of the screen.
2. Automatically resize the active application window (e.g., a game or desktop app) to fit the remaining visible area, preventing any UI overlap.
3. Maintain this docked behavior consistently.

The Ideal Behavior (as seen on native Windows handhelds):

The Current Problem: The Floating Keyboard Dilemma

In the headless setup with Apollo/Artemis, invoking the OSK consistently results in a floating keyboard. This behavior presents several usability issues:

1. It obscures critical UI elements of the underlying application.
2. The application window does not resize, making it impossible to see input fields or content covered by the OSK.
3. The OSK often requires manual repositioning, which is cumbersome, especially in full-screen applications or games.

What I'm Currently Experiencing:

Exhaustive Troubleshooting: A Multi-Pronged Approach

Understanding that Windows 11 largely automates "Tablet Mode" based on hardware cues (which are absent in a headless system), I've undertaken extensive efforts to programmatically induce the desired docked OSK behavior. Here’s a summary of the strategies explored:

1. Deep Registry Modifications

Numerous PowerShell scripts (Force-TabletMode.ps1, Force-TabletMode-Enhanced.ps1, Force-Docked-OSK.ps1) were developed to manipulate a wide array of registry keys. Key targets included:

# HKCU\Software\Microsoft\TabletTip\1.7
Set-ItemProperty -Path "HKCU:\Software\Microsoft\TabletTip\1.7" -Name "KeyboardUIMode" -Value 1 # (1 for Docked)
Set-ItemProperty -Path "HKCU:\Software\Microsoft\TabletTip\1.7" -Name "DockedState" -Value 1
Set-ItemProperty -Path "HKCU:\Software\Microsoft\TabletTip\1.7" -Name "TipbandDesiredVisibility" -Value 1
Set-ItemProperty -Path "HKCU:\Software\Microsoft\TabletTip\1.7" -Name "EdgeTargetDockedState" -Value 1 # New key explored

# HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\ImmersiveShell
Set-ItemProperty -Path "HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\ImmersiveShell" -Name "TabletMode" -Value 1
Set-ItemProperty -Path "HKCU:\SOFTWARE\Microsoft\Windows\CurrentVersion\ImmersiveShell" -Name "SignInMode" -Value 1 # (1 for Tablet, 2 for Desktop)

# HKLM Hardware Simulation (Attempted)
Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\PriorityControl" -Name "ConvertibleSlateMode" -Value 1
Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\PriorityControl" -Name "KeyboardPresent" -Value 0 # Simulate no physical keyboard

Outcome: While settings were applied, the OSK remained floating. This suggested that registry settings alone are insufficient in Windows 11 for this purpose.

2. Direct Win32 API Interventions

A PowerShell script with embedded C# (Direct-Win32-Docking.ps1) was created to send direct messages to the IPTip_Main_Window (TabTip.exe window) using SendMessage with WM_USER + 94 (a known message for influencing docking).

// Snippet from embedded C# in Direct-Win32-Docking.ps1
public static bool ForceKeyboardMode(TabTipKeyboardMode mode) {
    IntPtr tabTipHwnd = FindWindow("IPTip_Main_Window", null);
    if (tabTipHwnd != IntPtr.Zero) {
        // WM_USER + 94 is understood to influence docking state
        SendMessage(tabTipHwnd, /*WM_USER*/ 0x0400 + 94, (int)mode, 0);
        return true;
    }
    return false;
}

Outcome: Messages were sent, but the fundamental behavior didn't change, indicating deeper OS-level decisions overriding these messages.

3. COM Interface Manipulation

A C# application (ForceDockedOSK.cs) and PowerShell scripts (Direct-Force-Docked-OSK.ps1) attempted to use COM interfaces like UIHostNoLaunch and ITipInvocation, inspired by WPF TabTip helper implementations.

// From ForceDockedOSK.cs
[ComImport, Guid("4ce576fa-83dc-4F88-951c-9d0782b4e376")] // UIHostNoLaunch
class UIHostNoLaunch {}

[ComImport, Guid("37c994e7-432b-4834-a2f7-dce1f13b834b")] // ITipInvocation
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
interface ITipInvocation { void Toggle(IntPtr hwnd); }

Outcome: Consistently failed to instantiate UIHostNoLaunch. Logs (e.g., DirectForceOSK_20250608_030248.log) showed:
Error creating TabTip COM interface: Cannot find type [TabTipForce.UIHostNoLaunch]: verify that the assembly containing this type is loaded.
This suggests these COM interfaces might be deprecated, internal, or their registration/accessibility has changed in recent Windows 11 versions.

4. Combined Strategies: Process Management & Settings Broadcasts

Scripts were enhanced to:

• Stop/Start TabTip.exe and explorer.exe at strategic points.
• Broadcast system-wide setting changes using SendMessageTimeout(HWND_BROADCAST, WM_SETTINGCHANGE, ...).
  Outcome: Settings were broadcast and processes restarted, but the OSK behavior persisted.

5. Systematic Discovery of Settings

A PowerShell script (Discover-TabletMode-OSK-Settings.ps1) was used to perform an exhaustive search of the file system and registry for keywords related to tablet mode and the OSK.
Log Excerpt (TabletMode_OSK_Discovery_Log_20250608_012751.txt):

[Phase 3: Searching Registry]
...
Found Registry Key: HKEY_CURRENT_USER\Software\Microsoft\TabletTip\1.7
  Value: KeyboardUIMode, Type: REG_DWORD, Data: 1
  Value: DockedState, Type: REG_DWORD, Data: 1
...
Found Registry Key: HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\ImmersiveShell
  Value: TabletMode, Type: REG_DWORD, Data: 1
...

Outcome: Confirmed the registry keys being targeted but also highlighted the sheer number of potentially related settings, many of which are likely interdependent or influenced by hardware state.

6. HID Device Manipulation (Conceptual)

The idea of programmatically disabling physical HID Keyboard and Mouse drivers was considered to make Windows believe it's in a touch-only, slate-like state.

# Conceptual:
# Get-PnpDevice -Class Keyboard | Disable-PnpDevice -Confirm:$false
# Get-PnpDevice -Class Mouse | Disable-PnpDevice -Confirm:$false
# ... apply OSK settings ...
# Get-PnpDevice -Class Keyboard | Enable-PnpDevice -Confirm:$false
# Get-PnpDevice -Class Mouse | Enable-PnpDevice -Confirm:$false

Outcome: This remains a more invasive step, potentially risky for a headless system if not handled carefully, and still might not overcome how the display itself is perceived.

Core Conclusion: It's Likely a Driver/Hardware-Reporting Issue

Despite all software-level manipulations (registry, API calls, COM), Windows 11 in a headless configuration via Apollo/Sunshine stubbornly refuses to dock the OSK and resize applications.

This leads to a strong hypothesis: The behavior is dictated by how the virtual display driver (provided by Sunshine/Apollo's underlying stack) reports its capabilities to Windows. If the OS sees a standard, non-touch, non-convertible desktop display, it will always default to the floating OSK, irrespective of user-level registry tweaks. Native handhelds achieve the docked behavior because their actual hardware and drivers signal "tablet" or "convertible" capabilities.

Key Questions for the Apollo/Sunshine Developers & Community:

1. Virtual Display Capabilities: Is it feasible for the virtual display driver used by Sunshine/Apollo to report different EDID information or hardware capabilities to Windows, such as:
   • Touch-screen capability?
   • "Convertible" or "Slate" chassis type?
2. Simulating Tablet Hardware: Are there lower-level mechanisms (beyond user-mode registry/API) that could make Windows perceive the headless session as being on tablet-like hardware?
3. Client-Side Input Reporting: Could touch input from the Artemis client be translated in a way that signals to the Windows host that the primary interaction is touch, influencing OSK behavior?
4. Alternative OSK Invocation: Does TabTip.exe accept any command-line arguments or undocumented methods to force a specific layout or docking state upon launch?

A solution to this would significantly enhance the usability of Apollo/Artemis for anyone aiming for a seamless, handheld-like experience on a headless Windows host. Any insights, suggestions, or collaboration on this front would be immensely appreciated.

All scripts and detailed logs mentioned are available from my previous interactions if they can be of help.

Thanks for your time and for these fantastic projects!

[ClassicOldSong]

This is a requested and planned feature. It requires to report the physical screen size in the EDID of the virtual display but it requires to update the driver, and EDID parsing and editing isn't implemented yet.

[candemancan]

Please and thank you!

I was able to get this work briefly using this:
https://migueltek.com/tool/simple-edid-editor-for-win-11/

Seems to revert after disconnect though. Will await implementation.