Fix iPadOS hardware-keyboard CJK input and visual indicators (refs #18)#24
Open
ab300819 wants to merge 4 commits into
Open
Fix iPadOS hardware-keyboard CJK input and visual indicators (refs #18)#24ab300819 wants to merge 4 commits into
ab300819 wants to merge 4 commits into
Conversation
The current routing policy only forwards printable keys to UIKit when hasActiveIMEComposition is true, but hasActiveIMEComposition only becomes true after the IME has accepted a first keystroke. On iPadOS hardware keyboard with a CJK input method active, the first printable key therefore stays on the direct terminal path and composition never has a chance to start — the candidate window never appears. Reintroduce isCurrentInputMethodCJK (derived from textInputMode's primary language) and add a policy rule that routes printable keys to the system text input as soon as a CJK input method is selected, rather than waiting for composition to already be underway. Non-CJK paths are unchanged: modifier chords, fallback keys (arrows, Enter, Esc, etc.), CapsLock toggle, and in-progress composition continue to behave exactly as before. Existing policy tests are updated to pass the new parameter; their assertions are unchanged. Refs vivy-company#18 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…switch toast Coordinated changes in GhosttyTerminalView+iOS.swift so the iPadOS Magic Keyboard CJK visual indicators all render correctly on top of the existing hardware-key routing fix: 1. Move the IME proxy frame from (-10000, -10000) to (0, 0). `caretRect(for:)` returns coordinates from `ghostty_surface_ime_point` in the terminal view's coordinate space; UIKit then offsets by the proxy's frame.origin to anchor the candidate window. With an off-screen origin the candidate rendered 10,000 points outside the screen. 2. Raise proxy `alpha` from 0.01 to 1.0. iPadOS suppresses the Caps Lock "拼音" / "ABC" IME-switch toast when the focused text input has alpha below ~1. The proxy stays invisible via clear background, clear text/tint, and a no-op `draw(_:)`. 3. Override `caretRect(for:)` to return a rect at the terminal view's center when there is no active composition. iOS anchors the Caps Lock toast at this rect, so without an explicit center anchor the toast appeared at (0, 0) and clipped against the status bar (or followed the terminal cursor when composition was active and clipped at edges). Composition still anchors at the ghostty cursor for the candidate window. 4. Return nil from `textInputContextIdentifier`. A non-nil identifier makes iOS persist a per-view IME and treats globe-key switches as "automatic", suppressing the system "拼音" / "English" toast that Notes / Mail show. VVTerm has no need for per-view IME memory. Refs vivy-company#18
The previous commit raised the IME proxy `UITextView`'s alpha to 1.0 to unlock the iPadOS Caps Lock IME-switch toast. With the proxy now on screen at full alpha, an empty 1×1 view at the terminal's top-left corner could surface to VoiceOver and other accessibility clients as a focusable text input. Setting `accessibilityElementsHidden = true` keeps the IME bridge functional while excluding it (and its children) from the accessibility tree. Refs vivy-company#18
…licy When a CJK input method is active, the routing policy still needs to keep modifier-bearing keystrokes (Ctrl-C, Cmd-V, Alt-arrow, etc.) on the direct ghostty path so terminal shortcuts are never swallowed by the system IME. The "modifier returns false" branch sits before the CJK branch in `shouldRoutePressToSystemTextInput`; this test pins that ordering so a future refactor that reorders the conditionals fails loudly instead of silently breaking shortcuts under CJK layouts. Refs vivy-company#18
|
Thank you for your contribution! Before we can merge your pull request, we need you to sign our Contributor License Agreement (CLA). Please read and sign the CLA at: https://github.com/vivy-company/vvterm/blob/main/CLA.md Once signed, comment on this PR with: |
Author
|
I have read the CLA Document and I hereby sign the CLA |
Contributor
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
Contributor
|
@ab300819 hi there, any updates on this? did you figure our solution for the issue? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
UITextViewIME proxy on iPad.Why these files are necessary
Routing (
5022fb3)VVTerm/GhosttyTerminal/TerminalHardwareTextInputRoutingPolicy.swift— centralizes when a hardware press should bypass the direct ghostty path and flow through UIKit's IME composition. Without this, CJK IMEs never start composition (the first keystroke goes straight to ghostty as raw Latin).VVTermTests/TerminalHardwareTextInputRoutingPolicyTests.swift— decision-matrix coverage for the routing policy.Visibility (
6027d02)VVTerm/GhosttyTerminal/GhosttyTerminalView+iOS.swift— four coordinated tweaks to the IME proxyUITextViewso iPadOS' IME UI subsystem accepts it:(-10000, -10000)to(0, 0)so UIKit converts ourcaretRect(for:)value (in terminal-view coords fromghostty_surface_ime_point) back into the right window position. With an offscreen origin the candidate window anchored ~10000 points outside the screen.alphafrom0.01to1.0. iPadOS suppresses the Caps Lock "拼音" / "ABC" toast when the focused text input has alpha < 1. The proxy stays visually silent via clear background, clear text/tint, and a no-opdraw(_:).caretRect(for:)to return a center anchor when there is no active composition. iOS anchors the Caps Lock toast on this rect, so without an explicit center anchor the toast clipped against the status bar (or followed the terminal cursor when composing and clipped at edges). Composition still anchors at the ghostty cursor for the candidate window.nilfromtextInputContextIdentifier. A non-nil identifier makes iOS persist a per-view IME and treats globe-key switches as "automatic", suppressing the system "拼音" / "English" toast that Notes / Mail show. VVTerm has only one text input, so per-view IME memory has no benefit.Hardening from review (
705c6d6,3855731)VVTerm/GhosttyTerminal/GhosttyTerminalView+iOS.swift— setaccessibilityElementsHidden = trueon the proxy. With the proxy now on-screen atalpha = 1.0, an empty 1×1 element should not surface to VoiceOver as a focusable text input.VVTermTests/TerminalHardwareTextInputRoutingPolicyTests.swift— pin the modifier-priority precedence: even when a CJK input method is active,Ctrl-,Cmd-, andAlt-keystrokes must stay on the direct ghostty path so terminal shortcuts (Ctrl-C, Cmd-V, etc.) are never captured by the system IME.Verification
Tested on iPad Air 5 (M1) + Apple Magic Keyboard, iPadOS 26.4.2, real device.
n i→ marked text + candidate window at cursor你)TerminalHardwareTextInputRoutingPolicyTests(suite extended in commit 4)Local `xcodebuild test` against the suite remains blocked by my local provisioning/signing environment, unrelated to these changes.
What's still broken (continues on #18, no new issue opened)
Software keyboard CJK composition does not work after this PR. Logs from on-device debugging show iPadOS Pinyin / Kana refuse to call
setMarkedTextagainst the offscreenUITextViewproxy at all — they downgrade to directinsertTextand iOS auto-switches the input mode back to en-US after each keystroke. This is a separate architectural limitation of the proxy approach, not the same bug as the hardware keyboard path.The likely fix is structural: have
GhosttyTerminalViewitself implementUITextInputdirectly, mirroring Ghostty's macOS path, instead of routing through an offscreenUITextView. That's a larger structural change kept out of this PR to stay focused.(Earlier I'd reported the soft keyboard symptom as "IME inverted" in the issue thread — that diagnosis was wrong; the actual symptom is that composition never engages.)
Refs #18