Improved management of multi-char keyboards (Vietnamese and Korean)

[holzschu]

These changes improve upon PR #409 in the following way:

• allow Vietnamese keyboard input (Tiếng Việt Telex). To test:

  • activate the Tiếng Việt Telex keyboard
  • type R a s t [space] v u i [space] d d u w o j c [space] g a j w p [space] b a j n
  • observe on screen: Rất vui được gặp bạn
  • see e.g. https://vietnameselab.com/blog/vietnamese-typing for example for more information.

• improve Korean keybard input. To test:

  • activate Korean keyboard
  • type ㅇ ㅜ ㅇ
  • observe 웅 (that one wasn't working with Allow keyboard input on iOS for languages with multi-char input (Japanese, Chinese, Korean, Hindi) #409, but I didn't detect it because, well, I'm not good in Korean).

The way these work is by calling resetInputBuffer() after every text insertion in insertText(), but only for Vietnamese and Korean keyboards. I tried enabling it for other multi-char keyboards (Japanese), but it causes a crash elsewhere (which may be due to a different issue, but we'll deal with it later).

[migueldeicaza]

Oh this is interesting - is this the expectation that folks have when using those keyboards?

and can you tell me more about the Japanese crash?

[migueldeicaza]

So one fear that I have is that this is not using UITextInput correctly, and it is working behind its back. I wonder if a proper UITExtInput support would address these

[holzschu]

Oh this is interesting - is this the expectation that folks have when using those keyboards?

Yes, definitely. That's one of the oldest feature request for a-Shell (proper support of Vietnamese and Korean keyboards). That is the most common way to enter the diacritics used by Vietnamese, as the "long-press-on-key-to-make-diacritics-appear" method is not convenient for them.

and can you tell me more about the Japanese crash?

If I call resetInputBuffer() also for Japanese keyboards, that is:

if (keyboardLanguage.hasPrefix("vi") || keyboardLanguage.hasPrefix("ko") || keyboardLanguage.hasPrefix("ja")) {
    resetInputBuffer() 
}

then the second set of Japanese ideograms I enter causes a crash in this function:

 func fullRange(in baseString: String) -> Range<String.Index> {
    let beginIndex = baseString.index(baseString.startIndex, offsetBy: startPosition.offset)
    let endIndex = baseString.index(beginIndex, offsetBy: endPosition.offset - startPosition.offset)
    return beginIndex..<endIndex
 }

itself called by:

            textInputStorage.replaceSubrange(rangeToReplace.fullRange(in: textInputStorage), with: newText)

The error is: "String index is out of bounds", because textInputStorage is "" and rangeToReplace is not empty (which is weird).

[migueldeicaza]

How could I test the Japanese behavior here?

Alternatively, what is the value of startPosition? I wonder if just being defensive about that, making sure that beginIndex..<endIndex is within the range of the baseString is enough. Because what is happening is that the startPosition is out of sync with the request.

startPosition is something that UIKit asks us to create.

[holzschu]

The easiest way to check for yourself for Japanese crash is to insert this snippet between lines 1110 and 1111 of Sources/SwiftTerm/iOS/iOSTerminalView.swift:

else {
    // resetInputBuffer is *required* for Tiếng Việt Telex and Korean keyboards
    // it causes a crash with Japanese keyboards.
    if let keyboardLanguage = self.textInputMode?.primaryLanguage {
        if (keyboardLanguage.hasPrefix("vi") || keyboardLanguage.hasPrefix("ko") || keyboardLanguage.hasPrefix("ja")) {
            resetInputBuffer() 
        }
    }
    self.send(txt: text)
}

[migueldeicaza]

Could you try this patch? I tried using the Vietnamese telex keyboard and it works:

https://gist.github.com/migueldeicaza/30c3a5e47ef64192127478eb0654e060

This works with the scenario you showed above for Vietnamese Telex, with a couple of caveats:

• The "Rats" Turns into "R'at" (accent on the a) but not the other character. I tried this on iMessages, and the first time I got it the way you showed it, but since then I can no longer get it to render again, so I suspect we are correct?

[holzschu]

I'm going to try the patch. In my experiments, letter+s turning into accented letter is relatively robust (it happens all the time), but dd is more fragile: without the call to resetInputBuffer(), dd first turns into đ, then turns back into "dd" as I type the next letter. That's why I had to add the call to resetInputBuffer().

[holzschu]

The last beginTextInputEdit() on the patch, 3 lines from the end, feels weird. Shouldn't it be endTextInputEdit()?

[migueldeicaza]

I am not sure I follow, this is what I see at the end, and it is an endTextInputEdit:

[holzschu]

Ah yes, thank you, that was a mistake on my end (I pasted in the wrong buffer).

Here are the results of my tests:

• Chinese-Japanese input: it's much better with your patch. I can now delete keys that I typed, and escape when I have entered a sequence I don't like.
• Vietnamese input: it works fine. I've entered "dduwojp" (a difficult sequence) several times, and got the expected result each time.
• Korean input: ㅇ followed by ㅜ produces 우 (as expected), but ㅇ ㅜ ㅇ produces 우ㅇ instead of 웅.

[migueldeicaza]

Ok, I can not seem to trick UITextInput into not composing the next value, despite repeated efforts.

I instrumented the code, and I can not see in the logs the input method system using setMarkedText/unmarkText

After "우" there is no replace sequence triggered with selectedTextRange -> deleteBackward -> insertText before
the final consonant. Sadly, this protocol has historically been terribly documented.

I think that I will accept the band-aid at this point, but likely on top of the patch I pasted above.

[migueldeicaza]

Can you try the branch input-update, it should have special casing for Korean.

[holzschu]

The branch input-update works nicely. It's great to have something that works for all languages (in my test set), without any special cases (well, except for Korean). Thank you very much for your efforts on this.

As you say, it would be great if those protocols were actually documented. It's insane that the same code works for Tiếng Việt Telex and doesn't work for Korean.

[migueldeicaza]

Ok merged the alternative.