Description
CollationElements::upcoming
currently holds CharacterAndClassAndTrieValue
. Logically its last element (when not at the end of the stream) needs to hold CharacterAndTrieValue
(which currently doesn't exist in the collator but exists in the normalizer) and the other elements can be CharacterAndClass
. (When we have CharacterAndClass
but want CharacterAndTrieValue
, we can synthesize the trie value from the class.)
Refactor upcoming
so that there's a SmallVec
of CharacterAndClass
and a separate Option<CharacterAndTrieValue>
(that logically comes after the contents of the SmallVec
).
This would also remove the special case where at the end of the stream we find an already-decomposed non-starter in the last slot of upcoming
and just put it back. After the refactoring, the Option
would be None
in this case instead of the nature of the last slot changing upon the end of the stream.