Skip to content

Refactor the upcoming buffer in the collator #2386

Open
@hsivonen

Description

@hsivonen

CollationElements::upcoming currently holds CharacterAndClassAndTrieValue. Logically its last element (when not at the end of the stream) needs to hold CharacterAndTrieValue (which currently doesn't exist in the collator but exists in the normalizer) and the other elements can be CharacterAndClass. (When we have CharacterAndClass but want CharacterAndTrieValue, we can synthesize the trie value from the class.)

Refactor upcoming so that there's a SmallVec of CharacterAndClass and a separate Option<CharacterAndTrieValue> (that logically comes after the contents of the SmallVec).

This would also remove the special case where at the end of the stream we find an already-decomposed non-starter in the last slot of upcoming and just put it back. After the refactoring, the Option would be None in this case instead of the nature of the last slot changing upon the end of the stream.

Metadata

Metadata

Assignees

Labels

A-performanceArea: Performance (CPU, Memory)C-collatorComponent: Collation, normalizationS-mediumSize: Less than a week (larger bug fix or enhancement)T-techdebtType: ICU4X code health and tech debt

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions