FTS5 CJK Support (ICU or other options)? #3560
Replies: 1 comment 4 replies
-
Hi!
I'm also wondering where that came from. There are ICU extensions for SQLite, but they're meant to add locale-aware implementations of functions like So it's definitely possible to build SQLite with extensions linked into it (and I'm happy to help with more pointers if you need that, unfortunately it requires a custom build at the moment). But I'm not sure if the ICU extensions from the SQLite project would be helpful for better tokenization here. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Heya, thanks again for all your fantastic work with this library and speaking to us at Superlist!
I had a question: I'm looking at adding a full search index to our app. I was interested in trying the ICU tokenizer for FTS5. I saw in a comment that ICU support might be interesting, but I did not see it listed on the official extensions page.
If we wanted to try ICU tokenization, what would you recommendation be? Should we follow the examples in https://github.com/simolus3/sqlite3.dart/tree/e682c9792cc1ea094281a7d2edfc7db5ca802b05/sqlite3 to compile a version with ICU support?
Thanks again,
Brian
Beta Was this translation helpful? Give feedback.
All reactions