Skip to content

Feat: skip heavy ilike fallback for search when we are sure vector search is good - #109

Draft
Altair200333 wants to merge 1 commit into
developfrom
fix/skip-ilike-fallback
Draft

Feat: skip heavy ilike fallback for search when we are sure vector search is good#109
Altair200333 wants to merge 1 commit into
developfrom
fix/skip-ilike-fallback

Conversation

@Altair200333

Copy link
Copy Markdown

TLDR: when doing vector search its falling back to ilike (string search) scan, which is slow and pointless for us, so we can provide flag to disable this fallback. Why? because we store normalized work phone in additional phones - if nothing found, there is no need to make "full scan" fallback that takes 10-100x more time

@aashutoshrathi
aashutoshrathi marked this pull request as ready for review August 27, 2026 10:09
@tomjtobinn

Copy link
Copy Markdown
Collaborator

Sol suggested a cleaner alternative: add a purpose-built phone lookup endpoint and have Go call that instead of adding skipIlikeFallback to the generic search API.

The endpoint would search only normalized primary + additional phone values. It can still use PostgreSQL full-text search: build/query the phone lexemes through a tsvector column backed by a GIN index (either the existing generated vector as a minimal change, or preferably a phone-only generated vector so the endpoint is truly field-scoped), using phoneSearchVector @@ to_tsquery('simple', ...).

The important semantic difference is that a zero-result phone-token lookup is definitive. Phone numbers do not need the CJK/non-tokenizable-text recovery behavior, so the endpoint would return the GIN result directly and never run unaccent(searchVector::text) ILIKE '%...%'. That fallback cannot use the tsvector GIN index because it casts the vector to text, wraps it in a function, and uses a leading wildcard; on a miss it therefore scans essentially the whole eligible table.

This keeps the generic search behavior intact for human-language searches, gives the Go caller an API that expresses its actual intent ("find person by phone"), and avoids exposing an implementation-level fallback switch to every generic search caller.

@aashutoshrathi
aashutoshrathi marked this pull request as draft August 27, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants