Commit 97ed43e
committed
feat: Optimize Scopus backend queries with functional indexes (issue #60)
## Problem
Scopus backend was taking 37ms per cached query, 40x slower than other
cached backends (<1ms). This was caused by SQL queries using LOWER()
functions without corresponding functional indexes, forcing full table
scans of Scopus's 30k+ journal records.
## Root Cause
The `search_journals_by_name()` method uses `LOWER(normalized_name)` and
`LOWER(display_name)` for case-insensitive matching, but no functional
indexes existed on these expressions. SQLite could not use the regular
indexes on these columns when wrapped in LOWER(), resulting in:
- Full table scan of 30,272 Scopus journals
- 37ms query time vs <1ms for other backends
## Solution
Add functional indexes on `LOWER(normalized_name)` and `LOWER(display_name)`
to enable efficient case-insensitive lookups for large datasets.
## Changes
- Add `idx_journals_normalized_name_lower` functional index
- Add `idx_journals_display_name_lower` functional index
## Performance Impact
Scopus cached queries: 37ms → <2ms (20x+ faster)
Before:
✓ scopus: BackendStatus.FOUND [cached] (37.38ms)
After:
✓ scopus: BackendStatus.FOUND [cached] (1.64ms)
This brings Scopus performance in line with other cached backends:
- doaj: 0.81ms
- retraction_watch: 0.71ms
- scopus: 1.64ms ✓
## Testing
All 248 tests pass. Performance verified with multiple queries through CLI.
Resolves #60
Part of #52 - Performance: Optimize caching to eliminate redundant API calls1 parent c070a4f commit 97ed43e
1 file changed
+2
-0
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
154 | 154 | | |
155 | 155 | | |
156 | 156 | | |
| 157 | + | |
| 158 | + | |
157 | 159 | | |
158 | 160 | | |
159 | 161 | | |
| |||
0 commit comments