sql: rewrite scid string literals to use scid() for index support#8984
Open
vincenzopalazzo wants to merge 2 commits intoElementsProject:masterfrom
Open
sql: rewrite scid string literals to use scid() for index support#8984vincenzopalazzo wants to merge 2 commits intoElementsProject:masterfrom
vincenzopalazzo wants to merge 2 commits intoElementsProject:masterfrom
Conversation
vincenzopalazzo
force-pushed
the
claude/eloquent-meitner
branch
from
06ba1c8 to
922b1e1
Compare
vincenzopalazzo
force-pushed
the
claude/eloquent-meitner
branch
from
922b1e1 to
8627c69
Compare
Short channel IDs were stored as TEXT strings (e.g., "735095x480x1") in
SQLite, which prevented efficient use of indexes on SCID columns. Change
storage to INTEGER (the u64 encoding), using a custom "SCID" column type
so the result-reading code can detect these columns and format them back
as "NNNxNNNxNNN" strings for backward-compatible JSON output.
Add two new SQL functions:
- scid('NNNxNNNxNNN') -> integer: for efficient WHERE clause filtering
- fmt_scid(integer) -> 'NNNxNNNxNNN': for formatting in SQL expressions
Fixes ElementsProject#8941
When users query with WHERE in_channel='735095x480x1', the string
literal is compared directly against the integer SCID column, forcing
SQLite to perform a full table scan even when an index exists.
Automatically rewrite scid string literals (matching NNNxNNNxNNN format)
to use the scid() function before passing the query to SQLite, so
'735095x480x1' becomes scid('735095x480x1'). This allows SQLite to
use indexes on SCID columns transparently.
Queries already using scid() explicitly are detected and left unchanged.
Changelog-Fixed: sql plugin now automatically translates short_channel_id string literals to integers for efficient index usage.
Fixes: ElementsProject#8941
vincenzopalazzo
force-pushed
the
claude/eloquent-meitner
branch
from
8627c69 to
84562df
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
WHERE in_channel='735095x480x1', the string literal was compared directly against the integer SCID column, forcing SQLite into a full table scan even with an index presentjson_sql()passes user queries directly tosqlite3_prepare_v2()without translating scid string literals to integersNNNxNNNxNNNformat to use thescid()function before query execution, so'735095x480x1'becomesscid('735095x480x1')transparentlyFixes #8941
Changelog-Fixed: sql plugin now automatically translates short_channel_id string literals to integers for efficient index usage.
Test plan
SELECT * FROM forwards WHERE in_channel='735095x480x1'now use indexesscid('...')explicitly are left unchanged