Skip to content

Comments

EXPLAIN QUERY PLAN: fix incorrect join order, add missing annotations, convert some snapshot tests to EQP-only#5511

Merged
jussisaurio merged 10 commits intomainfrom
explain-query-plan-fix
Feb 21, 2026
Merged

EXPLAIN QUERY PLAN: fix incorrect join order, add missing annotations, convert some snapshot tests to EQP-only#5511
jussisaurio merged 10 commits intomainfrom
explain-query-plan-fix

Conversation

@jussisaurio
Copy link
Collaborator

@jussisaurio jussisaurio commented Feb 21, 2026

EXPLAIN QUERY PLAN didn't show correct join order

  • EXPLAIN QUERY PLAN was showing original join order, not the actual join order after optimizer pass. Highly annoying and clankers get mega confused by it too.
  • Fixing the above issue revealed that left join ordering fuzz test (based on EQP output) was not accounting for LEFT->INNER optimizations, so fix that by only asserting on LEFT JOINs that survived the optimizer pass

EXPLAIN QUERY PLAN was missing stuff that SQLite has

  • EXPLAIN QUERY PLAN was missing annotations for GROUP BY, DISTINCT, subqueries + it had an incorrect USE TEMP B-TREE FOR ORDER BY annotation because we never use temp b-tree for sorting, so instead print USE SORTER FOR ORDER BY. Also add turso-custom prints like USE HASH TABLE FOR count(DISTINCT) (sqlite doesn't have hash table)

Add new snapshot format that only prints EQP instead of full bytecode, in cases where we don't much care about the full bytecode

  • Add new snapshot-eqp snapshot test format that just prints the EQP, not the bytecode
  • Convert a bunch of snapshots to use snapshot-eqp (for example, most TPC-H tests; in many of them we just care about the join order and maybe order by elimination)
  • Note: the massive diff is due to the above - main change commits are smaller

jussisaurio and others added 2 commits February 21, 2026 21:07
EXPLAIN QUERY PLAN was iterating joined_tables() in original SQL order
instead of the optimizer's join_order. This made EQP output misleading
when the optimizer reordered joins. Now builds visit_order from
join_order first, then appends hash-join build tables.

Also fixes the left join ordering fuzz test to account for the
LEFT-to-INNER join simplification optimization that was added after
the test was written.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a new 'snapshot-eqp' directive that captures only the EXPLAIN
QUERY PLAN output without bytecode. Useful for tests that verify
join ordering and access method selection where bytecode details
are noise.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jussisaurio and others added 8 commits February 21, 2026 21:36
…ggregates

- Emit "USE SORTER FOR GROUP BY" when GROUP BY uses a sorter
- Emit "USE SORTER FOR ORDER BY" / "USE TEMP B-TREE FOR ORDER BY"
  depending on whether sorter or heap sort is used
- Emit "SCALAR SUBQUERY N" / "CORRELATED SCALAR SUBQUERY N" for
  scalar subqueries
- Emit "LIST SUBQUERY N" / "CORRELATED LIST SUBQUERY N" for IN
  subqueries
- Emit "USE HASH TABLE FOR func(DISTINCT)" for distinct aggregates

Annotations reflect actual data structures used (sorter, hash table)
rather than copying SQLite's "TEMP B-TREE" phrasing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Bytecode snapshots now include USE SORTER FOR GROUP BY/ORDER BY,
USE HASH TABLE FOR count(DISTINCT), and SCALAR SUBQUERY annotations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… annotations

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jussisaurio jussisaurio force-pushed the explain-query-plan-fix branch from 817ff39 to 3e13ff5 Compare February 21, 2026 19:36
@jussisaurio jussisaurio changed the title Explain query plan fix EXPLAIN QUERY PLAN: fix incorrect join order, add missing annotations, convert some snapshot tests to EQP-only Feb 21, 2026
@jussisaurio jussisaurio merged commit c58b2f6 into main Feb 21, 2026
80 of 81 checks passed
@jussisaurio jussisaurio deleted the explain-query-plan-fix branch February 21, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants