diff --git a/backlog/tasks/task-49 - Specializer-SQL-support-builtin-long-tail-similarity-string-subscripts-wave-3.md b/backlog/tasks/task-49 - Specializer-SQL-support-builtin-long-tail-similarity-string-subscripts-wave-3.md index 639f9db..7614d6a 100644 --- a/backlog/tasks/task-49 - Specializer-SQL-support-builtin-long-tail-similarity-string-subscripts-wave-3.md +++ b/backlog/tasks/task-49 - Specializer-SQL-support-builtin-long-tail-similarity-string-subscripts-wave-3.md @@ -3,10 +3,10 @@ id: TASK-49 title: >- Specializer SQL support: builtin long tail + similarity + string subscripts (wave 3) -status: In Progress +status: Done assignee: [] created_date: '2026-07-26 16:45' -updated_date: '2026-07-26 17:44' +updated_date: '2026-07-26 18:15' labels: [] milestone: m-7 dependencies: diff --git "a/backlog/tasks/task-50 - Specializer-SQL-support-join-forms-\342\200\224-USING-residual-ON-semi-joins-comma-rewrite-wave-4.md" "b/backlog/tasks/task-50 - Specializer-SQL-support-join-forms-\342\200\224-USING-residual-ON-semi-joins-comma-rewrite-wave-4.md" new file mode 100644 index 0000000..986ccf1 --- /dev/null +++ "b/backlog/tasks/task-50 - Specializer-SQL-support-join-forms-\342\200\224-USING-residual-ON-semi-joins-comma-rewrite-wave-4.md" @@ -0,0 +1,47 @@ +--- +id: TASK-50 +title: >- + Specializer SQL support: join forms — USING, residual ON, semi joins, comma + rewrite (wave 4) +status: In Progress +assignee: [] +created_date: '2026-07-26 18:15' +updated_date: '2026-07-26 19:24' +labels: [] +milestone: m-7 +dependencies: + - TASK-49 +documentation: + - docs/superpowers/specs/2026-07-25-sql-specializer-design.md +type: feature +ordinal: 44000 +--- + +## Description + + +Census-itemized (2026-07-26, post-wave-3: 71 join-blocked cases). STAGE A — no new execution model (~43 cases): JOIN USING desugar incl. chains and repeated statics (8), all-key/semi joins by lifting the no-value-columns restriction (4), star expansion over joined tables incl. key columns reconstructed from the dynamic side (2), comma-join rewrite to INNER equi-probe for dynamic-x-static with equi WHERE conjuncts + residual WHERE (13), equi-ON with residual predicates preserving 0-or-1 multiplicity for INNER and LEFT (12: ON l.id=r.id AND l.val>1 / AND true / AND false / constant equalities), cross join to a PROVABLY 1-row static (4). STAGE B (design gate — present before building): output multiplicity (Emit-continuation in the IR + both backends) for pure non-equi ON and range comma-joins (~20 cases). DESCOPED: dynamic self-joins (~9, needs batch-as-build-side), FULL OUTER (1, build-driven output), NATURAL (2, deeper-blocked by COLUMNS()). Pins fleet FIRST: USING output-column semantics (dedup, qualification), LEFT-join residual-ON vs WHERE placement, ON constant conditions, comma-join equivalence to cross-then-filter. + + +## Acceptance Criteria + +- [x] #1 Join-form pins measured BEFORE implementation (USING star/dedup/qualification; residual ON semantics on INNER and LEFT incl. constants; comma-join = cross-then-filter equivalence; 1-row cross join) — JSON + spec addendum committed +- [x] #2 Stage A ships: USING, all-key/semi, star-over-join, comma equi rewrite, equi-ON residuals — both backends, differential green +- [x] #3 Multiplicity (stage B) gets a written design presented at the stop point — NOT built without explicit go +- [x] #4 Corpus replay: flips recorded, zero FAILs +- [x] #5 mise gate-specializer green + + +## Implementation Notes + + +Pins committed (7bca38c) BEFORE implementation. Fleet: 3 agents, 223k tokens. Headlines: comma == INNER probe bit-exactly (rewrite safe, WHERE conjuncts split freely); USING merged col stays at LEFT position with LEFT value (exact under Inner/Left), both quals addressable, chained USING binds merged, case-insensitive even quoted, USING-after-ON must ERROR; LEFT keeps residual-failing key-matches with all-NULL side, AND false => all-NULL side; ERROR-EAGERNESS trap: single-side residuals are scan-pushed (eager traps in duck) while both-sides are pair-lazy => v0 requires single-side residuals conservatively trap-free, both-sides may trap; cross-to-1-row = empty-keys probe with compile-time 1-entry check (0-row annihilation is free); duplicate star output names survive to Arrow => our typed model cannot represent them, SELECT * both-keys shapes stay honestly unsupported (revised flip estimate ~30-37, not 43). STAGE-A DESIGN FROZEN: JoinSpec += residual SExpr (hit-guarded lowering, match = hit AND residual), SKind::JoinHit for key-column reconstruction, empty val_cols allowed, empty keys = cross with materialization check, USING desugar in bind_from with using-col star suppression. NO IR changes. + +STAGE A COMPLETE. Corpus 383 -> 395 of 678, zero FAILs (deterministic); gate green (753 + 13 xfail); 15 new join oracle tests all green first run. Join-blocked pool 71 -> 13: only dynamic self-joins (10), NATURAL (2, deeper-blocked by COLUMNS()), FULL OUTER (1) remain — the other 58 either match or reclassified to their TRUE deeper blocker (aggregation, NULL-in-value-column for the nulltable cross cases, non-equi -> the cross path where multi-row statics reject via duplicate-map-key, which is CORRECT: a 1-row static would genuinely serve). Implementation: bind_on equality/residual split with the scan_residual totality/sidedness walk; hit-guarded residual lowering threads probe value lanes through block params + the live stack (strict SSA violation caught by corpus replay, fixed); USING desugar with merged-column suppression; key_lane reconstruction (INNER = key expr, LEFT = CASE JoinHit); all-key semi joins via one-side-empty map statics (grammar + verifier extended, round-trip fuzz green); comma rewrite consuming WHERE equi conjuncts with leftover-WHERE reconstruction; UBIGINT/HUGEINT static payloads reject cleanly. STAGE B WRITTEN DESIGN (not built, per AC #3): (1) Term::EmitTo{then,args} — emit current out-row, continue at block (plain Emit stays as row-end); (2) interp: push row + jump; cranelift: emit becomes an h_emit(cx) helper call (per-row ABI return then means only row-done); (3) lowering wraps the row program in a loop over static entries (block-param induction + a ProbeAt inst reading entry idx), residual evaluated per entry; INNER emits per match, LEFT tracks any-match and emits the NULL-side row after the loop; (4) contract change: output multiplicity becomes data-dependent — batch-level API only, breaks SpecializedTransform 1:1 row mapping; (5) cost O(|static|) per row. Unlocks ~13 non-equi/self-adjacent cases + N-row cross. + + +## Final Summary + + +Wave-4 stage A complete: JOIN USING (merged-column semantics measured and mirrored — left position, left value, both quals addressable, chained binding, silent dedup), residual ON predicates on INNER and LEFT (match = key_hit AND residual, hit-guarded per-pair evaluation exactly matching DuckDB's measured laziness; single-side residuals restricted to trap-free shapes because DuckDB scan-pushes those), all-key/semi joins (one-side-empty map statics with text-format round-trip), star expansion over joined tables with key-column reconstruction (INNER = key expr, LEFT = CASE JoinHit THEN key ELSE NULL), comma-join rewrite to INNER probes with WHERE-conjunct key consumption, and cross join to 1-row statics via empty-key maps (0-row annihilates; multi-row rejects via the duplicate-key check). Corpus 383 -> 395 of 678, zero FAILs; join-blocked pool 71 -> 13 (self-joins, NATURAL, FULL OUTER — all named). Stage B (output multiplicity via Term::EmitTo + per-entry static scan) delivered as a written design, not built, per the design gate. + diff --git a/docs/superpowers/specs/2026-07-26-wave4-join-pins.md b/docs/superpowers/specs/2026-07-26-wave4-join-pins.md new file mode 100644 index 0000000..eb1bed1 --- /dev/null +++ b/docs/superpowers/specs/2026-07-26-wave4-join-pins.md @@ -0,0 +1,72 @@ +# Wave-4 join-form pins — DuckDB 1.5.5, measured 2026-07-26 + +The implementation contract for TASK-50 stage A. Three probe agents, +native tables, full pin tables in `pins-wave4/*.json`. Nothing inferred +from documentation. + +## Comma joins / cross / star (pins-wave4/comma_cross_star.json) + +- `FROM t, u WHERE t.k = u.k` is bit-identical to `INNER JOIN ON` (rows + AND star column order); WHERE conjuncts split freely into equi keys + + residuals; three-way comma == nested INNER; the comma→probe rewrite is + safe. Row ORDER of cross products is an artifact (flips with the + projection shape — measured) — multiset comparison is mandatory. +- Cross join to a 1-row static: output rowcount == driving rowcount; a + 0-row static annihilates (INNER semantics) — so an empty-keys probe + with a compile-time exactly-one-entry check is exact for 1 row and a + clean reject beyond. LEFT JOIN ON TRUE to an empty table keeps rows. +- Star expands FROM-order, declared column order, duplicate names KEPT + verbatim into the Arrow schema (`[id, lv, id, rv]`) — our typed output + model cannot hold duplicate field names, so duplicate-output-name star + shapes stay cleanly unsupported (documented model constraint). +- `FROM a, a a2`: bare `a` binds the unaliased occurrence (aliasing + normally shadows the base name entirely). Schema-qualified same-name + statics (`s1.t1, s2.t1`) bind per-schema — out of v0's bare-name + static catalog, stays unsupported. + +## USING (pins-wave4/using_desugar.json) + +- `USING (a)` == `ON t1.a = t2.a` row-wise for INNER and LEFT; NULL keys + never match. +- Star: the merged column appears ONCE, at the LEFT table's declared + position (NOT hoisted to the front — refutes the PostgreSQL habit); + USING-list order is irrelevant; output spelling = the LEFT table's + declared capitalization; matching is case-insensitive EVEN QUOTED. +- Merged value = COALESCE(left, right) ≡ the LEFT value under + INNER/LEFT-only — bit-exact simplification. `t1.a` and `t2.a` both + stay addressable; `t2.a` is NULL on a LEFT miss (not coalesced). +- Chained `USING (a)` binds the MERGED column (≡ left under our kinds); + `USING (a)` after a prior ON join that left duplicate visible `a`s is + a binder AMBIGUITY error — never silently pick a side. +- `USING (a, a)` silently dedupes; USING a col missing on one side has + side-specific error texts (left checked first); same static joined + twice under different aliases is legal. + +## Residual ON predicates (pins-wave4/equi_on_residuals.json) + +- INNER: ON residual ≡ WHERE (rows and columns) — the rewrite target. +- LEFT: a key-matching row whose residual fails SURVIVES with an + all-NULL right side (incl. the right key column) — ON filters matches, + never rows. `AND false` ⇒ every driving row with NULL side; `AND + true` no-op; `ON NULL = 2` binds fine (INNER empty / LEFT all-NULL). +- Match rule for the probe plan: `match = key_hit AND residual` with + 3VL residual collapse (NULL ⇒ non-match). +- ERROR EAGERNESS (the trap): DuckDB pushes SINGLE-side residuals into + that side's scan — a trapping residual fires on rows/build-entries + whose key never matches. BOTH-sides residuals are lazy per candidate + pair (both kinds) — identical to our hit-guarded evaluation. v0 rule: + single-side residual conjuncts must be conservatively TRAP-FREE + (columns, literals, comparisons, IS NULL, logic) or the join rejects + by name; both-sides residuals may trap (evaluated per key-matched + pair, matching DuckDB's laziness bit-exactly). + +## Stage-A scope consequences + +- All-key statics: lift the no-value-columns rejection (empty probe dst + list); join hit becomes first-class in the plan (SKind::JoinHit) for + key-column reconstruction: `r.id` ≡ CASE match THEN dyn-key ELSE NULL. +- Expected non-flips recorded honestly: `SELECT *` shapes that produce + duplicate output names (both-keys star, self comma joins) and + schema-qualified statics stay cleanly unsupported. +- Stage B (output multiplicity — pure non-equi ON, range comma joins, + N-row cross) is design-gated: written proposal before any build. diff --git a/docs/superpowers/specs/pins-wave4/comma_cross_star.json b/docs/superpowers/specs/pins-wave4/comma_cross_star.json new file mode 100644 index 0000000..b44b140 --- /dev/null +++ b/docs/superpowers/specs/pins-wave4/comma_cross_star.json @@ -0,0 +1,2245 @@ +{ + "family": "comma_cross_star", + "duckdb_version": "1.5.5", + "decisions": [ + "comma-equi-equals-inner-join: FROM t, u WHERE t.k = u.k is bit-identical to FROM t INNER JOIN u ON t.k = u.k -- same row set, same row order, same SELECT * column names and order. Safe to rewrite comma->INNER JOIN.", + "comma-residual-equals-on-residual: FROM t, u WHERE t.k = u.k AND t.x > u.y is identical to INNER JOIN ... ON t.k = u.k AND t.x > u.y (equality + residual). The WHERE conjuncts may be split into equi-key + residual freely.", + "comma-nonequi-equals-cross-filter: FROM t, u WHERE t.x < u.y is identical (rows AND order) to INNER JOIN ON t.x < u.y; both are cross-product-then-filter. No equi key required for the rewrite.", + "star-column-order: SELECT * over any join expands tables in FROM order, each table's columns in declared order. Duplicate column names (join keys, self-joins) are KEPT verbatim -- cursor.description and Arrow schema both show e.g. [id, lv, id, rv]. No _1 suffixing, no dedup.", + "star-keeps-both-keys-on-ON-join: SELECT * FROM l JOIN r ON l.id = r.id outputs BOTH id columns (INNER and LEFT). On LEFT miss rows, r.id (and all r columns) are NULL. Only USING coalesces to a single key column.", + "cross-product-row-order-is-artifact: measured order for FROM t, u (no WHERE): LEFT table varies FASTEST (blocks of |t| rows per u row) -- but the order FLIPS with projection shape (SELECT * vs SELECT * EXCLUDE over the same tables flipped which side varies fastest, reproducibly). Deterministic per query in-process, NOT contractual. Engine must compare as multisets; do not pin an order.", + "left-join-order-artifact: LEFT JOIN emits matched rows first (probe order), unmatched left rows appended last (observed [1,3,2] for l=[1,2,3], r={1,3}). Also an artifact; multiset comparison only.", + "cross-join-one-row-multiplicity: comma or CROSS JOIN to a 1-row table yields exactly rowcount == driving rowcount (nulltable shape: 5 t-rows -> 5 output rows), columns of both sides usable in expressions, NULL propagates through arithmetic (n + t.x IS NULL).", + "cross-join-empty-static-annihilates: comma/CROSS JOIN to a 0-row table -> 0 output rows (INNER semantics). Empty DRIVING side also -> 0 rows. Contrast pinned: LEFT JOIN ON TRUE to the empty table keeps all driving rows with NULLs.", + "cross-join-n-row-multiplicity: 2-row static via comma -> exactly 2x driving rows (beyond v0 0-or-1 contract; pinned for the later stage).", + "self-join-binding: FROM a, a a2 -- bare 'a' remains addressable alongside 'a2'; a.rowid and a2.rowid resolve independently per binding (full cross of rowids). FROM a, a with NO alias errors: BinderException: Binder Error: Ambiguous reference to table \"memory.main.a\" (duplicate alias \"memory.main.a\", explicitly alias one of the tables using \"AS my_alias\").", + "alias-shadows-base-name: FROM a a2 makes 'a' unaddressable: BinderException: Binder Error: Referenced table \"a\" not found! Candidate tables: \"a2\". So in FROM a, a a2 the bare 'a' binds ONLY to the unaliased occurrence.", + "schema-qualified-binding: FROM s1.t1, s2.t1 binds both; columns are addressed as s1.t1.col / s2.t1.col (works in SELECT and WHERE). Bare t1.col errors: BinderException: Binder Error: Ambiguous reference to table \"t1\" (use: s1.t1 or s2.t1).", + "exclude-semantics: EXCLUDE (s1.t1.t) removes ONLY that binding's column (remaining: p, t, q). EXCLUDE (bare_name) removes the column from EVERY table that has it (EXCLUDE (t) -> p, q; EXCLUDE (id) on l/r join -> lv, rv). EXCLUDE (t1.t) with two schemas both aliased 't1' removes BOTH (alias matches both bindings). EXCLUDE (r.id) on an ON-join removes only r's key.", + "ambiguous-column-error: unqualified column present in both sides of a comma join (SELECT or WHERE) and of an explicit ON-join errors identically: BinderException: Binder Error: Ambiguous reference to column name \"k\" (use: \"t.k\" or \"u.k\") followed by newline-newline LINE : and a caret line. Only USING legalizes the bare key name.", + "three-way-comma: FROM t, u, v WHERE t.k = u.k AND u.k = v.k binds fine and is identical (rows, order, star column order t|u|v) to nested INNER JOINs. Equalities anchored differently (t.k = u.k AND t.k = v.k) give the same result; extra residual conjuncts compose.", + "table-star: t.* over a comma join projects only t's columns; SELECT u.*, t.* orders output by the SELECT list, not FROM order." + ], + "probes": [ + { + "sql": "SELECT * FROM t, u WHERE t.k = u.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "comma join, equality where", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t INNER JOIN u ON t.k = u.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "inner join on, same key", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u WHERE t.k = u.k AND t.x > u.y", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "comma join, equality + residual conjunct in WHERE", + "result": [ + [ + 2, + 20, + "b", + 2, + 15 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t INNER JOIN u ON t.k = u.k AND t.x > u.y", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "inner join, equality + residual in ON", + "result": [ + [ + 2, + 20, + "b", + 2, + 15 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "pure cross product via comma, row order + col order", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 3, + 30, + "c", + 2, + 15 + ], + [ + null, + 40, + "d", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t CROSS JOIN u", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "explicit CROSS JOIN, compare order", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 3, + 30, + "c", + 2, + 15 + ], + [ + null, + 40, + "d", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u WHERE t.x < u.y", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "comma join, non-equi only predicate", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t INNER JOIN u ON t.x < u.y", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "inner join with non-equi ON", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT t.k, t.x + 1 AS x1, n, n + t.x AS nx FROM t, nulltable", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE nulltable(n INTEGER)", + "INSERT INTO nulltable VALUES (NULL)" + ], + "note": "cross join to 1-row NULL table, mixed expressions; expect rowcount == |t| == 5", + "result": [ + [ + 1, + 11, + null, + null + ], + [ + 2, + 21, + null, + null + ], + [ + 3, + 31, + null, + null + ], + [ + null, + 41, + null, + null + ], + [ + 2, + 6, + null, + null + ] + ], + "columns": [ + "k", + "x1", + "n", + "nx" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, nulltable", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE nulltable(n INTEGER)", + "INSERT INTO nulltable VALUES (NULL)" + ], + "note": "star over t x nulltable", + "result": [ + [ + 1, + 10, + "a", + null + ], + [ + 2, + 20, + "b", + null + ], + [ + 3, + 30, + "c", + null + ], + [ + null, + 40, + "d", + null + ], + [ + 2, + 5, + "e", + null + ] + ], + "columns": [ + "k", + "x", + "name", + "n" + ], + "error": null + }, + { + "sql": "SELECT t.k, n FROM t, nulltable", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE nulltable(n INTEGER)" + ], + "note": "cross join to EMPTY table; expect 0 rows", + "result": [], + "columns": [ + "k", + "n" + ], + "error": null + }, + { + "sql": "SELECT t.k, n FROM t LEFT JOIN nulltable ON TRUE", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE nulltable(n INTEGER)" + ], + "note": "contrast: LEFT JOIN ON TRUE to empty table keeps t rows with NULL n", + "result": [ + [ + 1, + null + ], + [ + 2, + null + ], + [ + 3, + null + ], + [ + null, + null + ], + [ + 2, + null + ] + ], + "columns": [ + "k", + "n" + ], + "error": null + }, + { + "sql": "SELECT * FROM a, a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "same table twice, star", + "result": [ + [ + 1, + 10, + 1, + 10 + ], + [ + 2, + 20, + 1, + 10 + ], + [ + 1, + 10, + 2, + 20 + ], + [ + 2, + 20, + 2, + 20 + ] + ], + "columns": [ + "id", + "v", + "id", + "v" + ], + "error": null + }, + { + "sql": "SELECT a.id, a2.id FROM a, a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "bare 'a' still addressable alongside alias a2?", + "result": [ + [ + 1, + 1 + ], + [ + 2, + 1 + ], + [ + 1, + 2 + ], + [ + 2, + 2 + ] + ], + "columns": [ + "id", + "id" + ], + "error": null + }, + { + "sql": "SELECT a.rowid, a2.rowid FROM a, a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "rowid per binding", + "result": [ + [ + 0, + 0 + ], + [ + 1, + 0 + ], + [ + 0, + 1 + ], + [ + 1, + 1 + ] + ], + "columns": [ + "rowid", + "rowid" + ], + "error": null + }, + { + "sql": "SELECT a.rowid AS r1, a2.rowid AS r2 FROM a, a a2 WHERE a.rowid = a2.rowid", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "rowid equality across the two bindings", + "result": [ + [ + 0, + 0 + ], + [ + 1, + 1 + ] + ], + "columns": [ + "r1", + "r2" + ], + "error": null + }, + { + "sql": "SELECT * FROM a, a", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "same table twice with NO alias -> error text?", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to table \"memory.main.a\" (duplicate alias \"memory.main.a\", explicitly alias one of the tables using \"AS my_alias\")" + }, + { + "sql": "SELECT id FROM a, a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "unqualified id when both sides have it -> ambiguous?", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"id\" (use: \"a.id\" or \"a2.id\")\n\nLINE 1: SELECT id FROM a, a a2\n ^" + }, + { + "sql": "SELECT * FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "two same-named tables from different schemas, star", + "result": [ + [ + 1, + 11, + 1, + 21 + ], + [ + 2, + 12, + 1, + 21 + ], + [ + 1, + 11, + 3, + 23 + ], + [ + 2, + 12, + 3, + 23 + ] + ], + "columns": [ + "t", + "p", + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT s1.t1.t, s2.t1.q FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "schema-qualified column reference works?", + "result": [ + [ + 1, + 21 + ], + [ + 2, + 21 + ], + [ + 1, + 23 + ], + [ + 2, + 23 + ] + ], + "columns": [ + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT t1.t FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "bare t1.t -> ambiguous between the two bindings?", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to table \"t1\" (use: s1.t1 or s2.t1)" + }, + { + "sql": "SELECT * FROM s1.t1, s2.t1 WHERE s1.t1.t = s2.t1.t", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "equality via schema-qualified refs", + "result": [ + [ + 1, + 11, + 1, + 21 + ] + ], + "columns": [ + "t", + "p", + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (s1.t1.t) FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "EXCLUDE with schema-qualified column", + "result": [ + [ + 11, + 1, + 21 + ], + [ + 11, + 3, + 23 + ], + [ + 12, + 1, + 21 + ], + [ + 12, + 3, + 23 + ] + ], + "columns": [ + "p", + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (t) FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "EXCLUDE bare ambiguous name over same-named cols", + "result": [ + [ + 11, + 21 + ], + [ + 12, + 21 + ], + [ + 11, + 23 + ], + [ + 12, + 23 + ] + ], + "columns": [ + "p", + "q" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (t1.t) FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "EXCLUDE table-qualified (no schema)", + "result": [ + [ + 11, + 21 + ], + [ + 12, + 21 + ], + [ + 11, + 23 + ], + [ + 12, + 23 + ] + ], + "columns": [ + "p", + "q" + ], + "error": null + }, + { + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "star keeps BOTH id columns on inner join?", + "result": [ + [ + 1, + "l1", + 1, + "r1" + ], + [ + 3, + "l3", + 3, + "r3" + ] + ], + "columns": [ + "id", + "lv", + "id", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "star on left join: r.id NULL on miss rows?", + "result": [ + [ + 1, + "l1", + 1, + "r1" + ], + [ + 3, + "l3", + 3, + "r3" + ], + [ + 2, + "l2", + null, + null + ] + ], + "columns": [ + "id", + "lv", + "id", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * FROM l LEFT JOIN r USING (id)", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "contrast: USING coalesces the key in star output", + "result": [ + [ + 1, + "l1", + "r1" + ], + [ + 3, + "l3", + "r3" + ], + [ + 2, + "l2", + null + ] + ], + "columns": [ + "id", + "lv", + "rv" + ], + "error": null + }, + { + "sql": "SELECT k FROM t, u", + "setup": [ + "CREATE TABLE t(k INTEGER, w INTEGER)", + "INSERT INTO t VALUES (1, 10)", + "CREATE TABLE u(k INTEGER, z INTEGER)", + "INSERT INTO u VALUES (1, 100)" + ], + "note": "ambiguous k in SELECT of comma join -> error text", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"k\" (use: \"t.k\" or \"u.k\")\n\nLINE 1: SELECT k FROM t, u\n ^" + }, + { + "sql": "SELECT * FROM t, u WHERE k = 1", + "setup": [ + "CREATE TABLE t(k INTEGER, w INTEGER)", + "INSERT INTO t VALUES (1, 10)", + "CREATE TABLE u(k INTEGER, z INTEGER)", + "INSERT INTO u VALUES (1, 100)" + ], + "note": "ambiguous k in WHERE -> error text", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"k\" (use: \"t.k\" or \"u.k\")\n\nLINE 1: SELECT * FROM t, u WHERE k = 1\n ^" + }, + { + "sql": "SELECT k FROM t INNER JOIN u ON t.k = u.k", + "setup": [ + "CREATE TABLE t(k INTEGER, w INTEGER)", + "INSERT INTO t VALUES (1, 10)", + "CREATE TABLE u(k INTEGER, z INTEGER)", + "INSERT INTO u VALUES (1, 100)" + ], + "note": "ambiguous k in SELECT of explicit equi ON join (NOT USING) -> error? (USING would legalize)", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"k\" (use: \"t.k\" or \"u.k\")\n\nLINE 1: SELECT k FROM t INNER JOIN u ON t.k = u.k\n ^" + }, + { + "sql": "SELECT * FROM t, u, v WHERE t.k = u.k AND u.k = v.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE v(k INTEGER, z INTEGER)", + "INSERT INTO v VALUES (1, 1000), (2, 2000)" + ], + "note": "3-way comma, chained equalities", + "result": [ + [ + 1, + 10, + "a", + 1, + 100, + 1, + 1000 + ], + [ + 2, + 20, + "b", + 2, + 15, + 2, + 2000 + ], + [ + 2, + 5, + "e", + 2, + 15, + 2, + 2000 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y", + "k", + "z" + ], + "error": null + }, + { + "sql": "SELECT * FROM t INNER JOIN u ON t.k = u.k INNER JOIN v ON u.k = v.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE v(k INTEGER, z INTEGER)", + "INSERT INTO v VALUES (1, 1000), (2, 2000)" + ], + "note": "equivalent nested inner joins", + "result": [ + [ + 1, + 10, + "a", + 1, + 100, + 1, + 1000 + ], + [ + 2, + 20, + "b", + 2, + 15, + 2, + 2000 + ], + [ + 2, + 5, + "e", + 2, + 15, + 2, + 2000 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y", + "k", + "z" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u, v WHERE t.k = u.k AND t.k = v.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE v(k INTEGER, z INTEGER)", + "INSERT INTO v VALUES (1, 1000), (2, 2000)" + ], + "note": "3-way comma, equalities both anchored on t", + "result": [ + [ + 1, + 10, + "a", + 1, + 100, + 1, + 1000 + ], + [ + 2, + 20, + "b", + 2, + 15, + 2, + 2000 + ], + [ + 2, + 5, + "e", + 2, + 15, + 2, + 2000 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y", + "k", + "z" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u, v WHERE t.k = u.k AND u.k = v.k AND t.x > u.y", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE v(k INTEGER, z INTEGER)", + "INSERT INTO v VALUES (1, 1000), (2, 2000)" + ], + "note": "3-way comma with residual", + "result": [ + [ + 2, + 20, + "b", + 2, + 15, + 2, + 2000 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y", + "k", + "z" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "cross product order, run A (stability check)", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 3, + 30, + "c", + 2, + 15 + ], + [ + null, + 40, + "d", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT * FROM t, u", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "cross product order, run B (stability check)", + "result": [ + [ + 1, + 10, + "a", + 1, + 100 + ], + [ + 2, + 20, + "b", + 1, + 100 + ], + [ + 3, + 30, + "c", + 1, + 100 + ], + [ + null, + 40, + "d", + 1, + 100 + ], + [ + 2, + 5, + "e", + 1, + 100 + ], + [ + 1, + 10, + "a", + 2, + 15 + ], + [ + 2, + 20, + "b", + 2, + 15 + ], + [ + 3, + 30, + "c", + 2, + 15 + ], + [ + null, + 40, + "d", + 2, + 15 + ], + [ + 2, + 5, + "e", + 2, + 15 + ], + [ + 1, + 10, + "a", + 4, + 400 + ], + [ + 2, + 20, + "b", + 4, + 400 + ], + [ + 3, + 30, + "c", + 4, + 400 + ], + [ + null, + 40, + "d", + 4, + 400 + ], + [ + 2, + 5, + "e", + 4, + 400 + ] + ], + "columns": [ + "k", + "x", + "name", + "k", + "y" + ], + "error": null + }, + { + "sql": "SELECT t.* FROM t, u WHERE t.k = u.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "t.* over comma join", + "result": [ + [ + 1, + 10, + "a" + ], + [ + 2, + 20, + "b" + ], + [ + 2, + 5, + "e" + ] + ], + "columns": [ + "k", + "x", + "name" + ], + "error": null + }, + { + "sql": "SELECT u.*, t.* FROM t, u WHERE t.k = u.k", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)" + ], + "note": "projection order follows SELECT list (u.* first), not FROM order", + "result": [ + [ + 1, + 100, + 1, + 10, + "a" + ], + [ + 2, + 15, + 2, + 20, + "b" + ], + [ + 2, + 15, + 2, + 5, + "e" + ] + ], + "columns": [ + "k", + "y", + "k", + "x", + "name" + ], + "error": null + }, + { + "sql": "SELECT l.*, r.* FROM l LEFT JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "l.*, r.* on LEFT join, miss row NULLs", + "result": [ + [ + 1, + "l1", + 1, + "r1" + ], + [ + 3, + "l3", + 3, + "r3" + ], + [ + 2, + "l2", + null, + null + ] + ], + "columns": [ + "id", + "lv", + "id", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (r.id) FROM l LEFT JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "EXCLUDE alias-qualified key col on ON-join: removes only r.id?", + "result": [ + [ + 1, + "l1", + "r1" + ], + [ + 3, + "l3", + "r3" + ], + [ + 2, + "l2", + null + ] + ], + "columns": [ + "id", + "lv", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (id) FROM l LEFT JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "EXCLUDE bare 'id' on ON-join: removes both?", + "result": [ + [ + "l1", + "r1" + ], + [ + "l3", + "r3" + ], + [ + "l2", + null + ] + ], + "columns": [ + "lv", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id", + "setup": [ + "CREATE TABLE l(id INTEGER, lv VARCHAR)", + "INSERT INTO l VALUES (1, 'l1'), (2, 'l2'), (3, 'l3')", + "CREATE TABLE r(id INTEGER, rv VARCHAR)", + "INSERT INTO r VALUES (1, 'r1'), (3, 'r3')" + ], + "note": "arrow: dup id names kept?", + "result": [ + [ + 1, + "l1", + 1, + "r1" + ], + [ + 3, + "l3", + 3, + "r3" + ], + [ + 2, + "l2", + null, + null + ] + ], + "columns": [ + "id", + "lv", + "id", + "rv" + ], + "error": null + }, + { + "sql": "SELECT * FROM a, a a2 WHERE a.id = a2.id", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "self equi comma join", + "result": [ + [ + 1, + 10, + 1, + 10 + ], + [ + 2, + 20, + 2, + 20 + ] + ], + "columns": [ + "id", + "v", + "id", + "v" + ], + "error": null + }, + { + "sql": "SELECT * FROM a, a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "arrow: self comma join dup names", + "result": [ + [ + 1, + 10, + 1, + 10 + ], + [ + 2, + 20, + 1, + 10 + ], + [ + 1, + 10, + 2, + 20 + ], + [ + 2, + 20, + 2, + 20 + ] + ], + "columns": [ + "id", + "v", + "id", + "v" + ], + "error": null + }, + { + "sql": "SELECT t.k, m FROM t, two", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE two(m INTEGER)", + "INSERT INTO two VALUES (7), (8)" + ], + "note": "cross to 2-row table -> 2x multiplicity (beyond v0; pinned)", + "result": [ + [ + 1, + 7 + ], + [ + 2, + 7 + ], + [ + 3, + 7 + ], + [ + null, + 7 + ], + [ + 2, + 7 + ], + [ + 1, + 8 + ], + [ + 2, + 8 + ], + [ + 3, + 8 + ], + [ + null, + 8 + ], + [ + 2, + 8 + ] + ], + "columns": [ + "k", + "m" + ], + "error": null + }, + { + "sql": "SELECT k, n FROM e, nulltable", + "setup": [ + "CREATE TABLE e(k INTEGER)", + "CREATE TABLE nulltable(n INTEGER)", + "INSERT INTO nulltable VALUES (NULL)" + ], + "note": "empty driving table x 1-row static -> 0 rows", + "result": [], + "columns": [ + "k", + "n" + ], + "error": null + }, + { + "sql": "SELECT t.k, n FROM t CROSS JOIN nulltable", + "setup": [ + "CREATE TABLE t(k INTEGER, x INTEGER, name VARCHAR)", + "INSERT INTO t VALUES (1, 10, 'a'), (2, 20, 'b'), (3, 30, 'c'), (NULL, 40, 'd'), (2, 5, 'e')", + "CREATE TABLE u(k INTEGER, y INTEGER)", + "INSERT INTO u VALUES (1, 100), (2, 15), (4, 400)", + "CREATE TABLE nulltable(n INTEGER)", + "INSERT INTO nulltable VALUES (NULL)" + ], + "note": "explicit CROSS JOIN keyword to nulltable, same as comma", + "result": [ + [ + 1, + null + ], + [ + 2, + null + ], + [ + 3, + null + ], + [ + null, + null + ], + [ + 2, + null + ] + ], + "columns": [ + "k", + "n" + ], + "error": null + }, + { + "sql": "SELECT * FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "star rerun: which side varies fastest?", + "result": [ + [ + 1, + 11, + 1, + 21 + ], + [ + 2, + 12, + 1, + 21 + ], + [ + 1, + 11, + 3, + 23 + ], + [ + 2, + 12, + 3, + 23 + ] + ], + "columns": [ + "t", + "p", + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT * EXCLUDE (s1.t1.t) FROM s1.t1, s2.t1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "EXCLUDE rerun: order flip reproducible?", + "result": [ + [ + 11, + 1, + 21 + ], + [ + 11, + 3, + 23 + ], + [ + 12, + 1, + 21 + ], + [ + 12, + 3, + 23 + ] + ], + "columns": [ + "p", + "t", + "q" + ], + "error": null + }, + { + "sql": "SELECT s2.t1.t FROM s1.t1, s2.t1 WHERE s1.t1.t = 1", + "setup": [ + "CREATE SCHEMA s1", + "CREATE SCHEMA s2", + "CREATE TABLE s1.t1(t INTEGER, p INTEGER)", + "INSERT INTO s1.t1 VALUES (1, 11), (2, 12)", + "CREATE TABLE s2.t1(t INTEGER, q INTEGER)", + "INSERT INTO s2.t1 VALUES (1, 21), (3, 23)" + ], + "note": "schema-qualified in WHERE too", + "result": [ + [ + 1 + ], + [ + 3 + ] + ], + "columns": [ + "t" + ], + "error": null + }, + { + "sql": "SELECT a.id FROM a a2", + "setup": [ + "CREATE TABLE a(id INTEGER, v INTEGER)", + "INSERT INTO a VALUES (1, 10), (2, 20)" + ], + "note": "aliased table: original name NOT addressable -> error text", + "result": null, + "columns": null, + "error": "BinderException: Binder Error: Referenced table \"a\" not found!\nCandidate tables: \"a2\"\n\nLINE 1: SELECT a.id FROM a a2\n ^" + } + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave4/equi_on_residuals.json b/docs/superpowers/specs/pins-wave4/equi_on_residuals.json new file mode 100644 index 0000000..eb3a6e4 --- /dev/null +++ b/docs/superpowers/specs/pins-wave4/equi_on_residuals.json @@ -0,0 +1,927 @@ +{ + "family": "equi_on_residuals", + "duckdb_version": "1.5.5", + "setup": [ + "CREATE TABLE l(id INTEGER, val INTEGER, amount INTEGER)", + "INSERT INTO l VALUES (1,5,100),(2,0,200),(3,2,1000),(NULL,7,300),(5,9,400)", + "CREATE TABLE r(id INTEGER UNIQUE, category INTEGER, budget INTEGER)", + "INSERT INTO r VALUES (1,1,50),(2,2,60),(3,1,500),(4,1,999)", + "CREATE TABLE rz(id INTEGER UNIQUE, budget INTEGER)", + "INSERT INTO rz VALUES (1,50),(99,0)", + "CREATE TABLE rz2(id INTEGER UNIQUE, budget INTEGER)", + "INSERT INTO rz2 VALUES (1,50),(2,0)", + "CREATE TABLE l2(id INTEGER, v INTEGER)", + "INSERT INTO l2 VALUES (1,10),(99,0)", + "CREATE TABLE l3(id INTEGER, v INTEGER)", + "INSERT INTO l3 VALUES (1,0),(3,10)", + "CREATE TABLE l4(id INTEGER, a INTEGER)", + "INSERT INTO l4 VALUES (1,100),(2,5)", + "CREATE TABLE r4(id INTEGER UNIQUE, b INTEGER)", + "INSERT INTO r4 VALUES (1,50),(2,60)", + "CREATE TABLE test(b INTEGER)", + "INSERT INTO test VALUES (1),(2),(NULL)", + "CREATE TABLE test2(b INTEGER)", + "INSERT INTO test2 VALUES (1),(2),(3)" + ], + "decisions": [ + "INNER: residual in ON is identical to the same predicate as WHERE after the join \u2014 same rows, same column names/order (probes inner_residual_lval vs inner_where_lval, python-equal=true).", + "LEFT: a driving row whose key matches but whose ON residual fails SURVIVES with an all-NULL right side (including the right key column). ON-residual failure = non-match, never a dropped row. WHERE after the LEFT join DROPS the row instead (probes left_residual_lval vs left_where_lval).", + "LEFT: right-side residual (r.category = 1) behaves the same: key-match + residual-fail => NULL right side (probe left_residual_rcat); INNER drops the row (inner_residual_rcat).", + "Constant residuals: 'AND true' is a no-op on both join types. 'AND false': INNER => empty result (schema/column names preserved on the empty result); LEFT => every driving row exactly once with all-NULL right side (probes *_and_true, *_and_false).", + "Residual referencing both sides (l.amount + r.budget > 1100) evaluates per candidate pair; LEFT keeps residual-failing key-matches with NULL right side (probes inner_both_sides, left_both_sides).", + "Left-side-only conjunct (ON test2.b = test.b AND test.b = 2) binds fine and stays part of the match condition: on LEFT, a row that key-matches but fails the conjunct is KEPT with NULL right side (test.b=1 row); on INNER it acts exactly like a WHERE filter (probes *_test_const_left_side).", + "ON NULL = 2 alone is accepted by the binder (no error): INNER => empty; LEFT => every driving row once with NULL right side (probes *_on_null_eq_2).", + "NULL driving keys: NULL = x is a non-match regardless of residual; INNER drops the row, LEFT keeps it with NULL right side (probes inner_null_key, left_null_key).", + "SELECT * column order: all left-table columns in table order, then all right-table columns in table order; duplicate names are KEPT verbatim (two 'id' columns in duckdb description and arrow schema; arrow types stay int32, right side merely nullable). On LEFT misses every right column is NULL including the key column (probe arrow_schema_left_join_star).", + "ERROR EAGERNESS (trap-laziness pins): (a) single-side residual under INNER is pushed into that side's scan and evaluated on EVERY row of that side \u2014 log(0) errors surface even when the poison row's key never matches (probes inner_lazy_right_poison_unmatched, inner_lazy_left_poison_unmatched); (b) right-side-only residual under LEFT is likewise pushed into the right/build side \u2014 errors surface for build rows that never match (probe left_lazy_right_poison_unmatched); (c) left-side residual under LEFT is LAZY \u2014 evaluated only on key-matched candidate pairs; unmatched poison row survives with NULL side, no error (left_lazy_left_poison_unmatched vs left_lazy_left_poison_matched); (d) residual referencing BOTH sides is lazy on both INNER and LEFT \u2014 a poison value arising only on a non-matching key pair is never evaluated, no error (probes *_both_sides_poison_nonmatching_pair).", + "Engine consequence of the eagerness pins: evaluating residuals only on hash-probe candidate pairs is bit-exact for both-sides residuals (all join types) and left-side residuals under LEFT; but for single-side residuals under INNER and right-side residuals under LEFT, duckdb evaluates the residual over the ENTIRE side \u2014 to reproduce error behavior the engine must pre-evaluate those residuals across the whole static table (build-time filter) / whole driving batch, not just on candidates.", + "Scalar side-pins: log(0)/ln(0) raise OutOfRangeException 'Out of Range Error: cannot take logarithm of zero'; 1/0 => Infinity (double, NO error); 1//0 => NULL (NO error). Division is NOT a usable error trap in duckdb." + ], + "probes": [ + { + "name": "inner_residual_lval", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND l.val > 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ] + ], + "error": null, + "note": "left-side residual in ON of INNER join" + }, + { + "name": "inner_where_lval", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id WHERE l.val > 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ] + ], + "error": null, + "note": "same predicate as WHERE after INNER join; should be identical" + }, + { + "name": "inner_on_vs_where_identical", + "sql": null, + "setup": null, + "columns": null, + "result": true, + "error": null, + "note": "python equality of the two INNER probes above (rows AND column names)" + }, + { + "name": "left_residual_lval", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND l.val > 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 2, + 0, + 200, + null, + null, + null + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "id=2 matches key but val=0<=1: must SURVIVE with NULL right side" + }, + { + "name": "left_where_lval", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id WHERE l.val > 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "contrast: WHERE after LEFT join DROPS the id=2 row entirely" + }, + { + "name": "left_residual_rcat", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND r.category = 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 2, + 0, + 200, + null, + null, + null + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "id=2 matches key but r.category=2: NULL right side on LEFT" + }, + { + "name": "inner_residual_rcat", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND r.category = 1 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ] + ], + "error": null, + "note": "same on INNER: id=2 row dropped" + }, + { + "name": "inner_and_true", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND true ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 2, + 0, + 200, + 2, + 2, + 60 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ] + ], + "error": null, + "note": "" + }, + { + "name": "left_and_true", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND true ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 2, + 0, + 200, + 2, + 2, + 60 + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "" + }, + { + "name": "inner_and_false", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND false ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [], + "error": null, + "note": "expect empty; capture column names of empty result" + }, + { + "name": "left_and_false", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND false ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + null, + null, + null + ], + [ + 2, + 0, + 200, + null, + null, + null + ], + [ + 3, + 2, + 1000, + null, + null, + null + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "expect every l row with NULL right side" + }, + { + "name": "inner_both_sides", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND l.amount + r.budget > 1100 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ] + ], + "error": null, + "note": "only id=3 (1000+500=1500) passes" + }, + { + "name": "left_both_sides", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND l.amount + r.budget > 1100 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + null, + null, + null + ], + [ + 2, + 0, + 200, + null, + null, + null + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "id=1,2 match key but fail residual: NULL right side" + }, + { + "name": "inner_test_const_left_side", + "sql": "SELECT * FROM test INNER JOIN test2 ON test2.b = test.b AND test.b = 2 ORDER BY test.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "b", + "b" + ], + "result": [ + [ + 2, + 2 + ] + ], + "error": null, + "note": "test.b=2 references left table only; INNER: acts like WHERE" + }, + { + "name": "left_test_const_left_side", + "sql": "SELECT * FROM test LEFT JOIN test2 ON test2.b = test.b AND test.b = 2 ORDER BY test.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "b", + "b" + ], + "result": [ + [ + 1, + null + ], + [ + 2, + 2 + ], + [ + null, + null + ] + ], + "error": null, + "note": "test.b=1 key-matches test2.b=1 but const eq fails: NULL side expected, row kept" + }, + { + "name": "inner_on_null_eq_2", + "sql": "SELECT * FROM test INNER JOIN test2 ON NULL = 2 ORDER BY test.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "b", + "b" + ], + "result": [], + "error": null, + "note": "ON NULL=2 alone: NULL condition = non-match; expect empty" + }, + { + "name": "left_on_null_eq_2", + "sql": "SELECT * FROM test LEFT JOIN test2 ON NULL = 2 ORDER BY test.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "b", + "b" + ], + "result": [ + [ + 1, + null + ], + [ + 2, + null + ], + [ + null, + null + ] + ], + "error": null, + "note": "expect every test row once with NULL test2 side" + }, + { + "name": "left_lazy_right_poison_unmatched", + "sql": "SELECT * FROM l LEFT JOIN rz ON l.id = rz.id AND log(rz.budget) > 0 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "rz has (99,0); id=99 never key-matches. Does log(0) still get evaluated?" + }, + { + "name": "inner_lazy_right_poison_unmatched", + "sql": "SELECT * FROM l INNER JOIN rz ON l.id = rz.id AND log(rz.budget) > 0 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "INNER variant: right-side residual may be PUSHED DOWN onto rz scan -> error?" + }, + { + "name": "left_lazy_right_poison_matched", + "sql": "SELECT * FROM l LEFT JOIN rz2 ON l.id = rz2.id AND log(rz2.budget) > 0 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "rz2 has (2,0) and l has id=2: residual evaluated on a real candidate -> error expected" + }, + { + "name": "inner_lazy_right_poison_matched", + "sql": "SELECT * FROM l INNER JOIN rz2 ON l.id = rz2.id AND log(rz2.budget) > 0 ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "" + }, + { + "name": "inner_lazy_left_poison_unmatched", + "sql": "SELECT * FROM l2 INNER JOIN r ON l2.id = r.id AND log(l2.v) > 0 ORDER BY l2.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "l2 has (99,0); 99 not in r. Left-side residual pushdown on INNER -> error?" + }, + { + "name": "left_lazy_left_poison_unmatched", + "sql": "SELECT * FROM l2 LEFT JOIN r ON l2.id = r.id AND log(l2.v) > 0 ORDER BY l2.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "v", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 10, + 1, + 1, + 50 + ], + [ + 99, + 0, + null, + null, + null + ] + ], + "error": null, + "note": "LEFT: left-side residual cannot filter l. Row (99,0) has no key match anyway. Error or NULL side?" + }, + { + "name": "inner_lazy_left_poison_matched", + "sql": "SELECT * FROM l3 INNER JOIN r ON l3.id = r.id AND log(l3.v) > 0 ORDER BY l3.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "l3 (1,0) key-matches r.id=1: candidate pair -> residual evaluated -> error expected" + }, + { + "name": "left_lazy_left_poison_matched", + "sql": "SELECT * FROM l3 LEFT JOIN r ON l3.id = r.id AND log(l3.v) > 0 ORDER BY l3.rowid", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "" + }, + { + "name": "inner_lazy_both_sides_poison_nonmatching_pair", + "sql": "SELECT * FROM l4 INNER JOIN r4 ON l4.id = r4.id AND log(l4.a + r4.b - 55) > 0 ORDER BY l4.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "a", + "id", + "b" + ], + "result": [ + [ + 1, + 100, + 1, + 50 + ], + [ + 2, + 5, + 2, + 60 + ] + ], + "error": null, + "note": "log(0) only for NON-matching key pair (2,1): cannot be pushed to either side; expect NO error if residual is evaluated only on key-matched candidate pairs" + }, + { + "name": "left_lazy_both_sides_poison_nonmatching_pair", + "sql": "SELECT * FROM l4 LEFT JOIN r4 ON l4.id = r4.id AND log(l4.a + r4.b - 55) > 0 ORDER BY l4.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "a", + "id", + "b" + ], + "result": [ + [ + 1, + 100, + 1, + 50 + ], + [ + 2, + 5, + 2, + 60 + ] + ], + "error": null, + "note": "LEFT variant of the same" + }, + { + "name": "inner_null_key", + "sql": "SELECT * FROM l INNER JOIN r ON l.id = r.id AND l.val > 1 WHERE l.id IS NULL ORDER BY l.rowid", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [], + "error": null, + "note": "NULL l.id under INNER+residual: expect empty" + }, + { + "name": "left_null_key", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND l.val > 1 ORDER BY l.rowid NULLS LAST", + "setup": "shared (see setup field at top level)", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + [ + 1, + 5, + 100, + 1, + 1, + 50 + ], + [ + 2, + 0, + 200, + null, + null, + null + ], + [ + 3, + 2, + 1000, + 3, + 1, + 500 + ], + [ + null, + 7, + 300, + null, + null, + null + ], + [ + 5, + 9, + 400, + null, + null, + null + ] + ], + "error": null, + "note": "NULL l.id row (val=7 passes residual) must appear with NULL right side" + }, + { + "name": "scalar_log_zero", + "sql": "SELECT log(0)", + "setup": "shared (see setup field at top level)", + "columns": null, + "result": null, + "error": "OutOfRangeException: Out of Range Error: cannot take logarithm of zero", + "note": "the error function used by laziness probes" + }, + { + "name": "scalar_div_zero", + "sql": "SELECT 1/0", + "setup": "shared (see setup field at top level)", + "columns": [ + "(1 / 0)" + ], + "result": [ + [ + "Infinity" + ] + ], + "error": null, + "note": "NOT an error in duckdb: returns Infinity (double)" + }, + { + "name": "scalar_intdiv_zero", + "sql": "SELECT 1//0", + "setup": "shared (see setup field at top level)", + "columns": [ + "(1 // 0)" + ], + "result": [ + [ + null + ] + ], + "error": null, + "note": "NOT an error in duckdb: returns NULL" + }, + { + "name": "arrow_schema_left_join_star", + "sql": "SELECT * FROM l LEFT JOIN r ON l.id = r.id AND l.val > 1", + "setup": "shared", + "columns": [ + "id", + "val", + "amount", + "id", + "category", + "budget" + ], + "result": [ + "int32", + "int32", + "int32", + "int32", + "int32", + "int32" + ], + "error": null, + "note": "arrow schema names+types for SELECT *; duplicate 'id' expected (l cols then r cols)" + } + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave4/using_desugar.json b/docs/superpowers/specs/pins-wave4/using_desugar.json new file mode 100644 index 0000000..88362e3 --- /dev/null +++ b/docs/superpowers/specs/pins-wave4/using_desugar.json @@ -0,0 +1,1797 @@ +{ + "family": "join-using-desugar", + "duckdb_version": "1.5.5", + "decisions": [ + "ROW SETS: t1 JOIN t2 USING (a) produces exactly the same rows as ON t1.a = t2.a, for both INNER and LEFT. NULL keys never match: INNER drops the NULL-key input row, LEFT emits it with all right columns NULL. [P01,P02,P03,P04,P29,P30]", + "STAR SHAPE: ON star emits BOTH key columns with duplicate names (a,x,a,y); USING star emits the key exactly ONCE. Duplicate column names in a result schema are legal in DuckDB (also y,y in P20) - the engine must not assume unique output names. [P01,P02,P20]", + "STAR POSITION (refutes PostgreSQL hoisting): the merged USING column stays at the LEFT table's declared position - w1(x,a) JOIN t2 USING(a) yields (x,a,y), NOT (a,x,y). Output order = left table's columns in declared order (using cols in place), then right table's non-using columns in right declared order. USING-list order is irrelevant: USING(a,b) and USING(b,a) give identical star order and rows. [F01,F02,F03,F04,F05,P10,P11]", + "MERGED VALUE: the exposed using column is semantically COALESCE(left, right): INNER -> equal on both sides; LEFT -> always the left value (miss rows show the LEFT value, not NULL; the NULL-key left row shows NULL because left is NULL); RIGHT -> right value on left-miss; FULL -> true COALESCE. For an INNER/LEFT-only engine, 'merged col := left value' is bit-exact. [P03,P06,P27,P28,P28b]", + "QUALIFIED ACCESS SURVIVES: t1.a and t2.a both remain addressable in SELECT and WHERE after USING. On a LEFT miss, t2.a is NULL (not coalesced) and t2.* expands to all-NULL including a. Unqualified a binds the merged column with no ambiguity, in SELECT and WHERE alike (WHERE a=3 keeps the miss row). [P05,P05b,P06,P07,P08,P26a,P26b]", + "CHAINED USING binds the MERGED column of the previous join, not a specific side: in t1 FULL JOIN t2r USING(a) JOIN t3f USING(a), the row with t1.a NULL / t2r.a=9 matches t3f.a=9 (witness z9). Under LEFT chains this is observationally identical to binding t1.a. The chain star still shows a single a column. Unqualified a in a later ON condition also resolves to the merged column. [F06,F06b,F07,F08,P12,P13,P14]", + "USING AFTER AN ON JOIN ERRORS: once a prior ON join leaves two visible a columns, a later USING(a) fails with the same ambiguity error as SELECT a: 'Binder Error: Ambiguous reference to column name \"a\" (use: \"t1.a\" or \"t2.a\")'. USING resolves names against the merged visible scope. [P15,P09]", + "ERROR PINS: col only on left -> 'Binder Error: Column \"x\" does not exist on right side of join!'; col only on right (and fully nonexistent cols) -> 'Binder Error: Column \"y\" does not exist on left side of join!' (left side is checked first, so a nonexistent name reports the LEFT side); USING () -> ParserException 'syntax error at or near \")\"'. [P16,P17,P19b,P18]", + "USING (a, a) duplicate entries are silently deduplicated: no error, output columns and rows identical to USING (a). [P19 vs P01]", + "SAME STATIC TWICE: joining one table twice under different aliases with USING works for INNER and LEFT chains; the star contains the non-key column twice under the same name (y,y); alias-qualified u1.y / u2.y disambiguate. [P20,P21,P21b]", + "CAPITALIZATION: USING identifiers match columns case-insensitively even when QUOTED (\"a\" matches a column declared \"A\", and unquoted a matches \"A\"). The merged output column keeps the LEFT table's declared spelling regardless of the USING-list spelling or the right side's spelling ('A' when left declares \"A\"; 'a' when left declares a and right declares \"A\"). [P22,P23,P24,P24b,P25]" + ], + "probes": [ + { + "sql": "SELECT * FROM t1 JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 2, + "x2", + "y2" + ] + ], + "error": null, + "note": "P01 INNER USING star: 'a' appears once; which side + position" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 ON t1.a = t2.a ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + 1, + "y1" + ], + [ + 2, + "x2", + 2, + "y2" + ] + ], + "error": null, + "note": "P02 INNER ON star: both 'a' columns present, duplicate names" + }, + { + "sql": "SELECT * FROM t1 LEFT JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 2, + "x2", + "y2" + ], + [ + 3, + "x3", + null + ], + [ + null, + "xN", + null + ] + ], + "error": null, + "note": "P03 LEFT USING star: THE coalescing pin - using col value on miss rows (a=3, a=NULL)" + }, + { + "sql": "SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + 1, + "y1" + ], + [ + 2, + "x2", + 2, + "y2" + ], + [ + 3, + "x3", + null, + null + ], + [ + null, + "xN", + null, + null + ] + ], + "error": null, + "note": "P04 LEFT ON star: both a cols, right a NULL on miss" + }, + { + "sql": "SELECT t1.a, t2.a FROM t1 JOIN t2 USING (a) ORDER BY t1.a", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "a", + "INTEGER" + ] + ], + "result": [ + [ + 1, + 1 + ], + [ + 2, + 2 + ] + ], + "error": null, + "note": "P05 qualified t1.a and t2.a both addressable after INNER USING" + }, + { + "sql": "SELECT a FROM t1 JOIN t2 USING (a) ORDER BY a", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ] + ], + "result": [ + [ + 1 + ], + [ + 2 + ] + ], + "error": null, + "note": "P05b unqualified a after INNER USING: no ambiguity error" + }, + { + "sql": "SELECT a AS ua, t1.a AS l_a, t2.a AS r_a, x, y FROM t1 LEFT JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "ua", + "INTEGER" + ], + [ + "l_a", + "INTEGER" + ], + [ + "r_a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 1, + 1, + "x1", + "y1" + ], + [ + 2, + 2, + 2, + "x2", + "y2" + ], + [ + 3, + 3, + null, + "x3", + null + ], + [ + null, + null, + null, + "xN", + null + ] + ], + "error": null, + "note": "P06 LEFT USING: unqualified a vs t1.a vs t2.a per row (miss rows show the coalesce direction)" + }, + { + "sql": "SELECT x FROM t1 LEFT JOIN t2 USING (a) WHERE a = 3", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "x", + "VARCHAR" + ] + ], + "result": [ + [ + "x3" + ] + ], + "error": null, + "note": "P07 unqualified a in WHERE after LEFT USING: matches miss row a=3 -> binds to left value" + }, + { + "sql": "SELECT x FROM t1 LEFT JOIN t2 USING (a) WHERE t2.a IS NULL ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "x", + "VARCHAR" + ] + ], + "result": [ + [ + "x3" + ], + [ + "xN" + ] + ], + "error": null, + "note": "P08 t2.a IS NULL in WHERE after LEFT USING: anti-join filter on right key works" + }, + { + "sql": "SELECT a FROM t1 JOIN t2 ON t1.a = t2.a ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"a\" (use: \"t1.a\" or \"t2.a\")\n\nLINE 1: SELECT a FROM t1 JOIN t2 ON t1.a = t2.a ORDER BY x\n ^", + "note": "P09 unqualified a after INNER ON: ambiguity? exact error" + }, + { + "sql": "SELECT * FROM m1 JOIN m2 USING (a, b) ORDER BY x", + "setup": [ + "CREATE TABLE m1(a INTEGER, b INTEGER, x VARCHAR)", + "INSERT INTO m1 VALUES (1,10,'x1'),(2,20,'x2'),(3,30,'x3')", + "CREATE TABLE m2(b INTEGER, a INTEGER, y VARCHAR)", + "INSERT INTO m2 VALUES (10,1,'y1'),(99,2,'y2')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "b", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 10, + "x1", + "y1" + ] + ], + "error": null, + "note": "P10 USING (a,b) star: where do using cols land; right table declares (b,a,y)" + }, + { + "sql": "SELECT * FROM m1 JOIN m2 USING (b, a) ORDER BY x", + "setup": [ + "CREATE TABLE m1(a INTEGER, b INTEGER, x VARCHAR)", + "INSERT INTO m1 VALUES (1,10,'x1'),(2,20,'x2'),(3,30,'x3')", + "CREATE TABLE m2(b INTEGER, a INTEGER, y VARCHAR)", + "INSERT INTO m2 VALUES (10,1,'y1'),(99,2,'y2')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "b", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 10, + "x1", + "y1" + ] + ], + "error": null, + "note": "P11 USING (b,a) reversed list: star order follows USING-list order or table order?" + }, + { + "sql": "SELECT * FROM m1 LEFT JOIN m2 USING (b, a) ORDER BY x", + "setup": [ + "CREATE TABLE m1(a INTEGER, b INTEGER, x VARCHAR)", + "INSERT INTO m1 VALUES (1,10,'x1'),(2,20,'x2'),(3,30,'x3')", + "CREATE TABLE m2(b INTEGER, a INTEGER, y VARCHAR)", + "INSERT INTO m2 VALUES (10,1,'y1'),(99,2,'y2')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "b", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 10, + "x1", + "y1" + ], + [ + 2, + 20, + "x2", + null + ], + [ + 3, + 30, + "x3", + null + ] + ], + "error": null, + "note": "P11b USING (b,a) LEFT: miss row col values" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING (a) JOIN t3 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + "z1" + ] + ], + "error": null, + "note": "P12 chained INNER USING(a) twice: star shape" + }, + { + "sql": "SELECT * FROM t1 LEFT JOIN t2 USING (a) JOIN t3 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + "z1" + ], + [ + 3, + "x3", + null, + "z3" + ] + ], + "error": null, + "note": "P13 chain witness: t1 LEFT t2 USING(a) then INNER t3 USING(a); a=3 misses t2 - does it still match t3 (binds left/coalesced) or not (binds t2.a=NULL)?" + }, + { + "sql": "SELECT a AS ua, t1.a AS a1, t2.a AS a2, t3.a AS a3, x, y, z FROM t1 LEFT JOIN t2 USING (a) LEFT JOIN t3 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "ua", + "INTEGER" + ], + [ + "a1", + "INTEGER" + ], + [ + "a2", + "INTEGER" + ], + [ + "a3", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 1, + 1, + 1, + "x1", + "y1", + "z1" + ], + [ + 2, + 2, + 2, + null, + "x2", + "y2", + null + ], + [ + 3, + 3, + null, + 3, + "x3", + null, + "z3" + ], + [ + null, + null, + null, + null, + "xN", + null, + null + ] + ], + "error": null, + "note": "P14 chained LEFT-LEFT: all three quals + unqualified per row" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 ON t1.a = t2.a JOIN t3 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "BinderException: Binder Error: Ambiguous reference to column name \"a\" (use: \"t1.a\" or \"t2.a\")", + "note": "P15 USING(a) after an ON join that left both a's visible: ambiguous? exact behavior/error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING (x)", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "BinderException: Binder Error: Column \"x\" does not exist on right side of join!", + "note": "P16 USING col exists only on LEFT side (x is t1-only): exact error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING (y)", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "BinderException: Binder Error: Column \"y\" does not exist on left side of join!", + "note": "P17 USING col exists only on RIGHT side (y is t2-only): exact error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING ()", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "ParserException: Parser Error: syntax error at or near \")\"\n\nLINE 1: SELECT * FROM t1 JOIN t2 USING ()\n ^", + "note": "P18 empty USING (): exact error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING (a, a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 2, + "x2", + "y2" + ] + ], + "error": null, + "note": "P19 duplicate col in USING (a, a): behavior/error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 USING (nope)", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": null, + "result": null, + "error": "BinderException: Binder Error: Column \"nope\" does not exist on left side of join!", + "note": "P19b nonexistent col USING (nope): exact error" + }, + { + "sql": "SELECT * FROM t1 JOIN t2 AS u1 USING (a) JOIN t2 AS u2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + "y1" + ], + [ + 2, + "x2", + "y2", + "y2" + ] + ], + "error": null, + "note": "P20 same table twice via aliases, INNER USING: works? star names (y twice?)" + }, + { + "sql": "SELECT * FROM t1 LEFT JOIN t2 AS u1 USING (a) LEFT JOIN t2 AS u2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + "y1" + ], + [ + 2, + "x2", + "y2", + "y2" + ], + [ + 3, + "x3", + null, + null + ], + [ + null, + "xN", + null, + null + ] + ], + "error": null, + "note": "P21 same table twice via aliases, LEFT USING chain" + }, + { + "sql": "SELECT u1.y AS y1, u2.y AS y2 FROM t1 JOIN t2 AS u1 USING (a) JOIN t2 AS u2 USING (a) ORDER BY u1.y", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "y1", + "VARCHAR" + ], + [ + "y2", + "VARCHAR" + ] + ], + "result": [ + [ + "y1", + "y1" + ], + [ + "y2", + "y2" + ] + ], + "error": null, + "note": "P21b same table twice: qualified u1.y / u2.y addressable" + }, + { + "sql": "SELECT * FROM c1 JOIN c2 USING (a)", + "setup": [ + "CREATE TABLE c1(\"A\" INTEGER, x VARCHAR)", + "INSERT INTO c1 VALUES (1,'x1')", + "CREATE TABLE c2(\"A\" INTEGER, y VARCHAR)", + "INSERT INTO c2 VALUES (1,'y1')" + ], + "columns": [ + [ + "A", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ] + ], + "error": null, + "note": "P22 both sides declared \"A\", unquoted lowercase USING (a): works? output col name/case" + }, + { + "sql": "SELECT * FROM c1 JOIN c2 USING (\"A\")", + "setup": [ + "CREATE TABLE c1(\"A\" INTEGER, x VARCHAR)", + "INSERT INTO c1 VALUES (1,'x1')", + "CREATE TABLE c2(\"A\" INTEGER, y VARCHAR)", + "INSERT INTO c2 VALUES (1,'y1')" + ], + "columns": [ + [ + "A", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ] + ], + "error": null, + "note": "P23 both sides \"A\", quoted USING (\"A\")" + }, + { + "sql": "SELECT * FROM d1 JOIN d2 USING (a)", + "setup": [ + "CREATE TABLE d1(a INTEGER, x VARCHAR)", + "INSERT INTO d1 VALUES (1,'x1')", + "CREATE TABLE d2(\"A\" INTEGER, y VARCHAR)", + "INSERT INTO d2 VALUES (1,'y1')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ] + ], + "error": null, + "note": "P24 left declares a, right declares \"A\", USING (a): cross-case match? output name" + }, + { + "sql": "SELECT * FROM d1 JOIN d2 USING (\"A\")", + "setup": [ + "CREATE TABLE d1(a INTEGER, x VARCHAR)", + "INSERT INTO d1 VALUES (1,'x1')", + "CREATE TABLE d2(\"A\" INTEGER, y VARCHAR)", + "INSERT INTO d2 VALUES (1,'y1')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ] + ], + "error": null, + "note": "P24b same mixed setup, USING (\"A\")" + }, + { + "sql": "SELECT * FROM c1 JOIN c2 USING (\"a\")", + "setup": [ + "CREATE TABLE c1(\"A\" INTEGER, x VARCHAR)", + "INSERT INTO c1 VALUES (1,'x1')", + "CREATE TABLE c2(\"A\" INTEGER, y VARCHAR)", + "INSERT INTO c2 VALUES (1,'y1')" + ], + "columns": [ + [ + "A", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ] + ], + "error": null, + "note": "P25 both sides \"A\", quoted lowercase USING (\"a\"): quoted is case-sensitive?" + }, + { + "sql": "SELECT t1.* FROM t1 LEFT JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1" + ], + [ + 2, + "x2" + ], + [ + 3, + "x3" + ], + [ + null, + "xN" + ] + ], + "error": null, + "note": "P26a SELECT t1.* after LEFT USING: includes t1.a" + }, + { + "sql": "SELECT t2.* FROM t1 LEFT JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "y1" + ], + [ + 2, + "y2" + ], + [ + null, + null + ], + [ + null, + null + ] + ], + "error": null, + "note": "P26b SELECT t2.* after LEFT USING: includes t2.a (NULL on miss)?" + }, + { + "sql": "SELECT * FROM t1 RIGHT JOIN t2r USING (a) ORDER BY y", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 9, + null, + "y9" + ] + ], + "error": null, + "note": "P27 RIGHT JOIN USING star: exposed a on left-miss row (a=9) - right's value, refutes 'always left'" + }, + { + "sql": "SELECT * FROM t1 FULL JOIN t2r USING (a) ORDER BY x NULLS LAST", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 2, + "x2", + null + ], + [ + 3, + "x3", + null + ], + [ + null, + "xN", + null + ], + [ + 9, + null, + "y9" + ] + ], + "error": null, + "note": "P28 FULL JOIN USING star: exposed a = COALESCE(l,r)?" + }, + { + "sql": "SELECT a AS ua, t1.a AS l_a, t2r.a AS r_a, x, y FROM t1 FULL JOIN t2r USING (a) ORDER BY x NULLS LAST", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')" + ], + "columns": [ + [ + "ua", + "INTEGER" + ], + [ + "l_a", + "INTEGER" + ], + [ + "r_a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 1, + 1, + "x1", + "y1" + ], + [ + 2, + 2, + null, + "x2", + null + ], + [ + 3, + 3, + null, + "x3", + null + ], + [ + null, + null, + null, + "xN", + null + ], + [ + 9, + null, + 9, + null, + "y9" + ] + ], + "error": null, + "note": "P28b FULL JOIN USING quals: a vs t1.a vs t2r.a" + }, + { + "sql": "SELECT t1.a, x, y FROM t1 LEFT JOIN t2 ON t1.a = t2.a ORDER BY x", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1" + ], + [ + 2, + "x2", + "y2" + ], + [ + 3, + "x3", + null + ], + [ + null, + "xN", + null + ] + ], + "error": null, + "note": "P29 LEFT ON with explicit COALESCE-free projection to compare against P03 row set" + }, + { + "sql": "SELECT count(*) FROM t1 JOIN t2 USING (a)", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(2,'x2'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE t3(a INTEGER, z VARCHAR)", + "INSERT INTO t3 VALUES (1,'z1'),(3,'z3')" + ], + "columns": [ + [ + "count_star()", + "BIGINT" + ] + ], + "result": [ + [ + 2 + ] + ], + "error": null, + "note": "P30 NULL key never matches: inner USING drops the NULL-key row (see P01), LEFT keeps it (see P03) - explicit count" + }, + { + "sql": "SELECT * FROM w1 JOIN t2 USING (a)", + "setup": [ + "CREATE TABLE w1(x VARCHAR, a INTEGER)", + "INSERT INTO w1 VALUES ('x1',1),('x3',3)", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE w2(y VARCHAR, a INTEGER)", + "INSERT INTO w2 VALUES ('y1',1)" + ], + "columns": [ + [ + "x", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + "x1", + 1, + "y1" + ] + ], + "error": null, + "note": "F01 star POSITION pin: left table declares (x,a) - does 'a' stay at its left-table position or get hoisted to front?" + }, + { + "sql": "SELECT * FROM w1 LEFT JOIN t2 USING (a) ORDER BY x", + "setup": [ + "CREATE TABLE w1(x VARCHAR, a INTEGER)", + "INSERT INTO w1 VALUES ('x1',1),('x3',3)", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE w2(y VARCHAR, a INTEGER)", + "INSERT INTO w2 VALUES ('y1',1)" + ], + "columns": [ + [ + "x", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + "x1", + 1, + "y1" + ], + [ + "x3", + 3, + null + ] + ], + "error": null, + "note": "F02 star POSITION pin LEFT + miss: (x,a) left, miss row a=3" + }, + { + "sql": "SELECT * FROM w1 JOIN w2 USING (a)", + "setup": [ + "CREATE TABLE w1(x VARCHAR, a INTEGER)", + "INSERT INTO w1 VALUES ('x1',1),('x3',3)", + "CREATE TABLE t2(a INTEGER, y VARCHAR)", + "INSERT INTO t2 VALUES (1,'y1'),(2,'y2')", + "CREATE TABLE w2(y VARCHAR, a INTEGER)", + "INSERT INTO w2 VALUES ('y1',1)" + ], + "columns": [ + [ + "x", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + "x1", + 1, + "y1" + ] + ], + "error": null, + "note": "F03 star position with using col last on BOTH sides (x,a) join (y,a)" + }, + { + "sql": "SELECT * FROM m3 JOIN m2 USING (a, b) ORDER BY x", + "setup": [ + "CREATE TABLE m3(x VARCHAR, b INTEGER, a INTEGER)", + "INSERT INTO m3 VALUES ('x1',10,1),('x2',20,2)", + "CREATE TABLE m2(b INTEGER, a INTEGER, y VARCHAR)", + "INSERT INTO m2 VALUES (10,1,'y1'),(99,2,'y2')" + ], + "columns": [ + [ + "x", + "VARCHAR" + ], + [ + "b", + "INTEGER" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + "x1", + 10, + 1, + "y1" + ] + ], + "error": null, + "note": "F04 multi-col position: left (x,b,a), right (b,a,y), USING (a,b) - star order?" + }, + { + "sql": "SELECT * FROM m3 JOIN m2 USING (b, a) ORDER BY x", + "setup": [ + "CREATE TABLE m3(x VARCHAR, b INTEGER, a INTEGER)", + "INSERT INTO m3 VALUES ('x1',10,1),('x2',20,2)", + "CREATE TABLE m2(b INTEGER, a INTEGER, y VARCHAR)", + "INSERT INTO m2 VALUES (10,1,'y1'),(99,2,'y2')" + ], + "columns": [ + [ + "x", + "VARCHAR" + ], + [ + "b", + "INTEGER" + ], + [ + "a", + "INTEGER" + ], + [ + "y", + "VARCHAR" + ] + ], + "result": [ + [ + "x1", + 10, + 1, + "y1" + ] + ], + "error": null, + "note": "F05 multi-col position: same, USING (b,a) reversed list - does list order matter now?" + }, + { + "sql": "SELECT * FROM t1 FULL JOIN t2r USING (a) JOIN t3f USING (a) ORDER BY z", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')", + "CREATE TABLE t3f(a INTEGER, z VARCHAR)", + "INSERT INTO t3f VALUES (1,'z1'),(9,'z9'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + "z1" + ], + [ + 3, + "x3", + null, + "z3" + ], + [ + 9, + null, + "y9", + "z9" + ] + ], + "error": null, + "note": "F06 chained-USING binding witness via FULL: t1 FULL t2r USING(a) then JOIN t3f USING(a); row a=9 has t1.a NULL, t2r.a=9 - matching z9 proves 2nd USING binds the COALESCED col, not t1.a" + }, + { + "sql": "SELECT a AS ua, t1.a AS a1, t2r.a AS a2, t3f.a AS a3, x, y, z FROM t1 FULL JOIN t2r USING (a) JOIN t3f USING (a) ORDER BY z", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')", + "CREATE TABLE t3f(a INTEGER, z VARCHAR)", + "INSERT INTO t3f VALUES (1,'z1'),(9,'z9'),(3,'z3')" + ], + "columns": [ + [ + "ua", + "INTEGER" + ], + [ + "a1", + "INTEGER" + ], + [ + "a2", + "INTEGER" + ], + [ + "a3", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + 1, + 1, + 1, + "x1", + "y1", + "z1" + ], + [ + 3, + 3, + null, + 3, + "x3", + null, + "z3" + ], + [ + 9, + null, + 9, + 9, + null, + "y9", + "z9" + ] + ], + "error": null, + "note": "F06b same chain, all quals + unqualified" + }, + { + "sql": "SELECT z FROM t1 FULL JOIN t2r USING (a) JOIN t3f USING (a) WHERE a = 9", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')", + "CREATE TABLE t3f(a INTEGER, z VARCHAR)", + "INSERT INTO t3f VALUES (1,'z1'),(9,'z9'),(3,'z3')" + ], + "columns": [ + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + "z9" + ] + ], + "error": null, + "note": "F07 after chained USING, unqualified a in WHERE binds coalesced: filter a=9 keeps the left-miss row" + }, + { + "sql": "SELECT * FROM t1 FULL JOIN t2r USING (a) JOIN t3f ON t3f.a = a ORDER BY z", + "setup": [ + "CREATE TABLE t1(a INTEGER, x VARCHAR)", + "INSERT INTO t1 VALUES (1,'x1'),(3,'x3'),(NULL,'xN')", + "CREATE TABLE t2r(a INTEGER, y VARCHAR)", + "INSERT INTO t2r VALUES (1,'y1'),(9,'y9')", + "CREATE TABLE t3f(a INTEGER, z VARCHAR)", + "INSERT INTO t3f VALUES (1,'z1'),(9,'z9'),(3,'z3')" + ], + "columns": [ + [ + "a", + "INTEGER" + ], + [ + "x", + "VARCHAR" + ], + [ + "y", + "VARCHAR" + ], + [ + "a", + "INTEGER" + ], + [ + "z", + "VARCHAR" + ] + ], + "result": [ + [ + 1, + "x1", + "y1", + 1, + "z1" + ], + [ + 3, + "x3", + null, + 3, + "z3" + ], + [ + 9, + null, + "y9", + 9, + "z9" + ] + ], + "error": null, + "note": "F08 USING then explicit ON to third table referencing unqualified a: which a?" + } + ] +} \ No newline at end of file diff --git a/src/duckdb/mod.rs b/src/duckdb/mod.rs index 86c1a94..e6daaa3 100644 --- a/src/duckdb/mod.rs +++ b/src/duckdb/mod.rs @@ -86,7 +86,13 @@ fn materialize_map( } keys.push(match ty { Ty::I1 => KeyBits::I1(v.extract()?), - Ty::I64 => KeyBits::I64(v.extract()?), + Ty::I64 => KeyBits::I64(v.extract().map_err(|_| { + build_err(format!( + "unsupported: static table '{}' key column '{name}' value \ + outside BIGINT range (UBIGINT/HUGEINT payloads)", + spec.table + )) + })?), Ty::F64 => KeyBits::F64(v.extract::()?.to_bits()), Ty::Str => KeyBits::Str(v.extract()?), }); @@ -103,7 +109,13 @@ fn materialize_map( } vals.push(match ty { Ty::I1 => ScalarVal::I1(v.extract()?), - Ty::I64 => ScalarVal::I64(v.extract()?), + Ty::I64 => ScalarVal::I64(v.extract().map_err(|_| { + build_err(format!( + "unsupported: static table '{}' value column '{name}' value \ + outside BIGINT range (UBIGINT/HUGEINT payloads)", + spec.table + )) + })?), Ty::F64 => ScalarVal::F64(v.extract()?), Ty::Str => ScalarVal::Str(v.extract()?), }); diff --git a/src/specializer/fold.rs b/src/specializer/fold.rs index 67e40ec..971030a 100644 --- a/src/specializer/fold.rs +++ b/src/specializer/fold.rs @@ -52,7 +52,8 @@ pub fn fold(e: SExpr) -> SExpr { let SExpr { kind, ty, nullable } = e; let e = |kind| SExpr { kind, ty, nullable }; match kind { - SKind::Col(_) | SKind::StaticCol { .. } | SKind::Lit(_) | SKind::NullOf => e(kind), + SKind::Col(_) | SKind::StaticCol { .. } | SKind::Lit(_) | SKind::NullOf + | SKind::JoinHit(_) => e(kind), SKind::IntToFloat(inner) => { let inner = fold(*inner); match as_const(&inner) { diff --git a/src/specializer/frontend.rs b/src/specializer/frontend.rs index 918fb71..c3f4920 100644 --- a/src/specializer/frontend.rs +++ b/src/specializer/frontend.rs @@ -102,10 +102,10 @@ pub fn frontend( return Err(unsup("GROUP BY / HAVING / aggregation")); } - let (binder, joins) = bind_from(select, this_name, in_cols, statics)?; + let (binder, joins, leftover_where) = bind_from(select, this_name, in_cols, statics)?; let mut rel = Rel::Scan; - if let Some(pred) = &select.selection { + if let Some(pred) = &leftover_where { let pred = fold(bool_context(binder.expr(pred)?, "WHERE predicate")?); rel = Rel::Filter { input: Box::new(rel), @@ -195,12 +195,9 @@ fn bind_from<'a>( this_name: &str, in_cols: &'a [Col], statics: &'a [StaticTable], -) -> Result<(Binder<'a>, Vec), PrepareError> { - let [table] = select.from.as_slice() else { - return Err(match select.from.len() { - 0 => unsup("FROM-less SELECT"), - _ => unsup("multiple FROM relations (comma join)"), - }); +) -> Result<(Binder<'a>, Vec, Option), PrepareError> { + let Some((table, comma_rels)) = select.from.split_first() else { + return Err(unsup("FROM-less SELECT")); }; let dyn_name = match &table.relation { TableFactor::Table { name, alias, .. } => { @@ -244,12 +241,6 @@ fn bind_from<'a>( JoinOperator::Left(c) | JoinOperator::LeftOuter(c) => (JoinKind::Left, c), other => return Err(unsup(format!("join type {other:?}"))), }; - let on = match constraint { - JoinConstraint::On(e) => e, - JoinConstraint::Using(_) => return Err(unsup("JOIN USING")), - JoinConstraint::Natural => return Err(unsup("NATURAL JOIN")), - JoinConstraint::None => return Err(unsup("JOIN without ON (cross join)")), - }; let (raw_name, scope_name) = match &join.relation { TableFactor::Table { name, alias, .. } => { let n = name.to_string(); @@ -264,22 +255,7 @@ fn bind_from<'a>( if raw_name.eq_ignore_ascii_case(this_name) { return Err(unsup("joining the dynamic table to itself")); } - let mut table_idx = None; - for (i, st) in statics.iter().enumerate() { - if st.name.eq_ignore_ascii_case(&raw_name) { - if table_idx.is_some() { - return Err(PrepareError::Bind(format!( - "ambiguous static table '{raw_name}'" - ))); - } - table_idx = Some(i); - } - } - let Some(table_idx) = table_idx else { - return Err(PrepareError::Bind(format!( - "table '{raw_name}' was not provided as a static table" - ))); - }; + let table_idx = resolve_static(statics, &raw_name)?; if binder.this_name.eq_ignore_ascii_case(&scope_name) || binder .joins @@ -292,15 +268,52 @@ fn bind_from<'a>( } let st = &statics[table_idx]; - let (keys, key_cols) = bind_on(&binder, st, &scope_name, on)?; + let (keys, key_cols, residual_raw, using) = match constraint { + JoinConstraint::On(e) => { + let (keys, key_cols, res) = bind_on(&binder, st, &scope_name, e)?; + (keys, key_cols, res, false) + } + // USING desugar (wave-4 pins): each column pairs the LEFT + // scope's binding with this table's column; duplicates in the + // list dedupe silently; ambiguity in the left scope (e.g. + // after a prior ON join) errors exactly like DuckDB. + JoinConstraint::Using(cols) => { + let mut keys = Vec::new(); + let mut key_cols = Vec::new(); + for obj in cols { + let [part] = obj.0.as_slice() else { + return Err(unsup("qualified name in JOIN USING")); + }; + let name = part + .as_ident() + .map(|i| i.value.clone()) + .ok_or_else(|| unsup("JOIN USING entry form"))?; + let mut col = None; + for (i, c) in st.cols.iter().enumerate() { + if c.name.eq_ignore_ascii_case(&name) { + col = Some(i as u32); + } + } + let Some(col) = col else { + return Err(PrepareError::Bind(format!( + "column \"{name}\" does not exist on right side of join!" + ))); + }; + if key_cols.contains(&col) { + continue; // USING (a, a) dedupes silently (measured) + } + let key = fold(binder.column(&name)?); + keys.push(promote_key(key, st, col)?); + key_cols.push(col); + } + (keys, key_cols, Vec::new(), true) + } + JoinConstraint::Natural => return Err(unsup("NATURAL JOIN")), + JoinConstraint::None => return Err(unsup("JOIN without ON (cross join)")), + }; let val_cols: Vec = (0..st.cols.len() as u32) .filter(|c| !key_cols.contains(c)) .collect(); - if val_cols.is_empty() { - return Err(unsup(format!( - "join to '{raw_name}' where every column is a key (no value columns)" - ))); - } binder.joins.push(ScopeJoin { name: scope_name, @@ -308,31 +321,226 @@ fn bind_from<'a>( kind, key_cols: key_cols.clone(), val_cols: val_cols.clone(), + keys: keys.clone(), + using, }); + // Residual conjuncts bind with THIS join in scope. + let j = (binder.joins.len() - 1) as u32; + let residual = bind_residual(&binder, j, &residual_raw)?; specs.push(JoinSpec { table: table_idx, kind, keys, key_cols, val_cols, + residual, + }); + } + + // Comma relations (measured: FROM t, u WHERE t.k = u.k is bit-identical + // to the INNER probe, star order included; residual WHERE placement is + // free under INNER). Equi conjuncts pairing the current scope with a + // comma table's column are consumed as its probe keys; everything else + // stays WHERE. A keyless comma table is a cross join — correct for a + // 1-row static via the empty-key map (the duplicate-key check enforces + // single-entry-ness at compile; a 0-row static annihilates, also + // measured). + let mut conjuncts: Vec<&SqlExpr> = Vec::new(); + if let Some(sel) = &select.selection { + collect_conjuncts(sel, &mut conjuncts); + } + let mut consumed = vec![false; conjuncts.len()]; + for rel in comma_rels { + if !rel.joins.is_empty() { + return Err(unsup("JOIN attached to a comma-joined relation")); + } + let (raw_name, scope_name) = match &rel.relation { + TableFactor::Table { name, alias, .. } => { + let n = name.to_string(); + let s = alias + .as_ref() + .map(|a| a.name.value.clone()) + .unwrap_or_else(|| n.clone()); + (n, s) + } + other => return Err(unsup(format!("FROM {other}"))), + }; + if raw_name.eq_ignore_ascii_case(this_name) { + return Err(unsup("joining the dynamic table to itself")); + } + // Unresolvable comma tables (schema-qualified names, table + // functions we didn't get as statics) stay CLEAN. + let table_idx = resolve_static(statics, &raw_name).map_err(|_| { + unsup(format!( + "comma-joined table '{raw_name}' is not a provided static table" + )) + })?; + if binder.this_name.eq_ignore_ascii_case(&scope_name) + || binder + .joins + .iter() + .any(|j| j.name.eq_ignore_ascii_case(&scope_name)) + { + return Err(PrepareError::Bind(format!( + "duplicate table name '{scope_name}' in FROM" + ))); + } + let st = &statics[table_idx]; + let mut keys = Vec::new(); + let mut key_cols = Vec::new(); + for (ci, c) in conjuncts.iter().enumerate() { + if consumed[ci] { + continue; + } + let SqlExpr::BinaryOp { + left, + op: BinaryOperator::Eq, + right, + } = c + else { + continue; + }; + let l = static_col_of(left, st, &scope_name)?; + let r = static_col_of(right, st, &scope_name)?; + let (col, dyn_side, static_side) = match (l, r) { + (Some(c), None) => (c, right.as_ref(), left.as_ref()), + (None, Some(c)) => (c, left.as_ref(), right.as_ref()), + _ => continue, // stays WHERE + }; + if let SqlExpr::Identifier(id) = static_side { + if binder.column(&id.value).is_ok() { + return Err(PrepareError::Bind(format!( + "ambiguous column '{}' in WHERE (qualify it)", + id.value + ))); + } + } + // The dynamic side must bind in the scope BEFORE this table — + // if it references this or a later comma table, leave the + // conjunct in WHERE (a later table may consume it). + let Ok(key) = binder.expr(dyn_side) else { + continue; + }; + keys.push(promote_key(fold(key), st, col)?); + key_cols.push(col); + consumed[ci] = true; + } + let val_cols: Vec = (0..st.cols.len() as u32) + .filter(|c| !key_cols.contains(c)) + .collect(); + binder.joins.push(ScopeJoin { + name: scope_name, + table: st, + kind: JoinKind::Inner, + key_cols: key_cols.clone(), + val_cols: val_cols.clone(), + keys: keys.clone(), + using: false, + }); + specs.push(JoinSpec { + table: table_idx, + kind: JoinKind::Inner, + keys, + key_cols, + val_cols, + residual: None, }); } - Ok((binder, specs)) + + // Rebuild the WHERE from unconsumed conjuncts (identity when nothing + // was consumed — the single-relation path always takes this shape). + let leftover = if comma_rels.is_empty() { + select.selection.clone() + } else { + let mut acc: Option = None; + for (ci, c) in conjuncts.iter().enumerate() { + if consumed[ci] { + continue; + } + acc = Some(match acc { + None => (*c).clone(), + Some(p) => ast_bin(BinaryOperator::And, p, (*c).clone()), + }); + } + acc + }; + Ok((binder, specs, leftover)) } -/// Bind a JOIN ... ON condition: a conjunction of ` = ` -/// equalities. Returns the dynamic-side key expressions (promoted to the -/// map's key types) and the static key columns they match, aligned. -fn bind_on( +fn resolve_static(statics: &[StaticTable], raw_name: &str) -> Result { + let mut table_idx = None; + for (i, st) in statics.iter().enumerate() { + if st.name.eq_ignore_ascii_case(raw_name) { + if table_idx.is_some() { + return Err(PrepareError::Bind(format!( + "ambiguous static table '{raw_name}'" + ))); + } + table_idx = Some(i); + } + } + table_idx.ok_or_else(|| { + PrepareError::Bind(format!( + "table '{raw_name}' was not provided as a static table" + )) + }) +} + +/// Bind the non-key ON conjuncts of join `j` and AND them into the spec's +/// residual, enforcing the wave-4 evaluation-order rule: single-side +/// residuals must be conservatively trap-free (DuckDB scan-pushes them — +/// different error timing); both-sides residuals may trap (DuckDB +/// evaluates them per candidate pair, exactly our hit-guarded lowering). +fn bind_residual( + binder: &Binder<'_>, + j: u32, + raw: &[&SqlExpr], +) -> Result, PrepareError> { + let mut acc: Option = None; + for c in raw { + let bound = bool_context(fold(binder.expr(c)?), "JOIN ON condition")?; + let (mut right, mut left, mut total, mut known) = (false, false, true, true); + scan_residual(&bound, j, &mut right, &mut left, &mut total, &mut known); + if !(total || (left && right && known)) { + return Err(unsup(format!( + "JOIN ON condition '{c}' (single-side residual with trapping \ + ops: DuckDB's scan-pushed evaluation order differs)" + ))); + } + acc = Some(match acc { + None => bound, + Some(p) => { + let nullable = p.nullable || bound.nullable; + SExpr { + kind: SKind::And { + a: Box::new(p), + b: Box::new(bound), + }, + ty: Ty::I1, + nullable, + } + } + }); + } + Ok(acc) +} + +/// Bind a JOIN ... ON condition. Equalities pairing a dynamic-side +/// expression with a static column become probe keys; every OTHER conjunct +/// (non-equalities, constant equalities, both-sides-static equalities) is +/// returned raw for residual binding once the join is in scope (wave-4: +/// `match = key_hit AND residual`). +fn bind_on<'e>( binder: &Binder<'_>, st: &StaticTable, scope_name: &str, - on: &SqlExpr, -) -> Result<(Vec, Vec), PrepareError> { + on: &'e SqlExpr, +) -> Result<(Vec, Vec, Vec<&'e SqlExpr>), PrepareError> { let mut conjuncts = Vec::new(); collect_conjuncts(on, &mut conjuncts); let mut keys = Vec::new(); let mut key_cols = Vec::new(); + let mut residual = Vec::new(); for c in conjuncts { let SqlExpr::BinaryOp { left, @@ -340,24 +548,20 @@ fn bind_on( right, } = c else { - return Err(unsup(format!( - "JOIN ON condition '{c}' (only AND-ed equalities are supported)" - ))); + residual.push(c); + continue; }; let l = static_col_of(left, st, scope_name)?; let r = static_col_of(right, st, scope_name)?; let (col, dyn_side, static_side) = match (l, r) { (Some(c), None) => (c, right.as_ref(), left.as_ref()), (None, Some(c)) => (c, left.as_ref(), right.as_ref()), - (Some(_), Some(_)) => { - return Err(unsup(format!( - "JOIN ON '{c}': both sides are columns of '{scope_name}'" - ))) - } - (None, None) => { - return Err(unsup(format!( - "JOIN ON '{c}': neither side is a column of '{scope_name}'" - ))) + // Both sides this table (r.a = r.b) or neither (test.b = 2, + // NULL = 2): a residual match condition, not a key (measured — + // DuckDB binds these fine and they filter matches). + _ => { + residual.push(c); + continue; } }; // A bare identifier on the static side that also binds in the outer @@ -371,27 +575,87 @@ fn bind_on( } } let key = fold(binder.expr(dyn_side)?); - let col_ty = st.cols[col as usize].ty.ty; - let key = match (key.ty, col_ty) { - (a, b) if a == b => key, - (Ty::I64, Ty::F64) => promote_f64(key), - // Static-side ints promote at materialization: the map key type - // (the key expression's type) becomes F64 and the build side is - // converted while the probe table is built. - (Ty::F64, Ty::I64) => key, - (a, b) => { - return Err(PrepareError::Bind(format!( - "cannot join {} with {} (ON '{}')", - a.name(), - b.name(), - st.cols[col as usize].name - ))) - } - }; + let key = promote_key(key, st, col)?; keys.push(key); key_cols.push(col); } - Ok((keys, key_cols)) + Ok((keys, key_cols, residual)) +} + +/// Promote a dynamic-side key expression to the map's key type. +fn promote_key(key: SExpr, st: &StaticTable, col: u32) -> Result { + let col_ty = st.cols[col as usize].ty.ty; + match (key.ty, col_ty) { + (a, b) if a == b => Ok(key), + (Ty::I64, Ty::F64) => Ok(promote_f64(key)), + // Static-side ints promote at materialization: the map key type + // (the key expression's type) becomes F64 and the build side is + // converted while the probe table is built. + (Ty::F64, Ty::I64) => Ok(key), + (a, b) => Err(PrepareError::Bind(format!( + "cannot join {} with {} (ON '{}')", + a.name(), + b.name(), + st.cols[col as usize].name + ))), + } +} + +/// One traversal answering the wave-4 residual questions about a bound ON +/// residual: `right`/`left` — does it reference THIS join's columns / any +/// other scope; `total` — is every node in the conservative trap-free +/// allowlist (columns, literals, comparisons, IS NULL, logic); `known` — +/// was every node classifiable at all. Acceptance rule at the call site: +/// `total || (left && right && known)` — measured: DuckDB scan-pushes +/// single-side residuals (eager trap timing) but evaluates both-sides +/// residuals per candidate pair, which our hit-guarded lowering matches. +fn scan_residual( + e: &SExpr, + j: u32, + right: &mut bool, + left: &mut bool, + total: &mut bool, + known: &mut bool, +) { + match &e.kind { + SKind::StaticCol { join, .. } | SKind::JoinHit(join) => { + if *join == j { + *right = true; + } else { + *left = true; + } + } + SKind::Col(_) => *left = true, + SKind::Lit(_) | SKind::NullOf => {} + SKind::Cmp { a, b, .. } | SKind::And { a, b } | SKind::Or { a, b } => { + scan_residual(a, j, right, left, total, known); + scan_residual(b, j, right, left, total, known); + } + SKind::Not(a) | SKind::IsNull { inner: a, .. } | SKind::IntToFloat(a) => { + scan_residual(a, j, right, left, total, known); + } + SKind::Arith { a, b, .. } => { + *total = false; + scan_residual(a, j, right, left, total, known); + scan_residual(b, j, right, left, total, known); + } + SKind::Case { arms, default } => { + *total = false; + for (c, r) in arms { + scan_residual(c, j, right, left, total, known); + scan_residual(r, j, right, left, total, known); + } + if let Some(d) = default { + scan_residual(d, j, right, left, total, known); + } + } + // Anything else: not classifiable — the caller must reject rather + // than risk the permissive both-sides path on a wrong guess. + _ => { + *total = false; + *known = false; + } + } } fn collect_conjuncts<'e>(e: &'e SqlExpr, out: &mut Vec<&'e SqlExpr>) { @@ -463,13 +727,22 @@ fn default_name(e: &SqlExpr) -> String { } /// One joined static table in scope: how it is named, which of its columns -/// are probe values (bindable) vs keys (ON-clause only). +/// are probe values (bindable directly) vs keys (reconstructed from the +/// dynamic side: `r.id` ≡ CASE match THEN dyn-key ELSE NULL — wave-4). struct ScopeJoin<'a> { name: String, table: &'a StaticTable, kind: JoinKind, key_cols: Vec, val_cols: Vec, + /// Dynamic-side key expressions aligned with `key_cols` — the material + /// for key-column reconstruction. + keys: Vec, + /// USING join: the static side's using (key) columns are merged into + /// the left occurrence — hidden from bare-name binds and star + /// expansion (measured: merged col sits at the LEFT position with the + /// LEFT value; `t2.a` stays addressable and is NULL on a LEFT miss). + using: bool, } struct Binder<'a> { @@ -623,13 +896,31 @@ impl Binder<'_> { )); } } - for sj in &self.joins { - if qualifier.is_none_or(|q| q.eq_ignore_ascii_case(&sj.name)) { - let key = &sj.table.cols[sj.key_cols[0] as usize].name; - return Err(unsup(format!( - "star expansion over joined table '{}' (includes join key column '{key}')", - sj.name - ))); + // Joined tables expand in FROM order, columns in DECLARED order + // (measured): value columns as probe lanes, key columns via the + // dynamic-side reconstruction, USING keys suppressed (merged into + // the left occurrence). Duplicate output names across the star are + // caught by the existing duplicate-name check — DuckDB emits them + // verbatim, our typed output model cannot (documented constraint). + for (j, sj) in self.joins.iter().enumerate() { + if !qualifier.is_none_or(|q| q.eq_ignore_ascii_case(&sj.name)) { + continue; + } + matched = true; + for (ci, c) in sj.table.cols.iter().enumerate() { + let ci = ci as u32; + if let Some(pos) = sj.val_cols.iter().position(|&v| v == ci) { + cols.push((c.name.clone(), self.static_lane(j, pos))); + } else { + let kp = sj + .key_cols + .iter() + .position(|&k| k == ci) + .expect("column is key or value"); + if !sj.using { + cols.push((c.name.clone(), self.key_lane(j, kp))); + } + } } } if !matched { @@ -904,6 +1195,33 @@ impl Binder<'_> { } } + /// KEY column `key_pos` of join `j`, reconstructed from the dynamic + /// side (measured: on a match the static key equals the probe key; + /// on a LEFT miss it is NULL): INNER rows all matched, so the key + /// expression itself is exact; LEFT wraps it in CASE match THEN key + /// ELSE NULL. + fn key_lane(&self, j: usize, key_pos: usize) -> SExpr { + let sj = &self.joins[j]; + let key = sj.keys[key_pos].clone(); + if sj.kind == JoinKind::Inner { + return key; + } + let ty = key.ty; + let hit = SExpr { + kind: SKind::JoinHit(j as u32), + ty: Ty::I1, + nullable: false, + }; + SExpr { + kind: SKind::Case { + arms: vec![(hit, key)], + default: None, + }, + ty, + nullable: true, + } + } + /// Case-insensitive, spelling-preserving bare-column bind over the whole /// scope: the dynamic table plus every joined static table's value /// columns (DuckDB semantics; ambiguity is an error). @@ -918,7 +1236,6 @@ impl Binder<'_> { }); } } - let mut key_only = false; for (j, sj) in self.joins.iter().enumerate() { for pos in 0..sj.val_cols.len() { if sj.table.cols[sj.val_cols[pos] as usize] @@ -928,16 +1245,21 @@ impl Binder<'_> { hits.push(self.static_lane(j, pos)); } } - key_only |= sj - .key_cols - .iter() - .any(|&ci| sj.table.cols[ci as usize].name.eq_ignore_ascii_case(name)); + // Key columns resolve via reconstruction. A USING join's key + // is MERGED into the left occurrence (measured) — the static + // side contributes no separate binding; an ON join's key + // contributes one, so a bare shared key name is ambiguous, + // exactly like DuckDB. + if !sj.using { + for (kp, &ci) in sj.key_cols.iter().enumerate() { + if sj.table.cols[ci as usize].name.eq_ignore_ascii_case(name) { + hits.push(self.key_lane(j, kp)); + } + } + } } match hits.len() { 1 => Ok(hits.pop().expect("len checked")), - 0 if key_only => Err(unsup(format!( - "referencing join key column '{name}' outside its ON clause" - ))), // Real DuckDB features we don't model reject cleanly, not as // bind errors: the rowid pseudo-column, and DuckDB's lateral // alias extension (a SELECT alias visible inside WHERE). @@ -999,14 +1321,13 @@ impl Binder<'_> { if let Some(pos) = hit { return Ok(self.static_lane(j, pos)); } - if sj - .key_cols - .iter() - .any(|&ci| sj.table.cols[ci as usize].name.eq_ignore_ascii_case(name)) - { - return Err(unsup(format!( - "referencing join key column '{table}.{name}' outside its ON clause" - ))); + // Qualified key access reconstructs from the dynamic side — + // measured to stay addressable even after USING (NULL on a + // LEFT miss, never coalesced). + for (kp, &ci) in sj.key_cols.iter().enumerate() { + if sj.table.cols[ci as usize].name.eq_ignore_ascii_case(name) { + return Ok(self.key_lane(j, kp)); + } } return Err(PrepareError::Bind(format!( "column '{name}' does not exist in '{table}'" diff --git a/src/specializer/ir/parse.rs b/src/specializer/ir/parse.rs index 3e010fd..e1cb8b0 100644 --- a/src/specializer/ir/parse.rs +++ b/src/specializer/ir/parse.rs @@ -574,12 +574,22 @@ impl Parser { Ok(StaticTy::Scalar(ct)) } "map" => { + // Either side may be empty (wave-4): `map() -> (i64)` is a + // cross-join single-entry map, `map(i64) -> ()` a semi join. self.expect(Tok::LParen)?; - let keys = self.ty_list()?; + let keys = if *self.peek() == Tok::RParen { + Vec::new() + } else { + self.ty_list()? + }; self.expect(Tok::RParen)?; self.expect(Tok::Arrow)?; self.expect(Tok::LParen)?; - let values = self.ty_list()?; + let values = if *self.peek() == Tok::RParen { + Vec::new() + } else { + self.ty_list()? + }; self.expect(Tok::RParen)?; Ok(StaticTy::Map { keys, values }) } diff --git a/src/specializer/ir/tests.rs b/src/specializer/ir/tests.rs index c8f9e12..29662b3 100644 --- a/src/specializer/ir/tests.rs +++ b/src/specializer/ir/tests.rs @@ -589,11 +589,12 @@ fn deep_cfg_verifies_without_crashing() { verify(&p).expect("a deep linear CFG is legal"); } -/// `map() -> (..)` / `map(..) -> ()` cannot be expressed by the grammar, so -/// a verified program containing one could not be reloaded. +/// Wave-4: one-sided empty map signatures are legal (cross-join and +/// semi-join shapes) and round-trip through the text format; only the +/// BOTH-empty map — which carries no information — is rejected. #[test] fn rejects_empty_map_static_signatures() { - use super::{StaticTy, Ty}; + use super::{parse::parse, print::print, StaticTy, Ty}; for st in [ StaticTy::Map { keys: vec![], @@ -605,14 +606,25 @@ fn rejects_empty_map_static_signatures() { }, ] { let p = api_program(vec![st], "f", vec![store_emit_block()]); - let errs = verify(&p).expect_err("empty map signature must not verify"); - assert!( - errs.iter() - .any(|e| e.to_string().contains("at least one key and one value")), - "wrong errors: {:?}", - errs.iter().map(|e| e.to_string()).collect::>() - ); + verify(&p).expect("one-sided empty map signatures verify"); + let text = print(&p); + assert_eq!(parse(&text).unwrap(), p, "round-trip failed:\n{text}"); } + let p = api_program( + vec![StaticTy::Map { + keys: vec![], + values: vec![], + }], + "f", + vec![store_emit_block()], + ); + let errs = verify(&p).expect_err("both-empty map must not verify"); + assert!( + errs.iter() + .any(|e| e.to_string().contains("neither keys nor values")), + "wrong errors: {:?}", + errs.iter().map(|e| e.to_string()).collect::>() + ); } /// A non-identifier (or empty) function name prints as unparseable text. diff --git a/src/specializer/ir/verify.rs b/src/specializer/ir/verify.rs index 7582c99..4a2e0ef 100644 --- a/src/specializer/ir/verify.rs +++ b/src/specializer/ir/verify.rs @@ -83,14 +83,18 @@ fn check_structure(p: &Program, errs: &mut Vec) { format!("function name '{}' must be an identifier", p.name), ); } + // Wave-4: maps may have EMPTY keys (cross join to a table whose + // single-entry-ness the duplicate-key check enforces at compile) and + // EMPTY values (all-key/semi joins — the probe carries only the hit). + // A map that is empty on BOTH axes carries no information at all. for (i, st) in p.statics.iter().enumerate() { if let StaticTy::Map { keys, values } = st { - if keys.is_empty() || values.is_empty() { + if keys.is_empty() && values.is_empty() { err( errs, None, None, - format!("@{i}: map statics need at least one key and one value column"), + format!("@{i}: map static with neither keys nor values"), ); } } diff --git a/src/specializer/lower.rs b/src/specializer/lower.rs index ef0fcab..ae09a87 100644 --- a/src/specializer/lower.rs +++ b/src/specializer/lower.rs @@ -59,7 +59,7 @@ pub fn lower( } }; - let mut fb = FB::new(in_cols, joins); + let mut fb = FB::new(in_cols, joins, catalog); // Joins run before WHERE (SQL order), each in FROM order: probe, and for // INNER skip the row on a miss. A LEFT join's probe is also forced here @@ -188,16 +188,22 @@ struct FB<'a> { cur: usize, in_cols: &'a [Col], joins: &'a [JoinSpec], + catalog: &'a [StaticTable], } impl<'a> FB<'a> { - fn new(in_cols: &'a [Col], joins: &'a [JoinSpec]) -> FB<'a> { + fn new( + in_cols: &'a [Col], + joins: &'a [JoinSpec], + catalog: &'a [StaticTable], + ) -> FB<'a> { FB { b: Builder::new(), blocks: vec![PB::new(vec![])], cur: 0, in_cols, joins, + catalog, } } @@ -859,6 +865,13 @@ impl<'a> FB<'a> { val: dst, }) } + SKind::JoinHit(j) => { + let (match_v, _) = self.emit_probe(*j, live)?; + Ok(Lane { + flag: None, + val: match_v, + }) + } SKind::SLen { bytes, a } => { let l = self.emit(a, live)?; let dst = self.fresh(); @@ -919,7 +932,7 @@ impl<'a> FB<'a> { let spec = &self.joins[j as usize]; let hit = self.fresh(); - let dsts: Vec = spec.val_cols.iter().map(|_| self.b.fresh()).collect(); + let mut dsts: Vec = spec.val_cols.iter().map(|_| self.b.fresh()).collect(); self.inst(Inst::Probe { static_id: j, hit, @@ -930,10 +943,87 @@ impl<'a> FB<'a> { None => hit, Some(f) => self.bin(BinOp::And, f, hit), }; + // Cache the RAW probe first: the residual below references this + // join's own columns, and its StaticCol emissions must find the + // lanes instead of recursing. self.blocks[self.cur] .probes .insert(j, (valid_hit, dsts.clone())); - Ok((valid_hit, dsts)) + let residual = self.joins[j as usize].residual.clone(); + let match_v = match residual { + None => valid_hit, + Some(res) => { + // Hit-guarded lazy evaluation (wave-4 pins: DuckDB + // evaluates both-sides residuals per candidate PAIR) — + // the mini-case shape: brif raw_hit -> eval, else false. + // The strict block-param SSA means the probe's value lanes + // must ride the params (and, during residual emission, the + // live stack — nested CASE machinery rebinds them). + let spec = &self.joins[j as usize]; + let dst_tys: Vec = spec + .val_cols + .iter() + .map(|&c| self.catalog[spec.table].cols[c as usize].ty.ty) + .collect(); + let live_width = Self::live_types(live).len(); + let mut join_tys = Self::live_types(live); + join_tys.extend(dst_tys.iter().copied()); + join_tys.push(Ty::I1); + let (join, join_params) = self.create_block(&join_tys); + let mut eval_tys = Self::live_types(live); + eval_tys.extend(dst_tys.iter().copied()); + let (eval, eval_params) = self.create_block(&eval_tys); + let mut then_args = Self::live_args(live); + then_args.extend(dsts.iter().copied()); + let mut miss_args = Self::live_args(live); + miss_args.extend(dsts.iter().copied()); + let f = self.const_i1(false); + miss_args.push(f); + self.term(Term::Brif { + cond: valid_hit, + then_to: BlockId(eval as u32), + then_args, + else_to: BlockId(join as u32), + else_args: miss_args, + }); + self.switch(eval); + Self::rebind_live(live, &eval_params[..live_width]); + // Ride the dsts on the live stack through the residual — + // and seed the cache with hit = TRUE (by construction in + // the guarded branch) so StaticCol/JoinHit resolve here. + let eval_dsts: Vec = eval_params[live_width..].to_vec(); + for (&d, &ty) in eval_dsts.iter().zip(dst_tys.iter()) { + live.push((Lane { flag: None, val: d }, ty)); + } + let t = self.const_i1(true); + self.blocks[self.cur].probes.insert(j, (t, eval_dsts)); + let rl = self.emit(&res, live)?; + // 3VL collapse: NULL residual is a non-match. + let rv = match rl.flag { + None => rl.val, + Some(f) => self.bin(BinOp::And, f, rl.val), + }; + let cur_dsts: Vec = live + .drain(live.len() - dst_tys.len()..) + .map(|(l, _)| l.val) + .collect(); + let mut hit_args = Self::live_args(live); + hit_args.extend(cur_dsts.iter().copied()); + hit_args.push(rv); + self.term(Term::Jump { + to: BlockId(join as u32), + args: hit_args, + }); + self.switch(join); + Self::rebind_live(live, &join_params[..live_width]); + dsts = join_params[live_width..live_width + dst_tys.len()].to_vec(); + join_params[live_width + dst_tys.len()] + } + }; + // Downstream consumers (INNER skip, LEFT flags, StaticCol, JoinHit) + // read the cache: store the final MATCH there. + self.blocks[self.cur].probes.insert(j, (match_v, dsts.clone())); + Ok((match_v, dsts)) } /// Branchless Kleene AND/OR from flag algebra. With the lane contract diff --git a/src/specializer/plan.rs b/src/specializer/plan.rs index 456959e..8d76c6a 100644 --- a/src/specializer/plan.rs +++ b/src/specializer/plan.rs @@ -51,9 +51,17 @@ pub struct JoinSpec { /// aligned with `keys`. pub key_cols: Vec, /// The remaining columns, in table order — the probe's value lanes. + /// May be EMPTY (all-key/semi joins — wave-4 pins). /// ponytail: all non-key columns become map values even if unreferenced; /// prune to referenced columns when codegen makes the width measurable. pub val_cols: Vec, + /// Non-key ON conjuncts, ANDed: `match = key_hit AND residual` with + /// 3VL collapse (NULL => non-match). Evaluated HIT-GUARDED — exactly + /// DuckDB's per-candidate-pair laziness for both-sides residuals; + /// single-side residuals are restricted to trap-free shapes at bind + /// (wave-4 pins: DuckDB scan-pushes those, different error timing). + /// May reference this join's own columns via StaticCol. + pub residual: Option, } /// A bound, typed scalar expression. `nullable` is the frontend's @@ -234,6 +242,10 @@ pub enum SKind { }, /// strip_accents — Str -> Str, total (oracle table + Hangul compose). StripAccents(Box), + /// TRUE iff join `join` MATCHED this row (key hit AND residual) — + /// i1, never NULL. The building block for key-column reconstruction + /// (`r.id` ≡ CASE JoinHit THEN dyn-key ELSE NULL) and semi joins. + JoinHit(u32), } /// SQL-level arithmetic. `Div` is DuckDB's `/` — ALWAYS float division diff --git a/src/specializer/tests.rs b/src/specializer/tests.rs index 6fde49d..bfbb061 100644 --- a/src/specializer/tests.rs +++ b/src/specializer/tests.rs @@ -446,20 +446,41 @@ fn star_expands_in_declared_order_with_exclude() { #[test] fn star_over_joined_table_rejects_by_name() { + // Wave-4: joined-table stars expand (key columns reconstruct from the + // dynamic side); only DUPLICATE output names still reject — DuckDB + // emits them verbatim, the typed output model cannot hold them. let schema = cols(&[("a", Ty::I64, false)]); let st = stat("dim", &[("id", Ty::I64, false), ("v", Ty::F64, false)]); - for sql in [ - "SELECT * FROM __THIS__ JOIN dim ON a = dim.id", - "SELECT dim.* FROM __THIS__ JOIN dim ON a = dim.id", + for (sql, want) in [ + ( + "SELECT * FROM __THIS__ JOIN dim ON a = dim.id", + vec!["a", "id", "v"], + ), + ( + "SELECT dim.* FROM __THIS__ JOIN dim ON a = dim.id", + vec!["id", "v"], + ), ] { - match prepare(sql, "__THIS__", &schema, std::slice::from_ref(&st)) { - Err(PrepareError::Unsupported(m)) => assert!( - m.contains("star expansion over joined table") && m.contains("id"), - "'{sql}': got '{m}'" - ), - Err(other) => panic!("'{sql}': wrong error kind: {other}"), - Ok(_) => panic!("'{sql}': unexpectedly prepared"), + let p = prepare(sql, "__THIS__", &schema, std::slice::from_ref(&st)) + .unwrap_or_else(|e| panic!("'{sql}': {e}")) + .program; + let names: Vec<&str> = p.out_cols.iter().map(|c| c.name.as_str()).collect(); + assert_eq!(names, want, "'{sql}'"); + } + // A star that would produce duplicate names still rejects cleanly. + // (The bare `id = dim.id` spelling is an ambiguity error in DuckDB + // too — qualify the dynamic side.) + let clash = cols(&[("id", Ty::I64, false)]); + match prepare( + "SELECT * FROM __THIS__ JOIN dim ON __THIS__.id = dim.id", + "__THIS__", + &clash, + std::slice::from_ref(&st), + ) { + Err(PrepareError::Unsupported(m)) => { + assert!(m.contains("duplicate output column"), "got '{m}'") } + other => panic!("wanted duplicate-name unsup, got {:?}", other.err()), } // Qualified star over the ROW table under a join is fine. let p = prepare( @@ -476,7 +497,10 @@ fn star_over_joined_table_rejects_by_name() { fn unsupported_constructs_are_named_cleanly() { let schema = cols(&[("a", Ty::I64, false)]); for (sql, needle) in [ - ("SELECT a FROM __THIS__ JOIN t USING (a)", "USING"), + ( + "SELECT a FROM __THIS__ FULL OUTER JOIN t ON a = t.a", + "join type", + ), // Bare aggregates parse as plain function calls; they reject via the // function arm until the catalogue distinguishes aggregation. ("SELECT sum(a) FROM __THIS__", "aggregate function sum"), @@ -712,29 +736,28 @@ fn join_programs_are_canonical_ir() { #[test] fn join_shape_errors() { + // Wave-4: both former rejections now prepare — the key column + // reconstructs from the dynamic side, and all-key (semi) joins probe + // with zero value lanes. let schema = cols(&[("k", Ty::I64, false)]); - // A join key column referenced outside its ON clause is a clean unsup. let dim = stat("dim", &[("id", Ty::I64, false), ("name", Ty::Str, false)]); - match prepare( + let p = prepare( "SELECT dim.id FROM __THIS__ JOIN dim ON k = dim.id", "__THIS__", &schema, &[dim], - ) { - Err(PrepareError::Unsupported(msg)) => assert!(msg.contains("ON clause"), "got: {msg}"), - other => panic!("wrong outcome: {:?}", other.err()), - } - // A join where every column is a key has no value columns to produce. + ) + .expect("key reconstruction prepares") + .program; + assert_eq!(p.out_cols[0].name, "id"); let keys_only = stat("dim", &[("id", Ty::I64, false)]); - match prepare( + prepare( "SELECT k FROM __THIS__ JOIN dim ON k = dim.id", "__THIS__", &schema, &[keys_only], - ) { - Err(PrepareError::Unsupported(msg)) => assert!(msg.contains("value columns"), "got: {msg}"), - other => panic!("wrong outcome: {:?}", other.err()), - } + ) + .expect("all-key join prepares"); } #[test] diff --git a/tests/test_duckdb_interpreter.py b/tests/test_duckdb_interpreter.py index 768af35..7900d8b 100644 --- a/tests/test_duckdb_interpreter.py +++ b/tests/test_duckdb_interpreter.py @@ -576,12 +576,15 @@ def test_star_qualified_over_row_table_under_join(): ) -def test_star_over_joined_table_rejects_by_name(): - with pytest.raises(ValueError, match="star expansion over joined table"): - DuckDBInferFn( - "SELECT * FROM __THIS__ JOIN dim ON k = dim.id", - row_tables={"__THIS__": _row_model({"k": "int"})}, - static_tables={"dim": DIM}, +def test_star_over_joined_table_expands(): + # Wave-4: joined-table stars expand, the key column reconstructed from + # the dynamic side — oracle-checked end to end (INNER and LEFT). + for join in ["JOIN", "LEFT JOIN"]: + duck_check( + f"SELECT * FROM __THIS__ {join} dim ON k = dim.id", + {"k": "int"}, + [{"k": 1}, {"k": 2}, {"k": 3}], + {"dim": DIM}, ) diff --git a/tests/test_duckdb_wave4_joins.py b/tests/test_duckdb_wave4_joins.py new file mode 100644 index 0000000..af892c8 --- /dev/null +++ b/tests/test_duckdb_wave4_joins.py @@ -0,0 +1,221 @@ +"""Wave-4 join forms vs the duckdb oracle. + +Pins: docs/superpowers/specs/2026-07-26-wave4-join-pins.md — USING +desugar (merged column = left value at the left position), residual ON +predicates (LEFT keeps residual-failing key-matches with a NULL side), +all-key/semi joins, star-over-join key reconstruction, comma-join +rewrite, cross join to a 1-row static. +""" + +from __future__ import annotations + +import pytest +from test_duckdb_interpreter import duck_check, static + +R = static( + {"id": "int", "category": "int", "budget": "int"}, + [ + {"id": 1, "category": 1, "budget": 100}, + {"id": 2, "category": 2, "budget": 1000}, + {"id": 3, "category": 1, "budget": 50}, + ], +) +L_ROWS = [ + {"lid": 1, "val": 0, "amount": 200}, + {"lid": 1, "val": 5, "amount": 200}, + {"lid": 2, "val": 9, "amount": 500}, + {"lid": 4, "val": 9, "amount": 500}, + {"lid": None, "val": 1, "amount": 1}, +] +L = {"lid": "int?", "val": "int", "amount": "int"} + + +def test_inner_residual_matches_where_placement(): + duck_check( + "SELECT lid, val, r.budget AS b FROM __THIS__ JOIN r ON lid = r.id AND val > 1", + L, + L_ROWS, + {"r": R}, + ) + + +def test_left_residual_keeps_row_with_null_side(): + # THE pin: key matches but residual fails -> row SURVIVES, right side + # NULL (val=0 row); WHERE placement would drop it. + duck_check( + "SELECT lid, val, r.budget AS b, r.category AS c FROM __THIS__ " + "LEFT JOIN r ON lid = r.id AND val > 1", + L, + L_ROWS, + {"r": R}, + ) + + +def test_left_right_side_residual(): + duck_check( + "SELECT lid, r.budget AS b FROM __THIS__ " + "LEFT JOIN r ON lid = r.id AND r.category = 1", + L, + L_ROWS, + {"r": R}, + ) + + +def test_constant_residuals_true_false(): + for c in ["true", "false"]: + duck_check( + f"SELECT lid, r.budget AS b FROM __THIS__ " + f"LEFT JOIN r ON lid = r.id AND {c}", + L, + L_ROWS, + {"r": R}, + ) + duck_check( + f"SELECT lid, r.budget AS b FROM __THIS__ JOIN r ON lid = r.id AND {c}", + L, + L_ROWS, + {"r": R}, + ) + + +def test_both_sides_residual(): + duck_check( + "SELECT lid, r.budget AS b FROM __THIS__ " + "LEFT JOIN r ON lid = r.id AND amount + r.budget > 1100", + L, + L_ROWS, + {"r": R}, + ) + + +def test_left_only_constant_equality_residual(): + duck_check( + "SELECT lid, val, r.budget AS b FROM __THIS__ JOIN r ON r.id = lid AND val = 9", + L, + L_ROWS, + {"r": R}, + ) + + +def test_all_key_semi_join(): + keys = static({"id": "int"}, [{"id": 1}, {"id": 3}]) + duck_check( + "SELECT lid, val FROM __THIS__ JOIN k ON lid = k.id", + L, + L_ROWS, + {"k": keys}, + ) + + +def test_star_over_join_reconstructs_key(): + # Star includes r's key column (named distinctly to keep the typed + # output model happy); LEFT misses show it NULL. + r2 = static( + {"rid": "int", "budget": "int"}, + [{"rid": 1, "budget": 100}, {"rid": 2, "budget": 1000}], + ) + duck_check( + "SELECT * FROM __THIS__ LEFT JOIN r2 ON lid = r2.rid", + L, + L_ROWS, + {"r2": r2}, + ) + duck_check( + "SELECT r2.rid AS k FROM __THIS__ JOIN r2 ON lid = r2.rid", + L, + L_ROWS, + {"r2": r2}, + ) + + +def test_using_merges_key_and_binds_both_quals(): + r3 = static( + {"lid": "int", "budget": "int"}, + [{"lid": 1, "budget": 100}, {"lid": 2, "budget": 1000}], + ) + # Merged star column once, left value; r3.lid NULL on LEFT miss. + duck_check( + "SELECT * FROM __THIS__ LEFT JOIN r3 USING (lid)", + L, + L_ROWS, + {"r3": r3}, + ) + duck_check( + "SELECT lid, r3.lid AS rk, budget FROM __THIS__ LEFT JOIN r3 USING (lid)", + L, + L_ROWS, + {"r3": r3}, + ) + duck_check( + "SELECT lid, budget FROM __THIS__ JOIN r3 USING (lid)", + L, + L_ROWS, + {"r3": r3}, + ) + + +def test_comma_join_equi_rewrite(): + duck_check( + "SELECT lid, val, budget FROM __THIS__, r WHERE lid = r.id AND val > 1", + L, + L_ROWS, + {"r": R}, + ) + + +def test_comma_join_three_way(): + dim = static( + {"category": "int", "label": "str"}, + [ + {"category": 1, "label": "low"}, + {"category": 2, "label": "high"}, + ], + ) + duck_check( + "SELECT lid, budget, label FROM __THIS__, r, dim " + "WHERE lid = r.id AND r.category = dim.category", + L, + L_ROWS, + {"r": R, "dim": dim}, + ) + + +def test_cross_join_to_one_row_static(): + one = static({"base": "int"}, [{"base": 7}]) + duck_check( + "SELECT lid, val + base AS vb FROM __THIS__, one", + L, + L_ROWS, + {"one": one}, + ) + + +def test_cross_join_to_empty_static_annihilates(): + empty = static({"base": "int"}, []) + duck_check( + "SELECT lid, base FROM __THIS__, empty", + L, + L_ROWS, + {"empty": empty}, + ) + + +def test_cross_join_to_multirow_static_rejects_cleanly(): + two = static({"base": "int"}, [{"base": 1}, {"base": 2}]) + with pytest.raises(ValueError, match="duplicate map key"): + duck_check( + "SELECT lid, base FROM __THIS__, two", + L, + L_ROWS, + {"two": two}, + ) + + +def test_single_side_trapping_residual_rejects_cleanly(): + with pytest.raises(ValueError, match="trapping"): + duck_check( + "SELECT lid FROM __THIS__ JOIN r ON lid = r.id AND log(r.budget) > 0", + L, + L_ROWS, + {"r": R}, + )