Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
id: TASK-47
title: 'Specializer SQL support: workload builtins & predicates wave 1'
status: In Progress
status: Done
assignee: []
created_date: '2026-07-26 11:42'
updated_date: '2026-07-26 12:20'
updated_date: '2026-07-26 15:05'
labels: []
milestone: m-7
dependencies:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
id: TASK-48
title: 'Specializer SQL support: LIKE, dynamic-table alias, comma-join rewrite (wave 2)'
status: In Progress
assignee: []
created_date: '2026-07-26 15:05'
updated_date: '2026-07-26 15:05'
labels: []
milestone: m-7
dependencies:
- TASK-47
documentation:
- docs/superpowers/specs/2026-07-25-sql-specializer-design.md
- docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md
type: feature
ordinal: 42000
---

## Description

<!-- SECTION:DESCRIPTION:BEGIN -->
The dual-axis rung plus two structural cheap wins, ~120 corpus first-blockers in reach. (1) LIKE / NOT LIKE with % and _ wildcards and the ESCAPE clause — DuckDB semantics pinned FIRST (case sensitivity, codepoint vs byte matching for _, escape edge cases, NULL propagation, degenerate patterns); a compiled-pattern op on both backends via one shared matcher; SIMILAR TO / regexp reject by name unless the pins show LIKE-only covers the corpus head. Closes the workload ladder's last gap (title extraction, device normalization patterns). (2) Alias on the dynamic table — scope plumbing in the frontend binder (30 cases). (3) Comma-join rewrite: FROM t, dim WHERE <equi-conjuncts> rewrites to the INNER equi-join the engine already serves, with non-equi/cross shapes rejecting cleanly by name (up to 50 join-form cases share this first blocker).
<!-- SECTION:DESCRIPTION:END -->

## Acceptance Criteria
<!-- AC:BEGIN -->
- [x] #1 LIKE semantics pinned by measurement (duck_check tests recorded before implementation: wildcards, ESCAPE, unicode, NULL, degenerate patterns) and both backends agree via a shared matcher (differential extended)
- [x] #2 Dynamic-table alias binds (qualified refs via alias, original name behavior measured and mirrored)
- [x] #3 RE-SCOPED by census (recorded below): the corpus comma-join cases are SELF-joins of the dynamic table (FROM integers i1, integers i2 ...), not dynamic-x-static shapes — a rewrite would flip ~0 cases. Comma-join stays cleanly unsupported; dynamic self-join is a distinct future feature (needs row-table-as-static materialization).
- [x] #4 Corpus replay: wave-2 first-blocker cases flip to match or a named deeper blocker; zero FAILs; tally recorded here
- [x] #5 mise gate-specializer green
<!-- AC:END -->

## Implementation Notes

<!-- SECTION:NOTES:BEGIN -->
Census before code (2026-07-26): the 30 alias-first-blocker cases are mostly alias + a DEEPER blocker (self-joins, USING, rowid, column-renaming aliases); the ~30 comma-join cases are nearly all SELF cross-products of the dynamic table — the planned equi-WHERE rewrite would flip approximately zero, so it was dropped from scope honestly rather than shipped as dead code. Alias landed: the alias REPLACES the original name (measured: qualified refs through the original are DuckDB binder errors) — implemented by making the alias the binder's this_name, which yields exactly that scoping for qualified refs, star expansion, and error messages; AS/bare forms identical; column-renaming alias t(a,b) rejects by name. LIKE landed: byte-based O(n*m) two-pointer matcher with codepoint _, leftmost-first backtracking reproducing the DATA-DEPENDENT dangling-escape error rows, full ESCAPE semantics (any single byte, doubled = literal, never self-matching, '' = none, NULL = NULL), NOT LIKE, and ILIKE via double simple-casemap fold (exhaustively verified vs DuckDB lower()). The corpus caught a live divergence during replay: DuckDB's ILIKE result for NUL-containing rows depends on SIBLING rows (stats-selected kernels; the generic one NUL-truncates) — irreproducible row-locally, recorded as the repo's first documented oracle divergence (_KNOWN_DIVERGENT_SOURCES) with the measurement in the pins. SIMILAR TO rejects by name (DuckDB binds it to regexp_full_match). CORPUS: 240 -> 265 match of 678, zero FAILs, gate green.
<!-- SECTION:NOTES:END -->

## Final Summary

<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Wave 2 complete: LIKE/NOT LIKE/ILIKE with full ESCAPE semantics on both backends (pins-first; the matcher reproduces even DuckDB's data-dependent error rows while avoiding its measured O(n^k) blowup), dynamic-table alias (alias replaces the original name), comma-join honestly descoped by census (corpus cases are self-joins). Corpus 240 -> 265/678, zero FAILs; first documented oracle divergence recorded with proof of row-local irreproducibility (stats-dependent ILIKE kernels).
<!-- SECTION:FINAL_SUMMARY:END -->
21 changes: 21 additions & 0 deletions docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,24 @@ differs.
DOUBLE column path, as with substr).
- Repo doc drift found during verification: interp.rs header still says
fcmp is IEEE-with-NaN-false; it is duck_fcmp. Fix in this wave.

## Wave-2 addendum: LIKE (pins-wave1/pins_like.json)

Byte-based matcher, codepoint `_`, no normalization; backslash is a LITERAL
(no implicit escape — unlike Postgres); ESCAPE takes any single BYTE (a
2-byte codepoint errors), doubled = literal, the escape char never matches
itself unescaped, ESCAPE '' = none, ESCAPE NULL = NULL. The dangling-escape
error is DATA-DEPENDENT per row (string exhausted -> plain false; bytes
remaining when the matcher examines it -> SyntaxException) — reproduced by
leftmost-first backtracking. ILIKE = fold both sides with the measured
simple casemap then LIKE (exhaustive all-codepoint sweep, zero exceptions);
DuckDB's oracle-side caveats: (1) a stats-dependent ASCII kernel whose only
divergence surface is U+212A/U+0130 in patterns, and (2) the generic
kernel's fold NUL-truncates inconsistently, making a NUL-containing row's
result depend on SIBLING rows — statistics-dependent semantics are
irreproducible row-locally, so the engine is NUL-transparent and the one
corpus case is a documented divergence (test_corpus_replay
_KNOWN_DIVERGENT_SOURCES). SIMILAR TO rejects by name: DuckDB binds it to
regexp_full_match (RE2), not SQL wildcards. DuckDB's own column-pattern
LIKE is naive-recursive (measured 23s on one pathological row); ours is the
O(n*m) two-pointer restart with identical booleans and error rows.
Loading