You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: backlog/tasks/task-53 - Specializer-wave-B-—-regexp-family-regexp_-functions-~-~-SIMILAR-TO-deferred-star-forms.md
+32-8Lines changed: 32 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,9 +3,10 @@ id: TASK-53
3
3
title: >-
4
4
Specializer wave B — regexp family (regexp_* functions, ~/!~/SIMILAR TO,
5
5
deferred star forms)
6
-
status: In Progress
6
+
status: Done
7
7
assignee: []
8
8
created_date: '2026-07-26 23:50'
9
+
updated_date: '2026-07-27 00:35'
9
10
labels: []
10
11
milestone: m-7
11
12
dependencies:
@@ -31,11 +32,34 @@ Pins-first; wave-3 over-generalization precedent applies — every claim needs a
31
32
32
33
## Acceptance Criteria
33
34
<!-- AC:BEGIN -->
34
-
-[]#1 Pins-first: wave-B pins spec (md + JSON) committed before implementation, incl. the duckdb-vs-rust-regex differential battery and the crate decision
35
-
-[]#2 regexp_matches / regexp_full_match / regexp_extract / regexp_replace serve per pins (options strings, group semantics, backrefs, NULL handling); list/table forms classify clean
36
-
-[]#3 Operators ~ / !~ and SIMILAR TO on values serve per pins (anchoring semantics measured, not assumed)
37
-
-[]#4 Deferred star forms serve: * SIMILAR TO name filter (incl. the pinned NOT asymmetry) and COLUMNS('re') expansion
38
-
-[]#5 Patterns whose semantics differ between RE2 and rust-regex classify clean-unsupported (guard at bind time from the pinned divergence list) — never a wrong answer
39
-
-[]#6 Corpus replay: three-outcome contract holds, zero FAILs, match count reported
40
-
-[]#7 Gate green both backends, clippy clean, serving-bench parity gate passes
35
+
-[x]#1 Pins-first: wave-B pins spec (md + JSON) committed before implementation, incl. the duckdb-vs-rust-regex differential battery and the crate decision
36
+
-[x]#2 regexp_matches / regexp_full_match / regexp_extract / regexp_replace serve per pins (options strings, group semantics, backrefs, NULL handling); list/table forms classify clean
37
+
-[x]#3 Operators ~ / !~ and SIMILAR TO on values serve per pins (anchoring semantics measured, not assumed)
38
+
-[x]#4 Deferred star forms serve: * SIMILAR TO name filter (incl. the pinned NOT asymmetry) and COLUMNS('re') expansion
39
+
-[x]#5 Patterns whose semantics differ between RE2 and rust-regex classify clean-unsupported (guard at bind time from the pinned divergence list) — never a wrong answer
40
+
-[x]#6 Corpus replay: three-outcome contract holds, zero FAILs, match count reported
41
+
-[x]#7 Gate green both backends, clippy clean, serving-bench parity gate passes
41
42
<!-- AC:END -->
43
+
44
+
## Implementation Plan
45
+
46
+
<!-- SECTION:PLAN:BEGIN -->
47
+
Stages (each lands with tests + corpus replay green):
48
+
1. Regex infrastructure: regex = "1" dep (RegexBuilder::octal(true), default Unicode — unicode(false) measured to BREAK (?i) parity); translation module (perl-class rewrite \d->(?-u:\d) etc incl. in-class variants, reject list: \B, (?<>), dup group names, bounds>1000, stacked quantifiers, \u, \Q\E; options parser c/i/l/s live, m/n/p no-ops, g replace-only; replacement translation \N->${N}, $->$$, with the out-of-range->identity and bad-escape global/non-global asymmetry resolved at BIND time); Program-level regex table; IR ops ReMatch/ReExtract/ReReplace + print/parse/verify + both backends.
49
+
2. Functions + operators: regexp_matches (search) / regexp_full_match / regexp_extract (''-on-no-match, flat 0..9 group check) / regexp_replace; ~ / !~ = full match (NOT Postgres search!); SIMILAR TO = raw-pattern full match (no % translation); NULL rules incl. the options-arg asymmetry.
50
+
3. Star forms: * SIMILAR TO (unanchored search) / NOT SIMILAR TO (NOT full match — independent predicates) via rewrite.rs marker codes; COLUMNS('re') interception + declared-order expansion.
51
+
4. Census + bench parity + close-out + PR.
52
+
Spec: docs/superpowers/specs/2026-07-27-waveB-regexp-pins.md (pins committed b003122 before implementation).
53
+
<!-- SECTION:PLAN:END -->
54
+
55
+
## Final Summary
56
+
57
+
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
58
+
Wave B shipped: the regexp family, corpus 484 -> 505 bit-exact of 678 (zero wrong answers, zero replay FAILs). Pins-first: 6-agent fleet (5 DuckDB areas + the RE2-vs-rust-regex DIFFERENTIAL battery) committed as docs/superpowers/specs/2026-07-27-waveB-regexp-pins.md + pins-waveB/*.json before implementation.
59
+
60
+
The engine is the rust `regex` crate (new direct dependency) behind the measured parity recipe: RegexBuilder::octal(true) + default Unicode mode (unicode(false) was measured to BREAK (?i) folding parity — the 'obvious' fix is the wrong one), a bind-time Perl-class rewrite in retrans.rs (\d -> (?-u:\d) etc, in-class variants included) closing the whole RE2-ASCII-vs-rust-Unicode gap, a reject list for irreconcilables (\B unservable in DuckDB itself, (?<name>), duplicate group names, bounds > 1000, stacked quantifiers a*+ — silently WRONG in rust, not just error-shaped different, \u, \Q\E), and replacement-template translation with RE2's invalid-rewrite quirks resolved at bind (out-of-range backref = identity; global bad escape = consume-with-prefix). With that applied all 98 differential battery entries were byte-identical or identically-rejected.
61
+
62
+
Infrastructure: Program grew a prepare-time regex table (print/parse round-trips it; verify checks indices and rewrite presence); ReMatch/ReExtract/ReReplace on both backends (interp closures own their compiled Regex; cranelift helpers read a CompiledRe table through Cx). Semantics per pins: regexp_matches = unanchored SEARCH, regexp_full_match / ~ / !~ / SIMILAR TO = FULL match (~ is NOT the Postgres search — the DuckDB binder error names regexp_full_match; SIMILAR TO translates NO wildcards), regexp_extract '' on no-match with the flat 0..9 group check and NULL-group -> '', regexp_replace backslash backrefs with $-literals and the NULL-options -> NULL asymmetry, options alphabet c/i/l/s live with m/n/p as measured no-ops, constant patterns compile at prepare (pinned eagerness). Star forms: * SIMILAR TO = unanchored name search, * NOT SIMILAR TO = NOT full-match (independent predicates — pinned non-complement asymmetry), bare COLUMNS('re')/COLUMNS(*) expand in declared order with alias stamping into the wave-5 dedup. List-valued forms (extract_all, split_to_array) and COLUMNS-in-expression stay clean-unsupported.
63
+
64
+
Gate: 154 Rust + 557 py green, interp backend identical (548 minus the backend-identity guards), clippy clean on wave files, serving-bench parity gate passed with spec in the usual 1.4-2x band. Remaining pool after the wave: aggregation 45 + table-fns 24 (out of scope), lists/structs 25 (wave C), dynamic self-joins 10, small tails.
0 commit comments