diff --git a/backlog/tasks/task-46 - Specializer-SQL-support-SELECT-star-expansion.md b/backlog/tasks/task-46 - Specializer-SQL-support-SELECT-star-expansion.md index 7bfce87..c074b51 100644 --- a/backlog/tasks/task-46 - Specializer-SQL-support-SELECT-star-expansion.md +++ b/backlog/tasks/task-46 - Specializer-SQL-support-SELECT-star-expansion.md @@ -1,10 +1,10 @@ --- id: TASK-46 title: 'Specializer SQL support: SELECT * star expansion' -status: In Progress +status: Done assignee: [] created_date: '2026-07-26 11:42' -updated_date: '2026-07-26 11:55' +updated_date: '2026-07-26 12:20' labels: [] milestone: m-7 dependencies: @@ -46,3 +46,9 @@ Measured pins: expansion order is FROM order then declared column order per tabl Landed in one pass: Binder::expand_star (frontend.rs) expands `*` and `tbl.*` at the single projection site using the existing SKind::Col / static_lane constructors — no IR/backend/boundary changes, exactly as scoped. EXCLUDE handles bare + paren forms case-insensitively (sqlparser 0.62 carries entries as ObjectName; single-part idents only, qualified entries reject by name); unknown-column and duplicate-entry EXCLUDE mirror DuckDB's binder errors; exclude-all falls through to the existing empty-SELECT bind error (same shape as DuckDB's "SELECT list is empty after resolving * expressions"). Star items covering a joined static table reject cleanly naming the join-key column (DuckDB includes the key AND emits duplicate output names there — both unrepresentable; `__THIS__.*` under a join works). ILIKE/EXCEPT/REPLACE/RENAME/COLUMNS() reject by name. CORPUS RESULT (AC #2): 53 -> 172 match of 678 (+119, a 3.2x coverage jump), 0 FAILs, 506 clean-unsupported. New first-blocker head: BETWEEN 31, comma join 30, dynamic-table alias 30, table-function FROM 24, then the builtin catalogue (array_slice 23, contains 18, damerau_levenshtein 17, ...) — the TASK-47 target list confirmed. Gate green: cargo 129 + pytest 627 (13 xfail incl. the pre-existing substr const-fold pin from master). + +## Final Summary + + +Merged as PR #30 (2026-07-26). Star expansion at the frontend's single projection site tripled corpus coverage: 53 -> 172 match of 678, zero FAILs. All measured edges pinned (order, EXCLUDE case-folding + binder errors, star-over-join rejection naming the join key, macro forms rejected by name). Gate green cargo 129 / pytest 627. Unblocked TASK-47's target list: post-star first-blocker head is BETWEEN 31, comma join 30, dynamic-table alias 30, then the builtin catalogue. + diff --git a/backlog/tasks/task-47 - Specializer-SQL-support-workload-builtins-and-predicates-wave-1.md b/backlog/tasks/task-47 - Specializer-SQL-support-workload-builtins-and-predicates-wave-1.md index a2ec160..bfeb34e 100644 --- a/backlog/tasks/task-47 - Specializer-SQL-support-workload-builtins-and-predicates-wave-1.md +++ b/backlog/tasks/task-47 - Specializer-SQL-support-workload-builtins-and-predicates-wave-1.md @@ -1,9 +1,10 @@ --- id: TASK-47 title: 'Specializer SQL support: workload builtins & predicates wave 1' -status: To Do +status: In Progress assignee: [] created_date: '2026-07-26 11:42' +updated_date: '2026-07-26 12:20' labels: [] milestone: m-7 dependencies: @@ -23,9 +24,31 @@ Close the workload ladder measured by the serving-bench scenarios (benchmarks/se ## Acceptance Criteria -- [ ] #1 Each shipped function/predicate has measured DuckDB pins recorded as duck_check tests before its implementation landed (domain edges, NULL, special floats) -- [ ] #2 Interpreter and cranelift agree byte-identically on all new ops (shared semantic fns; 500-seed differential extended to cover them) -- [ ] #3 The four serving scenarios' compromises lists re-audited: every gap this wave claims to close is exercised by an upgraded scenario feature or a new duck_check -- [ ] #4 Corpus replay: predicate/function first-blocker cases flip to match or to a named deeper blocker; zero FAILs; new tally recorded here -- [ ] #5 mise gate-specializer green +- [x] #1 Each shipped function/predicate has measured DuckDB pins recorded as duck_check tests before its implementation landed (domain edges, NULL, special floats) +- [x] #2 Interpreter and cranelift agree byte-identically on all new ops (shared semantic fns; 500-seed differential extended to cover them) +- [x] #3 The four serving scenarios' compromises lists re-audited: every gap this wave claims to close is exercised by an upgraded scenario feature or a new duck_check +- [x] #4 Corpus replay: predicate/function first-blocker cases flip to match or to a named deeper blocker; zero FAILs; new tally recorded here +- [x] #5 mise gate-specializer green + +## Implementation Plan + + +Stretch plan (recorded 2026-07-26). Measurement BEFORE implementation, per the builtin-pins discipline; a parallel measurement fleet pins each family against DuckDB 1.5.5 first, and lowering decisions are finalized from those pins. +1. Measurement fleet (6 families, parallel): log/ln/log2/log10/exp; floor/ceil/trunc + round(x, digits); pow/sqrt (+ the ^ operator); sin/cos; string search (instr/position/strpos/contains/starts_with/ends_with/length); predicates (BETWEEN incl. NULL bounds and NOT, IN-list incl. NULL-element three-valued logic and NOT IN, least/greatest NULL policy). Each family delivers: measured pin table (edges: 0/negative domains, NULL, NaN, +-0.0, +-inf, int vs float overloads, return types, error messages), draft duck_check tests, and a lowering proposal (frontend desugar vs new IR op with the exact Rust semantics fn). +2. Frontend desugars first (no IR changes where the pins allow): BETWEEN -> >= AND <= under Kleene; IN-list -> OR chain of equalities (three-valued logic makes this exact); NOT variants; least/greatest per measured NULL policy (CASE-chain if NULL-ignoring is expressible, else an IR op). Corpus predicates flip here. +3. New IR ops for the rest: math unaries/binaries and string-search ops, implemented ONCE as shared semantic functions used by both backends (interp closures + cranelift helpers), fuzz generator extended so the 500-seed differential covers every new op; catalogue entries with the pinned edge/trap behavior. +4. Re-audit the four serving scenarios' compromises lists (AC #3): upgrade scenario features the wave unblocks (log1p amount/fare/sales, floor decade bins, instr title extraction, IN-list flags), keeping three-way parity green; corpus re-tally into this ticket (AC #4); gate green (AC #5). + + +## Implementation Notes + + +Progress (2026-07-26, branch claude/specializer-builtins-wave1). Landed, each oracle-pinned before implementation and green on both backends: (1) BETWEEN/IN as exact K3 desugars with whole-construct f64 unification (corpus 172->179; remaining BETWEEN cases moved to NAMED deeper blockers — COUNT aggregation shapes). (2) Fourteen math ops (ln/log/log2/log10/exp/sqrt/cbrt/sin/cos/tan/floor/ceil/trunc/pow + log(b,x)) — shared semantic fns, safe-masked NULL lowering (log family's trap domain includes the type default; Flogb masks under the COMBINED flag since NULL pre-empts every domain check), fuzz-covered incl. trap agreement; the ^ operator stays cleanly unsupported (sqlparser parses it below *, DuckDB binds it above — measured tree divergence), ** does not parse. (3) String search (instr/strpos/position incl. the needle-first SQL form, contains, starts_with/ends_with + prefix/suffix, length/char_length/len, strlen) — 1-based codepoint positions, empty-needle-matches-all, NULL-strict; the corpus REFUTED the fleet's contains-NULL blanket-error pin, re-measured to the real rule (NULL needle binds iff a non-literal Str haystack anchors resolution). (4) least/greatest as CASE+duck-order composition (NULL-ignoring, first-arg ties, NaN-above-inf) — no IR op. CORPUS: 53 (start of day) -> 172 (star) -> 179 (predicates) -> 237 match of 678, zero FAILs, gate green (cargo 129 / pytest 667). COMPLETE: round/trunc-with-digits landed with the oracle-extracted pow10 table (gen_pow10.py, ulp witnesses k=23/k=126 checked at generation; wrapping i64 semantics; round/trunc non-finite asymmetry preserved) — corpus 240/678, zero FAILs. Scenario re-audit done by a parity-gated 4-agent fleet: 47 famous-solution features restored (titanic 24->31, ames 42->54, fraud 41->57, rossmann 44->56 outputs) — ln(1+x) everywhere, decade buckets, Deotte cents, IN-set flags, domain parsing via starts_with/ends_with/instr, sin/cos cyclical encodings, least/greatest clamps; three-way parity green on all four. Final gate: cargo 129 / pytest 671 / corpus 240 match, 0 FAIL. + + +## Final Summary + + +Wave 1 complete on claude/specializer-builtins-wave1. Shipped, all measured-first: BETWEEN/IN/least/greatest as exact compositions (zero IR); 14 math ops + log(b,x) + round/trunc-with-digits (oracle pow10 table) + 6 string-search ops on both backends via shared semantic fns, fuzz-covered incl. trap agreement. Corpus 53 -> 240 match of 678 across the day (star 172, predicates 179, builtins 240), zero wrong answers at every step. Two measured rejections ship as named unsupported: the ^ operator (sqlparser precedence contradicts DuckDB) and ** (does not parse). One fleet pin was refuted by the corpus and re-measured (contains-NULL overload anchoring). The four serving scenarios regained 47 real famous-solution features under the standing three-way parity gate. + diff --git a/benchmarks/bench_serving.py b/benchmarks/bench_serving.py index 180f68e..aab6fc0 100644 --- a/benchmarks/bench_serving.py +++ b/benchmarks/bench_serving.py @@ -22,6 +22,9 @@ any timing; a scenario that disagrees aborts the bench. Run: uv run python -m benchmarks.bench_serving [--json out.json] +IMPORTANT: rebuild the wheel first (uv run --reinstall-package sql-transform +python -c pass) — a stale wheel inflates ONLY the engine rows and once +produced a phantom 7x regression (caught by bisection, 2026-07-26). """ from __future__ import annotations diff --git a/benchmarks/serving_scenarios/fraud_txn.py b/benchmarks/serving_scenarios/fraud_txn.py index ca6bbac..1566f04 100644 --- a/benchmarks/serving_scenarios/fraud_txn.py +++ b/benchmarks/serving_scenarios/fraud_txn.py @@ -3,8 +3,11 @@ Serving shape of the famous IEEE-CIS Fraud Detection winning solutions: uid/frequency/target encodings materialized at prepare time become plain LEFT-JOIN lookup tables at serve time; everything else is scalar math over -the raw transaction row. -""" +the raw transaction row — including the wave-1 builtin features the winning +solutions actually used: log1p amounts via ln(1.0 + x), the Deotte cents +feature via round(x, 2), amount decade via round(x, -1), cyclical hour/dow +sin/cos encodings, email-domain group flags via IN / starts_with / ends_with +/ instr, and C-column outlier clips via least().""" import math import random @@ -18,12 +21,14 @@ "IEEE-CIS Fraud Detection (Kaggle 2019, Deotte/Yakovlev winning-solution " "serving shape): composite uid (card1 x addr1) + card1/addr1/email-domain " "frequency+target encodings as join tables, amt ratio/delta vs card1 and " - "uid means, cents feature, D-column NULL flags, hour/day-of-week from " - "unix ts, amount buckets, ProductCD one-hots." + "uid means, log1p amount, rounded cents feature, amount decade, D-column " + "NULL flags, hour/day-of-week from unix ts + cyclical sin/cos encodings, " + "amount buckets, ProductCD one-hots, email-domain group/suspicious flags, " + "clipped C-columns." ) N_INPUT_COLS = 32 -N_OUTPUT_COLS = 41 +N_OUTPUT_COLS = 57 ROW_SCHEMA: dict[str, str] = { "txn_id": "int", @@ -303,7 +308,28 @@ def make_rows(seed: int, n: int) -> list[dict]: c1 / (c1 + c2 + c3 + c4 + c5 + c6 + 1) AS c1_share, coalesce(dist1, 0.0) AS dist1_filled, CASE WHEN m1 IS NULL THEN -1 WHEN m1 THEN 1 ELSE 0 END AS m1_flag, - CASE WHEN addr1 IS NULL THEN 0 ELSE 1 END AS addr_known + CASE WHEN addr1 IS NULL THEN 0 ELSE 1 END AS addr_known, + ln(1.0 + transaction_amt) AS amt_log1p, + round(transaction_amt, -1) AS amt_decade, + round(transaction_amt - floor(transaction_amt), 2) AS amt_cents_r, + sin(2.0 * pi() * (((transaction_dt % 86400) - (transaction_dt % 3600)) / 3600) / 24.0) AS hour_sin, + cos(2.0 * pi() * (((transaction_dt % 86400) - (transaction_dt % 3600)) / 3600) / 24.0) AS hour_cos, + sin(2.0 * pi() * ((CAST((transaction_dt - (transaction_dt % 86400)) / 86400 AS INTEGER) + 4) % 7) / 7.0) AS dow_sin, + cos(2.0 * pi() * ((CAST((transaction_dt - (transaction_dt % 86400)) / 86400 AS INTEGER) + 4) % 7) / 7.0) AS dow_cos, + CASE WHEN p_email_domain IN ('protonmail.com', 'protonmail.ch', 'anonymous.com', 'mail.com', 'qq.com') + THEN 1 ELSE 0 END AS p_dom_suspicious, + CASE WHEN p_email_domain IS NULL THEN 0 + WHEN starts_with(p_email_domain, 'gmail') THEN 1 ELSE 0 END AS p_is_gmail, + CASE WHEN p_email_domain IN ('hotmail.com', 'outlook.com', 'live.com', 'msn.com') + THEN 1 ELSE 0 END AS p_is_msft, + CASE WHEN p_email_domain IS NULL THEN 0 + WHEN ends_with(p_email_domain, '.net') THEN 1 ELSE 0 END AS p_dom_net, + CASE WHEN p_email_domain IS NULL THEN -1 + ELSE length(p_email_domain) - instr(p_email_domain, '.') END AS p_dom_suffix_len, + least(c1, 50) AS c1_clip, + least(c2, 40) AS c2_clip, + greatest(coalesce(d1, 0), coalesce(d2, 0), coalesce(d3, 0), coalesce(d4, 0)) AS d_max, + ln(1.0 + card1_stats.card1_txn_cnt) AS card1_cnt_log FROM __THIS__ LEFT JOIN card1_stats ON card1 = card1_stats.card1_id LEFT JOIN addr1_stats ON addr1 = addr1_stats.addr1_id @@ -314,6 +340,20 @@ def make_rows(seed: int, n: int) -> list[dict]: """ +def _round_half_away(x: float, n: int) -> float: + """SQL round(x, n): scale by 10**n, round half away from zero, unscale. + Bit-identical to the engine/DuckDB for this scenario's inputs (probed).""" + s = 10.0**n + y = x * s + return math.copysign(math.floor(abs(y) + 0.5), y) / s + + +_SUSPICIOUS_DOMAINS = frozenset( + {"protonmail.com", "protonmail.ch", "anonymous.com", "mail.com", "qq.com"} +) +_MSFT_DOMAINS = frozenset({"hotmail.com", "outlook.com", "live.com", "msn.com"}) + + def handcrafted(statics: dict[str, pa.Table]) -> Callable[[dict], dict]: """What a competent engineer hand-writes for the same features: hydrate the encoding tables into plain dicts once, then a per-row closure.""" @@ -342,6 +382,8 @@ def handcrafted(statics: dict[str, pa.Table]) -> Callable[[dict], dict]: def fn(row: dict) -> dict: amt = row["transaction_amt"] ts = row["transaction_dt"] + hour = (ts % 86400) // 3600 + dow = (ts // 86400 + 4) % 7 c1s = card1_map.get(row["card1"]) card1_amt_mean, card1_txn_cnt, card1_fraud_rate = ( @@ -405,8 +447,8 @@ def fn(row: dict) -> dict: "amt_x_pcd_rate": amt * pcd_fraud_rate if pcd_fraud_rate is not None else None, - "txn_hour": (ts % 86400) // 3600, - "txn_dow": (ts // 86400 + 4) % 7, + "txn_hour": hour, + "txn_dow": dow, "is_night": 1 if ts % 86400 < 21600 else 0, "amt_bucket": amt_bucket, "amt_cents": amt_cents, @@ -429,6 +471,30 @@ def fn(row: dict) -> dict: "dist1_filled": row["dist1"] if row["dist1"] is not None else 0.0, "m1_flag": -1 if m1 is None else (1 if m1 else 0), "addr_known": 0 if row["addr1"] is None else 1, + "amt_log1p": math.log(1.0 + amt), + "amt_decade": _round_half_away(amt, -1), + "amt_cents_r": _round_half_away(amt - math.floor(amt), 2), + "hour_sin": math.sin(2.0 * math.pi * hour / 24.0), + "hour_cos": math.cos(2.0 * math.pi * hour / 24.0), + "dow_sin": math.sin(2.0 * math.pi * dow / 7.0), + "dow_cos": math.cos(2.0 * math.pi * dow / 7.0), + # NULL IN (...) is NULL -> CASE falls through to 0, same as .get miss + "p_dom_suspicious": 1 if p_dom in _SUSPICIOUS_DOMAINS else 0, + "p_is_gmail": 1 if p_dom is not None and p_dom.startswith("gmail") else 0, + "p_is_msft": 1 if p_dom in _MSFT_DOMAINS else 0, + "p_dom_net": 1 if p_dom is not None and p_dom.endswith(".net") else 0, + # instr is 1-based codepoints; domains are ASCII with one dot + "p_dom_suffix_len": -1 + if p_dom is None + else len(p_dom) - (p_dom.index(".") + 1), + "c1_clip": min(row["c1"], 50), + "c2_clip": min(row["c2"], 40), + "d_max": max( + row["d1"] or 0, row["d2"] or 0, row["d3"] or 0, row["d4"] or 0 + ), + "card1_cnt_log": math.log(1.0 + card1_txn_cnt) + if card1_txn_cnt is not None + else None, } return fn diff --git a/benchmarks/serving_scenarios/house_prices.py b/benchmarks/serving_scenarios/house_prices.py index faaeb6b..beb562f 100644 --- a/benchmarks/serving_scenarios/house_prices.py +++ b/benchmarks/serving_scenarios/house_prices.py @@ -3,9 +3,17 @@ The canonical Kaggle House Prices feature set, expressed as the serve-time query a fitted pipeline reduces to: scalar expressions over the row plus LEFT JOINs to prepare-time encoding tables. This is the wide-arith stress -case: 43 input columns, 42 output features, almost all of them arithmetic +case: 43 input columns, 54 output features, almost all of them arithmetic and CASE over the row. +Originally authored before the wave-1 math/string builtins landed, so the +public-kernel log1p skew fixes, sqrt, decade bins, clamps, IN-set flags and +cyclical month encoding were dropped. They are back now that the engine +supports ln/sqrt/floor/least/greatest/IN/sin/cos/pi: log1p(TotalSF/LotArea/ +GrLivArea) as ln(1.0 + x), sqrt(GrLivArea), YearBuilt decade via +floor(x/10)*10, garage-age clamp via greatest, Ex/Gd quality IN-sets, the +rich-neighborhood and PUD MSSubClass membership flags, and MoSold sin/cos. + Realistic serving gotchas baked into the row distributions: - MSSubClass 150 and the GrnHill/Landmrk neighborhoods exist only outside the Kaggle train set -> LEFT JOIN misses at serve time (COALESCE to the @@ -17,6 +25,7 @@ per-neighborhood median via the join table (the classic trick). """ +import math import random from collections.abc import Callable @@ -28,12 +37,13 @@ "public-kernel feature set: TotalSF/TotalBath sums, age features, " "Ex..Po ordinal quality maps, porch total, has_* flags, nullif-guarded " "lot ratios, qual*cond crosses, neighborhood-median LotFrontage " - "imputation, and Neighborhood/MSSubClass target+frequency encodings " - "served as join tables." + "imputation, Neighborhood/MSSubClass target+frequency encodings " + "served as join tables, log1p skew fixes, decade bins, quality/" + "neighborhood/PUD membership flags, and cyclical MoSold encoding." ) N_INPUT_COLS = 43 -N_OUTPUT_COLS = 42 +N_OUTPUT_COLS = 54 ROW_SCHEMA = { "id": "int", @@ -346,7 +356,21 @@ def _ord_case(col: str) -> str: WHEN mo_sold >= 6 AND mo_sold <= 8 THEN 2 WHEN mo_sold >= 9 AND mo_sold <= 11 THEN 3 ELSE 0 END AS season_sold, - CASE WHEN mo_sold >= 5 AND mo_sold <= 7 THEN 1 ELSE 0 END AS is_peak_season + CASE WHEN mo_sold >= 5 AND mo_sold <= 7 THEN 1 ELSE 0 END AS is_peak_season, + ln(1.0 + (coalesce(total_bsmt_sf, 0.0) + first_flr_sf + second_flr_sf)) + AS log_total_sf, + ln(1.0 + lot_area) AS log_lot_area, + ln(1.0 + gr_liv_area) AS log_gr_liv_area, + sqrt(CAST(gr_liv_area AS DOUBLE)) AS sqrt_gr_liv_area, + floor(year_built / 10.0) * 10.0 AS decade_built, + greatest(yr_sold - garage_yr_blt, 0.0) AS garage_age_clamped, + CASE WHEN exter_qual IN ('Ex', 'Gd') THEN 1 ELSE 0 END AS exter_qual_good, + CASE WHEN kitchen_qual IN ('Ex', 'Gd') THEN 1 ELSE 0 END AS kitchen_qual_good, + CASE WHEN neighborhood IN ('NoRidge', 'NridgHt', 'StoneBr') THEN 1 ELSE 0 END + AS nbhd_rich, + CASE WHEN ms_sub_class IN (120, 150, 160, 180) THEN 1 ELSE 0 END AS is_pud, + sin(2.0 * pi() * mo_sold / 12.0) AS mo_sold_sin, + cos(2.0 * pi() * mo_sold / 12.0) AS mo_sold_cos FROM __THIS__ LEFT JOIN nbhd_price_enc ON neighborhood = nbhd_price_enc.nbhd LEFT JOIN subclass_enc ON ms_sub_class = subclass_enc.sub_class""" @@ -399,11 +423,10 @@ def fe(r: dict) -> dict: lff = nbe[2] if nbe is not None else None if lff is None: lff = 69.0 + tsf = (tb if tb is not None else 0.0) + r["first_flr_sf"] + r["second_flr_sf"] return { "id": r["id"], - "total_sf": (tb if tb is not None else 0.0) - + r["first_flr_sf"] - + r["second_flr_sf"], + "total_sf": tsf, "total_bath": r["full_bath"] + 0.5 * r["half_bath"] + (bf if bf is not None else 0) @@ -465,6 +488,21 @@ def fe(r: dict) -> dict: if 9 <= r["mo_sold"] <= 11 else 0, "is_peak_season": 1 if 5 <= r["mo_sold"] <= 7 else 0, + "log_total_sf": math.log(1.0 + tsf), + "log_lot_area": math.log(1.0 + la), + "log_gr_liv_area": math.log(1.0 + gla), + "sqrt_gr_liv_area": math.sqrt(gla), + "decade_built": math.floor(r["year_built"] / 10.0) * 10.0, + # greatest() is NULL-ignoring: greatest(NULL, 0.0) -> 0.0 + "garage_age_clamped": 0.0 if gy is None else max(r["yr_sold"] - gy, 0.0), + "exter_qual_good": 1 if r["exter_qual"] in ("Ex", "Gd") else 0, + "kitchen_qual_good": 1 if kq in ("Ex", "Gd") else 0, + "nbhd_rich": 1 + if r["neighborhood"] in ("NoRidge", "NridgHt", "StoneBr") + else 0, + "is_pud": 1 if r["ms_sub_class"] in (120, 150, 160, 180) else 0, + "mo_sold_sin": math.sin(2.0 * math.pi * r["mo_sold"] / 12.0), + "mo_sold_cos": math.cos(2.0 * math.pi * r["mo_sold"] / 12.0), } return fe diff --git a/benchmarks/serving_scenarios/store_sales.py b/benchmarks/serving_scenarios/store_sales.py index 493dfa4..58f0307 100644 --- a/benchmarks/serving_scenarios/store_sales.py +++ b/benchmarks/serving_scenarios/store_sales.py @@ -8,10 +8,18 @@ encodings (what a fitted mean encoding IS at serve time), with partial coverage and literal global priors as fallback; * day-of-week / month / region seasonal-factor encodings; - * competition-open-months and promo2-active-weeks arithmetic with - NULL-guarded clamping, promo interactions, day-of-week one-hots, - state-holiday CASEs, a store_type x promo cross, ratios vs store means, - and Walmart-style econ covariates (fuel/CPI/unemployment/markdowns). + * competition-open-months and promo2-active-weeks arithmetic clamped with + NULL-ignoring greatest(0, ...), promo interactions, day-of-week one-hots, + state-holiday IN-set CASEs, a store_type x promo cross, ratios vs store + means, and Walmart-style econ covariates (fuel/CPI/unemployment/markdowns); + * the log/cyclical features the famous solutions actually ship (previously + compromised away when the SQL surface lacked math builtins, now real): + ln(1+CompetitionDistance) and ln(1+markdown)/ln(1+mean_sales) log1p + transforms, competition-age-in-years via floor(), Rossmann week-of-month + via trunc((day-1)/7), sin/cos week-of-year + day-of-week cyclical + seasonality augmenting the seasonal-factor joins, the canonical + Jan/Apr/Jul/Oct PromoInterval renewal-month flag and a Nov/Dec peak-season + flag via IN lists, and a promo2-weeks saturation cap via least(). """ from __future__ import annotations @@ -27,11 +35,12 @@ "Rossmann Store Sales (+ Walmart Recruiting) serving path: store.csv dim join, " "competition-open-months / promo2-weeks arithmetic, per-store mean-target encodings " "with global priors, dow/month seasonal factors, store_type x promo cross, " - "holiday CASEs, Walmart econ covariates." + "holiday CASEs, Walmart econ covariates, log1p distance/markdown/sales " + "transforms, week-of-month, sin/cos cyclical seasonality, PromoInterval flag." ) N_INPUT_COLS = 21 -N_OUTPUT_COLS = 44 +N_OUTPUT_COLS = 56 ROW_SCHEMA: dict[str, str] = { "record_id": "int", # the Rossmann test.csv `Id`: unique serving-row key @@ -91,7 +100,8 @@ CASE WHEN __THIS__.state_holiday = 'a' THEN 1 ELSE 0 END AS state_hol_public, CASE WHEN __THIS__.state_holiday = 'b' THEN 1 ELSE 0 END AS state_hol_easter, CASE WHEN __THIS__.state_holiday = 'c' THEN 1 ELSE 0 END AS state_hol_christmas, - CASE WHEN __THIS__.state_holiday = '0' THEN 0 ELSE 1 END AS any_state_holiday, + CASE WHEN __THIS__.state_holiday IN ('a', 'b', 'c') THEN 1 ELSE 0 END + AS any_state_holiday, CASE WHEN __THIS__.state_holiday <> '0' AND __THIS__.school_holiday THEN 1 ELSE 0 END AS holiday_x_school, CASE WHEN __THIS__.day_of_week = 1 THEN 1 ELSE 0 END AS dow_mon, @@ -124,13 +134,9 @@ COALESCE(store_dim.competition_distance, {COMP_DIST_FILL}) AS comp_distance, 1.0 / (1.0 + COALESCE(store_dim.competition_distance, {COMP_DIST_FILL})) AS comp_distance_inv, - CASE WHEN {_COMP_MONTHS} IS NULL THEN 0 - WHEN {_COMP_MONTHS} < 0 THEN 0 - ELSE {_COMP_MONTHS} END AS comp_open_months, + greatest(0, {_COMP_MONTHS}) AS comp_open_months, CASE WHEN NOT __THIS__.promo2 THEN 0 - WHEN {_PROMO2_WEEKS} IS NULL THEN 0 - WHEN {_PROMO2_WEEKS} < 0 THEN 0 - ELSE {_PROMO2_WEEKS} END AS promo2_active_weeks, + ELSE greatest(0, {_PROMO2_WEEKS}) END AS promo2_active_weeks, COALESCE(store_stats.mean_sales, {PRIOR_MEAN_SALES}) AS store_mean_sales, COALESCE(store_stats.mean_customers, {PRIOR_MEAN_CUSTOMERS}) AS store_mean_customers, @@ -161,7 +167,23 @@ AS days_since_promo_filled, CASE WHEN __THIS__.promo AND COALESCE(__THIS__.days_since_prev_promo, {DAYS_SINCE_PROMO_FILL}) < 7 - THEN 1 ELSE 0 END AS promo_fatigue + THEN 1 ELSE 0 END AS promo_fatigue, + ln(1.0 + COALESCE(store_dim.competition_distance, {COMP_DIST_FILL})) + AS comp_distance_log, + floor(greatest(0, {_COMP_MONTHS}) / 12.0) AS comp_open_years, + trunc((__THIS__.day_of_month - 1) / 7.0) + 1.0 AS week_of_month, + sin(2.0 * pi() * __THIS__.week_of_year / 52.0) AS woy_sin, + cos(2.0 * pi() * __THIS__.week_of_year / 52.0) AS woy_cos, + sin(2.0 * pi() * __THIS__.day_of_week / 7.0) AS dow_sin, + cos(2.0 * pi() * __THIS__.day_of_week / 7.0) AS dow_cos, + CASE WHEN __THIS__.promo2 AND __THIS__.month IN (1, 4, 7, 10) + THEN 1 ELSE 0 END AS promo2_interval_month, + CASE WHEN __THIS__.month IN (11, 12) THEN 1 ELSE 0 END AS peak_season, + ln(1.0 + COALESCE(__THIS__.markdown_total, 0.0)) AS markdown_log, + ln(1.0 + COALESCE(store_stats.mean_sales, {PRIOR_MEAN_SALES})) + AS store_mean_sales_log, + least(CASE WHEN NOT __THIS__.promo2 THEN 0 + ELSE greatest(0, {_PROMO2_WEEKS}) END, 156) AS promo2_weeks_capped FROM __THIS__ LEFT JOIN store_dim ON __THIS__.store_id = store_dim.store_id LEFT JOIN store_stats ON __THIS__.store_id = store_stats.store_id @@ -377,21 +399,25 @@ def fn(row: dict) -> dict: sort = d["assortment"] if d is not None else None comp_dist = d["competition_distance"] if d is not None else COMP_DIST_FILL + # greatest(0, x) with NULL-ignoring greatest: join-miss NULL months + # collapse to the 0 floor, negatives clamp to 0. if d is None: comp_open_months = 0 else: - cm = (row["year"] - d["competition_open_since_year"]) * 12 + ( - row["month"] - d["competition_open_since_month"] + comp_open_months = max( + 0, + (row["year"] - d["competition_open_since_year"]) * 12 + + (row["month"] - d["competition_open_since_month"]), ) - comp_open_months = 0 if cm < 0 else cm if not row["promo2"] or d is None: promo2_active_weeks = 0 else: - pw = (row["year"] - d["promo2_since_year"]) * 52 + ( - row["week_of_year"] - d["promo2_since_week"] + promo2_active_weeks = max( + 0, + (row["year"] - d["promo2_since_year"]) * 52 + + (row["week_of_year"] - d["promo2_since_week"]), ) - promo2_active_weeks = 0 if pw < 0 else pw mean_sales = s["mean_sales"] if s is not None else PRIOR_MEAN_SALES mean_customers = s["mean_customers"] if s is not None else PRIOR_MEAN_CUSTOMERS @@ -417,7 +443,7 @@ def fn(row: dict) -> dict: "state_hol_public": 1 if hol == "a" else 0, "state_hol_easter": 1 if hol == "b" else 0, "state_hol_christmas": 1 if hol == "c" else 0, - "any_state_holiday": 0 if hol == "0" else 1, + "any_state_holiday": 1 if hol in ("a", "b", "c") else 0, "holiday_x_school": 1 if hol != "0" and row["school_holiday"] else 0, "dow_mon": 1 if dow == 1 else 0, "dow_tue": 1 if dow == 2 else 0, @@ -491,6 +517,30 @@ def fn(row: dict) -> dict: * (unemp if unemp is not None else PRIOR_UNEMP), "days_since_promo_filled": ds_filled, "promo_fatigue": 1 if promo and ds_filled < 7 else 0, + # -- wave-1 builtin features (the famous-solution versions) -- + # ln(1 + CompetitionDistance): the classic Rossmann skew fix + "comp_distance_log": math.log(1.0 + comp_dist), + # competition age bucketed to whole years; floor(double) is a + # double in SQL, so the twin must produce a float too + "comp_open_years": float(math.floor(comp_open_months / 12.0)), + # trunc((day-1)/7) + 1: Rossmann week-of-month, kept as the + # double the SQL trunc() yields (1.0 .. 4.0) + "week_of_month": math.trunc((row["day_of_month"] - 1) / 7.0) + 1.0, + # cyclical year/week encodings; evaluation order mirrors the SQL + # left-to-right ((2.0 * pi) * k) / period exactly + "woy_sin": math.sin(2.0 * math.pi * row["week_of_year"] / 52.0), + "woy_cos": math.cos(2.0 * math.pi * row["week_of_year"] / 52.0), + "dow_sin": math.sin(2.0 * math.pi * dow / 7.0), + "dow_cos": math.cos(2.0 * math.pi * dow / 7.0), + # canonical 'Jan,Apr,Jul,Oct' PromoInterval renewal-month flag + "promo2_interval_month": ( + 1 if row["promo2"] and row["month"] in (1, 4, 7, 10) else 0 + ), + "peak_season": 1 if row["month"] in (11, 12) else 0, + "markdown_log": math.log(1.0 + md_filled), + "store_mean_sales_log": math.log(1.0 + mean_sales), + # least(x, 156): promo2 effect saturates after ~3 years + "promo2_weeks_capped": min(promo2_active_weeks, 156), } return fn diff --git a/benchmarks/serving_scenarios/titanic.py b/benchmarks/serving_scenarios/titanic.py index 588d62d..bc4ffef 100644 --- a/benchmarks/serving_scenarios/titanic.py +++ b/benchmarks/serving_scenarios/titanic.py @@ -3,6 +3,11 @@ Reproduces the canonical public-kernel feature pipeline at serve time: scalar expressions over the passenger row + LEFT JOINs to fitted encoding tables (a target-mean encoding IS a join table at serve time). + +Wave-1 builtins version: the log1p fare skew transform, round(fare, -1) +decade bucket, Sehgal ordinal age code (floor + least), fare winsorizing +(greatest/least), rare-title IN flag, upper-deck IN flag and the +contains()-based multi-cabin flag are now real SQL — no compromises left. """ from __future__ import annotations @@ -19,11 +24,13 @@ "FamilySize/IsAlone, FarePerPerson, deck letter from Cabin + HasCabin, " "Age-imputation flag + Age*Pclass interaction, age bins, Embarked one-hots, " "sex x pclass survival target-mean encoding and title-group mean-fare/survival " - "encodings served as fitted join tables (incl. the famous unseen-'Dona' miss)." + "encodings served as fitted join tables (incl. the famous unseen-'Dona' miss); " + "plus log1p(Fare), Fare decade + IQR winsorizing, the Sehgal 16-year ordinal " + "age code, rare-title / upper-deck membership flags and the multi-cabin flag." ) N_INPUT_COLS = 10 -N_OUTPUT_COLS = 24 +N_OUTPUT_COLS = 31 ROW_SCHEMA = { "passenger_id": "int", @@ -280,7 +287,28 @@ def make_rows(seed: int, n: int) -> list[dict]: coalesce(td.group_survival_rate, 0.383838) AS title_rate, td.group_mean_fare AS title_fare_mean, coalesce(__THIS__.fare, 14.4542) - td.group_mean_fare AS fare_minus_title_mean, - coalesce(em.survival_rate, 0.383838) AS embarked_rate + coalesce(em.survival_rate, 0.383838) AS embarked_rate, + ln(1.0 + coalesce(__THIS__.fare, 14.4542)) AS fare_log1p, + round(coalesce(__THIS__.fare, 14.4542), -1) AS fare_decade, + least(greatest(coalesce(__THIS__.fare, 14.4542), 7.91), 31.0) AS fare_winsor, + least(floor(coalesce(__THIS__.age, 28.0) / 16.0), 4.0) AS age_code, + CASE + WHEN __THIS__.title IN ('Dr', 'Rev', 'Col', 'Major', 'Capt', + 'Sir', 'Lady', 'Don', 'Countess', 'Jonkheer') + THEN 1 ELSE 0 + END AS title_is_rare, + CASE + WHEN (CASE + WHEN __THIS__.cabin IS NULL THEN 'U' + ELSE upper(substr(trim(__THIS__.cabin), 1, 1)) + END) IN ('A', 'B', 'C') + THEN 1 ELSE 0 + END AS is_upper_deck, + CASE + WHEN __THIS__.cabin IS NULL THEN 0 + WHEN contains(trim(__THIS__.cabin), ' ') THEN 1 + ELSE 0 + END AS multi_cabin FROM __THIS__ LEFT JOIN sex_pclass_enc AS sp ON __THIS__.sex = sp.sex AND __THIS__.pclass = sp.pclass @@ -289,6 +317,22 @@ def make_rows(seed: int, n: int) -> list[dict]: """ +# The rare-title training set (title_dim's 'Rare' group). Unseen 'Dona' is +# NOT here — the flag misses it exactly as a fitted membership list would, +# while title_group still falls back to 'Rare' via the join-miss coalesce. +_RARE_TITLES = frozenset(t for t, g in _TITLE_TO_GROUP.items() if g == "Rare") + + +def _round_neg1(x: float) -> float: + """DuckDB round(x, -1) for finite x: (x / 10.0) rounded half-away + from zero, times 10.0 — bit-identical to the engine's pow-table path.""" + m = x / 10.0 + f = math.floor(m) + d = m - f + r = f + 1.0 if d > 0.5 or (d == 0.5 and m > 0.0) else float(f) + return r * 10.0 + + def handcrafted(statics: dict[str, pa.Table]) -> Callable[[dict], dict]: """What a competent engineer hand-writes for a Python microservice: plain-dict lookups prepared once, a per-row closure after that.""" @@ -315,6 +359,7 @@ def infer(row: dict) -> dict: fare_filled = fare if fare is not None else 14.4542 age_filled = age if age is not None else 28.0 t = td.get(row["title"]) + deck = "U" if cabin is None else cabin.strip(" ")[:1].upper() if age is None: age_bin = "unknown" @@ -344,7 +389,7 @@ def infer(row: dict) -> dict: "age_x_pclass": age_filled * pclass, "age_bin": age_bin, "has_cabin": 0 if cabin is None else 1, - "deck": "U" if cabin is None else cabin.strip(" ")[:1].upper(), + "deck": deck, "embarked_s": 1 if emb == "S" else 0, "embarked_c": 1 if emb == "C" else 0, "embarked_q": 1 if emb == "Q" else 0, @@ -356,6 +401,18 @@ def infer(row: dict) -> dict: fare_filled - t["group_mean_fare"] if t is not None else None ), "embarked_rate": em.get(emb, 0.383838), + # log1p fare skew transform (fare >= 0, so no <= -1 guard). + "fare_log1p": math.log(1.0 + fare_filled), + "fare_decade": _round_neg1(fare_filled), + # Winsorize fare to the famous qcut quartile edges [7.91, 31.0]. + "fare_winsor": min(max(fare_filled, 7.91), 31.0), + # Sehgal ordinal age code: floor(age/16) clamped to 4. + "age_code": min(float(math.floor(age_filled / 16.0)), 4.0), + "title_is_rare": 1 if row["title"] in _RARE_TITLES else 0, + "is_upper_deck": 1 if deck in ("A", "B", "C") else 0, + "multi_cabin": ( + 0 if cabin is None else (1 if " " in cabin.strip(" ") else 0) + ), } return infer diff --git a/docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md b/docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md new file mode 100644 index 0000000..814c58b --- /dev/null +++ b/docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md @@ -0,0 +1,114 @@ +# Wave-1 builtin & predicate pins — DuckDB 1.5.5, measured 2026-07-26 + +The implementation contract for TASK-47. Every claim below was MEASURED through +the vectorized path (table columns, not literals) by a six-family fleet; the +full pin tables (300+ rows incl. exact reprs, result types, and error heads) +are the JSON files in `pins-wave1/`. Where the constant-fold path diverges it +is noted. Nothing here is inferred from documentation. + +## Decisions (implementation-shaping) + +### Frontend desugars — NO new IR (proven exact against the pins) +- `x BETWEEN lo AND hi` → `(x >= lo) AND (x <= hi)`; `NOT BETWEEN` → `NOT(...)`. + K3 (Kleene) AND over `duck_fcmp` reproduces every pinned row, including + `(5, NULL, 4) → FALSE` (NULL AND FALSE) and NaN-above-inf ordering. No + empty-range special case — and none may be added (short-circuiting the + NULL/FALSE interplay diverges). +- `x IN (a, b, …)` → K3 OR-chain of equalities; `NOT IN` → `NOT(...)`. + Truth table pinned: `1 IN (1,NULL)`=TRUE, `3 IN (1,NULL)`=NULL. Type + unification is WHOLE-EXPRESSION (one common type across x and every + element; a single DOUBLE drags the list to DOUBLE) — unify int→f64 like + our cmp coercion; VARCHAR↔numeric mixing has exec-time cast semantics we + do not model → clean-unsupported. `x IN ()` is a parse error upstream. +- `power(x,y)`, `x ^ y`, `x ** y` → one `pow` op. DuckDB `^` IS pow (not + xor); unary minus binds TIGHTER (`-2^2 = +4.0`); `^`/`**` are + LEFT-associative (`2^3^2 = 64`). Verify sqlparser reproduces this + precedence/associativity — if not, fix at the frontend. +- `pi()` → fold to the f64 literal 0x400921FB54442D18. +- Aliases → one op each: instr = strpos = 2-arg "position" (haystack, + needle); the SQL form `position(n IN s)` is needle-first; prefix = + starts_with, suffix = ends_with; len = char_length = character_length = + length; `^@` operator = starts_with. +- `least/greatest(n-ary)` → left fold of BINARY Least/Greatest IR ops + (first-arg-wins on ties makes the left fold exact). + +### New IR ops +Math unary (f64→f64, int inputs cast to f64 first; VARCHAR/BOOLEAN columns +are binder errors — no implicit cast): +- `Ln, Log2, Log10`: TRAP on x ≤ 0 — "cannot take logarithm of zero" / + "cannot take logarithm of a negative number" (exact heads in pins); + -0.0 hits the ZERO check first. NaN (either sign) passes THROUGH to + libm-NaN (comparison-based guard, not is_finite). NULL pre-empts every + domain check. Otherwise bit-exact platform libm. +- `Exp`: TOTAL — never errors. exp(1000)=inf, exp(-inf)=+0.0 (positive + zero), underflow passes through denormals (exp(-745)=5e-324). +- `Sqrt`: TRAP on any negative incl. -inf ("cannot take square root of a + negative number"); sqrt(-0.0) = -0.0 (not a trap — IEEE). +- `Cbrt`: TOTAL; cbrt(-8) = -2.0 exactly (NOT pow(x,1/3) which is NaN). + CI-discovered addendum: DuckDB's own wheels disagree with each other on + cbrt by one ulp across platforms (Windows wheel == Rust/ucrt bit-exact; + Linux wheel's bundled std::cbrt returns e.g. 3.0000000000000004 for + cbrt(27)). The engine stays deterministic (Rust cbrt); oracle parity for + cbrt is pinned to <= 1 ulp, not repr-exact — the only such exception. +- `SinF64/CosF64/TanF64`: TRAP on ±inf ("Out of Range Error: input value + inf is out of range for numeric function"); NaN passes through + BIT-EXACTLY (payload+sign preserved — check is_nan() BEFORE calling + libm); finite inputs bit-match platform libm incl. 1e300 (full argument + reduction; 0/10000 fuzz mismatches). tan(pi()/2) is finite 1.633…e16. +- `FloorF64/CeilF64/TruncF64/RoundF64`: Rust `f64::floor/ceil/trunc/round` + are bit-exact (round is half-AWAY-from-zero, incl. the + 0.49999999999999994 → 0.0 exactness — no +0.5 tricks). floor/ceil have + NO integer overloads (int → lossy f64); single-arg trunc/round on + integers are identity. +Math binary: +- `LogBase(b, x)`: EXACTLY log10(x)/log10(b) — NOT ln-ratio (refuted + 6989/20000) or log2-ratio. Base domain-checked BEFORE x; base==1 has a + dedicated error (DuckDB's typo verbatim in pins); sign-of-zero leaks + from the division (log(0.5,1.0) = -0.0) and must be preserved. +- `Pow(x, y)`: TOTAL, pure IEEE: pow(NaN,0)=1, pow(1,NaN)=1, pow(0,-1)=inf, + negative^fractional=NaN, overflow=inf. Result always DOUBLE (int operands + cast first — no integer pow, no overflow trap). +- `Least/Greatest` (binary): NULL-IGNORING (result NULL only if BOTH are); + ties return the FIRST argument (pinned via -0.0 vs 0.0); NaN sorts above + +inf under duck order (greatest(x, NaN) = NaN). +- `Round2/Trunc2 (x, n)` — the nasty one, own ops: + - round(x, n<0): NaN/±inf → +0.0 (!); n ≤ -309 → 0.0 for every finite x. + - trunc(x, n<0) non-finite fallback returns the INPUT (differs from round). + - Integer round(int, n<0) WRAPS at i64 (round(i64::MAX,-2) = + -9223372036854775700); |n| ≥ 19 collapses to 0. Never errors. + - The f64 scale factor is DuckDB's own std::pow(10,n) which is NOT + correctly rounded (1-ulp off strtod at n=23) — extract an + oracle-pinned pow10 table (n ∈ [-323, 308]) with a generator script + rather than calling any libm (same playbook as the casemap table). + - round(DOUBLE, BIGINT-column) does not bind in DuckDB — digits must be + INTEGER; constant integer digits (the corpus shape) bind fine. +String search (all TOTAL, all NULL-strict in every argument — validity = +AND of input validities, no three-valued special cases): +- `StrFind` (instr/strpos/position): 1-based CODEPOINT index (not bytes, + not graphemes), 0 = not found, empty needle → 1 (matches even in ''). +- `SContains`, `SStartsWith`, `SEndsWith`: empty needle → TRUE. Byte-wise + `str` ops reproduce every pin (no normalization, no case folding). +- `SLenChars` (length: codepoints), `SLenBytes` (strlen: UTF-8 bytes). + bit_length = 8×strlen (desugar if wanted; not in wave scope). + +### Catalogue rejections (do NOT ship) +log1p/expm1 do not exist in DuckDB 1.5.5 (catalog error — scenarios use +ln(1+x) instead); `truncate` does not exist; contains(numeric,…) etc. have +no implicit numeric→VARCHAR casts (binder errors); `'2^-2'` is a bind-time +error in DuckDB (lexer eats `^-`) — mirror as clean unsupported if sqlparser +differs. + +### Cross-cutting, engine-shaping +- Trap laziness is already correct by construction: WHERE filters and + untaken CASE branches genuinely prevent evaluation in DuckDB (pinned), + and our lowering evaluates the projection only after the filter branch; + a domain-violating row that IS evaluated aborts the whole call — also + matching DuckDB. +- Bare SQL float literals are DECIMAL (no signed zero; exact comparisons + against BIGINT) — pins used `'…'::DOUBLE`/columns to dodge it; our + frontend maps decimal literals to f64 (known, documented v0 divergence). +- Literal-vs-vectorized: zero divergences in string-search, trig, log, + pow families; DECIMAL-literal round differs by design (we follow the + DOUBLE column path, as with substr). +- Repo doc drift found during verification: interp.rs header still says + fcmp is IEEE-with-NaN-false; it is duck_fcmp. Fix in this wave. diff --git a/docs/superpowers/specs/pins-wave1/pins_floor-ceil-trunc-round.json b/docs/superpowers/specs/pins-wave1/pins_floor-ceil-trunc-round.json new file mode 100644 index 0000000..798a791 --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_floor-ceil-trunc-round.json @@ -0,0 +1,336 @@ +{ + "family": "floor-ceil-trunc-round", + "pins": [ + { + "expr": "floor(x)", + "input_desc": "DOUBLE column 2.5", + "result": "2.0", + "result_type": "DOUBLE", + "note": "IEEE floor; Rust f64::floor is bit-exact (0 mismatches in 9028-case fuzz incl. specials)" + }, + { + "expr": "ceil(x) / ceiling(x)", + "input_desc": "DOUBLE column 2.5", + "result": "3.0", + "result_type": "DOUBLE", + "note": "ceiling is a pure alias of ceil; IEEE ceil, f64::ceil bit-exact" + }, + { + "expr": "trunc(x)", + "input_desc": "DOUBLE column 2.5 / -2.999", + "result": "2.0 / -2.0", + "result_type": "DOUBLE", + "note": "toward zero; f64::trunc bit-exact" + }, + { + "expr": "round(x)", + "input_desc": "DOUBLE column 2.5, 3.5, -2.5, -3.5", + "result": "3.0, 4.0, -3.0, -4.0", + "result_type": "DOUBLE", + "note": "half AWAY from zero, NOT banker's; identical to Rust f64::round at every magnitude" + }, + { + "expr": "round(x)", + "input_desc": "DOUBLE column 0.49999999999999994 (largest double < 0.5)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "proves NO naive x+0.5-then-floor (that would give 1.0); exact std::round" + }, + { + "expr": "round(x)", + "input_desc": "DOUBLE column 4503599627370495.5 (largest double with .5 fraction, 2^52-0.5)", + "result": "4503599627370496.0", + "result_type": "DOUBLE", + "note": "negative twin gives -4503599627370496.0" + }, + { + "expr": "floor(x), ceil(x)", + "input_desc": "BIGINT column 7", + "result": "7.0", + "result_type": "DOUBLE", + "note": "floor/ceil have NO integer overloads: every int width is implicitly cast to DOUBLE" + }, + { + "expr": "floor(x), ceil(x)", + "input_desc": "BIGINT column 9223372036854775807 (i64::MAX)", + "result": "9.223372036854776e+18", + "result_type": "DOUBLE", + "note": "LOSSY int->double cast; floor(BIGINT 9007199254740993 = 2^53+1) -> 9007199254740992.0" + }, + { + "expr": "trunc(x), round(x)", + "input_desc": "BIGINT column 9223372036854775807; INTEGER column 7", + "result": "9223372036854775807; 7", + "result_type": "BIGINT; INTEGER", + "note": "single-arg trunc/round on integer types are exact IDENTITY preserving the input type (asymmetric with floor/ceil)" + }, + { + "expr": "round(x, n)", + "input_desc": "DOUBLE 2.567, n=2 (INTEGER col)", + "result": "2.57", + "result_type": "DOUBLE", + "note": "round(DOUBLE, INTEGER) -> DOUBLE; n accepts TINYINT/SMALLINT/INTEGER but round(DOUBLE, BIGINT) is a BINDER ERROR" + }, + { + "expr": "round(x, n) algorithm", + "input_desc": "DOUBLE x, n>=0", + "result": "r = round_half_away(x * m) / m where m = std::pow(10.0, n) of the DuckDB binary; if r is inf or NaN return x", + "result_type": "DOUBLE", + "note": "bit-exact-verified on 9028 fuzz pairs + 6400 targeted cases; the modifier table is the binary's libm pow, NOT correctly-rounded 1e{n} (differs at n=23) and NOT ucrt pow (differs at n=126)" + }, + { + "expr": "round(x, n) algorithm", + "input_desc": "DOUBLE x, n<0", + "result": "m = pow(10.0, -n); r = round_half_away(x / m) * m; if r is inf or NaN return +0.0", + "result_type": "DOUBLE", + "note": "the n<0 fallback returns 0.0 (positive zero), not the input \u2014 this is why NaN/inf/overflow cases below yield 0.0" + }, + { + "expr": "round(x, 2)", + "input_desc": "DOUBLE column 2.675 (= 2.67499999999999982...)", + "result": "2.68", + "result_type": "DOUBLE", + "note": "NOT the Python-style 2.67: double(2.675)*100 rounds to exactly 267.5 in the multiply, then half-away -> 268" + }, + { + "expr": "round(x, 2)", + "input_desc": "DOUBLE column 0.285", + "result": "0.28", + "result_type": "DOUBLE", + "note": "product 28.499999999999996 rounds down; DECIMAL-literal path gives 0.29 \u2014 see surprises" + }, + { + "expr": "round(x, 2)", + "input_desc": "DOUBLE columns 1.005, 0.145, 1.015, 2.665, 1.255, 1.115, 0.045, 0.125, 0.375", + "result": "1.0, 0.14, 1.01, 2.67, 1.25, 1.12, 0.05, 0.13, 0.38", + "result_type": "DOUBLE", + "note": "each decided solely by the IEEE rounding of x*100 (1.115 and 1.255 both have true value ...4999 yet go opposite ways)" + }, + { + "expr": "round(x, 23)", + "input_desc": "DOUBLE x = float.fromhex('0x1.9fc4c8ce7ceb8p+8') = 415.7686890654545", + "result": "415.76868906545445 (0x1.9fc4c8ce7ceb7p+8)", + "result_type": "DOUBLE", + "note": "pow-table ulp pin: only m = 0x1.52d02c7e14af7p+76 (bits 0x44B52D02C7E14AF7) explains this; correctly-rounded 1e23 (0x...af6p+76) would give 415.76868906545457" + }, + { + "expr": "round(x, 23)", + "input_desc": "DOUBLE x = float.fromhex('0x1.bc25ab024d0ffp+8') = 444.14714064008143", + "result": "444.14714064008143 (unchanged)", + "result_type": "DOUBLE", + "note": "second discriminating witness; strtod-1e23 modifier would give 444.1471406400814. DuckDB's pow(10,126) = 0x1.7a2ecc414a03fp+418 (bits 0x5A17A2ECC414A03F) = strtod 1e126, differing from ucrt pow" + }, + { + "expr": "round(x, -2)", + "input_desc": "DOUBLE 1234.5", + "result": "1200.0", + "result_type": "DOUBLE", + "note": "34.5 < 50 rounds down \u2014 no cascaded rounding of the .5 first" + }, + { + "expr": "round(x, -2)", + "input_desc": "DOUBLE 1250.0 / -1250.0", + "result": "1300.0 / -1300.0", + "result_type": "DOUBLE", + "note": "half away from zero at negative scales too; round(55.0,-1)=60.0, round(-55.0,-1)=-60.0" + }, + { + "expr": "round(x, -2)", + "input_desc": "DOUBLE -44.0", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "signed zero preserved through the scale-round-unscale pipeline: round(-0.4)->-0.0, round(-0.04,1)->-0.0, round(-4.0,-1)->-0.0" + }, + { + "expr": "round(x, n), n <= -309", + "input_desc": "DOUBLE 2.675, n=-400 (also n=-1000, n=-2147483648)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "modifier overflows to inf -> x/inf=0 -> 0*inf=NaN -> fallback 0.0 for EVERY finite x" + }, + { + "expr": "round(x, -308)", + "input_desc": "DOUBLE 1.7e308", + "result": "0.0", + "result_type": "DOUBLE", + "note": "rounded value 2*1e308 overflows to inf -> fallback 0.0; but round(9.9e307, -308) = 1e+308 (fits)" + }, + { + "expr": "round(x, n), n >= 309", + "input_desc": "DOUBLE 2.675, n=309/400/1000/2147483647", + "result": "2.675 (input returned)", + "result_type": "DOUBLE", + "note": "x*inf = inf -> positive-branch fallback returns INPUT; likewise round(1e308, 2) -> 1e+308 and round(5e-324, 324) -> 5e-324" + }, + { + "expr": "round(NaN, -2)", + "input_desc": "DOUBLE 'nan' column, n=-2 (also -400)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "SURPRISE: NaN in, +0.0 out. round(NaN, 2)/round(NaN)/floor/ceil/trunc(NaN) all stay NaN; only the n<0 branch converts" + }, + { + "expr": "round(inf, -2), round(-inf, -2)", + "input_desc": "DOUBLE +/-inf column, n<0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "both infinities become +0.0; round(inf, 2) = inf, round(inf) = inf, floor/ceil/trunc(+/-inf) = +/-inf" + }, + { + "expr": "trunc(x, n)", + "input_desc": "DOUBLE 2.675 n=2; 25.67 n=-1; -25.67 n=-1; 1250.0 n=-2", + "result": "2.67; 20.0; -20.0; 1200.0", + "result_type": "DOUBLE", + "note": "2-arg trunc EXISTS (floor/ceil 2-arg do not); same scale algorithm but std::trunc, and BOTH branches' non-finite fallback returns the INPUT" + }, + { + "expr": "trunc(NaN, -2) / trunc(inf, -2) / trunc(2.675, -400)", + "input_desc": "DOUBLE column, negative n", + "result": "nan / inf / 2.675", + "result_type": "DOUBLE", + "note": "contrast with round's 0.0: trunc returns input on NaN/inf/modifier-overflow in both branches; trunc(1.7e308, -308) = 1e+308 (finite path)" + }, + { + "expr": "round(i, n) BIGINT, n>=0", + "input_desc": "BIGINT 1234, n=2; BIGINT i64::MAX, n=2", + "result": "1234; 9223372036854775807", + "result_type": "BIGINT", + "note": "positive/zero n on integers is identity, never converts to DOUBLE" + }, + { + "expr": "round(i, -2) BIGINT", + "input_desc": "BIGINT 1250 / -1250 / 1249 / 49 / 50 / -49", + "result": "1300 / -1300 / 1200 / 0 / 100 / 0", + "result_type": "BIGINT", + "note": "integer-domain half-away: y = x +/- power/2 (sign of x), then trunc-div and re-multiply" + }, + { + "expr": "round(i, -2) BIGINT at bounds", + "input_desc": "BIGINT 9223372036854775807 (MAX); -9223372036854775808 (MIN)", + "result": "-9223372036854775700; 9223372036854775700", + "result_type": "BIGINT", + "note": "NO TRAP \u2014 the half-power add WRAPS at i64: wrapping_add(MAX,50)=-9223372036854775759 -> /100*100. Verified bit-exact on 7391-pair fuzz; round(MAX,-1)=-9223372036854775800, round(MAX,-18)=-8000000000000000000, round(MIN,-18)=8000000000000000000" + }, + { + "expr": "round(i, n) BIGINT, n <= -19", + "input_desc": "any BIGINT (incl. 6000000000000000000, MAX, MIN), n=-19/-20/-2147483648", + "result": "0", + "result_type": "BIGINT", + "note": "hard cutoff: 10^19 > i64::MAX so |n|>=19 returns 0 unconditionally (no rounding to 1e19)" + }, + { + "expr": "trunc(i, -2) BIGINT", + "input_desc": "BIGINT MAX / MIN / 1299 / -1299; n=-19", + "result": "9223372036854775800 / -9223372036854775800 / 1200 / -1200; 0", + "result_type": "BIGINT", + "note": "trunc has no half-add so it never wraps; same >= 19 cutoff to 0; positive n identity" + }, + { + "expr": "floor(x)/ceil(x)/round(x)/trunc(x)", + "input_desc": "DOUBLE 9.3e18 (> i64::MAX)", + "result": "9.3e+18", + "result_type": "DOUBLE", + "note": "result stays DOUBLE \u2014 no i64 conversion, no trap anywhere in the family" + }, + { + "expr": "floor(5e-324), ceil(5e-324), ceil(-5e-324)", + "input_desc": "DOUBLE denormal column", + "result": "0.0, 1.0, -0.0", + "result_type": "DOUBLE", + "note": "denormals behave per IEEE; round(5e-324, 2) = 0.0" + }, + { + "expr": "floor(-0.5), ceil(-0.5)", + "input_desc": "DOUBLE column", + "result": "-1.0, -0.0", + "result_type": "DOUBLE", + "note": "ceil produces negative zero" + }, + { + "expr": "all functions, bound -0.0", + "input_desc": "DOUBLE column holding true -0.0 (parameter-bound)", + "result": "-0.0 from floor, ceil, trunc, round, round(x,2), round(x,-2), trunc(x,2)", + "result_type": "DOUBLE", + "note": "sign of zero fully preserved (bit-verified). CAUTION: the SQL literal -0.0 is DECIMAL(2,1) and becomes +0.0 when cast to DOUBLE; '-0.0'::DOUBLE and -(0.0::DOUBLE) keep the sign" + }, + { + "expr": "every function, NULL in any argument", + "input_desc": "NULL x and/or NULL n via columns", + "result": "None", + "result_type": "input-dependent (DOUBLE for floor/ceil; input type for trunc/round)", + "note": "strictly NULL-propagating in every position; round(NULL::INT, -2) -> NULL INTEGER" + }, + { + "expr": "round(x, n) FLOAT input", + "input_desc": "FLOAT column 2.567, n=2", + "result": "2.569999933242798", + "result_type": "FLOAT", + "note": "computed in f32 (float32 of 2.57); round(FLOAT NaN, -2) -> 0.0f32. Engine never sees f32 ('float' schema = float64) but any FLOAT-typed SQL cast would hit this" + }, + { + "expr": "constant-fold vs vectorized", + "input_desc": "round(CAST(2.675 AS DOUBLE), 2), round(CAST(0.285 AS DOUBLE), 2), literal round(9223372036854775807, -2), const-n vs column-n on 10 edge cases incl. n=23 ulp witnesses", + "result": "identical to the column path in every measured case", + "result_type": "DOUBLE/BIGINT", + "note": "no fold divergence once types match; ALL divergences come from bare literals being DECIMAL, not from folding" + } + ], + "errors": [ + { + "expr": "SELECT round(d, b) FROM tt -- d DOUBLE, b BIGINT column", + "error_head": "Binder Error: No function matches the given name and argument types 'round(DOUBLE, BIGINT)'.", + "kind": "binder" + }, + { + "expr": "SELECT floor(d, n) FROM tt -- 2-arg floor", + "error_head": "Binder Error: No function matches the given name and argument types 'floor(DOUBLE, INTEGER)'.", + "kind": "binder" + }, + { + "expr": "SELECT ceil(d, n) FROM tt -- 2-arg ceil", + "error_head": "Binder Error: No function matches the given name and argument types 'ceil(DOUBLE, INTEGER)'.", + "kind": "binder" + }, + { + "expr": "SELECT ceiling(d, n) FROM tt -- 2-arg ceiling", + "error_head": "Binder Error: No function matches the given name and argument types 'ceiling(DOUBLE, INTEGER)'.", + "kind": "binder" + }, + { + "expr": "SELECT floor(v) FROM tt -- v VARCHAR (no implicit VARCHAR->DOUBLE; same for round/trunc/ceil and BOOLEAN)", + "error_head": "Binder Error: No function matches the given name and argument types 'floor(VARCHAR)'.", + "kind": "binder" + }, + { + "expr": "SELECT round(dec, n) FROM tt -- DECIMAL x with NON-CONSTANT n (NotImplementedException at bind time)", + "error_head": "Not implemented Error: ROUND(DECIMAL, INTEGER) with non-constant precision is not supported", + "kind": "binder" + }, + { + "expr": "SELECT truncate(d) FROM tt -- CatalogException; only 'trunc' exists", + "error_head": "Catalog Error: Scalar Function with name truncate does not exist!", + "kind": "binder" + }, + { + "expr": "SELECT round(d, 1, 2) FROM tt", + "error_head": "Binder Error: No function matches the given name and argument types 'round(DOUBLE, INTEGER_LITERAL, INTEGER_LITERAL)'.", + "kind": "binder" + } + ], + "duck_check_tests": "NAN = float(\"nan\")\nINF = float(\"inf\")\n\n\ndef test_round_family_single_arg_double():\n vals = [2.5, 3.5, -2.5, -3.5, 0.5, -0.5, 1.5, 0.49999999999999994,\n -0.49999999999999994, 2.675, -0.4, -0.04, 0.0, -0.0, 5e-324,\n -5e-324, 4503599627370495.5, -4503599627370495.5, 9.3e18, -9.3e18,\n 1.7976931348623157e308, 12345.678901234567, NAN, INF, -INF, None]\n duck_check(\n \"SELECT floor(x) AS f, ceil(x) AS c, ceiling(x) AS c2,\"\n \" trunc(x) AS t, round(x) AS r FROM __THIS__\",\n {\"x\": \"float?\"},\n [{\"x\": v} for v in vals],\n )\n\n\ndef test_round_double_with_digits_constant_n():\n vals = [2.675, 0.285, 1.005, 0.145, 1.015, 2.665, 0.125, 0.375, 1.255,\n 0.045, 1.115, 1234.5, -1234.5, 1250.0, -1250.0, -44.0, -0.04,\n 1.7e308, 9.9e307, 5e-324, -0.0, NAN, INF, -INF, None]\n duck_check(\n \"SELECT round(x, 2) AS r2, round(x, 0) AS r0, round(x, -1) AS rn1,\"\n \" round(x, -2) AS rn2, round(x, 309) AS rbig, round(x, -308) AS rn308,\"\n \" round(x, -400) AS rneg, trunc(x, 2) AS t2, trunc(x, -1) AS tn1,\"\n \" trunc(x, -400) AS tneg FROM __THIS__\",\n {\"x\": \"float?\"},\n [{\"x\": v} for v in vals],\n )\n\n\ndef test_round_double_pow_table_ulp_witnesses():\n # Only DuckDB's own pow(10,23) modifier (bits 0x44B52D02C7E14AF7) passes these.\n rows = [{\"x\": float.fromhex(\"0x1.9fc4c8ce7ceb8p+8\")}, # -> 415.76868906545445\n {\"x\": float.fromhex(\"0x1.bc25ab024d0ffp+8\")}, # -> 444.14714064008143\n {\"x\": float.fromhex(\"-0x1.a279f7d13a32ep+8\")}] # -> -418.4764376418197\n duck_check(\"SELECT round(x, 23) AS r FROM __THIS__\", {\"x\": \"float\"}, rows)\n\n\ndef test_round_variable_n_from_column():\n # engine 'int' columns are BIGINT: round(DOUBLE, BIGINT) does not bind, so CAST.\n rows = [{\"x\": 2.675, \"n\": 2}, {\"x\": 1234.5, \"n\": -2}, {\"x\": NAN, \"n\": -2},\n {\"x\": INF, \"n\": -2}, {\"x\": -INF, \"n\": -2}, {\"x\": 2.5, \"n\": None},\n {\"x\": None, \"n\": 2}, {\"x\": 1.7e308, \"n\": -308}, {\"x\": 2.675, \"n\": 400},\n {\"x\": 2.675, \"n\": -400}, {\"x\": -44.0, \"n\": -2}]\n duck_check(\n \"SELECT round(x, CAST(n AS INTEGER)) AS r,\"\n \" trunc(x, CAST(n AS INTEGER)) AS t FROM __THIS__\",\n {\"x\": \"float?\", \"n\": \"int?\"},\n rows,\n )\n\n\ndef test_round_family_int64_identity_and_negative_n():\n vals = [1250, -1250, 1249, -1249, 25, -25, 49, 50, -49, -50, 0, 15,\n 9223372036854775807, -9223372036854775808, 9223372036854775757,\n 9223372036854775800, 5000000000000000000, None]\n duck_check(\n \"SELECT trunc(i) AS t1, round(i) AS r1, round(i, 2) AS rpos,\"\n \" round(i, -2) AS rn2, round(i, -18) AS rn18, round(i, -19) AS rcut,\"\n \" trunc(i, -2) AS tn2, trunc(i, -19) AS tcut FROM __THIS__\",\n {\"i\": \"int?\"},\n [{\"i\": v} for v in vals],\n )\n\n\ndef test_floor_ceil_int64_go_through_double():\n rows = [{\"i\": v} for v in [7, 9007199254740993, 9223372036854775807,\n -9223372036854775808, None]]\n duck_check(\n \"SELECT floor(i) AS f, ceil(i) AS c, ceiling(i) AS c2 FROM __THIS__\",\n {\"i\": \"int?\"},\n rows,\n )\n", + "lowering": "New IR ops \u2014 do NOT desugar. NULL handling is ordinary strict propagation (safe for the frontend), but the scalar semantics need dedicated ops because of the non-finite fallback branches, the oracle-extracted pow table, and i64 wrapping.\n\nOps and exact Rust semantics (every line below is fuzz-verified bit-exact against DuckDB 1.5.5):\n\n1. FloorF64 / CeilF64 / TruncF64 / RoundF64 (f64 -> f64):\n x.floor() / x.ceil() / x.trunc() / x.round()\n Rust f64::round is round-half-away-from-zero computed exactly \u2014 identical to std::round (verified incl. 0.49999999999999994, 2^52-0.5, NaN/inf/-0.0/denormals). Cranelift note: nearest/trunc/floor/ceil FP instructions exist but `nearest` is half-to-EVEN \u2014 do NOT use it for round; call a helper or emit the away-from-zero sequence.\n Integer inputs: floor/ceil lower as FloorF64(cast i64->f64) \u2014 frontend inserts the lossy cast, result stays f64 (matches DuckDB's DOUBLE). trunc/round on i64 lower to NOTHING (identity passthrough, result i64).\n\n2. RoundPrecF64(x: f64, n: i32) -> f64:\n fn round_prec_f64(x: f64, n: i32) -> f64 {\n if n >= 0 {\n let m = pow10(n as i64); // table lookup, inf beyond 308\n let r = (x * m).round() / m;\n if r.is_infinite() || r.is_nan() { x } else { r }\n } else {\n let m = pow10((n as i64).unsigned_abs() as i64); // i32::MIN safe via i64\n let r = (x / m).round() * m;\n if r.is_infinite() || r.is_nan() { 0.0 } else { r }\n }\n }\n pow10(k): const [f64; 309] table for k=0..=308, else f64::INFINITY. The table MUST be extracted from the oracle (SELECT pow(10.0, k::DOUBLE)) \u2014 it equals float(\"1e{k}\") for all k EXCEPT k=23 = f64::from_bits(0x44B52D02C7E14AF7) (one ulp above 1e23; ucrt agrees) and note k=126 = f64::from_bits(0x5A17A2ECC414A03F) (equals 1e126 strtod; one ulp below ucrt pow). Do not compute with f64::powi/powf at runtime.\n\n3. TruncPrecF64(x: f64, n: i32) -> f64: same shape as RoundPrecF64 with .trunc() instead of .round(), and BOTH branches' fallback returns x (never 0.0). This asymmetry with round is measured, not a bug.\n\n4. RoundPrecI64(x: i64, n: i32) -> i64:\n fn round_prec_i64(x: i64, n: i32) -> i64 {\n if n >= 0 { return x; }\n let p = (n as i64).unsigned_abs();\n if p >= 19 { return 0; }\n let power = 10i64.pow(p as u32);\n let half = power / 2;\n let y = if x >= 0 { x.wrapping_add(half) } else { x.wrapping_sub(half) };\n (y / power) * power // Rust / truncates toward zero, matching C++; no overflow possible after the div\n }\n The wrapping_add/sub is REQUIRED: DuckDB wraps at i64 bounds instead of trapping (round(i64::MAX,-2) = -9223372036854775700). Zero trap conditions in this whole family.\n\n5. TruncPrecI64(x: i64, n: i32): same minus the half-add: if n>=0 {x} else if p>=19 {0} else {(x / power) * power} \u2014 never wraps.\n\nFrontend notes: map ceiling->ceil, reject truncate (CatalogException parity). round's n argument must be INTEGER-width: an engine int64 column used as n must be CAST to INTEGER in SQL or it's a binder error (mirror DuckDB's message). n is i32 in the IR; the two ops take n as a value (vectorized n verified identical to constant n). FLOAT(f32) inputs never reach the engine ('float' schema = f64) \u2014 if f32 columns ever land, DuckDB computes these in f32, a separate op family.", + "surprises": [ + "round(x, n) with NEGATIVE n turns NaN, +inf and -inf into +0.0 (the n<0 branch's overflow guard returns 0, the n>=0 branch returns the input). round(NaN, -2) = 0.0 is the single most counterintuitive pin in the family.", + "round(x, n<=-309) returns 0.0 for EVERY finite x (modifier overflows to inf), while trunc(x, n<=-309) returns x unchanged \u2014 round and trunc use different non-finite fallbacks (0.0 vs input) in the negative branch.", + "Integer round with negative n NEVER raises on overflow \u2014 it WRAPS at i64: round(9223372036854775807, -2) = -9223372036854775700 (wrapping half-power add, then trunc-div/mul). |n| >= 19 collapses every BIGINT to 0, even values that would mathematically round to 1e19. There are no runtime errors anywhere in this family.", + "floor/ceil have NO integer overloads (int input -> lossy cast to DOUBLE: floor(2^53+1) = 9007199254740992.0, floor(i64::MAX) = 9.223372036854776e+18 DOUBLE), while trunc/round single-arg on integers are exact identities preserving the integer type.", + "The double scale-then-round modifier is the DuckDB binary's own std::pow(10,n), which is neither correctly-rounded (differs from strtod 1e23 by 1 ulp at n=23) nor ucrt pow (differs at n=126); a 1-ulp modifier difference visibly changes round(x, 23) results (witness x=0x1.9fc4c8ce7ceb8p+8). The Rust table must be extracted from the oracle binary, and this is a cross-platform (Linux glibc vs bundled-libm) divergence hazard for CI.", + "round(2.675, 2) = 2.68 on a DOUBLE column NOT because of decimal semantics: double(2.675)*100 rounds UP to exactly 267.5 in the multiply. But round(0.285, 2) = 0.28 and round(1.005, 2) = 1.0. Two inputs whose true values both end in ...4999 can go opposite ways (1.115 -> 1.12 but 1.255 -> 1.25) \u2014 only the multiply-then-half-away model predicts all of them; 'round the true decimal value' does not.", + "Bare SQL numeric literals are DECIMAL, and DECIMAL round is exact decimal half-away: literal round(0.285, 2) = 0.29 vs DOUBLE column 0.28 for the same digits. Also round(DECIMAL, non-constant n) is a bind-time NotImplementedException. Frontends lowering literals to f64 diverge from DuckDB's literal typing here.", + "round(x) is half-AWAY-from-zero (2.5 -> 3, -2.5 -> -3), never banker's; and it is exact std::round semantics: round(0.49999999999999994) = 0.0, round(4503599627370495.5) = 4503599627370496.0 \u2014 any add-0.5 implementation fails these. Cranelift's `nearest` (half-to-even) must NOT be used.", + "round(DOUBLE, BIGINT) does not bind \u2014 the engine's int64 columns cannot be passed as the digits argument without CAST(... AS INTEGER); the corpus's constant digits (INTEGER literals) bind fine.", + "trunc has a 2-argument form trunc(x, n) (floor/ceil do not); 'truncate' does not exist (CatalogException suggesting trunc).", + "The SQL literal -0.0 silently loses its sign (DECIMAL(2,1) -> +0.0 as DOUBLE), but true -0.0 through a bound column is preserved bit-exactly by every function in the family, and the family PRODUCES negative zeros: ceil(-0.5) = -0.0, round(-0.4) = -0.0, round(-44.0, -2) = -0.0, trunc(-0.04, 1) = -0.0.", + "Constant-fold and vectorized paths agree bit-exactly in every measured case (including the pow-ulp witnesses at n=23) once the input is genuinely DOUBLE/BIGINT \u2014 all literal-vs-column divergences in this family are typing (DECIMAL literals), not folding." + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave1/pins_log-exp.json b/docs/superpowers/specs/pins-wave1/pins_log-exp.json new file mode 100644 index 0000000..a4250f1 --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_log-exp.json @@ -0,0 +1,586 @@ +{ + "family": "log-exp", + "pins": [ + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=1.0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "positive zero (repr shows no sign)" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=2.718281828459045 (e)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "exact 1.0" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=2.0", + "result": "0.6931471805599453", + "result_type": "DOUBLE", + "note": "== Python math.log(2.0) bit-exact" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=NaN", + "result": "nan", + "result_type": "DOUBLE", + "note": "NaN does NOT error \u2014 fails both ==0 and <0 checks, passes through libm" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=-NaN (bits 0xFFF8000000000000)", + "result": "nan", + "result_type": "DOUBLE", + "note": "negative NaN also passes the checks (IEEE compares false)" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=+inf", + "result": "inf", + "result_type": "DOUBLE", + "note": "no error" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "strict NULL propagation" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=1e308", + "result": "709.1962086421661", + "result_type": "DOUBLE", + "note": "matches platform libm" + }, + { + "expr": "ln(x)", + "input_desc": "DOUBLE col x=5e-324 (min denormal)", + "result": "-744.4400719213812", + "result_type": "DOUBLE", + "note": "note: differs in last ulp from log(e,5e-324)=-744.4400719213813" + }, + { + "expr": "ln(x)", + "input_desc": "INTEGER col x=2", + "result": "0.6931471805599453", + "result_type": "DOUBLE", + "note": "all int widths (TINYINT..HUGEINT), FLOAT, DECIMAL -> DOUBLE result" + }, + { + "expr": "ln(x)", + "input_desc": "BIGINT col x=9223372036854775807", + "result": "43.66827237527655", + "result_type": "DOUBLE", + "note": "i64::MAX converts to f64 then ln" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=10.0", + "result": "1.0", + "result_type": "DOUBLE", + "note": "1-arg log IS BASE 10 in DuckDB 1.5.5 (alias of log10), NOT natural log" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=2.0", + "result": "0.3010299956639812", + "result_type": "DOUBLE", + "note": "== log10(2.0)" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=1e308", + "result": "308.0", + "result_type": "DOUBLE", + "note": "exactly 308.0 \u2014 base-10 proof" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=5e-324", + "result": "-323.3062153431158", + "result_type": "DOUBLE", + "note": "== log10" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=2.718281828459045 (e)", + "result": "0.4342944819032518", + "result_type": "DOUBLE", + "note": "log10(e), NOT 1.0 \u2014 base-10 proof" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col, 20000 fuzzed positive values", + "result": "bit-identical to log10(x) on every sample", + "result_type": "DOUBLE", + "note": "log(x) == log10(x) == Python math.log10 bit-exact, 0/20000 mismatches" + }, + { + "expr": "log(x)", + "input_desc": "DOUBLE col x=NaN / +inf / NULL", + "result": "nan / inf / None", + "result_type": "DOUBLE", + "note": "same special handling as ln" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=2.0", + "result": "1.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=1.0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=2.718281828459045", + "result": "1.4426950408889634", + "result_type": "DOUBLE", + "note": "matches math.log2 bit-exact" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=1e308", + "result": "1023.1538532253076", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=5e-324", + "result": "-1074.0", + "result_type": "DOUBLE", + "note": "exactly -1074.0 (min denormal exponent)" + }, + { + "expr": "log2(x)", + "input_desc": "BIGINT col x=9223372036854775807", + "result": "63.0", + "result_type": "DOUBLE", + "note": "exactly 63.0 because i64::MAX rounds to 2^63 as f64" + }, + { + "expr": "log2(x)", + "input_desc": "DOUBLE col x=NaN / +inf / NULL", + "result": "nan / inf / None", + "result_type": "DOUBLE", + "note": "same special handling as ln" + }, + { + "expr": "log10(x)", + "input_desc": "DOUBLE col x=10.0 / 1.0 / e / 1e308 / 5e-324", + "result": "1.0 / 0.0 / 0.4342944819032518 / 308.0 / -323.3062153431158", + "result_type": "DOUBLE", + "note": "bit-identical to 1-arg log(x) on all 20000 fuzz samples" + }, + { + "expr": "log10(x)", + "input_desc": "DOUBLE col x=NaN / +inf / NULL", + "result": "nan / inf / None", + "result_type": "DOUBLE", + "note": "same special handling as ln" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=0.0", + "result": "1.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=-0.0", + "result": "1.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=1.0", + "result": "2.718281828459045", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=-1.0", + "result": "0.36787944117144233", + "result_type": "DOUBLE", + "note": "negative input fine \u2014 exp has NO domain errors at all" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=NaN / -NaN", + "result": "nan", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=+inf", + "result": "inf", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=-inf", + "result": "0.0", + "result_type": "DOUBLE", + "note": "POSITIVE zero (copysign(1,r)=+1.0)" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=1000.0", + "result": "inf", + "result_type": "DOUBLE", + "note": "overflow -> inf, NOT an error" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=1e308", + "result": "inf", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=709.782712893384", + "result": "1.7976931348622732e+308", + "result_type": "DOUBLE", + "note": "largest finite probe, just below f64::MAX; x=709.7827128933841 -> inf" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=-745.0", + "result": "5e-324", + "result_type": "DOUBLE", + "note": "underflows into denormals, not zero; also exp(-744.4400719213812)=5e-324" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=-746.0 / -1000.0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "full underflow -> +0.0" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col x=5e-324", + "result": "1.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "exp(x)", + "input_desc": "INTEGER col x=0 / BIGINT i64::MAX / BIGINT i64::MIN", + "result": "1.0 / inf / 0.0", + "result_type": "DOUBLE", + "note": "int input -> DOUBLE" + }, + { + "expr": "exp(x)", + "input_desc": "DOUBLE col, 20000 fuzzed values in [-1e6,1e6]", + "result": "bit-identical to Python math.exp (OverflowError treated as inf)", + "result_type": "DOUBLE", + "note": "0/20000 mismatches vs platform libm" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=2.0 x=8.0", + "result": "3.0", + "result_type": "DOUBLE", + "note": "arg order is log(base, x)" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=10.0 x=1000.0", + "result": "3.0", + "result_type": "DOUBLE", + "note": "PROOF of log10-ratio lowering: ln(x)/ln(b) would give 2.9999999999999996" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=e x=10.0", + "result": "2.302585092994046", + "result_type": "DOUBLE", + "note": "log2-ratio would give 2.3025850929940455 \u2014 rules out log2 lowering" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=e x=5e-324", + "result": "-744.4400719213813", + "result_type": "DOUBLE", + "note": "log(e,x) is NOT ln(x): ln(5e-324)=-744.4400719213812 (last-ulp difference)" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols, 20000 fuzzed (b,x) pairs", + "result": "bit-identical to log10(x)/log10(b) on every sample", + "result_type": "DOUBLE", + "note": "vs ln-ratio 6989/20000 mismatch, vs log2-ratio 9142/20000 \u2014 lowering is log10(x)/log10(b), settled" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=3.0 x=7.0", + "result": "1.7712437491614221", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=0.5 x=8.0", + "result": "-3.0", + "result_type": "DOUBLE", + "note": "base in (0,1) valid" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=0.5 x=1.0", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "NEGATIVE zero: +0.0/log10(0.5)= -0.0; b=2.0 x=1.0 gives +0.0" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=2.0 x=0.5", + "result": "-1.0", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=2.0 x=1e308 / x=5e-324", + "result": "1023.1538532253076 / -1074.0", + "result_type": "DOUBLE", + "note": "== log2 values here" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=NaN, x in {1.0,8.0,0.5,NaN,+inf,1e308,5e-324}", + "result": "nan", + "result_type": "DOUBLE", + "note": "NaN base passes domain checks; but x==0 or x<0 STILL errors even with NaN base" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=2.0 x=NaN", + "result": "nan", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=+inf, x=8.0 / 1.0 / 1e308", + "result": "0.0", + "result_type": "DOUBLE", + "note": "finite/inf = +0.0" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=+inf, x=0.5 / 5e-324", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "negative-numerator/inf = -0.0" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=+inf x=+inf", + "result": "nan", + "result_type": "DOUBLE", + "note": "inf/inf" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=0.5 x=+inf", + "result": "-inf", + "result_type": "DOUBLE", + "note": "inf/negative" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=2.0 x=+inf", + "result": "inf", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=1.0000000000000002 x=4.0", + "result": "6243314768165360.0", + "result_type": "DOUBLE", + "note": "base near 1 is NOT the base==1 error; b=0.9999999999999999 x=4.0 -> -1.2486629536330718e+16" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=5e-324 x=8.0", + "result": "-0.0027932960893854745", + "result_type": "DOUBLE", + "note": "denormal base valid" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=1e308 x=8.0", + "result": "0.00293211034737644", + "result_type": "DOUBLE", + "note": "" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols b=NULL, x anything incl 0.0/-4.0/-inf/NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "NULL propagation pre-empts ALL domain errors" + }, + { + "expr": "log(b, x)", + "input_desc": "DOUBLE cols x=NULL, b in {0.0,-0.0,-2.0,1.0,-inf}", + "result": "None", + "result_type": "DOUBLE", + "note": "even error-triggering bases yield NULL when x is NULL" + }, + { + "expr": "log(b, x)", + "input_desc": "INTEGER cols b=2 x=8", + "result": "3.0", + "result_type": "DOUBLE", + "note": "int/int -> DOUBLE" + }, + { + "expr": "CASE WHEN x > 0 THEN ln(x) ELSE NULL END", + "input_desc": "DOUBLE col rows [1.0,-1.0,4.0,0.0]", + "result": "[0.0, None, 1.3862943611198906, None]", + "result_type": "DOUBLE", + "note": "guarded rows do NOT trap \u2014 ln is only evaluated on taken branch" + }, + { + "expr": "SELECT ln(x) FROM t WHERE x > 0", + "input_desc": "DOUBLE col rows [1.0,2.0,-1.0,4.0]", + "result": "[0.0, 0.6931471805599453, 1.3862943611198906]", + "result_type": "DOUBLE", + "note": "filtered-out bad rows do NOT trap; unfiltered query with same data aborts entirely" + }, + { + "expr": "SELECT ln(0.0) etc. as pure literals", + "input_desc": "constant-fold path, all error and special cases re-probed as literals", + "result": "identical values and identical exceptions to the vectorized path", + "result_type": "DOUBLE", + "note": "no literal-vs-vectorized divergence found anywhere in this family" + } + ], + "errors": [ + { + "expr": "ln(x) / log(x) / log2(x) / log10(x), x=0.0 column", + "error_head": "Out of Range Error: cannot take logarithm of zero", + "kind": "runtime" + }, + { + "expr": "ln(x) / log(x) / log2(x) / log10(x), x=-0.0 column", + "error_head": "Out of Range Error: cannot take logarithm of zero", + "kind": "runtime" + }, + { + "expr": "ln(x) / log(x) / log2(x) / log10(x), x=-1.0 or -1000.0 column", + "error_head": "Out of Range Error: cannot take logarithm of a negative number", + "kind": "runtime" + }, + { + "expr": "ln(x) / log(x) / log2(x) / log10(x), x=-inf column", + "error_head": "Out of Range Error: cannot take logarithm of a negative number", + "kind": "runtime" + }, + { + "expr": "ln(x), INTEGER col x=0", + "error_head": "Out of Range Error: cannot take logarithm of zero", + "kind": "runtime" + }, + { + "expr": "ln(x), INTEGER col x=-1 / BIGINT col x=i64::MIN", + "error_head": "Out of Range Error: cannot take logarithm of a negative number", + "kind": "runtime" + }, + { + "expr": "log(b,x), b=0.0 or -0.0 column, ANY non-NULL x (incl NaN, +inf, -4.0)", + "error_head": "Out of Range Error: cannot take logarithm of zero", + "kind": "runtime" + }, + { + "expr": "log(b,x), b<0 (-2.0, -inf) column, ANY non-NULL x (incl 0.0)", + "error_head": "Out of Range Error: cannot take logarithm of a negative number", + "kind": "runtime" + }, + { + "expr": "log(b,x), b=1.0 column, ANY non-NULL x (incl 0.0, -4.0, NaN, inf)", + "error_head": "Out of Range Error: divison by zero in based logarithm", + "kind": "runtime" + }, + { + "expr": "log(b,x), valid b (incl NaN, +inf), x=0.0 or -0.0 column", + "error_head": "Out of Range Error: cannot take logarithm of zero", + "kind": "runtime" + }, + { + "expr": "log(b,x), valid b (incl NaN, +inf), x<0 or -inf column", + "error_head": "Out of Range Error: cannot take logarithm of a negative number", + "kind": "runtime" + }, + { + "expr": "log1p(1.0)", + "error_head": "Catalog Error: Scalar Function with name log1p does not exist!", + "kind": "binder" + }, + { + "expr": "expm1(1.0)", + "error_head": "Catalog Error: Scalar Function with name expm1 does not exist!", + "kind": "binder" + }, + { + "expr": "ln(x) with VARCHAR column", + "error_head": "Binder Error: No function matches the given name and argument types 'ln(VARCHAR)'. You might need to add explicit type casts.", + "kind": "binder" + }, + { + "expr": "ln(x) / exp(x) with BOOLEAN column", + "error_head": "Binder Error: No function matches the given name and argument types 'ln(BOOLEAN)'. You might need to add explicit type casts.", + "kind": "binder" + }, + { + "expr": "exp(x) \u2014 any input", + "error_head": "(exp NEVER raises: overflow->inf, underflow->0.0/denormal, negative/NaN/inf all fine)", + "kind": "runtime" + } + ], + "duck_check_tests": "import math\n\n\ndef test_logexp_basic_values():\n duck_check(\n \"SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": v} for v in [1.0, 2.718281828459045, 2.0, 10.0, 0.5, 1000.0,\n 1e308, 5e-324, 709.782712893384]],\n )\n\n\ndef test_logexp_nan_inf_pass_through():\n # NaN and +inf do NOT trip the domain checks; only <=0 does.\n duck_check(\n \"SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": float(\"nan\")}, {\"x\": float(\"inf\")}],\n )\n\n\ndef test_exp_never_errors():\n # overflow -> inf, underflow -> denormal then +0.0; -inf -> +0.0; boundary 709.782712893384 -> 1.7976931348622732e+308\n duck_check(\n \"SELECT exp(x) AS e FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": v} for v in [float(\"-inf\"), -1000.0, -746.0, -745.0, -744.4400719213812,\n 0.0, -0.0, 709.782712893384, 709.7827128933841, 710.0, 1000.0]],\n )\n\n\ndef test_logexp_null_propagation():\n duck_check(\n \"SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e,\"\n \" log(x, 2.0) AS f, log(2.0, x) AS g FROM __THIS__\",\n {\"x\": \"float?\"},\n [{\"x\": None}, {\"x\": 4.0}],\n )\n\n\ndef test_log_two_arg_is_log10_ratio():\n # log(b,x) = log10(x)/log10(b) bit-exact. Discriminators: (10,1000)->3.0 (ln-ratio\n # gives 2.9999999999999996), (e,10)->2.302585092994046 (log2-ratio differs),\n # (e,5e-324)->-744.4400719213813 != ln(5e-324). (0.5,1.0)->-0.0 and (inf,0.5)->-0.0\n # pin sign-of-zero; (inf,inf)->nan.\n duck_check(\n \"SELECT log(b, x) AS r FROM __THIS__\",\n {\"b\": \"float\", \"x\": \"float\"},\n [{\"b\": b, \"x\": x} for b, x in [\n (2.0, 8.0), (10.0, 1000.0), (3.0, 7.0), (0.5, 8.0), (0.5, 1.0), (2.0, 1.0),\n (2.718281828459045, 5e-324), (2.718281828459045, 10.0),\n (1.0000000000000002, 4.0), (0.9999999999999999, 4.0), (5e-324, 8.0), (1e308, 8.0),\n (float(\"nan\"), 8.0), (2.0, float(\"nan\")), (float(\"nan\"), float(\"nan\")),\n (float(\"inf\"), 8.0), (float(\"inf\"), 0.5), (float(\"inf\"), float(\"inf\")),\n (float(\"inf\"), 1.0), (2.0, float(\"inf\")), (2.0, 5e-324), (2.0, 1e308),\n ]],\n )\n\n\ndef test_log_two_arg_null_preempts_domain_errors():\n # NULL in either slot wins over every domain error, including base 0 / negative / 1.\n duck_check(\n \"SELECT log(b, x) AS r FROM __THIS__\",\n {\"b\": \"float?\", \"x\": \"float?\"},\n [{\"b\": None, \"x\": 8.0}, {\"b\": 2.0, \"x\": None}, {\"b\": None, \"x\": None},\n {\"b\": None, \"x\": -4.0}, {\"b\": 0.0, \"x\": None}, {\"b\": -2.0, \"x\": None},\n {\"b\": 1.0, \"x\": None}],\n )\n\n\ndef test_logexp_integer_input_yields_double():\n duck_check(\n \"SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e,\"\n \" log(x, 8) AS f FROM __THIS__\",\n {\"x\": \"int\"},\n [{\"x\": 2}, {\"x\": 9223372036854775807}],\n )\n\n\ndef test_logexp_guarded_rows_do_not_trap():\n # A bad row aborts an unguarded query, but WHERE / CASE genuinely prevent evaluation.\n duck_check(\n \"SELECT CASE WHEN x > 0 THEN ln(x) ELSE NULL END AS a FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": 1.0}, {\"x\": -1.0}, {\"x\": 4.0}, {\"x\": 0.0}],\n )\n duck_check(\n \"SELECT ln(x) AS a FROM __THIS__ WHERE x > 0\",\n {\"x\": \"float\"},\n [{\"x\": 1.0}, {\"x\": -1.0}, {\"x\": 4.0}, {\"x\": 0.0}],\n )", + "lowering": "NEW IR OPS \u2014 not a pure desugar. Five ops: Ln, Log2, Log10, Exp (unary), LogBase (binary). All take f64 (frontend inserts int->f64 cast for INTEGER/BIGINT/etc. inputs; VARCHAR/BOOLEAN are binder errors, no implicit cast), all return DOUBLE, all strict-NULL (any NULL arg -> NULL BEFORE any domain check \u2014 pinned by log(0.0, NULL) = NULL). Frontend name mapping: ln(x)->Ln(x); log(x)->Log10(x) [1-arg log IS base 10]; log10(x)->Log10(x); log2(x)->Log2(x); exp(x)->Exp(x); log(b,x)->LogBase(b,x). log1p/expm1 do not exist -> catalog/binder error.\n\nWhy ops, not desugar: (1) the log family traps per-row with DuckDB's exact messages; (2) LogBase cannot be Div(Log10(x), Log10(b)) \u2014 the base is checked FIRST (log(-2.0, 0.0) must say \"negative number\", but left-to-right arg evaluation would raise the \"zero\" error from x first), and base==1 must raise a dedicated error where float division would just produce +/-inf.\n\nRust semantics (matches every pin):\n\n #[inline]\n fn log_domain_check(v: f64) -> Result<(), Error> {\n if v == 0.0 { return Err(oor(\"cannot take logarithm of zero\")); } // catches -0.0 too\n if v < 0.0 { return Err(oor(\"cannot take logarithm of a negative number\")); } // catches -inf; NaN passes both\n Ok(())\n }\n fn op_ln(x: f64) -> Result { log_domain_check(x)?; Ok(x.ln()) } // NaN->NaN, +inf->inf\n fn op_log2(x: f64) -> Result { log_domain_check(x)?; Ok(x.log2()) }\n fn op_log10(x: f64) -> Result { log_domain_check(x)?; Ok(x.log10()) }\n fn op_exp(x: f64) -> f64 { x.exp() } // NEVER traps: exp(1000)=inf, exp(-inf)=+0.0, exp(-745)=5e-324\n fn op_log_base(b: f64, x: f64) -> Result {\n log_domain_check(b)?; // base checked FIRST\n if b == 1.0 { return Err(oor(\"divison by zero in based logarithm\")); } // sic \u2014 DuckDB's typo, mirror it\n log_domain_check(x)?;\n Ok(x.log10() / b.log10()) // bit-exact log10 ratio (NOT ln ratio, NOT log2 ratio) \u2014 20k fuzz, 0 mismatches\n }\n\nError type mirrors duckdb.OutOfRangeException, full message = \"Out of Range Error: \" + head. Traps are per-row and LAZY: rows removed by WHERE or not taken by a CASE branch must not be evaluated (pinned). Constant-fold path is identical to vectorized \u2014 no special literal handling needed.\n\nTranscendental source: DuckDB calls the platform C++ std::log/log2/log10/exp. On this Windows box Rust's f64::ln/log2/log10/exp hit the same ucrt libm \u2014 Python's math matched DuckDB bit-exactly on 40k+ fuzz samples (0 mismatches), so plain f64 method calls are the right primitive. Cranelift has no transcendental instructions: emit calls from JIT code to the SAME extern \"C\" wrappers around these f64 methods that the interpreter uses, so both backends are bit-identical by construction. Cross-platform bit-exactness of libm is NOT guaranteed by IEEE \u2014 the differential oracle must run per-platform.", + "surprises": [ + "1-arg log(x) in DuckDB 1.5.5 is BASE 10 (bit-identical to log10, log(1e308)=308.0 exactly), not natural log \u2014 anyone assuming C/Python/numpy semantics ships a wrong transform.", + "The log family ERRORS on x<=0 (OutOfRangeException) instead of returning -inf/NaN like libm: x==0 and x==-0.0 both give 'cannot take logarithm of zero' (the zero check catches -0.0 first), any x<0 including -inf gives 'cannot take logarithm of a negative number'.", + "NaN input does NOT error \u2014 it fails both the ==0 and <0 checks and flows through libm to NaN; negative NaN too. So the domain guard is comparison-based, not is_finite-based.", + "Two-arg log(b,x) is bit-exactly log10(x)/log10(b) \u2014 NOT ln(x)/ln(b): log(10,1000)=3.0 while the ln ratio is 2.9999999999999996, and log(e,x) differs from ln(x) in the last ulp (log(e,5e-324)=-744.4400719213813 vs ln=-744.4400719213812). Fuzz: ln-ratio wrong on 35% of pairs, log2-ratio on 46%.", + "In two-arg log the BASE is domain-checked before x: log(-2.0, 0.0) raises the 'negative number' error, not 'zero'. And base==1.0 raises a dedicated error whose message contains DuckDB's typo verbatim: 'divison by zero in based logarithm' \u2014 the mirror must reproduce the typo.", + "NULL propagation pre-empts every domain check: log(0.0, NULL), log(-2.0, NULL), log(1.0, NULL) are all NULL, not errors.", + "exp NEVER raises: exp(1000)=inf (no overflow error), exp(-inf)=+0.0, exp(-745)=5e-324 (underflow passes through denormals), exp(709.782712893384)=1.7976931348622732e+308 just under f64::MAX.", + "log(0.5, 1.0) = -0.0 (negative zero) while log(2.0, 1.0) = +0.0, and log(inf, 0.5) = -0.0 \u2014 the division lowering leaks sign-of-zero, so the engine must preserve it (repr-level compare catches this).", + "log1p and expm1 do not exist in DuckDB 1.5.5 (CatalogException).", + "A single domain-violating row aborts the whole vectorized query, but WHERE filters and untaken CASE branches genuinely prevent evaluation (no trap) \u2014 the engine must evaluate lazily/masked, never eagerly over the full column.", + "No literal-vs-vectorized divergence exists in this family: constant folding produces the same values and the same OutOfRangeException messages.", + "All numeric input types (TINYINT..HUGEINT, FLOAT, DECIMAL) produce DOUBLE; VARCHAR and BOOLEAN are binder errors (no implicit cast), so the frontend must reject them at build time." + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave1/pins_pow-sqrt.json b/docs/superpowers/specs/pins-wave1/pins_pow-sqrt.json new file mode 100644 index 0000000..9efa4a8 --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_pow-sqrt.json @@ -0,0 +1,494 @@ +{ + "family": "pow-sqrt", + "pins": [ + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.0, 0.0)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "pow(0,0)=1, no error" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.0, -1.0)", + "result": "inf", + "result_type": "DOUBLE", + "note": "NO div-by-zero error; IEEE pow(+0, neg-odd) = +inf" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-0.0, -1.0)", + "result": "-inf", + "result_type": "DOUBLE", + "note": "sign of zero flows: pow(-0, neg-odd-int) = -inf" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.0, -2.0)", + "result": "inf", + "result_type": "DOUBLE", + "note": "pow(+0, neg-even) = +inf" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-0.0, -2.0)", + "result": "inf", + "result_type": "DOUBLE", + "note": "pow(-0, neg-even-int) = +inf" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-0.0, 3.0)", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "pow(-0, pos-odd-int) = -0.0 \u2014 negative zero result" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-0.0, 2.0)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "pow(-0, pos-even-int) = +0.0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-0.0, 0.5)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "pow(-0, non-integer pos) = +0.0 (contrast sqrt(-0.0) = -0.0)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-8.0, 0.3333333333333333)", + "result": "nan", + "result_type": "DOUBLE", + "note": "negative base + non-integer exponent = NaN, NOT an error; 1/3 in DuckDB is DOUBLE 0.3333333333333333 (int / is float division)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-2.0, 0.5)", + "result": "nan", + "result_type": "DOUBLE", + "note": "quiet NaN while sqrt(-2.0) would TRAP \u2014 pow(x,0.5) is NOT sqrt(x)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-2.0, -0.5)", + "result": "nan", + "result_type": "DOUBLE", + "note": "negative base, non-integer negative exponent = NaN" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (nan, 0.0) and (nan, -0.0)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "IEEE: pow(NaN, \u00b10) = 1 \u2014 NaN does NOT propagate" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (1.0, nan)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "IEEE: pow(1, NaN) = 1 \u2014 NaN does NOT propagate" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (1.0, inf)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "pow(1, anything) = 1" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (nan, nan) / (nan, 1.0) / (0.0, nan)", + "result": "nan", + "result_type": "DOUBLE", + "note": "NaN propagates in every case except y=\u00b10 or x=1" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (inf, 0.0) and (-inf, 0.0)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "pow(\u00b1inf, 0) = 1" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-1.0, inf) and (-1.0, -inf)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "IEEE: pow(-1, \u00b1inf) = 1" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (inf, -1.0)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "+0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-inf, -1.0)", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "pow(-inf, neg-odd-int) = -0.0 \u2014 negative zero" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-inf, -2.0)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "pow(-inf, neg-even) = +0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-inf, 3.0)", + "result": "-inf", + "result_type": "DOUBLE", + "note": "odd integer exponent keeps sign" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-inf, 2.0) and (-inf, 0.5)", + "result": "inf", + "result_type": "DOUBLE", + "note": "pow(-inf, pos non-odd) = +inf (0.5 too \u2014 no NaN for inf base)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.0, inf) and (-0.0, inf)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "+0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.0, -inf) and (-0.0, -inf)", + "result": "inf", + "result_type": "DOUBLE", + "note": "+inf, no error" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (2.0, inf) / (0.5, -inf)", + "result": "inf", + "result_type": "DOUBLE", + "note": "|x|>1 with +inf exp, |x|<1 with -inf exp" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (0.5, inf) / (2.0, -inf) / (-0.5, inf) / (-2.0, -inf)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "always +0.0 \u2014 negative base with inf exponent is NOT NaN" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (1e308, 2.0) and (2.0, 1024.0)", + "result": "inf", + "result_type": "DOUBLE", + "note": "overflow -> inf, NO error (unlike integer arithmetic overflow elsewhere)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (2.0, -1075.0)", + "result": "0.0", + "result_type": "DOUBLE", + "note": "underflow -> +0.0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-2.0, -1075.0)", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "underflow with odd-int exponent -> -0.0" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (2.0, 53.0)", + "result": "9007199254740992.0", + "result_type": "DOUBLE", + "note": "exact 2^53" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (10.0, 18.0)", + "result": "1e+18", + "result_type": "DOUBLE", + "note": "exact" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (5e-324, 0.5)", + "result": "2.2227587494850775e-162", + "result_type": "DOUBLE", + "note": "denormal input handled; equals sqrt(5e-324)" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols (-2.0, 3.0)", + "result": "-8.0", + "result_type": "DOUBLE", + "note": "negative base with exact integer exponent works" + }, + { + "expr": "pow(x,y)", + "input_desc": "DOUBLE cols, NULL in x / y / both", + "result": "None", + "result_type": "DOUBLE", + "note": "plain NULL propagation in every argument position; typeof still DOUBLE" + }, + { + "expr": "power(x,y)", + "input_desc": "DOUBLE cols (0,0), (-8,1/3), (nan,0)", + "result": "1.0 / nan / 1.0", + "result_type": "DOUBLE", + "note": "power === pow, identical on all spot checks" + }, + { + "expr": "x ^ y", + "input_desc": "DOUBLE cols (0,0)/(nan,0)/(0,-1)/(1e308,2); NULL either side", + "result": "1.0 / 1.0 / inf / inf; None", + "result_type": "DOUBLE", + "note": "^ operator === pow, identical on all spot checks" + }, + { + "expr": "x ** y", + "input_desc": "DOUBLE cols (0,0)/(nan,0)/(-8,1/3)", + "result": "1.0 / 1.0 / nan", + "result_type": "DOUBLE", + "note": "** operator === pow" + }, + { + "expr": "pow(x,y)", + "input_desc": "BIGINT cols (0,0)/(2,10)/(0,-1)/(-8,2)/(10,18)", + "result": "1.0 / 1024.0 / inf / 64.0 / 1e+18", + "result_type": "DOUBLE", + "note": "integer inputs -> DOUBLE result always; pow(0,-1) on ints is still inf, no error" + }, + { + "expr": "pow(x,y)", + "input_desc": "BIGINT cols (9223372036854775807, 1)", + "result": "9.223372036854776e+18", + "result_type": "DOUBLE", + "note": "i64::MAX cast to double (rounds UP to 2^63) before compute" + }, + { + "expr": "pow(x,y)", + "input_desc": "BIGINT cols (-9223372036854775808, 1)", + "result": "-9.223372036854776e+18", + "result_type": "DOUBLE", + "note": "i64::MIN -> exact -2^63 as double" + }, + { + "expr": "pow(x,y)", + "input_desc": "BIGINT cols (2,63) / (2,64) / (3,40)", + "result": "9.223372036854776e+18 / 1.8446744073709552e+19 / 1.2157665459056929e+19", + "result_type": "DOUBLE", + "note": "no i64 overflow trap ever \u2014 computed entirely in double domain; pow(3,40) != exact 3^40=12157665459056928801, it is the correctly-rounded double. Canary pin for cross-libm ulp drift." + }, + { + "expr": "pow(x,y)", + "input_desc": "BIGINT cols (9007199254740993, 1) i.e. 2^53+1", + "result": "9007199254740992.0", + "result_type": "DOUBLE", + "note": "BIGINT->DOUBLE cast (round-half-even) happens FIRST; the +1 is lost before pow runs" + }, + { + "expr": "x ^ y", + "input_desc": "BIGINT cols (5, 2)", + "result": "25.0", + "result_type": "DOUBLE", + "note": "^ is POWER not XOR (xor(5,2)=7 BIGINT is the xor); result DOUBLE even for int operands" + }, + { + "expr": "pow(x,y)", + "input_desc": "INTEGER cols (2,10); mixed BIGINT,DOUBLE and DOUBLE,BIGINT", + "result": "1024.0 / nan(-8,1/3) / 6.25(2.5,2)", + "result_type": "DOUBLE", + "note": "every numeric overload promotes to DOUBLE" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col 4.0 / 2.0 / 0.0", + "result": "2.0 / 1.4142135623730951 / 0.0", + "result_type": "DOUBLE", + "note": "plain IEEE sqrt" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col -0.0", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "IEEE: sqrt(-0.0) = -0.0, NO trap \u2014 the negative guard is strictly x < 0" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col nan", + "result": "nan", + "result_type": "DOUBLE", + "note": "NaN passes the guard (NaN < 0 is false), no trap" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col inf", + "result": "inf", + "result_type": "DOUBLE", + "note": "+inf passes" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col NULL; BIGINT col NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "NULL propagates, no trap" + }, + { + "expr": "sqrt(x)", + "input_desc": "DOUBLE col 5e-324 (denormal) / 1e308", + "result": "2.2227587494850775e-162 / 1e+154", + "result_type": "DOUBLE", + "note": "denormals fine" + }, + { + "expr": "sqrt(x)", + "input_desc": "BIGINT col 4 / 9223372036854775807", + "result": "2.0 / 3037000499.97605", + "result_type": "DOUBLE", + "note": "int -> DOUBLE result; i64::MAX goes through double cast first (matches libm sqrt(9.223372036854776e18))" + }, + { + "expr": "sqrt(x)", + "input_desc": "BIGINT cols 9007199254740992 and 9007199254740993 (2^53, 2^53+1)", + "result": "94906265.62425156 (identical for both)", + "result_type": "DOUBLE", + "note": "proves lossy BIGINT->DOUBLE cast precedes sqrt" + }, + { + "expr": "cbrt(x)", + "input_desc": "DOUBLE col 8.0 / -8.0 / 27.0 / 2.0", + "result": "2.0 / -2.0 / 3.0 / 1.259921049894873", + "result_type": "DOUBLE", + "note": "cbrt EXISTS; handles negative base exactly (unlike pow(-8,1/3)=nan)" + }, + { + "expr": "cbrt(x)", + "input_desc": "DOUBLE col 0.0 / -0.0 / inf / -inf / nan / NULL", + "result": "0.0 / -0.0 / inf / -inf / nan / None", + "result_type": "DOUBLE", + "note": "total function, sign-preserving, no traps anywhere" + }, + { + "expr": "cbrt(x)", + "input_desc": "BIGINT col 27 / 9223372036854775807; DOUBLE 5e-324", + "result": "3.0 / 2097152.0 / 1.7031839360032603e-108", + "result_type": "DOUBLE", + "note": "int -> DOUBLE; cbrt(double(i64::MAX)) = cbrt(2^63) = 2^21 exactly" + }, + { + "expr": "-2^2 (literal) and -x^2 (x=2.0 DOUBLE col; also BIGINT col; also -x**2)", + "input_desc": "precedence probe", + "result": "4.0", + "result_type": "DOUBLE", + "note": "unary minus binds TIGHTER than ^ and **: parsed (-2)^2. Opposite of Python -2**2=-4. PostgreSQL rule." + }, + { + "expr": "2^3^2 (literal), 2**3**2, x^y^z cols (2,3,2)", + "input_desc": "associativity probe", + "result": "64.0", + "result_type": "DOUBLE", + "note": "LEFT-associative: (2^3)^2=64, not 2^(3^2)=512. Opposite of Python **." + }, + { + "expr": "2*3^2 / 2^3*2 / 2+3^2 (literals), 2*x^2 col x=3.0", + "input_desc": "precedence vs * and +", + "result": "18.0 / 16.0 / 11.0 / 18.0", + "result_type": "DOUBLE", + "note": "^ binds tighter than * and +" + }, + { + "expr": "2 ^ -2, 2^ -2, 2^(-2), x ^ -y, x^(-y), 2**-2, 2 **- 2, x ** -y", + "input_desc": "spaced/parenthesized negative exponent", + "result": "0.25", + "result_type": "DOUBLE", + "note": "all fine \u2014 only '^' IMMEDIATELY followed by '-' fails to lex (see errors)" + }, + { + "expr": "sqrt('-0.0'::DOUBLE) literal (constant-fold)", + "input_desc": "fold-vs-vectorized divergence check", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "fold agrees with column path; NO fold divergence found anywhere in this family. Beware: bare literal -0.0 is DECIMAL(2,1) and casts to +0.0 \u2014 it never carries the zero sign." + }, + { + "expr": "SELECT sqrt(x) FROM t WHERE x >= 0 (rows 4.0, -1.0); CASE WHEN x>=0 THEN sqrt(x) END", + "input_desc": "lazy-evaluation probe", + "result": "[2.0] / [2.0, None]", + "result_type": "DOUBLE", + "note": "rows removed by WHERE and untaken CASE branches are never evaluated \u2014 the sqrt trap must be lazy, not eager-columnar" + } + ], + "errors": [ + { + "expr": "SELECT sqrt(x) FROM t -- any x < 0: -1.0, -1e-300, -inf, BIGINT -1 (column path; one bad row fails the whole query even if other rows are fine)", + "error_head": "Out of Range Error: cannot take square root of a negative number", + "kind": "runtime" + }, + { + "expr": "SELECT sqrt(-1.0) -- literal form: PREPARE succeeds, error at EXECUTE (constant-fold is execution-time)", + "error_head": "Out of Range Error: cannot take square root of a negative number", + "kind": "runtime" + }, + { + "expr": "SELECT 2^-2 -- also -2^-2, 2 ^- 2, x^-y: '^' immediately followed by '-' lexes as one operator '^-'; raised at PREPARE (duckdb.CatalogException)", + "error_head": "Catalog Error: Scalar Function with name ^- does not exist! Did you mean \"^\"?", + "kind": "binder" + }, + { + "expr": "SELECT sqrt(x) FROM t -- x VARCHAR column (no implicit VARCHAR->DOUBLE for columns)", + "error_head": "Binder Error: No function matches the given name and argument types 'sqrt(VARCHAR)'", + "kind": "binder" + }, + { + "expr": "SELECT x^y FROM t -- VARCHAR ^ DOUBLE columns", + "error_head": "Binder Error: No function matches the given name and argument types '^(VARCHAR, DOUBLE)'", + "kind": "binder" + }, + { + "expr": "SELECT pow(x,y) FROM t -- BOOLEAN columns", + "error_head": "Binder Error: No function matches the given name and argument types 'pow(BOOLEAN, BOOLEAN)'", + "kind": "binder" + }, + { + "expr": "SELECT pow('a', 2) -- string LITERAL does implicitly cast (unlike VARCHAR columns); PREPARE succeeds, fails at EXECUTE", + "error_head": "Conversion Error: Could not convert string 'a' to DOUBLE", + "kind": "runtime" + }, + { + "expr": "SELECT pow(2) -- wrong arity", + "error_head": "Binder Error: No function matches the given name and argument types 'pow(INTEGER_LITERAL)'", + "kind": "binder" + }, + { + "expr": "SELECT sqrt(1, 2) -- wrong arity", + "error_head": "Binder Error: No function matches the given name and argument types 'sqrt(INTEGER_LITERAL, INTEGER_LITERAL)'", + "kind": "binder" + } + ], + "duck_check_tests": "NAN = float(\"nan\")\nINF = float(\"inf\")\n\nPOW_SPECIALS = [\n (0.0, 0.0), (0.0, -1.0), (-0.0, -1.0), (0.0, -2.0), (-0.0, -2.0),\n (-0.0, 3.0), (-0.0, 2.0), (-0.0, 0.5), (-8.0, 1.0 / 3.0), (-2.0, 0.5),\n (-2.0, -0.5), (NAN, 0.0), (NAN, -0.0), (1.0, NAN), (NAN, NAN),\n (NAN, 1.0), (0.0, NAN), (INF, 0.0), (-INF, 0.0), (-1.0, INF),\n (-1.0, -INF), (1.0, INF), (INF, -1.0), (-INF, -1.0), (-INF, -2.0),\n (-INF, 3.0), (-INF, 2.0), (-INF, 0.5), (0.0, INF), (-0.0, INF),\n (0.0, -INF), (-0.0, -INF), (2.0, INF), (0.5, INF), (2.0, -INF),\n (0.5, -INF), (-0.5, INF), (-2.0, -INF), (1e308, 2.0), (2.0, 1024.0),\n (2.0, -1075.0), (-2.0, -1075.0), (2.0, 53.0), (10.0, 18.0),\n (5e-324, 0.5), (-2.0, 3.0), (None, 2.0), (2.0, None), (None, None),\n]\n\n\ndef test_pow_double_specials():\n # pow / power / ^ / ** are one function: pin all four spellings per row.\n duck_check(\n \"SELECT pow(x, y) AS p, power(x, y) AS q, x ^ y AS r, x ** y AS s FROM __THIS__\",\n {\"x\": \"float?\", \"y\": \"float?\"},\n [{\"x\": x, \"y\": y} for x, y in POW_SPECIALS],\n )\n\n\ndef test_pow_bigint_inputs_yield_double():\n # ints promote to DOUBLE before compute: i64::MAX rounds to 2^63,\n # 2^53+1 loses the +1, 3^40 is the rounded double not the exact int.\n rows = [(0, 0), (2, 10), (0, -1), (-8, 2), (10, 18),\n (9223372036854775807, 1), (-9223372036854775808, 1),\n (2, 63), (2, 64), (3, 40), (9007199254740993, 1), (5, 2)]\n duck_check(\n \"SELECT pow(x, y) AS p, x ^ y AS r FROM __THIS__\", # x^y is pow, never xor\n {\"x\": \"int\", \"y\": \"int\"},\n [{\"x\": x, \"y\": y} for x, y in rows],\n )\n\n\ndef test_sqrt_non_negative_domain():\n # NOTE: any x < 0 (incl -inf) raises OutOfRange; keep those out of duck_check.\n xs = [4.0, 2.0, 0.0, -0.0, INF, NAN, None, 5e-324, 1e308]\n duck_check(\"SELECT sqrt(x) AS s FROM __THIS__\", {\"x\": \"float?\"}, [{\"x\": x} for x in xs])\n\n\ndef test_sqrt_cbrt_bigint():\n xs = [4, 0, 27, 9223372036854775807, 9007199254740992, 9007199254740993, None]\n duck_check(\n \"SELECT sqrt(x) AS s, cbrt(x) AS c FROM __THIS__\",\n {\"x\": \"int?\"},\n [{\"x\": x} for x in xs],\n )\n\n\ndef test_cbrt_total_function():\n # cbrt has NO domain restriction: negatives, -inf, -0.0 all flow through.\n xs = [8.0, -8.0, 27.0, 2.0, 0.0, -0.0, INF, -INF, NAN, None, 5e-324, -1.0]\n duck_check(\"SELECT cbrt(x) AS c FROM __THIS__\", {\"x\": \"float?\"}, [{\"x\": x} for x in xs])\n\n\ndef test_pow_operator_precedence_vectorized():\n # -x^2 == (-x)^2 (unary minus binds tighter), x^y^z left-assoc, ^ over *.\n rows = [{\"x\": 2.0, \"y\": 3.0, \"z\": 2.0}, {\"x\": 3.0, \"y\": 2.0, \"z\": 0.5}]\n duck_check(\n \"SELECT -x ^ 2 AS neg_first, x ^ y ^ z AS left_assoc, 2 * x ^ y AS pow_over_mul, \"\n \"-x ** 2 AS neg_first2, x ** y ** z AS left_assoc2, x ^ -y AS spaced_neg_exp \"\n \"FROM __THIS__\",\n {\"x\": \"float\", \"y\": \"float\", \"z\": \"float\"},\n rows,\n )\n\n\ndef test_sqrt_where_guard_is_lazy():\n # Filtered-out negative rows must NOT trap.\n duck_check(\n \"SELECT sqrt(x) AS s FROM __THIS__ WHERE x >= 0\",\n {\"x\": \"float\"},\n [{\"x\": 4.0}, {\"x\": -1.0}],\n )\n duck_check(\n \"SELECT CASE WHEN x >= 0 THEN sqrt(x) ELSE NULL END AS s FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": 4.0}, {\"x\": -1.0}],\n )\n\n\ndef test_sqrt_negative_is_runtime_error():\n # Not expressible as a duck_check equality: both engines must RAISE.\n # DuckDB: _duckdb.OutOfRangeException\n # 'Out of Range Error: cannot take square root of a negative number'\n with pytest.raises(Exception, match=\"cannot take square root of a negative number\"):\n duck_check(\"SELECT sqrt(x) AS s FROM __THIS__\", {\"x\": \"float\"}, [{\"x\": -1.0}])\n with pytest.raises(Exception, match=\"cannot take square root of a negative number\"):\n duck_check(\"SELECT sqrt(x) AS s FROM __THIS__\", {\"x\": \"float\"}, [{\"x\": float(\"-inf\")}])", + "lowering": "Three IR ops; pow's four spellings are a pure frontend desugar onto one op.\n\nFRONTEND DESUGAR (proven exact by pins): power(x,y), x ^ y, x ** y ==> pow(x,y). All four spellings returned bit-identical results and DOUBLE type on every probed special (including NaN/inf/zero-sign cases), and all have plain NULL-propagation (NULL in either arg -> NULL), so three-valued logic is untouched \u2014 it is the standard binary-scalar NULL rule, no CASE rewriting needed. All numeric inputs (BOOLEAN excluded \u2014 binder error) are cast to f64 BEFORE the op: BIGINT->DOUBLE is round-half-even (pins: i64::MAX -> 9.223372036854776e18, 2^53+1 -> 2^53). There is NO integer pow overload, hence NO i64-overflow trap anywhere in this family.\n\nIR OP 1 \u2014 PowF64(a: f64, b: f64) -> f64, total, never traps:\n fn pow_f64(a: f64, b: f64) -> f64 { a.powf(b) }\nRust f64::powf is IEEE-754 pow and matched all 46 measured pins bit-for-bit semantics: pow(x,\u00b10)=1 for ALL x incl NaN/inf; pow(1,y)=1 incl NaN; pow(\u00b10,neg)=\u00b1inf with odd-integer sign rule (NO div-by-zero trap); neg base + non-integer exp -> NaN (NO trap); overflow -> inf; underflow -> \u00b10 with odd-int sign. Interpreter: f64::powf. Cranelift: no pow instruction \u2014 emit a call to the same extern \"C\" helper wrapping f64::powf so both backends share bits. Caveat: IEEE does not guarantee correct rounding for pow's ordinary values; DuckDB uses its toolchain's std::pow and Rust powf uses platform libm. Measured agreement on this machine for the mantissa canaries (pow(3,40)=1.2157665459056929e+19, sqrt(double(i64::MAX))=3037000499.97605); keep those pins as the cross-libm drift alarm.\n\nIR OP 2 \u2014 SqrtChecked(a: f64) -> f64, TRAPS:\n fn sqrt_checked(x: f64) -> Result {\n if x < 0.0 { // false for NaN, false for -0.0 \u2014 exactly DuckDB's guard\n return Err(Trap::OutOfRange(\"cannot take square root of a negative number\"));\n }\n Ok(x.sqrt()) // sqrt(-0.0) = -0.0, sqrt(NaN)=NaN, sqrt(inf)=inf per IEEE\n }\nEngine must surface it with DuckDB's exact head \"Out of Range Error: cannot take square root of a negative number\" (duckdb.OutOfRangeException, runtime kind \u2014 PREPARE of sqrt(-1.0) succeeds, EXECUTE raises). Cranelift: guard branch to trap, then native fsqrt (IEEE requires sqrt correctly rounded, so hardware fsqrt is bit-exact with libm). TRAP LAZINESS IS OBSERVABLE: WHERE-filtered rows and untaken CASE branches never evaluate (measured) \u2014 an eager columnar sqrt over the unfiltered vector would diverge on [4.0, -1.0] WHERE x>=0.\nDo NOT desugar sqrt(x) to pow(x, 0.5): pow(-4,0.5)=NaN but sqrt(-4) traps; pow(-0.0,0.5)=+0.0 but sqrt(-0.0)=-0.0.\n\nIR OP 3 \u2014 CbrtF64(a: f64) -> f64, total, never traps:\n fn cbrt_f64(x: f64) -> f64 { x.cbrt() }\nSign-preserving everywhere: cbrt(-8)=-2, cbrt(\u00b10)=\u00b10, cbrt(\u00b1inf)=\u00b1inf, cbrt(NaN)=NaN. Cranelift: helper call (no instruction). Do NOT desugar to pow(x, 1/3): pow(-8, 1/3)=NaN but cbrt(-8)=-2, and 1/3 is inexact anyway.\n\nPARSER CONTRACT the frontend must mirror (validate sqlparser-rs explicitly against these \u2014 PG rules, not Python rules): (1) unary minus binds TIGHTER than ^/**: -2^2 = -x^2|x=2 = +4.0; (2) ^ and ** are LEFT-associative: 2^3^2 = 64; (3) ^ binds tighter than * and +: 2*3^2=18, 2+3^2=11; (4) '^' immediately followed by '-' ('2^-2', 'x^-y', '2 ^- 2') lexes as one unknown operator '^-' -> must be rejected at bind (\"Catalog Error: Scalar Function with name ^- does not exist! ...\"), while '2 ^ -2', '2^(-2)' = 0.25 are fine and '**' is immune ('2**-2' and '2 **- 2' both parse). If sqlparser-rs happily parses '2^-2' or gives ** right-associativity, the frontend must reject/re-associate or the backends diverge from the oracle.", + "surprises": [ + "DuckDB's ^ operator is POWER, not XOR (5^2 = 25.0 DOUBLE); xor is only the xor() function (xor(5,2)=7 BIGINT). Any engine treating ^ as bitwise xor diverges on every query.", + "Unary minus binds TIGHTER than ^ and **: -2^2 = +4.0 and -x^2 (col) = +4.0 \u2014 the PostgreSQL rule, opposite of Python where -2**2 = -4. Confirmed on both literal and vectorized paths.", + "^ and ** are LEFT-associative: 2^3^2 = 64.0 (Python's ** is right-associative and would give 512).", + "'2^-2' is a bind-time CatalogException \u2014 the PG-style lexer greedily consumes '^-' as one operator ('Scalar Function with name ^- does not exist'). '2 ^ -2' = 0.25 works, and '**' is immune ('2**-2' and even '2 **- 2' parse fine). Raised at PREPARE, i.e. binder kind.", + "sqrt of ANY negative (including -1e-300, -inf, and BIGINT -1) is a per-row RUNTIME OutOfRangeException 'cannot take square root of a negative number' \u2014 but pow(negative, 0.5) quietly returns NaN. The guard is exactly x < 0.0: sqrt(-0.0) = -0.0 (passes, keeps the sign) and sqrt(NaN) = NaN (passes).", + "pow NEVER errors on any input: pow(0,-1)=inf (no div-by-zero), overflow (1e308^2)=inf, underflow=\u00b10, negative base + fractional exponent = NaN. IEEE oddities hold exactly: pow(NaN,0)=1, pow(1,NaN)=1, pow(-1,\u00b1inf)=1, pow(-0.0,-1)=-inf, pow(-inf,-1)=-0.0.", + "Every overload returns DOUBLE \u2014 there is no integer pow, so no i64-overflow trap exists; BIGINT operands are cast to DOUBLE FIRST (pow(2^53+1, 1) = 9007199254740992.0 silently loses the +1; sqrt(2^53) == sqrt(2^53+1)).", + "cbrt exists and is total: cbrt(-8) = -2.0 exactly (while pow(-8, 1.0/3.0) = NaN), cbrt(-inf) = -inf, cbrt(-0.0) = -0.0 \u2014 cbrt is NOT pow(x, 1/3).", + "The bare literal -0.0 is DECIMAL(2,1) and becomes +0.0 as DOUBLE \u2014 it cannot carry the zero sign; only '-0.0'::DOUBLE or a real column value does. Once that red herring is removed, constant-fold and vectorized paths AGREE on every probe in this family (no fold divergence found).", + "String LITERALS implicitly cast into pow (pow('a',2) -> ConversionException at EXECUTE, and PREPARE succeeds \u2014 runtime kind), but VARCHAR COLUMNS do not (bind-time 'No function matches sqrt(VARCHAR)' / '^(VARCHAR, DOUBLE)'). Literal-vs-column asymmetry in the type system.", + "sqrt's trap is observably LAZY: SELECT sqrt(x) FROM t WHERE x >= 0 over rows [4.0, -1.0] returns [2.0] with no error, and an untaken CASE branch never traps \u2014 an eager columnar evaluation over pre-filter vectors would diverge from the oracle." + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave1/pins_predicates-between-not-b.json b/docs/superpowers/specs/pins-wave1/pins_predicates-between-not-b.json new file mode 100644 index 0000000..33e4aeb --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_predicates-between-not-b.json @@ -0,0 +1,358 @@ +{ + "family": "predicates: BETWEEN / NOT BETWEEN, IN / NOT IN, least(), greatest()", + "pins": [ + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "BIGINT cols (x,lo,hi)=(5,1,10)", + "result": "True", + "result_type": "BOOLEAN", + "note": "plain in-range; result type BOOLEAN in all BETWEEN/IN pins" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(5,10,1) \u2014 lo > hi", + "result": "False", + "result_type": "BOOLEAN", + "note": "empty range is FALSE, not NULL (falls out of K3 AND: 5>=10 FALSE)" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(NULL,1,10) and (NULL,10,1)", + "result": "None (both)", + "result_type": "BOOLEAN", + "note": "NULL x -> NULL even when the range is empty; NO empty-range shortcut to FALSE" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(5,NULL,10)", + "result": "None", + "result_type": "BOOLEAN", + "note": "NULL lo with satisfiable hi -> NULL (NULL AND TRUE)" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(5,NULL,4)", + "result": "False", + "result_type": "BOOLEAN", + "note": "NULL lo but hi fails -> FALSE (NULL AND FALSE = FALSE); the failing half decides" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(5,1,NULL) / (5,6,NULL)", + "result": "None / False", + "result_type": "BOOLEAN", + "note": "symmetric on the hi side; (NULL,NULL,NULL) -> None" + }, + { + "expr": "x BETWEEN lo AND hi", + "input_desc": "(5,5,5)", + "result": "True", + "result_type": "BOOLEAN", + "note": "both endpoints inclusive" + }, + { + "expr": "x NOT BETWEEN lo AND hi", + "input_desc": "same 12-row BIGINT table", + "result": "exact K3 NOT of BETWEEN: True<->False, None stays None", + "result_type": "BOOLEAN", + "note": "verified row-for-row; e.g. (5,NULL,4) -> True, (NULL,10,1) -> None" + }, + { + "expr": "xi BETWEEN lod AND hid", + "input_desc": "BIGINT x=9007199254740993 (2^53+1), DOUBLE bounds both 9007199254740992.0", + "result": "True", + "result_type": "BOOLEAN", + "note": "BIGINT-vs-DOUBLE comparison casts the int to double (lossy at 2^53): xi = lod is True, xi > lod is False" + }, + { + "expr": "x = 9007199254740992.0 (decimal literal)", + "input_desc": "BIGINT col x=9007199254740993", + "result": "False (and x > lit is True)", + "result_type": "BOOLEAN", + "note": "bare float-looking literal is DECIMAL(17,1); BIGINT-vs-DECIMAL compares EXACTLY. Same expr with ::DOUBLE cast gives True. v0 collapses decimal literals to f64 -> known divergence beyond 2^53" + }, + { + "expr": "x BETWEEN 1e0 AND 9007199254740992.0", + "input_desc": "BIGINT col x=9007199254740993 (1e0 is DOUBLE, other bound DECIMAL)", + "result": "True", + "result_type": "BOOLEAN", + "note": "WHOLE-TRIPLE unification: one common type across x,lo,hi (DOUBLE wins), NOT pairwise \u2014 pairwise exact-decimal on hi would give False" + }, + { + "expr": "x BETWEEN lo AND hi, NaN edges", + "input_desc": "DOUBLE cols, native table: (NaN,1,2) / (5,1,NaN) / (5,NaN,10) / (NaN,NaN,NaN) / (inf,1,NaN)", + "result": "False / True / False / True / True", + "result_type": "BOOLEAN", + "note": "DuckDB DOUBLE order: NaN = NaN and NaN sorts above +inf; literal (fold) path measured identical" + }, + { + "expr": "x BETWEEN lo AND hi, NaN x NULL bound", + "input_desc": "(NaN, NULL, 1.0) / (NaN, 1.0, NULL) / (-inf, NULL, 1.0)", + "result": "False / None / None", + "result_type": "BOOLEAN", + "note": "proves the desugar's K3 AND: NaN<=1.0 is FALSE so NULL AND FALSE = FALSE; NaN>=1.0 TRUE so TRUE AND NULL = NULL" + }, + { + "expr": "x BETWEEN lo AND hi, signed zeros", + "input_desc": "(0.0,-0.0,-0.0) and (-0.0,0.0,0.0) via '-0.0'::DOUBLE", + "result": "True / True", + "result_type": "BOOLEAN", + "note": "-0.0 = 0.0 in comparisons; (-inf,-inf,inf) and (inf,-inf,inf) both True" + }, + { + "expr": "x BETWEEN lo AND hi on VARCHAR", + "input_desc": "('b','a','c') / ('B','a','c') / ('Z','a','z') / ('\u00e9','a','z') / ('','','a') / ('a','c','a')", + "result": "True / False / False / False / True / False", + "result_type": "BOOLEAN", + "note": "raw byte order (memcmp): 'B'(0x42)<'a'(0x61), '\u00e9'(0xC3A9)>'z'; empty string sorts lowest" + }, + { + "expr": "x BETWEEN '1' AND '2'", + "input_desc": "BIGINT col x in {1,2}", + "result": "True / True", + "result_type": "BOOLEAN", + "note": "numeric-string LITERAL bounds cast to INT64; non-numeric literal 'a' is a runtime ConversionException; VARCHAR COLUMN bounds are a binder error instead" + }, + { + "expr": "x IN (1, NULL)", + "input_desc": "BIGINT col x in {1, 3, NULL}", + "result": "True / None / None", + "result_type": "BOOLEAN", + "note": "match wins over NULL element; miss + NULL element -> NULL; NULL x -> NULL" + }, + { + "expr": "x NOT IN (1, NULL)", + "input_desc": "same rows", + "result": "False / None / None", + "result_type": "BOOLEAN", + "note": "the classic footgun: NOT IN with any NULL element can never be True" + }, + { + "expr": "x IN (1, 2) / x NOT IN (1, 2)", + "input_desc": "x in {1,3,NULL}", + "result": "True,False,None / False,True,None", + "result_type": "BOOLEAN", + "note": "no NULL element: plain three-valued equality chain" + }, + { + "expr": "x IN (NULL)", + "input_desc": "x in {1,3,NULL}", + "result": "None for every row", + "result_type": "BOOLEAN", + "note": "single NULL element -> always NULL" + }, + { + "expr": "x IN (y, z)", + "input_desc": "elements from BIGINT COLUMNS: (1,1,NULL)/(3,1,NULL)/(3,1,2)/(NULL,1,2); also NOT IN", + "result": "True/None/False/None; NOT IN exact K3 NOT", + "result_type": "BOOLEAN", + "note": "measured identical to NOT(x=y OR x=z) row-for-row \u2014 IN is exactly the OR-chain" + }, + { + "expr": "x IN (1.0) / x IN (0.5, 1.0) / x IN (1.5)", + "input_desc": "BIGINT col x in {1,2,3,NULL}", + "result": "x=1: True/True/False; x=2: False/False/False; x=3: all False; NULL: None", + "result_type": "BOOLEAN", + "note": "decimal element compared exactly (1.5 does NOT round to 2); small values agree with f64 semantics" + }, + { + "expr": "x IN (9007199254740992.0) vs x IN (9007199254740992.0::DOUBLE)", + "input_desc": "BIGINT col x=9007199254740993", + "result": "False vs True", + "result_type": "BOOLEAN", + "note": "all-DECIMAL list compares exactly; a DOUBLE element flips to lossy int->double. v0-f64 gives True for both (divergence only in the all-decimal >2^53 case)" + }, + { + "expr": "x IN (9007199254740992.0, 1e0)", + "input_desc": "BIGINT col x=9007199254740993", + "result": "True", + "result_type": "BOOLEAN", + "note": "WHOLE-LIST unification: one DOUBLE element drags the whole list (and x) to DOUBLE" + }, + { + "expr": "x IN ('1')", + "input_desc": "BIGINT col x in {1,2}", + "result": "True / False", + "result_type": "BOOLEAN", + "note": "numeric-string literal element casts to INT64 (the column's side wins over VARCHAR)" + }, + { + "expr": "s IN (1)", + "input_desc": "VARCHAR col, rows '1' and 'a'", + "result": "row '1' alone: True; any 'a' row: ConversionException", + "result_type": "BOOLEAN", + "note": "VARCHAR loses unification to INTEGER: the COLUMN is cast to INT32, per-row runtime error on non-numeric strings; s IN ('a', 1) behaves the same (whole list goes INT32)" + }, + { + "expr": "s IN ('a','b') / s IN ('b', NULL)", + "input_desc": "VARCHAR col s in {'a','A',NULL}", + "result": "True,False,None / None,None,None", + "result_type": "BOOLEAN", + "note": "byte-exact string equality; case-sensitive" + }, + { + "expr": "x IN ('NaN'::DOUBLE) / x IN (1.0,'NaN'::DOUBLE) / x IN (0.0)", + "input_desc": "DOUBLE col x in {NaN, 1.0, -0.0}", + "result": "NaN IN (NaN)=True; both rows True for the 2-elem list; -0.0 IN (0.0)=True", + "result_type": "BOOLEAN", + "note": "IN inherits duck = semantics: NaN equals NaN, zeros equal; matches canon_f64_bits key classes" + }, + { + "expr": "x NOT IN (a, b)", + "input_desc": "DOUBLE cols x=NaN, a=NULL, b=1.0", + "result": "None", + "result_type": "BOOLEAN", + "note": "NOT(NULL OR FALSE) = NULL \u2014 NaN-vs-number miss plus NULL element" + }, + { + "expr": "p IN (true) / p BETWEEN false AND true", + "input_desc": "BOOLEAN col p in {true,false,NULL}", + "result": "True,False,None / True,True,None", + "result_type": "BOOLEAN", + "note": "bools compare with false < true; three-valued as usual" + }, + { + "expr": "x IN (9223372036854775807) / x IN (9223372036854775808)", + "input_desc": "BIGINT col {i64::MAX, i64::MIN}", + "result": "MAX row: True / False; MIN row: False / False", + "result_type": "BOOLEAN", + "note": "beyond-i64 literal types HUGEINT, comparison exact (never equal to any BIGINT); typeof(9223372036854775807)=BIGINT, typeof(...808)=HUGEINT" + }, + { + "expr": "least(a, b) / greatest(a, b)", + "input_desc": "BIGINT cols: (1,NULL)/(NULL,2)/(NULL,NULL)/(1,2)", + "result": "least: 1/2/None/1; greatest: 1/2/None/2", + "result_type": "BIGINT", + "note": "NULLs are IGNORED (Postgres-style); NULL only when ALL args NULL. least(a, NULL) literal-NULL arg same: 5" + }, + { + "expr": "least(a) / least(a, b, 3, 0)", + "input_desc": "BIGINT col a=1 or NULL; n-ary with literals", + "result": "least(a)=a (None passes through); n-ary -> 0 on every row", + "result_type": "BIGINT", + "note": "1-arg is identity; genuinely n-ary; left-fold with NULL-ignore reproduces it" + }, + { + "expr": "least(a,b) / greatest(a,b) at i64 bounds", + "input_desc": "BIGINT (i64::MAX, i64::MIN)", + "result": "-9223372036854775808 / 9223372036854775807", + "result_type": "BIGINT", + "note": "all-int stays exact i64, no double round-trip" + }, + { + "expr": "least(i, d) / greatest(i, d)", + "input_desc": "BIGINT i + DOUBLE d cols: (1,2.5)/(3,2.5)/(9007199254740993, 9007199254740994.0)/(i64::MAX,1.0)", + "result": "least: 1.0/2.5/9007199254740992.0/1.0; greatest: 2.5/3.0/9007199254740994.0/9.223372036854776e+18", + "result_type": "DOUBLE", + "note": "mixed int/double unifies to DOUBLE; DuckDB itself does the lossy i64->f64 cast (2^53+1 became 9007199254740992.0) \u2014 v0 f64 matches exactly" + }, + { + "expr": "least/greatest NaN ordering", + "input_desc": "DOUBLE cols: (NaN,1.0)/(1.0,NaN)/(NaN,inf)/(NaN,NaN)/(-inf,inf)/(NaN,NULL)", + "result": "least: 1.0/1.0/inf/nan/-inf/nan; greatest: nan/nan/nan/nan/inf/nan", + "result_type": "DOUBLE", + "note": "NaN sorts above +inf: always loses least, always wins greatest; NULL ignored so (NaN,NULL) -> NaN for both. greatest(NULL,NaN,1.0)=nan, least=1.0" + }, + { + "expr": "least/greatest signed-zero ties", + "input_desc": "DOUBLE cols via '-0.0'::DOUBLE: (-0.0, 0.0) and (0.0, -0.0)", + "result": "(-0.0,0.0): least=-0.0 AND greatest=-0.0; (0.0,-0.0): least=0.0 AND greatest=0.0", + "result_type": "DOUBLE", + "note": "-0.0 = 0.0 so it's a tie, and BOTH functions return the FIRST argument on ties (strict-compare fold). Literal fold path identical. -0.0 survives native INSERT and the arrow->CTAS path duck_check uses" + }, + { + "expr": "least/greatest on VARCHAR", + "input_desc": "('b','a')/('B','a')/('abc','ab')/('a',NULL)/('\u00e9','z')/('','a')", + "result": "least: a/B/ab/a/z/''; greatest: b/a/abc/a/\u00e9/a", + "result_type": "VARCHAR", + "note": "byte order again; NULL ignored" + }, + { + "expr": "least(p, true) / greatest(p, false) on BOOLEAN", + "input_desc": "p in {true,false,NULL}", + "result": "least: True/False/True; greatest: True/False/False", + "result_type": "BOOLEAN", + "note": "false < true; NULL ignored (NOT K3 AND/OR: least(NULL,true)=true but NULL AND true is NULL)" + }, + { + "expr": "least(a, 1) with a BOOLEAN", + "input_desc": "a=true", + "result": "1", + "result_type": "INTEGER", + "note": "bool+int unifies to int (true -> 1)" + }, + { + "expr": "least(i, 2.5)", + "input_desc": "BIGINT col i=1, bare decimal literal", + "result": "Decimal('1.0')", + "result_type": "DECIMAL(20,1)", + "note": "decimal literal contaminates the RESULT type \u2014 Python fetch yields decimal.Decimal, repr-distinct from 1.0; v0-f64 diverges on repr. least(i, 2.5, 1e0) -> DOUBLE 1.0 (a DOUBLE arg wins). least(x, 9223372036854775808) -> HUGEINT" + }, + { + "expr": "typeof literals", + "input_desc": "literal typing probe", + "result": "1.5 -> DECIMAL(2,1); 9007199254740992.0 -> DECIMAL(17,1); 1e0 -> DOUBLE; -0.0 -> DECIMAL(2,1) (sign lost); least(1,2) -> INTEGER; least(int32col,int64col) -> BIGINT", + "result_type": "(see result)", + "note": "scientific notation is the only bare-literal spelling of DOUBLE; bare -0.0 comes out +0.0" + } + ], + "errors": [ + { + "expr": "SELECT x IN () FROM t", + "error_head": "Parser Error: syntax error at or near \")\"", + "kind": "parse" + }, + { + "expr": "SELECT least()", + "error_head": "Binder Error: No function matches the given name and argument types 'least()'.", + "kind": "binder" + }, + { + "expr": "SELECT least(int_col, varchar_col) FROM ...", + "error_head": "Binder Error: Cannot combine types of BIGINT and VARCHAR - an explicit cast is required", + "kind": "binder" + }, + { + "expr": "SELECT int_col BETWEEN varchar_col AND varchar_col FROM ...", + "error_head": "Binder Error: Cannot mix values of type BIGINT and VARCHAR in BETWEEN clause - an explicit", + "kind": "binder" + }, + { + "expr": "SELECT varchar_col IN (1) FROM ... (row 'a')", + "error_head": "Conversion Error: Could not convert string 'a' to INT32 when casting from source column s", + "kind": "runtime" + }, + { + "expr": "SELECT int_col IN ('abc') FROM ...", + "error_head": "Conversion Error: Could not convert string 'abc' to INT64", + "kind": "runtime" + }, + { + "expr": "SELECT least(int_col, 'abc') FROM ...", + "error_head": "Conversion Error: Could not convert string 'abc' to INT64", + "kind": "runtime" + }, + { + "expr": "SELECT int_col BETWEEN 'a' AND 'z' FROM ...", + "error_head": "Conversion Error: Could not convert string 'a' to INT64", + "kind": "runtime" + } + ], + "duck_check_tests": "# Predicates family: BETWEEN / IN / least / greatest \u2014 DuckDB 1.5.5 pins, measured 2026-07-26.\n# Float literals in SQL avoid big-magnitude bare decimals (DuckDB types them DECIMAL and\n# compares exactly; v0 collapses to f64 \u2014 divergence only beyond 2^53, kept out of these).\nnan, inf = float(\"nan\"), float(\"inf\")\n\ndef test_between_three_valued_int():\n duck_check(\n \"SELECT x BETWEEN lo AND hi AS b, x NOT BETWEEN lo AND hi AS nb FROM __THIS__\",\n {\"x\": \"int?\", \"lo\": \"int?\", \"hi\": \"int?\"},\n [\n {\"x\": 5, \"lo\": 1, \"hi\": 10}, # True\n {\"x\": 5, \"lo\": 10, \"hi\": 1}, # empty range -> False\n {\"x\": None, \"lo\": 1, \"hi\": 10}, # NULL\n {\"x\": None, \"lo\": 10, \"hi\": 1}, # NULL x + empty range -> still NULL\n {\"x\": 5, \"lo\": None, \"hi\": 10}, # NULL\n {\"x\": 5, \"lo\": None, \"hi\": 4}, # False \u2014 failing half decides\n {\"x\": 5, \"lo\": 1, \"hi\": None}, # NULL\n {\"x\": 5, \"lo\": 6, \"hi\": None}, # False\n {\"x\": None, \"lo\": None, \"hi\": None}, # NULL\n {\"x\": 5, \"lo\": 5, \"hi\": 5}, # inclusive -> True\n ],\n )\n\ndef test_between_int_x_double_bounds_unifies_to_double():\n duck_check(\n \"SELECT x BETWEEN lo AND hi AS b FROM __THIS__\",\n {\"x\": \"int\", \"lo\": \"float\", \"hi\": \"float\"},\n [\n {\"x\": 5, \"lo\": 0.5, \"hi\": 5.5},\n {\"x\": 5, \"lo\": 5.1, \"hi\": 9.9},\n # int->double is lossy at 2^53 \u2014 DuckDB does the same cast: True\n {\"x\": 9007199254740993, \"lo\": 9007199254740992.0, \"hi\": 9007199254740992.0},\n ],\n )\n\ndef test_between_duck_double_order_specials():\n duck_check(\n \"SELECT x BETWEEN lo AND hi AS b, x NOT BETWEEN lo AND hi AS nb FROM __THIS__\",\n {\"x\": \"float?\", \"lo\": \"float?\", \"hi\": \"float?\"},\n [\n {\"x\": nan, \"lo\": 1.0, \"hi\": 2.0}, # False (nan<=2 False)\n {\"x\": 5.0, \"lo\": 1.0, \"hi\": nan}, # True (nan above everything)\n {\"x\": 5.0, \"lo\": nan, \"hi\": 10.0}, # False\n {\"x\": nan, \"lo\": nan, \"hi\": nan}, # True (nan = nan)\n {\"x\": inf, \"lo\": 1.0, \"hi\": nan}, # True (nan above inf)\n {\"x\": nan, \"lo\": None, \"hi\": 1.0}, # False (NULL AND False)\n {\"x\": nan, \"lo\": 1.0, \"hi\": None}, # NULL (True AND NULL)\n {\"x\": 0.0, \"lo\": -0.0, \"hi\": -0.0}, # True (zeros equal)\n {\"x\": -0.0, \"lo\": 0.0, \"hi\": 0.0}, # True\n {\"x\": -inf, \"lo\": -inf, \"hi\": inf}, # True\n ],\n )\n\ndef test_between_strings_byte_order():\n duck_check(\n \"SELECT x BETWEEN lo AND hi AS b FROM __THIS__\",\n {\"x\": \"str\", \"lo\": \"str\", \"hi\": \"str\"},\n [\n {\"x\": \"b\", \"lo\": \"a\", \"hi\": \"c\"}, # True\n {\"x\": \"B\", \"lo\": \"a\", \"hi\": \"c\"}, # False (0x42 < 0x61)\n {\"x\": \"Z\", \"lo\": \"a\", \"hi\": \"z\"}, # False\n {\"x\": \"\u00e9\", \"lo\": \"a\", \"hi\": \"z\"}, # False (0xC3A9 > 'z')\n {\"x\": \"abc\", \"lo\": \"ab\", \"hi\": \"abd\"}, # True\n {\"x\": \"\", \"lo\": \"\", \"hi\": \"a\"}, # True\n {\"x\": \"a\", \"lo\": \"c\", \"hi\": \"a\"}, # False\n ],\n )\n\ndef test_in_null_element_truth_table():\n duck_check(\n \"SELECT x IN (1, NULL) AS i, x NOT IN (1, NULL) AS ni,\"\n \" x IN (1, 2) AS i2, x NOT IN (1, 2) AS ni2, x IN (NULL) AS io FROM __THIS__\",\n {\"x\": \"int?\"},\n [{\"x\": 1}, {\"x\": 3}, {\"x\": None}],\n # i: T/N/N ni: F/N/N i2: T/F/N ni2: F/T/N io: N/N/N\n )\n\ndef test_in_column_elements_is_the_or_chain():\n duck_check(\n \"SELECT x IN (y, z) AS r, x NOT IN (y, z) AS nr FROM __THIS__\",\n {\"x\": \"int?\", \"y\": \"int?\", \"z\": \"int?\"},\n [\n {\"x\": 1, \"y\": 1, \"z\": None}, # True / False\n {\"x\": 3, \"y\": 1, \"z\": None}, # NULL / NULL\n {\"x\": 3, \"y\": 1, \"z\": 2}, # False / True\n {\"x\": None, \"y\": 1, \"z\": 2}, # NULL / NULL\n ],\n )\n\ndef test_in_int_col_float_literals_small_values():\n duck_check(\n \"SELECT x IN (1.0) AS a, x IN (0.5, 1.0) AS b, x IN (1.5) AS c FROM __THIS__\",\n {\"x\": \"int?\"},\n [{\"x\": 1}, {\"x\": 2}, {\"x\": 3}, {\"x\": None}], # 1.5 never rounds to 2\n )\n\ndef test_in_nan_and_signed_zero():\n duck_check(\n \"SELECT x IN ('NaN'::DOUBLE) AS n, x IN (1.0, 'NaN'::DOUBLE) AS m,\"\n \" x IN (0.0) AS z, x NOT IN ('NaN'::DOUBLE) AS nn FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": nan}, {\"x\": 1.0}, {\"x\": -0.0}], # NaN=NaN True; -0.0 IN (0.0) True\n )\n\ndef test_in_strings_and_bools():\n duck_check(\n \"SELECT s IN ('a', 'b') AS r, s IN ('b', NULL) AS rn FROM __THIS__\",\n {\"s\": \"str?\"},\n [{\"s\": \"a\"}, {\"s\": \"A\"}, {\"s\": None}],\n )\n duck_check(\n \"SELECT p IN (true) AS r, p NOT IN (true) AS nr,\"\n \" p BETWEEN false AND true AS bt FROM __THIS__\",\n {\"p\": \"bool?\"},\n [{\"p\": True}, {\"p\": False}, {\"p\": None}],\n )\n\ndef test_least_greatest_null_ignoring_int():\n duck_check(\n \"SELECT least(a, b) AS l, greatest(a, b) AS g,\"\n \" least(a) AS l1, least(a, b, 3, 0) AS ln FROM __THIS__\",\n {\"a\": \"int?\", \"b\": \"int?\"},\n [\n {\"a\": 1, \"b\": None}, # l=1 g=1 \u2014 NULL ignored\n {\"a\": None, \"b\": 2}, # l=2 g=2\n {\"a\": None, \"b\": None},# NULL only when ALL null\n {\"a\": 1, \"b\": 2},\n {\"a\": 9223372036854775807, \"b\": -9223372036854775808}, # exact i64\n ],\n )\n\ndef test_least_greatest_double_nan_and_tie_keeps_first():\n duck_check(\n \"SELECT least(a, b) AS l, greatest(a, b) AS g FROM __THIS__\",\n {\"a\": \"float?\", \"b\": \"float?\"},\n [\n {\"a\": nan, \"b\": 1.0}, # l=1.0 g=nan (nan above everything)\n {\"a\": 1.0, \"b\": nan},\n {\"a\": nan, \"b\": inf}, # l=inf g=nan\n {\"a\": nan, \"b\": None}, # nan / nan \u2014 NULL ignored\n {\"a\": -inf, \"b\": inf},\n {\"a\": -0.0, \"b\": 0.0}, # BOTH return first arg: l=-0.0 g=-0.0\n {\"a\": 0.0, \"b\": -0.0}, # l=0.0 g=0.0\n {\"a\": nan, \"b\": nan},\n ],\n )\n\ndef test_least_greatest_mixed_int_double_unifies_to_double():\n duck_check(\n \"SELECT least(i, d) AS l, greatest(i, d) AS g FROM __THIS__\",\n {\"i\": \"int\", \"d\": \"float\"},\n [\n {\"i\": 1, \"d\": 2.5},\n {\"i\": 3, \"d\": 2.5},\n # DuckDB casts the i64 to double (lossy): l = 9007199254740992.0\n {\"i\": 9007199254740993, \"d\": 9007199254740994.0},\n {\"i\": 9223372036854775807, \"d\": 1.0}, # g = 9.223372036854776e+18\n ],\n )\n\ndef test_least_greatest_strings_and_bools():\n duck_check(\n \"SELECT least(a, b) AS l, greatest(a, b) AS g FROM __THIS__\",\n {\"a\": \"str?\", \"b\": \"str?\"},\n [\n {\"a\": \"b\", \"b\": \"a\"},\n {\"a\": \"B\", \"b\": \"a\"}, # least='B' \u2014 byte order\n {\"a\": \"\u00e9\", \"b\": \"z\"}, # greatest='\u00e9'\n {\"a\": \"\", \"b\": \"a\"},\n {\"a\": \"a\", \"b\": None}, # NULL ignored\n ],\n )\n duck_check(\n \"SELECT least(p, true) AS l, greatest(p, false) AS g FROM __THIS__\",\n {\"p\": \"bool?\"},\n [{\"p\": True}, {\"p\": False}, {\"p\": None}], # NULL row: l=True g=False\n )", + "lowering": "Three constructs, three answers \u2014 BETWEEN and IN are pure frontend desugars, least/greatest wants new IR ops.\n\nBETWEEN \u2014 pure desugar, proven exact. `x BETWEEN lo AND hi` => `(x >= lo) AND (x <= hi)`; `x NOT BETWEEN lo AND hi` => `NOT ((x >= lo) AND (x <= hi))`. All 12 int rows plus the NaN/NULL interplay rows match the K3 AND of the two comparisons with zero special cases: empty range FALSE falls out of `5 >= 10` being FALSE; `(5, NULL, 4) -> FALSE` is `NULL AND FALSE`; `(NaN, 1.0, NULL) -> NULL` is `TRUE AND NULL` (given fcmp = exec::duck_fcmp, which Inst::Cmp already implements \u2014 NaN=NaN/highest, -0.0=0.0). CRITICAL binding rule: unify ALL THREE operands to ONE type BEFORE emitting the two comparisons (measured: `x BETWEEN 1e0 AND 9007199254740992.0` is True because the DECIMAL bound rides along to DOUBLE). In v0's type lattice: any F64 among {x,lo,hi} -> compare all as F64 (itof the i64s); else all-I64; Str only with Str; I1 only with I1; mixed Str/number columns must be PrepareError::Bind mirroring \"Cannot mix values of type BIGINT and VARCHAR in BETWEEN clause\". Bind x to a temp so it evaluates once (cost only; x is pure so semantics are unchanged).\n\nIN \u2014 pure desugar, proven exact. `x IN (e1..en)` => `(x = e1) OR ... OR (x = en)` left-assoc; NOT IN => `NOT (chain)`. Verified row-for-row identical to the OR-chain including NULL elements, NULL x, NaN elements, and column elements (the desugar-check probes ran both forms side by side). n=0 is unreachable (DuckDB parse error). Same WHOLE-LIST unification rule first: x plus every element to one common type (measured discriminator: `x IN (9007199254740992.0, 1e0)` True vs all-decimal list False). In v0: any F64 in {x, e1..en} -> all F64; else I64; Str lists with Str x; I1 with I1. Backends may later strength-reduce long literal lists onto the existing canon_f64_bits/KeyBits hash machinery \u2014 semantics-preserving because those key classes (NaN one class, -0.0=+0.0) are exactly duck `=`.\n\nleast/greatest \u2014 new IR ops, binary, folded n-ary by the frontend. The NULL policy (IGNORE nulls, result NULL only if all args NULL) is the opposite of every existing op's propagate-NULL contract, so a desugar needs IS-NULL-guarded CASE with duplicated args; a dedicated op over (validity, payload) pairs is cleaner and allocation-free. Ops: `least2.{i64,f64,str,i1}` / `greatest2.{...}`. Exact Rust semantics (reproduces every pin):\n\n fn least2(a: Option, b: Option, cmp: impl Fn(&T, &T) -> Ordering) -> Option {\n match (a, b) {\n (None, b) => b, // NULL ignored\n (Some(a), None) => Some(a),\n (Some(a), Some(b)) =>\n Some(if cmp(&b, &a) == Ordering::Less { b } else { a }), // tie keeps a (FIRST arg)\n }\n }\n // greatest2: Ordering::Greater. No traps.\n // cmp: f64 -> exec::duck_fcmp (NaN=NaN, NaN highest, -0.0==0.0);\n // i64/bool -> Ord; str -> byte order (&str cmp).\n\nFrontend folds left: `least(a,b,c)` => `least2(least2(a,b), c)` \u2014 the left fold provably preserves both NULL-ignoring (least2(NULL, c) = c) and first-arg-wins ties (the -0.0 pins: least AND greatest of (-0.0, 0.0) both return -0.0). Type unification: any F64 arg -> all F64 (DuckDB itself does the lossy i64->f64 cast here \u2014 least(2^53+1, 9007199254740994.0) = 9007199254740992.0 \u2014 so v0-f64 matches exactly); all-int -> I64 (exact at i64 bounds); Str only with Str; I1-only -> I1 result; I1 mixed with int -> int (measured least(true,1)=1); Str/number mix -> PrepareError::Bind (\"Cannot combine types of BIGINT and VARCHAR\"). Note the verifier currently rejects Cmp on I1 \u2014 least2.i1/greatest2.i1 sidestep that (they are NOT K3 AND/OR: least(NULL,true)=true but NULL AND true is NULL); for bool IN/BETWEEN the frontend can widen I1 to I64 around the comparisons since the BOOLEAN result values are unaffected.\n\nTrap conditions: none anywhere in this family. The only runtime errors are cast errors from literal strings in numeric positions, which belong to the cast machinery, not these ops.", + "surprises": [ + "Bare float-looking literals (1.5, -0.0, 9007199254740992.0) are DECIMAL, not DOUBLE \u2014 only scientific notation (1e0) is DOUBLE. BIGINT-vs-DECIMAL comparisons are EXACT (9007199254740993 = 9007199254740992.0 is FALSE) while BIGINT-vs-DOUBLE casts the int to double (same compare is TRUE). v0's decimal->f64 collapse therefore diverges in comparisons/IN/BETWEEN only for all-decimal operand sets beyond 2^53 \u2014 same known divergence class as arithmetic, now proven to reach predicates.", + "Type unification for BETWEEN and IN is WHOLE-EXPRESSION, not pairwise: one common type across x and all bounds/list elements. A single DOUBLE element drags the entire list to DOUBLE \u2014 x IN (9007199254740992.0, 1e0) is TRUE for x=2^53+1 while x IN (9007199254740992.0, 1.0) is FALSE.", + "least/greatest IGNORE NULLs (Postgres-style): least(1, NULL) = 1, NULL only when every arg is NULL. Anyone assuming propagate-NULL ships a divergence on every row with one NULL arg.", + "On ties BOTH least AND greatest return the FIRST argument: least(-0.0, 0.0) = -0.0 and greatest(-0.0, 0.0) = -0.0 (observable because -0.0 == 0.0 under duck order and -0.0 survives storage). A max-style 'keep b on >=' implementation diverges on signed zeros.", + "NaN sorts ABOVE +inf and equals itself in every predicate context: 5.0 BETWEEN 1.0 AND NaN is TRUE, NaN IN (NaN) is TRUE, NaN BETWEEN NaN AND NaN is TRUE, greatest(anything, NaN) = NaN.", + "BETWEEN needs no empty-range special case, but also must not add one: (x=5, lo=NULL, hi=4) is FALSE (K3 AND with a failing half) while (x=NULL, lo=10, hi=1) is NULL \u2014 an implementation that short-circuits lo>hi to FALSE diverges on NULL x.", + "VARCHAR loses type unification to every numeric type, and the CAST DIRECTION is column-vs-literal asymmetric: s_varchar IN (1) casts the COLUMN to INT32 giving PER-ROW ConversionException on non-numeric rows ('1' rows return TRUE); int_col IN ('1') casts the literal and works; VARCHAR COLUMNS (not literals) against ints in least/BETWEEN are binder errors.", + "Constant-cast conversion errors (int_col IN ('abc'), least(x,'abc'), x BETWEEN 'a' AND 'z') are EXECUTION-time, not bind-time: WHERE false suppresses them entirely \u2014 the corpus three-outcome contract must classify them runtime.", + "x IN () is a ParserException, not binder \u2014 no IR concern.", + "A bare -0.0 SQL literal loses its sign (DECIMAL has no negative zero); '-0.0'::DOUBLE keeps it, and -0.0 survives both native INSERT and the arrow->CTAS ingestion duck_check uses.", + "Beyond-i64 integer literals (9223372036854775808) type HUGEINT and compare exactly (never equal to any BIGINT); least(bigint_col, 2^63) returns HUGEINT \u2014 outside v0's scalar set, keep rejecting at the literal per existing policy.", + "Repo doc drift found while verifying: src/specializer/exec/interp.rs header (lines 11-13) still claims fcmp is IEEE-ordered with NaN-false predicates, but Inst::Cmp uses exec::duck_fcmp and exec/tests.rs pins NaN-gt \u2014 flagged as a background task chip; spec text must not copy the stale line.", + "least(int_col, 2.5) has RESULT type DECIMAL(20,1) (Python fetch: decimal.Decimal) \u2014 a duck_check written with bare decimal literals in least/greatest repr-mismatches the engine's f64 even when values agree; contract tests use ::DOUBLE casts or float columns instead." + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave1/pins_string-search-instr-strp.json b/docs/superpowers/specs/pins-wave1/pins_string-search-instr-strp.json new file mode 100644 index 0000000..df5adb6 --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_string-search-instr-strp.json @@ -0,0 +1,359 @@ +{ + "family": "string-search: instr, strpos, position(n IN s), contains, starts_with/prefix, ends_with/suffix, length/len/char_length/character_length, strlen, length_grapheme, bit_length (DuckDB 1.5.5, all measured through table columns)", + "pins": [ + { + "expr": "instr(s, n)", + "input_desc": "s='abc', n='b' (VARCHAR columns)", + "result": "2", + "result_type": "BIGINT", + "note": "1-based position" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='abc', n='z'", + "result": "0", + "result_type": "BIGINT", + "note": "not found -> 0 (never NULL, never -1)" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='abc', n=''", + "result": "1", + "result_type": "BIGINT", + "note": "empty needle -> 1, not 0" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='', n='a'", + "result": "0", + "result_type": "BIGINT", + "note": "empty haystack -> 0" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='', n=''", + "result": "1", + "result_type": "BIGINT", + "note": "empty-in-empty -> 1" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='abc', n='abcd'", + "result": "0", + "result_type": "BIGINT", + "note": "needle longer than haystack -> 0" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='abcabc', n='bc'", + "result": "2", + "result_type": "BIGINT", + "note": "first occurrence" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='aaa', n='aa'", + "result": "1", + "result_type": "BIGINT", + "note": "overlapping repeats: leftmost" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='abc', n='B'", + "result": "0", + "result_type": "BIGINT", + "note": "case-sensitive, no folding" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='h\u00e9llo', n='l'", + "result": "3", + "result_type": "BIGINT", + "note": "position in CODEPOINTS (byte position would be 4)" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='h\u00e9llo', n='\u00e9'", + "result": "2", + "result_type": "BIGINT", + "note": "multibyte char at codepoint 2" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='h\u00e9llo', n='\u00e9l'", + "result": "2", + "result_type": "BIGINT", + "note": "multibyte needle, codepoint position" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='a\\U0001F44Db', n='b'", + "result": "3", + "result_type": "BIGINT", + "note": "4-byte emoji counts as ONE codepoint (bytes would give 6)" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='\\U0001F468\u200d\\U0001F469\u200d\\U0001F467' (ZWJ family), n='\\U0001F467'", + "result": "5", + "result_type": "BIGINT", + "note": "codepoints, NOT graphemes: ZWJ counts, grapheme answer would be 1" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='\u00e9\u00e9a', n='a'", + "result": "3", + "result_type": "BIGINT", + "note": "codepoint count of multibyte prefix" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='xabc', n='abc'", + "result": "2", + "result_type": "BIGINT", + "note": "match not at start" + }, + { + "expr": "instr(s, n)", + "input_desc": "(NULL,'b'), ('abc',NULL), (NULL,NULL), (NULL,'')", + "result": "None in all 4 combos", + "result_type": "BIGINT", + "note": "NULL-strict in EVERY argument; NULL haystack + empty needle is still NULL" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='e\u0301' (decomposed), n='\u00e9' (precomposed)", + "result": "0", + "result_type": "BIGINT", + "note": "NO unicode normalization; byte-exact matching (both directions measured 0)" + }, + { + "expr": "instr(s, n)", + "input_desc": "s='e\u0301' (decomposed), n='\u0301' (bare combining mark)", + "result": "2", + "result_type": "BIGINT", + "note": "combining mark alone is findable at its codepoint index" + }, + { + "expr": "strpos(s, n)", + "input_desc": "all 19 corpus pairs above", + "result": "bit-identical to instr(s, n) on every pair", + "result_type": "BIGINT", + "note": "strpos IS instr (0 divergences measured); NOT PostgreSQL arg order \u2014 DuckDB strpos(haystack, needle)" + }, + { + "expr": "position(n IN s)", + "input_desc": "all 19 corpus pairs above", + "result": "bit-identical to instr(s, n) on every pair", + "result_type": "BIGINT", + "note": "PostgreSQL POSITION syntax works in DuckDB; desugars to position(s, n)" + }, + { + "expr": "\"position\"(s, n) two-arg call", + "input_desc": "('abc','b') and ('b','abc')", + "result": "2 and 0 respectively", + "result_type": "BIGINT", + "note": "2-arg call form binds (haystack, needle) \u2014 same order as instr, i.e. position('needle','hay') silently searches the wrong way" + }, + { + "expr": "contains(s, n)", + "input_desc": "('abc','b')/('abc','z')/('abc','B')", + "result": "True / False / False", + "result_type": "BOOLEAN", + "note": "haystack-first arg order; case-sensitive" + }, + { + "expr": "contains(s, n)", + "input_desc": "('abc','') / ('','') / ('','a')", + "result": "True / True / False", + "result_type": "BOOLEAN", + "note": "empty needle -> TRUE even for empty haystack" + }, + { + "expr": "contains(s, n)", + "input_desc": "('h\u00e9llo','\u00e9'), (ZWJ family,'\\U0001F467'), ('aaa','aa'), ('xabc','abc')", + "result": "True in all", + "result_type": "BOOLEAN", + "note": "byte-substring semantics" + }, + { + "expr": "contains(s, n)", + "input_desc": "NULL in either/both args (typed VARCHAR)", + "result": "None", + "result_type": "BOOLEAN", + "note": "NULL-strict; note bare-NULL LITERAL needle is a binder error (see errors)" + }, + { + "expr": "starts_with(s, n)", + "input_desc": "('abc','') / ('','') / ('aaa','aa') / ('abc','abc') / ('xabc','abc') / ('abc','b') / ('abc','abcd')", + "result": "True / True / True / True / False / False / False", + "result_type": "BOOLEAN", + "note": "empty needle TRUE; exact-match TRUE; interior match FALSE" + }, + { + "expr": "starts_with(s, n)", + "input_desc": "NULL either/both sides", + "result": "None", + "result_type": "BOOLEAN", + "note": "NULL-strict" + }, + { + "expr": "prefix(s, n)", + "input_desc": "all 19 corpus pairs", + "result": "bit-identical to starts_with on every pair", + "result_type": "BOOLEAN", + "note": "exact alias" + }, + { + "expr": "ends_with(s, n)", + "input_desc": "('abc','') / ('abcabc','bc') / ('a\\U0001F44Db','b') / ('xabc','abc') / ('abc','b') / ('e\u0301','\u0301')", + "result": "True / True / True / True / False / True", + "result_type": "BOOLEAN", + "note": "empty needle TRUE; byte-suffix; a bare combining mark is a valid suffix" + }, + { + "expr": "ends_with(s, n)", + "input_desc": "NULL either/both sides", + "result": "None", + "result_type": "BOOLEAN", + "note": "NULL-strict" + }, + { + "expr": "suffix(s, n)", + "input_desc": "all 19 corpus pairs", + "result": "bit-identical to ends_with on every pair", + "result_type": "BOOLEAN", + "note": "exact alias" + }, + { + "expr": "length(s)", + "input_desc": "'abc' / '' / 'h\u00e9llo' / 'a\\U0001F44Db' / ZWJ family / 'e\u0301' / NULL", + "result": "3 / 0 / 5 / 3 / 5 / 2 / None", + "result_type": "BIGINT", + "note": "CODEPOINTS: ZWJ family is 5 (grapheme would be 1); decomposed e-acute is 2" + }, + { + "expr": "len(s), char_length(s), character_length(s)", + "input_desc": "same 7 inputs", + "result": "identical to length(s) on every input", + "result_type": "BIGINT", + "note": "exact aliases of length" + }, + { + "expr": "strlen(s)", + "input_desc": "'abc' / '' / 'h\u00e9llo' / 'a\\U0001F44Db' / ZWJ family / 'e\u0301' / NULL", + "result": "3 / 0 / 6 / 6 / 18 / 3 / None", + "result_type": "BIGINT", + "note": "UTF-8 BYTES, not codepoints" + }, + { + "expr": "length_grapheme(s)", + "input_desc": "ZWJ family / 'e\u0301' / 'a\\U0001F44Db' / NULL", + "result": "1 / 1 / 3 / None", + "result_type": "BIGINT", + "note": "grapheme clusters; only member of the family that sees ZWJ sequences as one unit" + }, + { + "expr": "bit_length(s)", + "input_desc": "'abc' / 'h\u00e9llo' / ZWJ family / NULL", + "result": "24 / 48 / 144 / None", + "result_type": "BIGINT", + "note": "exactly 8 * strlen(s)" + }, + { + "expr": "n IN s (both VARCHAR columns)", + "input_desc": "('abc','b')/('abc','z')/('abc','')/('','')/NULL combos", + "result": "True / False / True / True / None", + "result_type": "BOOLEAN", + "note": "string IN string == contains(s, n), NULL-strict; ONLY binds when RHS is a column/expr \u2014 literal RHS is a parse error" + }, + { + "expr": "n NOT IN s (both VARCHAR columns)", + "input_desc": "same pairs", + "result": "False / True / False / False / None", + "result_type": "BOOLEAN", + "note": "NOT contains, NULL-strict (NULL stays NULL, not flipped)" + }, + { + "expr": "s ^@ n", + "input_desc": "('abc','ab') literal and column", + "result": "True", + "result_type": "BOOLEAN", + "note": "PostgreSQL starts_with operator exists in DuckDB, same semantics as starts_with" + }, + { + "expr": "contains(s, n) with unicode case pairs", + "input_desc": "('h\u00e9llo','\u00c9') / ('H\u00c9LLO','\u00e9') / ('Stra\u00dfe','SS')", + "result": "False / False / False (instr 0 on all)", + "result_type": "BOOLEAN", + "note": "no unicode case folding of any kind (no eszett expansion)" + }, + { + "expr": "length(repeat('a',1000000)), strlen(repeat(emoji,250000))", + "input_desc": "1M-char string", + "result": "1000000 / 1000000", + "result_type": "BIGINT", + "note": "sanity: large strings fine, results i64; string lengths cannot approach i64::MAX (VARCHAR capped far below), so no overflow edge exists" + }, + { + "expr": "literal-fold vs vectorized", + "input_desc": "all 8 two-arg functions x 19 pairs, literal (NULL::VARCHAR) vs column", + "result": "0 divergences", + "result_type": "-", + "note": "constant-fold path agrees with vectorized path everywhere in this family" + } + ], + "errors": [ + { + "expr": "SELECT contains('abc', NULL)", + "error_head": "Binder Error: Could not choose a best candidate function for the function call \"contains(STRING_LITERAL, \"NULL\")\". In order to select one, please add explicit type casts.", + "kind": "binder" + }, + { + "expr": "SELECT instr(1, '2') (also INTEGER column: 'instr(INTEGER, STRING_LITERAL)')", + "error_head": "Binder Error: No function matches the given name and argument types 'instr(INTEGER_LITERAL, STRING_LITERAL)'. You might need to add explicit type casts.", + "kind": "binder" + }, + { + "expr": "SELECT length(1) (same for DOUBLE/BOOLEAN args to any family member)", + "error_head": "Binder Error: No function matches the given name and argument types 'length(INTEGER_LITERAL)'. You might need to add explicit type casts.", + "kind": "binder" + }, + { + "expr": "SELECT strpos('a')", + "error_head": "Binder Error: No function matches the given name and argument types 'strpos(STRING_LITERAL)'. You might need to add explicit type casts.", + "kind": "binder" + }, + { + "expr": "SELECT octet_length(s) FROM t -- s VARCHAR", + "error_head": "Binder Error: No function matches the given name and argument types 'octet_length(VARCHAR)'. You might need to add explicit type casts. Candidate functions: octet_length(BLOB) -> BIGINT, octet_length(BIT)", + "kind": "binder" + }, + { + "expr": "SELECT 'b' IN 'abc' -- literal RHS", + "error_head": "Parser Error: syntax error at or near \"'abc'\"", + "kind": "parse" + }, + { + "expr": "SELECT position(s, 'b') FROM t -- unquoted 2-arg call", + "error_head": "Parser Error: syntax error at or near \",\"", + "kind": "parse" + } + ], + "duck_check_tests": "\n# string-search family - contract pins measured against DuckDB 1.5.5 (2026-07-26).\n# All values below were probed through the vectorized path (table columns);\n# literal-fold agreed on every pair (0 divergences).\n\nSTRSEARCH_SCHEMA = {\"s\": \"str\", \"n\": \"str\"}\nSTRSEARCH_ROWS = [\n {\"s\": \"abc\", \"n\": \"b\"}, # simple hit -> 2\n {\"s\": \"abc\", \"n\": \"z\"}, # miss -> 0\n {\"s\": \"abc\", \"n\": \"\"}, # empty needle -> 1 / TRUE\n {\"s\": \"\", \"n\": \"a\"}, # empty haystack -> 0 / FALSE\n {\"s\": \"\", \"n\": \"\"}, # empty-in-empty -> 1 / TRUE\n {\"s\": \"abc\", \"n\": \"abcd\"}, # needle longer -> 0 / FALSE\n {\"s\": \"abcabc\", \"n\": \"bc\"}, # first occurrence -> 2\n {\"s\": \"aaa\", \"n\": \"aa\"}, # overlapping -> 1\n {\"s\": \"abc\", \"n\": \"B\"}, # case-sensitive -> 0\n {\"s\": \"h\u00e9llo\", \"n\": \"l\"}, # CODEPOINT position -> 3 (bytes would be 4)\n {\"s\": \"h\u00e9llo\", \"n\": \"\u00e9\"}, # -> 2\n {\"s\": \"h\u00e9llo\", \"n\": \"\u00e9l\"}, # multibyte needle -> 2\n {\"s\": \"a\\U0001F44Db\", \"n\": \"b\"}, # emoji = 1 codepoint -> 3 (bytes would be 6)\n {\"s\": \"\\U0001F468\u200d\\U0001F469\u200d\\U0001F467\", \"n\": \"\\U0001F467\"}, # ZWJ -> 5, not 1\n {\"s\": \"\u00e9\u00e9a\", \"n\": \"a\"}, # -> 3\n {\"s\": \"xabc\", \"n\": \"abc\"}, # suffix hit -> 2\n {\"s\": \"abc\", \"n\": \"abc\"}, # exact -> 1\n {\"s\": \"e\u0301\", \"n\": \"\u00e9\"}, # NO normalization -> 0 / FALSE\n {\"s\": \"e\u0301\", \"n\": \"\u0301\"}, # bare combining mark -> 2\n {\"s\": \"abc\", \"n\": None}, # NULL-strict every argument\n {\"s\": None, \"n\": \"b\"},\n {\"s\": None, \"n\": None},\n {\"s\": None, \"n\": \"\"}, # NULL haystack + empty needle is still NULL\n]\n\n\ndef test_strsearch_positions():\n for fn_expr in [\"instr(s, n)\", \"strpos(s, n)\", \"position(n IN s)\"]:\n duck_check(\n f\"SELECT {fn_expr} AS r FROM __THIS__\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n\n\ndef test_strsearch_predicates():\n for fn in [\"contains\", \"starts_with\", \"prefix\", \"ends_with\", \"suffix\"]:\n duck_check(\n f\"SELECT {fn}(s, n) AS r FROM __THIS__\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n\n\ndef test_strsearch_length_family():\n duck_check(\n \"SELECT length(s) AS cp, len(s) AS cp2, char_length(s) AS cp3,\"\n \" character_length(s) AS cp4, strlen(s) AS by, bit_length(s) AS bits\"\n \" FROM __THIS__\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n\n\ndef test_strsearch_length_grapheme():\n # Separate: only implement if length_grapheme lands in scope (needs a\n # grapheme-segmentation table; ZWJ family -> 1, decomposed e-acute -> 1).\n duck_check(\n \"SELECT length_grapheme(s) AS gr FROM __THIS__\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n\n\ndef test_strsearch_in_operator_is_contains():\n # `n IN s` over VARCHAR columns is string-contains in DuckDB (NULL-strict);\n # the pure-literal form `'b' IN 'abc'` is a DuckDB PARSE error. Gate this\n # test on the frontend actually parsing string-IN (sqlparser may reject or\n # mis-parse it as an in-list) - verify before enabling.\n duck_check(\n \"SELECT (n IN s) AS r, (n NOT IN s) AS q FROM __THIS__\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n\n\ndef test_strsearch_three_valued_where():\n # NULL predicate rows must be dropped by WHERE, same as FALSE.\n duck_check(\n \"SELECT s FROM __THIS__ WHERE contains(s, n)\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n duck_check(\n \"SELECT s FROM __THIS__ WHERE instr(s, n) > 1\",\n STRSEARCH_SCHEMA, STRSEARCH_ROWS,\n )\n", + "lowering": "Recommend SIX new IR ops plus pure-frontend desugars for the aliases. The whole family is TOTAL over (str|NULL) args: no trap conditions, no runtime errors, and every function is NULL-strict in every argument (all combos measured), so validity = AND of input validities \u2014 the strict-NULL vectorized fast path applies with no three-valued special cases. Rust byte-wise semantics reproduced every measured pin including normalization/combining-mark edges (validated against the probe corpus):\n\n // all ops NULL-strict; out_valid = s_valid & n_valid\n fn str_find(s: &str, n: &str) -> i64 { // instr / strpos / position\n match s.find(n) { // byte index; \"\" -> Some(0)\n Some(b) => 1 + s[..b].chars().count() as i64, // 1-based CODEPOINT pos\n None => 0,\n }\n }\n fn str_contains(s: &str, n: &str) -> bool { s.contains(n) } // \"\" -> true\n fn str_starts_with(s: &str, n: &str) -> bool { s.starts_with(n) }\n fn str_ends_with(s: &str, n: &str) -> bool { s.ends_with(n) }\n fn str_len_codepoints(s: &str) -> i64 { s.chars().count() as i64 }\n fn str_len_bytes(s: &str) -> i64 { s.len() as i64 }\n\nByte-prefix/suffix/substring tests are exact because DuckDB does byte-exact matching with NO normalization and NO case folding (measured), and UTF-8 guarantees byte-prefix == codepoint-prefix.\n\nFrontend desugars (all proven exact by 0-divergence corpus runs):\n strpos(s,n), position(n IN s), \"position\"(s,n) -> StrFind(s,n) [note \"position\" 2-arg order is (haystack, needle)]\n prefix(s,n) -> StrStartsWith(s,n)\n suffix(s,n) -> StrEndsWith(s,n)\n len/char_length/character_length(s) -> StrLenCodepoints(s)\n bit_length(s) -> 8 * StrLenBytes(s) [mul propagates NULL; no overflow \u2014 VARCHAR length bounded far below i64::MAX/8]\n n IN s (VARCHAR,VARCHAR) -> StrContains(s, n) [args REVERSED; only if sqlparser parses string-IN \u2014 verify first]\n n NOT IN s -> NOT StrContains(s, n) [plain NOT: NULL stays NULL, measured]\n s ^@ n -> StrStartsWith(s, n)\n\nDo NOT desugar contains -> StrFind > 0: it is exact under three-valued logic (NULL>0 is NULL, matching contains' NULL), but StrFind pays an extra codepoint count of the prefix; keep StrContains as its own op. Do NOT lower length_grapheme via desugar \u2014 it needs Unicode grapheme segmentation (unicode-segmentation crate); recommend leaving it unimplemented (clean unsupported-function error) unless the corpus demands it. octet_length must NOT be mapped to StrLenBytes: DuckDB rejects octet_length(VARCHAR) at bind time, so binding it would be a divergence \u2014 mirror the binder error.", + "surprises": [ + "Positions are 1-based CODEPOINT indices, not bytes and not graphemes: instr('a\\U0001F44Db','b')=3 (bytes would say 6); instr(ZWJ-family,'\\U0001F467')=5 (graphemes would say 1); instr('h\u00e9llo','l')=3.", + "Empty needle matches EVERYTHING: instr(s,'')=1 and contains/starts_with/ends_with(s,'')=TRUE even when s itself is '' \u2014 but NULL haystack still wins: instr(NULL,'') is NULL.", + "contains('abc', NULL) with a bare NULL literal is a BINDER ERROR (ambiguous with contains(MAP,K) and contains(T[],T) overloads) while instr/strpos/position/starts_with/ends_with/prefix/suffix all accept bare NULL and return NULL. A typed NULL (NULL::VARCHAR or a NULL column) works fine in contains. The engine must not 'helpfully' bind string-contains for an untyped NULL.", + "`n IN s` where both sides are VARCHAR expressions is string-contains \u2014 contains(s,n), args reversed, NULL-strict, empty-needle TRUE \u2014 but the pure-literal form 'b' IN 'abc' is a DuckDB PARSE error. A syntax-level literal-vs-column divergence; sqlparser support must be verified separately before wiring this.", + "The 2-arg call form \"position\"(a,b) binds (haystack, needle) \u2014 identical to instr/strpos \u2014 even though the SQL-standard form position(n IN s) reads needle-first. position('b','abc')=0, position('abc','b')=2. Also the unquoted call position(a,b) does not even parse (POSITION is grammar-reserved).", + "length() counts CODEPOINTS, not graphemes: length(ZWJ family emoji)=5 while length_grapheme=1; decomposed 'e\u0301' has length 2, strlen 3, length_grapheme 1. strlen is UTF-8 bytes. bit_length is exactly 8*strlen. octet_length does NOT exist for VARCHAR (binder error; BLOB/BIT only).", + "Zero unicode intelligence anywhere: no normalization (precomposed \u00e9 never matches decomposed e+\u0301 in either direction), no case folding ('Stra\u00dfe' does not contain 'SS'), and a bare combining mark \u0301 is independently findable (instr('e\u0301','\u0301')=2). Plain Rust byte search reproduces every pin.", + "No implicit numeric-to-VARCHAR casts for any family member: instr(123,'2'), length(123), contains(INTEGER,...) are all binder errors \u2014 DuckDB will not stringify numbers here.", + "The family is completely total: no per-row runtime error exists for any input; every function is NULL-strict in every argument (all NULL combos measured), so the strict-validity-mask fast path is exact.", + "strpos/instr/position are bit-identical (strpos is NOT PostgreSQL's (needle,haystack) order \u2014 DuckDB flips it to (haystack,needle)); prefix==starts_with and suffix==ends_with exactly; len/char_length/character_length==length exactly (19-pair corpus, 0 divergences).", + "Literal-fold vs vectorized: 0 divergences across all 8 two-arg functions x 19 pairs \u2014 this family is safe from the constant-fold trap.", + "PostgreSQL's ^@ starts-with operator exists in DuckDB and equals starts_with." + ] +} \ No newline at end of file diff --git a/docs/superpowers/specs/pins-wave1/pins_trig-sin-x-cos-x-tan-x-p.json b/docs/superpowers/specs/pins-wave1/pins_trig-sin-x-cos-x-tan-x-p.json new file mode 100644 index 0000000..ca5919e --- /dev/null +++ b/docs/superpowers/specs/pins-wave1/pins_trig-sin-x-cos-x-tan-x-p.json @@ -0,0 +1,465 @@ +{ + "family": "trig \u2014 sin(x), cos(x), tan(x), pi() [DuckDB 1.5.5, win-amd64, measured via native-table column path]", + "pins": [ + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 0.0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "bits 0x0000000000000000" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = -0.0 (0x8000000000000000)", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "sign of zero PRESERVED through the vectorized path; bits 0x8000000000000000" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 1.0", + "result": "0.8414709848078965", + "result_type": "DOUBLE", + "note": "bits 0x3feaed548f090cee; bitwise == platform libm" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = -1.0", + "result": "-0.8414709848078965", + "result_type": "DOUBLE", + "note": "odd symmetry exact (sign-flip of sin(1.0) bits)" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 0.5", + "result": "0.479425538604203", + "result_type": "DOUBLE", + "note": "bits 0x3fdeaee8744b05f0" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 1.5707963267948966 (nearest double to pi/2, == pi()/2 bits)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "EXACTLY 1.0 (0x3ff0000000000000); sin(pi()/2)=1.0 both literal and column path" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = -1.5707963267948966", + "result": "-1.0", + "result_type": "DOUBLE", + "note": "exactly -1.0" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 3.141592653589793 (nearest double to pi)", + "result": "1.2246467991473532e-16", + "result_type": "DOUBLE", + "note": "NOT zero; bits 0x3ca1a62633145c07" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 6.283185307179586 (2*pi)", + "result": "-2.4492935982947064e-16", + "result_type": "DOUBLE", + "note": "negative; bits 0xbcb1a62633145c07" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 1e15", + "result": "0.8582727931702359", + "result_type": "DOUBLE", + "note": "bits 0x3feb76f88136ceba \u2014 full argument reduction, bitwise == platform libm (UCRT here)" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 9007199254740992.0 (2^53)", + "result": "-0.848925964814655", + "result_type": "DOUBLE", + "note": "bits 0xbfeb2a66c8f35586" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 1e300", + "result": "-0.8178819121159085", + "result_type": "DOUBLE", + "note": "bits 0xbfea2c16b010e385 \u2014 Payne-Hanek-class reduction fidelity, bitwise == platform libm" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 1.7976931348623157e+308 (DBL_MAX)", + "result": "0.004961954789184062", + "result_type": "DOUBLE", + "note": "bits 0x3f7452fc98b34e97" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 5e-324 (min denormal)", + "result": "5e-324", + "result_type": "DOUBLE", + "note": "returns input bits 0x0000000000000001 (sin x ~ x, no denormal flush)" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 2.2250738585072014e-308 (min normal)", + "result": "2.2250738585072014e-308", + "result_type": "DOUBLE", + "note": "identity at tiny magnitude" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = 9.223372036854776e+18 (i64::MAX rounded to f64 = 2^63)", + "result": "0.9999303766734422", + "result_type": "DOUBLE", + "note": "bits 0x3fefff6dfd42dc54" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = NaN (quiet 0x7ff8000000000000)", + "result": "nan", + "result_type": "DOUBLE", + "note": "output bits 0x7ff8000000000000 \u2014 IDENTICAL to input" + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = NaN with payload 0x7ff0deadbeef0001 (signaling pattern) and 0xfff8000000000000 (negative NaN)", + "result": "nan (input bits unchanged)", + "result_type": "DOUBLE", + "note": "DuckDB passes the EXACT NaN bits through (does not call libm on NaN); UCRT libm would quiet it to 0x7ff8deadbeef0001. Same for cos and tan." + }, + { + "expr": "sin(x)", + "input_desc": "DOUBLE column x = NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "NULL in -> NULL out; no error even mixed with valid rows" + }, + { + "expr": "sin(i)", + "input_desc": "INTEGER column i = 1", + "result": "0.8414709848078965", + "result_type": "DOUBLE", + "note": "implicit int->double cast then sin; result type DOUBLE for ALL integer widths" + }, + { + "expr": "sin(b)", + "input_desc": "BIGINT column b = 9223372036854775807 (i64::MAX)", + "result": "0.9999303766734422", + "result_type": "DOUBLE", + "note": "cast rounds i64::MAX to 9.223372036854776e18 (2^63) FIRST, then sin; == math.sin(float(2**63)) bitwise" + }, + { + "expr": "sin(b)", + "input_desc": "BIGINT column b = -9223372036854775808 (i64::MIN)", + "result": "-0.9999303766734422", + "result_type": "DOUBLE", + "note": "bits 0xbfefff6dfd42dc54" + }, + { + "expr": "sin(b)", + "input_desc": "BIGINT column b = NULL / INTEGER NULL", + "result": "None", + "result_type": "DOUBLE", + "note": "NULL propagates through the cast+fn" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 0.0 and x = -0.0", + "result": "1.0 (both)", + "result_type": "DOUBLE", + "note": "cos(-0.0) = +1.0, no sign artifact" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 1.0", + "result": "0.5403023058681398", + "result_type": "DOUBLE", + "note": "bits 0x3fe14a280fb5068c; cos(-1.0) identical (even symmetry)" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 0.5", + "result": "0.8775825618903728", + "result_type": "DOUBLE", + "note": "bits 0x3fec1528065b7d50" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 1.5707963267948966 (pi()/2)", + "result": "6.123233995736766e-17", + "result_type": "DOUBLE", + "note": "POSITIVE, not zero; bits 0x3c91a62633145c07; cos(-pi/2) gives the same positive value" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 3.141592653589793 (pi())", + "result": "-1.0", + "result_type": "DOUBLE", + "note": "exactly -1.0 (0xbff0000000000000)" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 6.283185307179586 (2*pi)", + "result": "1.0", + "result_type": "DOUBLE", + "note": "exactly 1.0" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 1e15", + "result": "-0.5131937377869702", + "result_type": "DOUBLE", + "note": "bits 0xbfe06c154609d33e" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 9007199254740992.0 (2^53)", + "result": "-0.5285117844130887", + "result_type": "DOUBLE", + "note": "bits 0xbfe0e9918bb35aac" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 1e300", + "result": "-0.5753861119575491", + "result_type": "DOUBLE", + "note": "bits 0xbfe2699022adc4c1" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 1.7976931348623157e+308 (DBL_MAX)", + "result": "-0.9999876894265599", + "result_type": "DOUBLE", + "note": "bits 0xbfefffe62ecfab75" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 5e-324 / 2.2250738585072014e-308", + "result": "1.0 (both)", + "result_type": "DOUBLE", + "note": "exactly 1.0" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = 9.223372036854776e+18", + "result": "0.011800076512800236", + "result_type": "DOUBLE", + "note": "bits 0x3f882aa375b3c33e" + }, + { + "expr": "cos(x)", + "input_desc": "DOUBLE column x = NaN / NULL", + "result": "nan (input bits preserved) / None", + "result_type": "DOUBLE", + "note": "same NaN-bit passthrough and NULL propagation as sin" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 0.0", + "result": "0.0", + "result_type": "DOUBLE", + "note": "+0.0" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = -0.0", + "result": "-0.0", + "result_type": "DOUBLE", + "note": "sign preserved, bits 0x8000000000000000" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 1.0", + "result": "1.5574077246549023", + "result_type": "DOUBLE", + "note": "bits 0x3ff8eb245cbee3a6; tan(-1.0) = exact negation" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 0.5", + "result": "0.5463024898437905", + "result_type": "DOUBLE", + "note": "bits 0x3fe17b4f5bf3474a" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 1.5707963267948966 (pi()/2)", + "result": "1.633123935319537e+16", + "result_type": "DOUBLE", + "note": "LARGE FINITE value, not inf, not an error; bits 0x434d02967c31cdb5; tan(-pi/2) = -1.633123935319537e+16" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 3.141592653589793 (pi())", + "result": "-1.2246467991473532e-16", + "result_type": "DOUBLE", + "note": "negative; bits 0xbca1a62633145c07" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 6.283185307179586 (2*pi)", + "result": "-2.4492935982947064e-16", + "result_type": "DOUBLE", + "note": "same bits as sin(2*pi) here" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 1e15", + "result": "-1.672414782127583", + "result_type": "DOUBLE", + "note": "bits 0xbffac23600a95be4" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 9007199254740992.0 (2^53)", + "result": "1.6062574002155612", + "result_type": "DOUBLE", + "note": "bits 0x3ff9b33af5ae241f" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 1e300", + "result": "1.4214488238747245", + "result_type": "DOUBLE", + "note": "bits 0x3ff6be411f37ac77" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 1.7976931348623157e+308 (DBL_MAX)", + "result": "-0.004962015874444895", + "result_type": "DOUBLE", + "note": "bits 0xbf74530cfe729484" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 5e-324 (min denormal)", + "result": "5e-324", + "result_type": "DOUBLE", + "note": "input bits returned (0x0000000000000001)" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = 9.223372036854776e+18", + "result": "84.73931296875567", + "result_type": "DOUBLE", + "note": "bits 0x40552f50e757941d" + }, + { + "expr": "tan(x)", + "input_desc": "DOUBLE column x = NaN / NULL", + "result": "nan (input bits preserved) / None", + "result_type": "DOUBLE", + "note": "same NaN-bit passthrough and NULL propagation as sin/cos" + }, + { + "expr": "pi()", + "input_desc": "niladic, any context (literal and inside vectorized x + pi())", + "result": "3.141592653589793", + "result_type": "DOUBLE", + "note": "bits 0x400921fb54442d18 == Python math.pi EXACTLY; pi()/2 == math.pi/2 bitwise (0x3ff921fb54442d18)" + }, + { + "expr": "sin(f)", + "input_desc": "FLOAT (REAL) column f = 1.5", + "result": "0.9974949866040544", + "result_type": "DOUBLE", + "note": "f32 widened to f64 first, then sin; == math.sin(1.5) bitwise; FLOAT -0.0 -> sin gives 0.0 for +0.0 input as expected" + }, + { + "expr": "sin(CAST(3 AS SMALLINT)) / sin(CAST(18446744073709551615 AS UBIGINT)) / sin(HUGEINT i128::MAX)", + "input_desc": "other integer widths", + "result": "0.1411200080598672 / 0.023598509904439558 / 0.6233855129558702", + "result_type": "DOUBLE", + "note": "every integer width implicitly casts to DOUBLE (round-nearest) then sin; result type always DOUBLE" + }, + { + "expr": "sin(1.5) / sin(CAST(1.5 AS DECIMAL(10,2)))", + "input_desc": "DECIMAL literal and typed DECIMAL", + "result": "0.9974949866040544", + "result_type": "DOUBLE", + "note": "DECIMAL implicitly casts to DOUBLE" + }, + { + "expr": "sin('1.5')", + "input_desc": "VARCHAR string LITERAL (not a column)", + "result": "0.9974949866040544", + "result_type": "DOUBLE", + "note": "string literals ARE implicitly castable to DOUBLE at bind time; a VARCHAR COLUMN is a binder error (see errors)" + }, + { + "expr": "CAST(-0.0 AS DOUBLE) vs -0.0::DOUBLE", + "input_desc": "SQL literal parsing of negative zero", + "result": "0.0 vs -0.0", + "result_type": "DOUBLE", + "note": "CAST(-0.0 AS DOUBLE) = +0.0 because unary minus applies to a DECIMAL (no signed zero); '::' binds tighter than unary minus so -0.0::DOUBLE = -(0.0::DOUBLE) = -0.0. Column-fed -0.0 is real -0.0. Aside from this parsing artifact, the constant-fold path and the vectorized path agreed BITWISE on every edge probed (incl. the inf error)." + }, + { + "expr": "sin/cos/tan differential vs platform libm", + "input_desc": "10000 random doubles: uniform +-1e3, +-1e9, +-1e16, and ldexp exponents -1020..1020", + "result": "0 bitwise mismatches for all three functions", + "result_type": "DOUBLE", + "note": "DuckDB's finite-input trig is bit-identical to the platform C runtime libm (UCRT on this win-amd64 box), including huge-argument reduction. Rust f64::sin/cos/tan lower to the same platform libm call, so bit-parity is expected per platform; cross-OS bit-identity is NOT guaranteed by this measurement." + } + ], + "errors": [ + { + "expr": "SELECT sin(x) FROM t -- row with x = +inf (same for cos, tan)", + "error_head": "Out of Range Error: input value inf is out of range for numeric function", + "kind": "runtime" + }, + { + "expr": "SELECT sin(x) FROM t -- row with x = -inf (same for cos, tan)", + "error_head": "Out of Range Error: input value -inf is out of range for numeric function", + "kind": "runtime" + }, + { + "expr": "SELECT sin(v) FROM s -- v VARCHAR column (even holding '1.5')", + "error_head": "Binder Error: No function matches the given name and argument types 'sin(VARCHAR)'", + "kind": "binder" + }, + { + "expr": "SELECT sin('abc') -- string literal, implicit cast attempted then fails", + "error_head": "Conversion Error: Could not convert string 'abc' to DOUBLE", + "kind": "binder" + }, + { + "expr": "SELECT sin(TRUE)", + "error_head": "Binder Error: No function matches the given name and argument types 'sin(BOOLEAN)'", + "kind": "binder" + }, + { + "expr": "SELECT sin(DATE '2020-01-01')", + "error_head": "Binder Error: No function matches the given name and argument types 'sin(DATE)'", + "kind": "binder" + }, + { + "expr": "SELECT sin()", + "error_head": "Binder Error: No function matches the given name and argument types 'sin()'", + "kind": "binder" + }, + { + "expr": "SELECT sin(1, 2)", + "error_head": "Binder Error: No function matches the given name and argument types 'sin(INTEGER_LITERAL, INTEGER_LITERAL)'", + "kind": "binder" + }, + { + "expr": "SELECT pi(1)", + "error_head": "Binder Error: No function matches the given name and argument types 'pi(INTEGER_LITERAL)'", + "kind": "binder" + } + ], + "duck_check_tests": "import math\n\n# doubles that exercise every value pin reachable through a column\nTRIG_EDGE_FLOATS = [\n 0.0, -0.0, 1.0, -1.0, 0.5,\n math.pi / 2, -math.pi / 2, math.pi, 2 * math.pi, # pi()-derived bit patterns\n 1e15, float(2**53), 1e300, 1.7976931348623157e308, # huge args: reduction fidelity\n 5e-324, 2.2250738585072014e-308, # denormal / min normal\n 9.223372036854776e18, # i64::MAX rounded to f64\n float(\"nan\"), # quiet NaN passthrough (repr 'nan' compares equal)\n]\n\n\ndef test_trig_double_column_edges():\n duck_check(\n \"SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": v} for v in TRIG_EDGE_FLOATS],\n ) # repr-compare distinguishes -0.0 from 0.0, so the sin/tan(-0.0) sign pins are covered\n\n\ndef test_trig_null_propagation():\n duck_check(\n \"SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__\",\n {\"x\": \"float?\"},\n [{\"x\": None}, {\"x\": 1.0}],\n )\n\n\ndef test_trig_integer_input_yields_double():\n duck_check(\n \"SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__\",\n {\"x\": \"int\"},\n [\n {\"x\": 0},\n {\"x\": 1},\n {\"x\": -1},\n {\"x\": 9223372036854775807}, # rounds to 2^63 as f64 BEFORE sin\n {\"x\": -9223372036854775808},\n ],\n )\n\n\ndef test_trig_integer_null():\n duck_check(\n \"SELECT sin(x) AS s FROM __THIS__\",\n {\"x\": \"int?\"},\n [{\"x\": None}, {\"x\": 3}],\n )\n\n\ndef test_pi_literal_and_vectorized():\n duck_check(\n \"SELECT pi() AS p, x + pi() AS xp, sin(pi() / 2) AS one, \"\n \"cos(pi()) AS neg_one, tan(pi() / 2) AS big, sin(-pi() / 2) AS neg_one_s \"\n \"FROM __THIS__\",\n {\"x\": \"float\"},\n [{\"x\": 0.0}, {\"x\": 1.0}],\n )\n\n\n# NOT expressible via duck_check (it asserts success on both sides) \u2014 engine needs\n# dedicated tests for these, asserting against a plain duckdb replay:\n# 1. sin/cos/tan(+-inf) must raise a per-row runtime error whose message mirrors\n# \"Out of Range Error: input value inf is out of range for numeric function\"\n# (and \"... value -inf ...\"); an inf row excluded by WHERE must NOT raise.\n# 2. NaN *payload* preservation (e.g. input bits 0x7ff0deadbeef0001 out unchanged):\n# repr collapses every NaN to 'nan', so compare struct.pack bits directly.", + "lowering": "New IR ops \u2014 there is nothing to desugar to (no existing IR primitive computes a transcendental). Add three unary f64->f64 ops: SinF64, CosF64, TanF64. pi() is the only pure-frontend piece: fold it to the f64 literal with bits 0x400921FB54442D18 (== Rust std::f64::consts::PI == Python math.pi, measured bit-equal to DuckDB). Integer/FLOAT/DECIMAL arguments reuse the existing cast-to-f64 op in front (round-nearest; i64::MAX -> 9.223372036854776e18, measured).\n\nRust semantics (identical shape for all three; only the libm call differs):\n\n #[inline]\n fn duck_sin(x: f64) -> Result {\n if x.is_nan() {\n return Ok(x); // MUST return the input bits untouched \u2014 DuckDB preserves\n // NaN payload+sign exactly (even sNaN 0x7ff0deadbeef0001);\n // calling libm here would quiet the payload and diverge.\n }\n if x.is_infinite() {\n return Err(EngineError::out_of_range(format!(\n \"input value {} is out of range for numeric function\",\n if x > 0.0 { \"inf\" } else { \"-inf\" },\n ))); // per-row runtime trap, mirrors DuckDB OutOfRangeException\n }\n Ok(x.sin()) // platform libm; 10k-sample differential vs DuckDB: 0 bit mismatches\n }\n\nTrap condition: ONLY +-inf. NaN is a non-trap passthrough; every finite double (incl. denormals, DBL_MAX, 1e300) is a plain libm call. NULL propagation is the generic unary null mask \u2014 and note rows removed by WHERE must never evaluate the op (measured: an inf row filtered out does not trap), so the JIT must not speculatively execute trig over inactive lanes.\n\nCranelift backend: there is no native sin/cos/tan instruction \u2014 emit a call to the same extern \"C\" wrapper functions the interpreter uses (nan-check + inf-trap + libm), NOT a bare libm call, so both backends share one symbol and NaN-payload/trap behavior is identical by construction.\n\nBinder-level rules the frontend must enforce: sin/cos/tan accept all numeric types (any int width, FLOAT, DOUBLE, DECIMAL) -> DOUBLE; VARCHAR columns and BOOLEAN are binder errors (no implicit cast), while string LITERALS are cast at bind time ('1.5' works, 'abc' is a bind-time conversion error); arity is fixed (sin(), sin(1,2), pi(1) are binder errors).", + "surprises": [ + "sin/cos/tan(+-inf) is a per-row RUNTIME ERROR in DuckDB (OutOfRangeException: 'Out of Range Error: input value inf is out of range for numeric function'), not NaN \u2014 contradicts IEEE-754/libm behavior (libm returns NaN). Message spells 'inf' / '-inf'.", + "NaN input passes through BIT-EXACTLY: payload and sign preserved, even a signaling-NaN pattern (0x7ff0deadbeef0001 in -> 0x7ff0deadbeef0001 out) \u2014 DuckDB never hands NaN to libm; libm (and a naive Rust x.sin()) would quiet it to 0x7ff8deadbeef0001. An is_nan()-first passthrough branch is mandatory for bit parity.", + "For every finite input, DuckDB trig is bitwise identical to the platform C-runtime libm: 0/10000 mismatches on random doubles spanning exponents up to ~2^1020, including 1e300 and DBL_MAX \u2014 full argument-reduction fidelity. Rust f64::sin/cos/tan call the same platform libm, so same-platform bit parity should be free; but the oracle is platform-libm-dependent, so cross-OS bit-identity of pins (e.g. sin(1e300) = -0.8178819121159085) must be re-verified on the CI/serving platform.", + "SQL literal -0.0 is +0.0: unary minus binds to a DECIMAL literal which has no signed zero, so CAST(-0.0 AS DOUBLE) = +0.0; but -0.0::DOUBLE = -0.0 because '::' binds tighter than unary minus. Column-fed -0.0 is genuine: sin(-0.0) = -0.0, tan(-0.0) = -0.0, cos(-0.0) = +1.0.", + "tan(pi()/2) is a large finite 1.633123935319537e+16 \u2014 not inf, not an error (nearest-double pi/2 is not the exact pole).", + "sin(pi()/2) is exactly 1.0 and cos(pi()) exactly -1.0, but sin(pi()) = 1.2246467991473532e-16 (not 0) and cos(pi()/2) = +6.123233995736766e-17 (positive, not 0).", + "pi() is bit-equal to Python math.pi / Rust f64 PI (0x400921fb54442d18); fold it in the frontend, no IR op needed.", + "VARCHAR COLUMNS are a binder error for trig (no implicit varchar->double cast from columns), yet string LITERALS are implicitly cast at bind time: sin('1.5') works, sin('abc') fails with a Conversion Error (not a Binder Error). BOOLEAN input is a binder error.", + "Aside from the -0.0 literal-parsing artifact, the constant-fold path and the vectorized column path agreed bitwise on every probed edge \u2014 no fold/vector divergence in this family.", + "Integer inputs of EVERY width (SMALLINT/INTEGER/BIGINT/UBIGINT/HUGEINT) and DECIMAL implicitly cast to DOUBLE; result type is always DOUBLE. i64::MAX rounds to 9.223372036854776e18 (2^63) before the trig call, so sin(i64::MAX) == sin(9.223372036854776e18)." + ] +} \ No newline at end of file diff --git a/scripts/gen_pow10.py b/scripts/gen_pow10.py new file mode 100644 index 0000000..ec7ca35 --- /dev/null +++ b/scripts/gen_pow10.py @@ -0,0 +1,57 @@ +"""Extract DuckDB's own pow(10, k) table into src/specializer/exec/pow10.rs. + +round(x, n) / trunc(x, n) scale by the DuckDB binary's std::pow(10, n), +which is neither correctly rounded (k=23 is one ulp above strtod's 1e23) +nor any fixed libm we can call at runtime — so the scale factors are +extracted from the oracle itself and baked as bit-exact constants, the +same playbook as the case-map exception table. + +Run: uv run python scripts/gen_pow10.py (idempotent) +""" + +from __future__ import annotations + +import struct +from pathlib import Path + +import duckdb + +OUT = Path(__file__).parent.parent / "src" / "specializer" / "exec" / "pow10.rs" + +# Witnesses measured 2026-07-26 (wave-1 pins): the table is NOT float("1e{k}") +# everywhere, and NOT ucrt pow everywhere — both must hold or extraction broke. +WITNESSES = {23: 0x44B52D02C7E14AF7, 126: 0x5A17A2ECC414A03F} + + +def main() -> None: + con = duckdb.connect() + bits: list[int] = [] + for k in range(309): + (v,) = con.execute("SELECT pow(10.0, CAST(? AS DOUBLE))", [k]).fetchone() + bits.append(struct.unpack(" f64 { a % b } +// Wave-1 math helpers: each delegates to the interpreter's shared semantic +// fn, so the backends physically cannot drift (pins spec 2026-07-26). +macro_rules! math1_h { + ($name:ident, $f:path) => { + extern "C" fn $name(p: *mut Cx, x: f64) -> f64 { + match $f(x) { + Ok(v) => v, + Err(t) => { + unsafe { cx(p) }.set_trap(t.0); + 0.0 + } + } + } + }; +} +math1_h!(h_ln, interp::duck_ln); +math1_h!(h_log2, interp::duck_log2); +math1_h!(h_log10, interp::duck_log10); +math1_h!(h_fsqrt, interp::duck_sqrt); +math1_h!(h_fsin, interp::duck_sin); +math1_h!(h_fcos, interp::duck_cos); +math1_h!(h_ftan, interp::duck_tan); + +extern "C" fn h_sfind(p: *mut Cx, ao: i64, al: i64, bo: i64, bl: i64) -> i64 { + let c = unsafe { cx(p) }; + let arena = unsafe { &*c.arena }; + interp::str_find(arena.get(span(ao, al)), arena.get(span(bo, bl))) +} + +extern "C" fn h_spred(p: *mut Cx, which: i64, ao: i64, al: i64, bo: i64, bl: i64) -> u8 { + let c = unsafe { cx(p) }; + let arena = unsafe { &*c.arena }; + let op = match which { + 0 => StrOp2::Contains, + 1 => StrOp2::Starts, + _ => StrOp2::Ends, + }; + interp::str_pred(op, arena.get(span(ao, al)), arena.get(span(bo, bl))) as u8 +} + +extern "C" fn h_slen(p: *mut Cx, bytes: i64, ao: i64, al: i64) -> i64 { + let c = unsafe { cx(p) }; + let s = unsafe { &*c.arena }.get(span(ao, al)); + if bytes != 0 { + s.len() as i64 + } else { + s.chars().count() as i64 + } +} + +extern "C" fn h_round2f(x: f64, n: i64, trunc: i64) -> f64 { + if trunc != 0 { + interp::trunc_prec_f64(x, n) + } else { + interp::round_prec_f64(x, n) + } +} + +extern "C" fn h_round2i(x: i64, n: i64, trunc: i64) -> i64 { + if trunc != 0 { + interp::trunc_prec_i64(x, n) + } else { + interp::round_prec_i64(x, n) + } +} + +extern "C" fn h_fexp(x: f64) -> f64 { + interp::duck_exp(x).expect("exp is total") +} + +extern "C" fn h_fcbrt(x: f64) -> f64 { + interp::duck_cbrt(x).expect("cbrt is total") +} + +extern "C" fn h_fpow(x: f64, y: f64) -> f64 { + interp::duck_pow(x, y).expect("pow is total") +} + +extern "C" fn h_flogb(p: *mut Cx, base: f64, x: f64) -> f64 { + match interp::duck_logb(base, x) { + Ok(v) => v, + Err(t) => { + unsafe { cx(p) }.set_trap(t.0); + 0.0 + } + } +} + extern "C" fn h_fround(a: f64) -> f64 { a.round() } @@ -883,13 +972,15 @@ fn translate_inst( BinOp::Fmul => b.ins().fmul(x, y), BinOp::Fdiv => b.ins().fdiv(x, y), BinOp::Frem => call_h(b, module, "h_frem", &[x, y]).unwrap(), + BinOp::Fpow => call_h(b, module, "h_fpow", &[x, y]).unwrap(), + BinOp::Flogb => call_h(b, module, "h_flogb", &[cxp, x, y]).unwrap(), BinOp::And => b.ins().band(x, y), BinOp::Or => b.ins().bor(x, y), BinOp::Xor => b.ins().bxor(x, y), }; if matches!( op, - BinOp::Iadd | BinOp::Isub | BinOp::Imul | BinOp::Idiv | BinOp::Irem + BinOp::Iadd | BinOp::Isub | BinOp::Imul | BinOp::Idiv | BinOp::Irem | BinOp::Flogb ) { trap_check(b); } @@ -990,6 +1081,43 @@ fn translate_inst( vals.insert(flag.0, V::S(f)); vals.insert(dst.0, V::S(v)); } + Inst::Round2f { trunc, dst, a, n } => { + let (x, nv) = (vals[&a.0].s(), vals[&n.0].s()); + let tv = icon(b, *trunc as i64); + let v = call_h(b, module, "h_round2f", &[x, nv, tv]).unwrap(); + vals.insert(dst.0, V::S(v)); + } + Inst::Round2i { trunc, dst, a, n } => { + let (x, nv) = (vals[&a.0].s(), vals[&n.0].s()); + let tv = icon(b, *trunc as i64); + let v = call_h(b, module, "h_round2i", &[x, nv, tv]).unwrap(); + vals.insert(dst.0, V::S(v)); + } + Inst::Str2 { op, dst, a, b: rhs } => { + let (ao, al) = vals[&a.0].str2(); + let (bo, bl) = vals[&rhs.0].str2(); + let v = match op { + StrOp2::Find => call_h(b, module, "h_sfind", &[cxp, ao, al, bo, bl]).unwrap(), + pred => { + let which = icon( + b, + match pred { + StrOp2::Contains => 0, + StrOp2::Starts => 1, + _ => 2, + }, + ); + call_h(b, module, "h_spred", &[cxp, which, ao, al, bo, bl]).unwrap() + } + }; + vals.insert(dst.0, V::S(v)); + } + Inst::SLen { bytes, dst, a } => { + let (ao, al) = vals[&a.0].str2(); + let bv = icon(b, *bytes as i64); + let v = call_h(b, module, "h_slen", &[cxp, bv, ao, al]).unwrap(); + vals.insert(dst.0, V::S(v)); + } Inst::Sconcat { dst, a, b: rhs } => { let (ao, al) = vals[&a.0].str2(); let (bo, bl) = vals[&rhs.0].str2(); @@ -1050,6 +1178,29 @@ fn translate_inst( } NumOp1::Fabs => b.ins().fabs(x), NumOp1::Fround => call_h(b, module, "h_fround", &[x]).unwrap(), + // Wave-1: floor/ceil/trunc are single CLIF instructions; + // total transcendentals are plain helpers; trapping ones + // get the standard flag check. + NumOp1::Ffloor => b.ins().floor(x), + NumOp1::Fceil => b.ins().ceil(x), + NumOp1::Ftrunc => b.ins().trunc(x), + NumOp1::Fexp => call_h(b, module, "h_fexp", &[x]).unwrap(), + NumOp1::Fcbrt => call_h(b, module, "h_fcbrt", &[x]).unwrap(), + trapping => { + let name = match trapping { + NumOp1::Ln => "h_ln", + NumOp1::Log2 => "h_log2", + NumOp1::Log10 => "h_log10", + NumOp1::Fsqrt => "h_fsqrt", + NumOp1::Fsin => "h_fsin", + NumOp1::Fcos => "h_fcos", + NumOp1::Ftan => "h_ftan", + other => unreachable!("non-trapping op {other:?} handled above"), + }; + let v = call_h(b, module, name, &[cxp, x]).unwrap(); + trap_check(b); + v + } }; vals.insert(dst.0, V::S(v)); } @@ -1317,6 +1468,22 @@ const HELPERS: &[(&str, *const u8)] = &[ ("h_store_str", h_store_str as *const u8), ("h_sload", h_sload as *const u8), ("h_probe", h_probe as *const u8), + ("h_ln", h_ln as *const u8), + ("h_log2", h_log2 as *const u8), + ("h_log10", h_log10 as *const u8), + ("h_fsqrt", h_fsqrt as *const u8), + ("h_fsin", h_fsin as *const u8), + ("h_fcos", h_fcos as *const u8), + ("h_ftan", h_ftan as *const u8), + ("h_fexp", h_fexp as *const u8), + ("h_fcbrt", h_fcbrt as *const u8), + ("h_fpow", h_fpow as *const u8), + ("h_flogb", h_flogb as *const u8), + ("h_sfind", h_sfind as *const u8), + ("h_spred", h_spred as *const u8), + ("h_slen", h_slen as *const u8), + ("h_round2f", h_round2f as *const u8), + ("h_round2i", h_round2i as *const u8), ]; fn helper_sig(name: &str, sig: &mut cranelift_codegen::ir::Signature, ptr: types::Type) { @@ -1329,7 +1496,17 @@ fn helper_sig(name: &str, sig: &mut cranelift_codegen::ir::Signature, ptr: types "h_const_str" => (&[ptr, I64, I64], Some(I64)), "h_iadd" | "h_isub" | "h_imul" | "h_idiv" | "h_irem" => (&[ptr, I64, I64], Some(I64)), "h_frem" => (&[F64, F64], Some(F64)), - "h_fround" => (&[F64], Some(F64)), + "h_fround" | "h_fexp" | "h_fcbrt" => (&[F64], Some(F64)), + "h_fpow" => (&[F64, F64], Some(F64)), + "h_ln" | "h_log2" | "h_log10" | "h_fsqrt" | "h_fsin" | "h_fcos" | "h_ftan" => { + (&[ptr, F64], Some(F64)) + } + "h_flogb" => (&[ptr, F64, F64], Some(F64)), + "h_sfind" => (&[ptr, I64, I64, I64, I64], Some(I64)), + "h_spred" => (&[ptr, I64, I64, I64, I64, I64], Some(I8)), + "h_slen" => (&[ptr, I64, I64, I64], Some(I64)), + "h_round2f" => (&[F64, I64, I64], Some(F64)), + "h_round2i" => (&[I64, I64, I64], Some(I64)), "h_iabs" => (&[ptr, I64], Some(I64)), "h_fcmp" => (&[F64, F64, I64], Some(I8)), "h_scmp" => (&[ptr, I64, I64, I64, I64, I64], Some(I8)), diff --git a/src/specializer/exec/interp.rs b/src/specializer/exec/interp.rs index 2a699c6..ddafd34 100644 --- a/src/specializer/exec/interp.rs +++ b/src/specializer/exec/interp.rs @@ -27,8 +27,8 @@ use std::collections::HashMap; use super::super::ir::verify::{verify, VerifyError}; use super::super::ir::{ - self, BinOp, CmpPred, Inst, NumOp1, Program, RoundMode, StaticTy, StrOp1, Term, TrimSide, Ty, - Value, + self, BinOp, CmpPred, Inst, NumOp1, Program, RoundMode, StaticTy, StrOp1, StrOp2, Term, + TrimSide, Ty, Value, }; use super::{ Arena, Batch, ColData, KeyBits, OutCol, RegVal, RunState, ScalarVal, StaticData, StrRef, Trap, @@ -631,6 +631,228 @@ impl std::fmt::Write for StackStr { } } +// ------------------------------------------------- wave-1 math semantics -- +// One shared fn per op, used verbatim by BOTH backends (pins: +// docs/superpowers/specs/2026-07-26-wave1-builtin-pins.md). Trap messages +// are DuckDB 1.5.5's own, measured — including its typo. + +fn log_guard(x: f64) -> Result<(), Trap> { + // Comparison-based, not is_finite-based: NaN fails both checks and + // flows through to libm; +inf passes (measured). + if x == 0.0 { + return Err(Trap("cannot take logarithm of zero".into())); + } + if x < 0.0 { + return Err(Trap("cannot take logarithm of a negative number".into())); + } + Ok(()) +} + +pub(super) fn duck_ln(x: f64) -> Result { + log_guard(x)?; + Ok(if x.is_nan() { x } else { x.ln() }) +} +pub(super) fn duck_log2(x: f64) -> Result { + log_guard(x)?; + Ok(if x.is_nan() { x } else { x.log2() }) +} +pub(super) fn duck_log10(x: f64) -> Result { + log_guard(x)?; + Ok(if x.is_nan() { x } else { x.log10() }) +} +/// log(base, x): bit-exactly log10(x)/log10(base) — NOT an ln ratio +/// (refuted on 20k fuzz samples). Base is domain-checked FIRST; base==1 +/// carries DuckDB's message verbatim, typo included. +pub(super) fn duck_logb(base: f64, x: f64) -> Result { + log_guard(base)?; + if base == 1.0 { + return Err(Trap("divison by zero in based logarithm".into())); + } + log_guard(x)?; + Ok(x.log10() / base.log10()) +} +pub(super) fn duck_exp(x: f64) -> Result { + // TOTAL: overflow -> inf, underflow -> denormals -> +0.0 (measured). + Ok(x.exp()) +} +pub(super) fn duck_sqrt(x: f64) -> Result { + if x < 0.0 { + return Err(Trap("cannot take square root of a negative number".into())); + } + Ok(x.sqrt()) // sqrt(-0.0) = -0.0 (not negative, not a trap); NaN passes +} +pub(super) fn duck_cbrt(x: f64) -> Result { + Ok(x.cbrt()) // TOTAL; cbrt(-8) = -2.0 exactly (NOT pow(x, 1/3)) +} +fn trig_guard(x: f64) -> Result<(), Trap> { + if x.is_infinite() { + return Err(Trap(format!( + "input value {} is out of range for numeric function", + if x > 0.0 { "inf" } else { "-inf" } + ))); + } + Ok(()) +} +pub(super) fn duck_sin(x: f64) -> Result { + trig_guard(x)?; + // NaN passes through BIT-EXACTLY (payload + sign) — never hand it to + // libm (measured: DuckDB preserves even signaling patterns). + Ok(if x.is_nan() { x } else { x.sin() }) +} +pub(super) fn duck_cos(x: f64) -> Result { + trig_guard(x)?; + Ok(if x.is_nan() { x } else { x.cos() }) +} +pub(super) fn duck_tan(x: f64) -> Result { + trig_guard(x)?; + Ok(if x.is_nan() { x } else { x.tan() }) +} +pub(super) fn duck_pow(x: f64, y: f64) -> Result { + // TOTAL, pure IEEE: pow(NaN,0)=1, pow(1,NaN)=1, pow(0,-1)=inf, + // negative^fractional=NaN, overflow=inf (all measured). + Ok(x.powf(y)) +} +pub(super) fn duck_floor(x: f64) -> Result { + Ok(x.floor()) +} +pub(super) fn duck_ceil(x: f64) -> Result { + Ok(x.ceil()) +} +pub(super) fn duck_trunc(x: f64) -> Result { + Ok(x.trunc()) +} + +/// The wave-1 f64 unaries as shared fn pointers (Iabs/Fabs/Fround keep +/// their original arms). +pub(super) fn math1_fn(op: NumOp1) -> fn(f64) -> Result { + match op { + NumOp1::Ln => duck_ln, + NumOp1::Log2 => duck_log2, + NumOp1::Log10 => duck_log10, + NumOp1::Fexp => duck_exp, + NumOp1::Fsqrt => duck_sqrt, + NumOp1::Fcbrt => duck_cbrt, + NumOp1::Fsin => duck_sin, + NumOp1::Fcos => duck_cos, + NumOp1::Ftan => duck_tan, + NumOp1::Ffloor => duck_floor, + NumOp1::Fceil => duck_ceil, + NumOp1::Ftrunc => duck_trunc, + NumOp1::Iabs | NumOp1::Fabs | NumOp1::Fround => { + unreachable!("legacy unaries keep dedicated arms") + } + } +} + +/// DuckDB's pow(10, k) — the oracle-extracted table; inf beyond 308. +fn pow10(k: i64) -> f64 { + if (0..=308).contains(&k) { + super::pow10::POW10[k as usize] + } else { + f64::INFINITY + } +} + +/// round(x, n) on f64 — scale-then-round with the oracle pow table. +/// Non-finite results fall back to the INPUT for n >= 0 and to +0.0 for +/// n < 0 (measured asymmetry: round(NaN, -2) = 0.0). +pub(super) fn round_prec_f64(x: f64, n: i64) -> f64 { + if n >= 0 { + let m = pow10(n); + let r = (x * m).round() / m; + if r.is_infinite() || r.is_nan() { + x + } else { + r + } + } else { + let m = pow10(n.unsigned_abs() as i64); + let r = (x / m).round() * m; + if r.is_infinite() || r.is_nan() { + 0.0 + } else { + r + } + } +} + +/// trunc(x, n) on f64: same shape as round, but BOTH branches fall back +/// to the input — the round/trunc asymmetry is measured, not a bug. +pub(super) fn trunc_prec_f64(x: f64, n: i64) -> f64 { + if n >= 0 { + let m = pow10(n); + let r = (x * m).trunc() / m; + if r.is_infinite() || r.is_nan() { + x + } else { + r + } + } else { + let m = pow10(n.unsigned_abs() as i64); + let r = (x / m).trunc() * m; + if r.is_infinite() || r.is_nan() { + x + } else { + r + } + } +} + +/// Integer round with digits: identity for n >= 0; n < 0 WRAPS at i64 +/// (measured: round(i64::MAX, -2) = -9223372036854775700) — never traps. +pub(super) fn round_prec_i64(x: i64, n: i64) -> i64 { + if n >= 0 { + return x; + } + let p = n.unsigned_abs(); + if p >= 19 { + return 0; + } + let power = 10i64.pow(p as u32); + let half = power / 2; + let y = if x >= 0 { + x.wrapping_add(half) + } else { + x.wrapping_sub(half) + }; + (y / power) * power +} + +/// Integer trunc with digits: identity for n >= 0, truncating scale for +/// n < 0 — no half-add, never wraps. +pub(super) fn trunc_prec_i64(x: i64, n: i64) -> i64 { + if n >= 0 { + return x; + } + let p = n.unsigned_abs(); + if p >= 19 { + return 0; + } + let power = 10i64.pow(p as u32); + (x / power) * power +} + +/// Wave-1 string search (pins: 1-based CODEPOINT positions, empty needle +/// matches everything, byte-wise comparison, zero unicode intelligence). +pub(super) fn str_find(s: &str, n: &str) -> i64 { + if n.is_empty() { + return 1; + } + match s.find(n) { + None => 0, + Some(byte) => s[..byte].chars().count() as i64 + 1, + } +} + +pub(super) fn str_pred(op: StrOp2, s: &str, n: &str) -> bool { + match op { + StrOp2::Contains => s.contains(n), + StrOp2::Starts => s.starts_with(n), + StrOp2::Ends => s.ends_with(n), + StrOp2::Find => unreachable!("find returns i64, handled separately"), + } +} + /// Value id -> dense register slot. fn sl(slots: &HashMap, v: Value) -> usize { slots[&v.0] as usize @@ -694,20 +916,30 @@ fn compile_inst(p: &Program, inst: &Inst, slots: &HashMap) -> InstFn { None => Err(Trap(format!("i64 overflow in {}", op.name()))), } }), - BinOp::Fadd | BinOp::Fsub | BinOp::Fmul | BinOp::Fdiv | BinOp::Frem => { - Box::new(move |ctx| { - let (x, y) = (as_f64(ctx.regs[a]), as_f64(ctx.regs[b])); - let v = match op { - BinOp::Fadd => x + y, - BinOp::Fsub => x - y, - BinOp::Fmul => x * y, - BinOp::Fdiv => x / y, - _ => x % y, - }; - ctx.regs[dst] = RegVal::F64(v); - Ok(()) - }) - } + BinOp::Fadd + | BinOp::Fsub + | BinOp::Fmul + | BinOp::Fdiv + | BinOp::Frem + | BinOp::Fpow => Box::new(move |ctx| { + let (x, y) = (as_f64(ctx.regs[a]), as_f64(ctx.regs[b])); + let v = match op { + BinOp::Fadd => x + y, + BinOp::Fsub => x - y, + BinOp::Fmul => x * y, + BinOp::Fdiv => x / y, + BinOp::Fpow => duck_pow(x, y)?, + _ => x % y, + }; + ctx.regs[dst] = RegVal::F64(v); + Ok(()) + }), + // log(base, x): a is the base (SQL argument order). + BinOp::Flogb => Box::new(move |ctx| { + let (base, x) = (as_f64(ctx.regs[a]), as_f64(ctx.regs[b])); + ctx.regs[dst] = RegVal::F64(duck_logb(base, x)?); + Ok(()) + }), BinOp::And | BinOp::Or | BinOp::Xor => Box::new(move |ctx| { let (x, y) = (as_i1(ctx.regs[a]), as_i1(ctx.regs[b])); let v = match op { @@ -840,6 +1072,60 @@ fn compile_inst(p: &Program, inst: &Inst, slots: &HashMap) -> InstFn { Ok(()) }) } + Inst::Round2f { trunc, dst, a, n } => { + let (dst, a, n) = (sl(slots, dst), sl(slots, a), sl(slots, n)); + let f = if trunc { + trunc_prec_f64 + } else { + round_prec_f64 + }; + Box::new(move |ctx| { + ctx.regs[dst] = RegVal::F64(f(as_f64(ctx.regs[a]), as_i64(ctx.regs[n]))); + Ok(()) + }) + } + Inst::Round2i { trunc, dst, a, n } => { + let (dst, a, n) = (sl(slots, dst), sl(slots, a), sl(slots, n)); + let f = if trunc { + trunc_prec_i64 + } else { + round_prec_i64 + }; + Box::new(move |ctx| { + ctx.regs[dst] = RegVal::I64(f(as_i64(ctx.regs[a]), as_i64(ctx.regs[n]))); + Ok(()) + }) + } + Inst::Str2 { op, dst, a, b } => { + let (dst, a, b) = (sl(slots, dst), sl(slots, a), sl(slots, b)); + match op { + StrOp2::Find => Box::new(move |ctx| { + let s = ctx.arena.get(as_str(ctx.regs[a])); + let n = ctx.arena.get(as_str(ctx.regs[b])); + ctx.regs[dst] = RegVal::I64(str_find(s, n)); + Ok(()) + }), + op => Box::new(move |ctx| { + let s = ctx.arena.get(as_str(ctx.regs[a])); + let n = ctx.arena.get(as_str(ctx.regs[b])); + ctx.regs[dst] = RegVal::I1(str_pred(op, s, n)); + Ok(()) + }), + } + } + Inst::SLen { bytes, dst, a } => { + let (dst, a) = (sl(slots, dst), sl(slots, a)); + Box::new(move |ctx| { + let s = ctx.arena.get(as_str(ctx.regs[a])); + let v = if bytes { + s.len() as i64 + } else { + s.chars().count() as i64 + }; + ctx.regs[dst] = RegVal::I64(v); + Ok(()) + }) + } Inst::Sconcat { dst, a, b } => { let (dst, a, b) = (sl(slots, dst), sl(slots, a), sl(slots, b)); Box::new(move |ctx| { @@ -934,6 +1220,13 @@ fn compile_inst(p: &Program, inst: &Inst, slots: &HashMap) -> InstFn { ctx.regs[dst] = RegVal::F64(as_f64(ctx.regs[a]).round()); Ok(()) }), + op => { + let f = math1_fn(op); + Box::new(move |ctx| { + ctx.regs[dst] = RegVal::F64(f(as_f64(ctx.regs[a]))?); + Ok(()) + }) + } } } Inst::Load { dst, col } => { diff --git a/src/specializer/exec/mod.rs b/src/specializer/exec/mod.rs index 73a3850..84099c5 100644 --- a/src/specializer/exec/mod.rs +++ b/src/specializer/exec/mod.rs @@ -13,6 +13,7 @@ pub mod casemap; pub mod cranelift; pub mod interp; +mod pow10; #[cfg(test)] mod tests; diff --git a/src/specializer/exec/pow10.rs b/src/specializer/exec/pow10.rs new file mode 100644 index 0000000..79f9e30 --- /dev/null +++ b/src/specializer/exec/pow10.rs @@ -0,0 +1,318 @@ +//! GENERATED by scripts/gen_pow10.py — do not edit. +//! +//! DuckDB 1.5.5's own std::pow(10, k) for k in 0..=308, bit-exact +//! (k=23 is one ulp above strtod 1e23; see the wave-1 pins spec). +//! round(x, n)/trunc(x, n) must scale by THESE values, never a +//! runtime powi/powf. + +pub(super) const POW10: [f64; 309] = [ + f64::from_bits(0x3FF0000000000000), // 1e0 + f64::from_bits(0x4024000000000000), // 1e1 + f64::from_bits(0x4059000000000000), // 1e2 + f64::from_bits(0x408F400000000000), // 1e3 + f64::from_bits(0x40C3880000000000), // 1e4 + f64::from_bits(0x40F86A0000000000), // 1e5 + f64::from_bits(0x412E848000000000), // 1e6 + f64::from_bits(0x416312D000000000), // 1e7 + f64::from_bits(0x4197D78400000000), // 1e8 + f64::from_bits(0x41CDCD6500000000), // 1e9 + f64::from_bits(0x4202A05F20000000), // 1e10 + f64::from_bits(0x42374876E8000000), // 1e11 + f64::from_bits(0x426D1A94A2000000), // 1e12 + f64::from_bits(0x42A2309CE5400000), // 1e13 + f64::from_bits(0x42D6BCC41E900000), // 1e14 + f64::from_bits(0x430C6BF526340000), // 1e15 + f64::from_bits(0x4341C37937E08000), // 1e16 + f64::from_bits(0x4376345785D8A000), // 1e17 + f64::from_bits(0x43ABC16D674EC800), // 1e18 + f64::from_bits(0x43E158E460913D00), // 1e19 + f64::from_bits(0x4415AF1D78B58C40), // 1e20 + f64::from_bits(0x444B1AE4D6E2EF50), // 1e21 + f64::from_bits(0x4480F0CF064DD592), // 1e22 + f64::from_bits(0x44B52D02C7E14AF7), // 1e23 + f64::from_bits(0x44EA784379D99DB4), // 1e24 + f64::from_bits(0x45208B2A2C280291), // 1e25 + f64::from_bits(0x4554ADF4B7320335), // 1e26 + f64::from_bits(0x4589D971E4FE8402), // 1e27 + f64::from_bits(0x45C027E72F1F1281), // 1e28 + f64::from_bits(0x45F431E0FAE6D721), // 1e29 + f64::from_bits(0x46293E5939A08CEA), // 1e30 + f64::from_bits(0x465F8DEF8808B024), // 1e31 + f64::from_bits(0x4693B8B5B5056E17), // 1e32 + f64::from_bits(0x46C8A6E32246C99C), // 1e33 + f64::from_bits(0x46FED09BEAD87C03), // 1e34 + f64::from_bits(0x4733426172C74D82), // 1e35 + f64::from_bits(0x476812F9CF7920E3), // 1e36 + f64::from_bits(0x479E17B84357691B), // 1e37 + f64::from_bits(0x47D2CED32A16A1B1), // 1e38 + f64::from_bits(0x48078287F49C4A1D), // 1e39 + f64::from_bits(0x483D6329F1C35CA5), // 1e40 + f64::from_bits(0x48725DFA371A19E7), // 1e41 + f64::from_bits(0x48A6F578C4E0A061), // 1e42 + f64::from_bits(0x48DCB2D6F618C879), // 1e43 + f64::from_bits(0x4911EFC659CF7D4C), // 1e44 + f64::from_bits(0x49466BB7F0435C9E), // 1e45 + f64::from_bits(0x497C06A5EC5433C6), // 1e46 + f64::from_bits(0x49B18427B3B4A05C), // 1e47 + f64::from_bits(0x49E5E531A0A1C873), // 1e48 + f64::from_bits(0x4A1B5E7E08CA3A8F), // 1e49 + f64::from_bits(0x4A511B0EC57E649A), // 1e50 + f64::from_bits(0x4A8561D276DDFDC0), // 1e51 + f64::from_bits(0x4ABABA4714957D30), // 1e52 + f64::from_bits(0x4AF0B46C6CDD6E3E), // 1e53 + f64::from_bits(0x4B24E1878814C9CE), // 1e54 + f64::from_bits(0x4B5A19E96A19FC41), // 1e55 + f64::from_bits(0x4B905031E2503DA9), // 1e56 + f64::from_bits(0x4BC4643E5AE44D13), // 1e57 + f64::from_bits(0x4BF97D4DF19D6057), // 1e58 + f64::from_bits(0x4C2FDCA16E04B86D), // 1e59 + f64::from_bits(0x4C63E9E4E4C2F344), // 1e60 + f64::from_bits(0x4C98E45E1DF3B015), // 1e61 + f64::from_bits(0x4CCF1D75A5709C1B), // 1e62 + f64::from_bits(0x4D03726987666191), // 1e63 + f64::from_bits(0x4D384F03E93FF9F5), // 1e64 + f64::from_bits(0x4D6E62C4E38FF872), // 1e65 + f64::from_bits(0x4DA2FDBB0E39FB47), // 1e66 + f64::from_bits(0x4DD7BD29D1C87A19), // 1e67 + f64::from_bits(0x4E0DAC74463A989F), // 1e68 + f64::from_bits(0x4E428BC8ABE49F64), // 1e69 + f64::from_bits(0x4E772EBAD6DDC73D), // 1e70 + f64::from_bits(0x4EACFA698C95390C), // 1e71 + f64::from_bits(0x4EE21C81F7DD43A7), // 1e72 + f64::from_bits(0x4F16A3A275D49491), // 1e73 + f64::from_bits(0x4F4C4C8B1349B9B5), // 1e74 + f64::from_bits(0x4F81AFD6EC0E1411), // 1e75 + f64::from_bits(0x4FB61BCCA7119916), // 1e76 + f64::from_bits(0x4FEBA2BFD0D5FF5B), // 1e77 + f64::from_bits(0x502145B7E285BF99), // 1e78 + f64::from_bits(0x50559725DB272F7F), // 1e79 + f64::from_bits(0x508AFCEF51F0FB5F), // 1e80 + f64::from_bits(0x50C0DE1593369D1B), // 1e81 + f64::from_bits(0x50F5159AF8044462), // 1e82 + f64::from_bits(0x512A5B01B605557B), // 1e83 + f64::from_bits(0x516078E111C3556D), // 1e84 + f64::from_bits(0x5194971956342AC8), // 1e85 + f64::from_bits(0x51C9BCDFABC1357A), // 1e86 + f64::from_bits(0x5200160BCB58C16C), // 1e87 + f64::from_bits(0x52341B8EBE2EF1C7), // 1e88 + f64::from_bits(0x526922726DBAAE39), // 1e89 + f64::from_bits(0x529F6B0F092959C7), // 1e90 + f64::from_bits(0x52D3A2E965B9D81D), // 1e91 + f64::from_bits(0x53088BA3BF284E24), // 1e92 + f64::from_bits(0x533EAE8CAEF261AD), // 1e93 + f64::from_bits(0x53732D17ED577D0C), // 1e94 + f64::from_bits(0x53A7F85DE8AD5C4F), // 1e95 + f64::from_bits(0x53DDF67562D8B363), // 1e96 + f64::from_bits(0x5412BA095DC7701E), // 1e97 + f64::from_bits(0x5447688BB5394C25), // 1e98 + f64::from_bits(0x547D42AEA2879F2E), // 1e99 + f64::from_bits(0x54B249AD2594C37D), // 1e100 + f64::from_bits(0x54E6DC186EF9F45C), // 1e101 + f64::from_bits(0x551C931E8AB87173), // 1e102 + f64::from_bits(0x5551DBF316B346E8), // 1e103 + f64::from_bits(0x558652EFDC6018A2), // 1e104 + f64::from_bits(0x55BBE7ABD3781ECA), // 1e105 + f64::from_bits(0x55F170CB642B133F), // 1e106 + f64::from_bits(0x5625CCFE3D35D80E), // 1e107 + f64::from_bits(0x565B403DCC834E12), // 1e108 + f64::from_bits(0x569108269FD210CB), // 1e109 + f64::from_bits(0x56C54A3047C694FE), // 1e110 + f64::from_bits(0x56FA9CBC59B83A3D), // 1e111 + f64::from_bits(0x5730A1F5B8132466), // 1e112 + f64::from_bits(0x5764CA732617ED80), // 1e113 + f64::from_bits(0x5799FD0FEF9DE8E0), // 1e114 + f64::from_bits(0x57D03E29F5C2B18C), // 1e115 + f64::from_bits(0x58044DB473335DEF), // 1e116 + f64::from_bits(0x583961219000356B), // 1e117 + f64::from_bits(0x586FB969F40042C5), // 1e118 + f64::from_bits(0x58A3D3E2388029BB), // 1e119 + f64::from_bits(0x58D8C8DAC6A0342A), // 1e120 + f64::from_bits(0x590EFB1178484135), // 1e121 + f64::from_bits(0x59435CEAEB2D28C1), // 1e122 + f64::from_bits(0x59783425A5F872F1), // 1e123 + f64::from_bits(0x59AE412F0F768FAD), // 1e124 + f64::from_bits(0x59E2E8BD69AA19CC), // 1e125 + f64::from_bits(0x5A17A2ECC414A03F), // 1e126 + f64::from_bits(0x5A4D8BA7F519C84F), // 1e127 + f64::from_bits(0x5A827748F9301D32), // 1e128 + f64::from_bits(0x5AB7151B377C247E), // 1e129 + f64::from_bits(0x5AECDA62055B2D9E), // 1e130 + f64::from_bits(0x5B22087D4358FC82), // 1e131 + f64::from_bits(0x5B568A9C942F3BA3), // 1e132 + f64::from_bits(0x5B8C2D43B93B0A8C), // 1e133 + f64::from_bits(0x5BC19C4A53C4E697), // 1e134 + f64::from_bits(0x5BF6035CE8B6203D), // 1e135 + f64::from_bits(0x5C2B843422E3A84D), // 1e136 + f64::from_bits(0x5C6132A095CE4930), // 1e137 + f64::from_bits(0x5C957F48BB41DB7C), // 1e138 + f64::from_bits(0x5CCADF1AEA12525B), // 1e139 + f64::from_bits(0x5D00CB70D24B7379), // 1e140 + f64::from_bits(0x5D34FE4D06DE5057), // 1e141 + f64::from_bits(0x5D6A3DE04895E46D), // 1e142 + f64::from_bits(0x5DA066AC2D5DAEC4), // 1e143 + f64::from_bits(0x5DD4805738B51A75), // 1e144 + f64::from_bits(0x5E09A06D06E26112), // 1e145 + f64::from_bits(0x5E400444244D7CAB), // 1e146 + f64::from_bits(0x5E7405552D60DBD6), // 1e147 + f64::from_bits(0x5EA906AA78B912CC), // 1e148 + f64::from_bits(0x5EDF485516E7577F), // 1e149 + f64::from_bits(0x5F138D352E5096AF), // 1e150 + f64::from_bits(0x5F48708279E4BC5B), // 1e151 + f64::from_bits(0x5F7E8CA3185DEB72), // 1e152 + f64::from_bits(0x5FB317E5EF3AB327), // 1e153 + f64::from_bits(0x5FE7DDDF6B095FF1), // 1e154 + f64::from_bits(0x601DD55745CBB7ED), // 1e155 + f64::from_bits(0x6052A5568B9F52F4), // 1e156 + f64::from_bits(0x60874EAC2E8727B1), // 1e157 + f64::from_bits(0x60BD22573A28F19D), // 1e158 + f64::from_bits(0x60F2357684599702), // 1e159 + f64::from_bits(0x6126C2D4256FFCC3), // 1e160 + f64::from_bits(0x615C73892ECBFBF4), // 1e161 + f64::from_bits(0x6191C835BD3F7D78), // 1e162 + f64::from_bits(0x61C63A432C8F5CD6), // 1e163 + f64::from_bits(0x61FBC8D3F7B3340C), // 1e164 + f64::from_bits(0x62315D847AD00087), // 1e165 + f64::from_bits(0x6265B4E5998400A9), // 1e166 + f64::from_bits(0x629B221EFFE500D4), // 1e167 + f64::from_bits(0x62D0F5535FEF2084), // 1e168 + f64::from_bits(0x630532A837EAE8A5), // 1e169 + f64::from_bits(0x633A7F5245E5A2CF), // 1e170 + f64::from_bits(0x63708F936BAF85C1), // 1e171 + f64::from_bits(0x63A4B378469B6732), // 1e172 + f64::from_bits(0x63D9E056584240FE), // 1e173 + f64::from_bits(0x64102C35F729689F), // 1e174 + f64::from_bits(0x6444374374F3C2C6), // 1e175 + f64::from_bits(0x647945145230B378), // 1e176 + f64::from_bits(0x64AF965966BCE056), // 1e177 + f64::from_bits(0x64E3BDF7E0360C36), // 1e178 + f64::from_bits(0x6518AD75D8438F43), // 1e179 + f64::from_bits(0x654ED8D34E547314), // 1e180 + f64::from_bits(0x6583478410F4C7EC), // 1e181 + f64::from_bits(0x65B819651531F9E8), // 1e182 + f64::from_bits(0x65EE1FBE5A7E7861), // 1e183 + f64::from_bits(0x6622D3D6F88F0B3D), // 1e184 + f64::from_bits(0x665788CCB6B2CE0C), // 1e185 + f64::from_bits(0x668D6AFFE45F818F), // 1e186 + f64::from_bits(0x66C262DFEEBBB0F9), // 1e187 + f64::from_bits(0x66F6FB97EA6A9D38), // 1e188 + f64::from_bits(0x672CBA7DE5054486), // 1e189 + f64::from_bits(0x6761F48EAF234AD4), // 1e190 + f64::from_bits(0x679671B25AEC1D89), // 1e191 + f64::from_bits(0x67CC0E1EF1A724EB), // 1e192 + f64::from_bits(0x680188D357087713), // 1e193 + f64::from_bits(0x6835EB082CCA94D7), // 1e194 + f64::from_bits(0x686B65CA37FD3A0D), // 1e195 + f64::from_bits(0x68A11F9E62FE4448), // 1e196 + f64::from_bits(0x68D56785FBBDD55A), // 1e197 + f64::from_bits(0x690AC1677AAD4AB1), // 1e198 + f64::from_bits(0x6940B8E0ACAC4EAF), // 1e199 + f64::from_bits(0x6974E718D7D7625A), // 1e200 + f64::from_bits(0x69AA20DF0DCD3AF1), // 1e201 + f64::from_bits(0x69E0548B68A044D6), // 1e202 + f64::from_bits(0x6A1469AE42C8560C), // 1e203 + f64::from_bits(0x6A498419D37A6B8F), // 1e204 + f64::from_bits(0x6A7FE52048590673), // 1e205 + f64::from_bits(0x6AB3EF342D37A408), // 1e206 + f64::from_bits(0x6AE8EB0138858D0A), // 1e207 + f64::from_bits(0x6B1F25C186A6F04C), // 1e208 + f64::from_bits(0x6B537798F4285630), // 1e209 + f64::from_bits(0x6B88557F31326BBB), // 1e210 + f64::from_bits(0x6BBE6ADEFD7F06AA), // 1e211 + f64::from_bits(0x6BF302CB5E6F642A), // 1e212 + f64::from_bits(0x6C27C37E360B3D35), // 1e213 + f64::from_bits(0x6C5DB45DC38E0C82), // 1e214 + f64::from_bits(0x6C9290BA9A38C7D1), // 1e215 + f64::from_bits(0x6CC734E940C6F9C6), // 1e216 + f64::from_bits(0x6CFD022390F8B837), // 1e217 + f64::from_bits(0x6D3221563A9B7323), // 1e218 + f64::from_bits(0x6D66A9ABC9424FEB), // 1e219 + f64::from_bits(0x6D9C5416BB92E3E6), // 1e220 + f64::from_bits(0x6DD1B48E353BCE70), // 1e221 + f64::from_bits(0x6E0621B1C28AC20C), // 1e222 + f64::from_bits(0x6E3BAA1E332D728F), // 1e223 + f64::from_bits(0x6E714A52DFFC6799), // 1e224 + f64::from_bits(0x6EA59CE797FB817F), // 1e225 + f64::from_bits(0x6EDB04217DFA61DF), // 1e226 + f64::from_bits(0x6F10E294EEBC7D2C), // 1e227 + f64::from_bits(0x6F451B3A2A6B9C76), // 1e228 + f64::from_bits(0x6F7A6208B5068394), // 1e229 + f64::from_bits(0x6FB07D457124123D), // 1e230 + f64::from_bits(0x6FE49C96CD6D16CC), // 1e231 + f64::from_bits(0x7019C3BC80C85C7F), // 1e232 + f64::from_bits(0x70501A55D07D39CF), // 1e233 + f64::from_bits(0x708420EB449C8843), // 1e234 + f64::from_bits(0x70B9292615C3AA54), // 1e235 + f64::from_bits(0x70EF736F9B3494E9), // 1e236 + f64::from_bits(0x7123A825C100DD11), // 1e237 + f64::from_bits(0x7158922F31411456), // 1e238 + f64::from_bits(0x718EB6BAFD91596B), // 1e239 + f64::from_bits(0x71C33234DE7AD7E3), // 1e240 + f64::from_bits(0x71F7FEC216198DDC), // 1e241 + f64::from_bits(0x722DFE729B9FF153), // 1e242 + f64::from_bits(0x7262BF07A143F6D4), // 1e243 + f64::from_bits(0x72976EC98994F489), // 1e244 + f64::from_bits(0x72CD4A7BEBFA31AB), // 1e245 + f64::from_bits(0x73024E8D737C5F0B), // 1e246 + f64::from_bits(0x7336E230D05B76CD), // 1e247 + f64::from_bits(0x736C9ABD04725481), // 1e248 + f64::from_bits(0x73A1E0B622C774D0), // 1e249 + f64::from_bits(0x73D658E3AB795204), // 1e250 + f64::from_bits(0x740BEF1C9657A686), // 1e251 + f64::from_bits(0x74417571DDF6C814), // 1e252 + f64::from_bits(0x7475D2CE55747A18), // 1e253 + f64::from_bits(0x74AB4781EAD1989E), // 1e254 + f64::from_bits(0x74E10CB132C2FF63), // 1e255 + f64::from_bits(0x75154FDD7F73BF3C), // 1e256 + f64::from_bits(0x754AA3D4DF50AF0B), // 1e257 + f64::from_bits(0x7580A6650B926D67), // 1e258 + f64::from_bits(0x75B4CFFE4E7708C0), // 1e259 + f64::from_bits(0x75EA03FDE214CAF1), // 1e260 + f64::from_bits(0x7620427EAD4CFED6), // 1e261 + f64::from_bits(0x7654531E58A03E8C), // 1e262 + f64::from_bits(0x768967E5EEC84E2F), // 1e263 + f64::from_bits(0x76BFC1DF6A7A61BB), // 1e264 + f64::from_bits(0x76F3D92BA28C7D15), // 1e265 + f64::from_bits(0x7728CF768B2F9C5A), // 1e266 + f64::from_bits(0x775F03542DFB8370), // 1e267 + f64::from_bits(0x779362149CBD3226), // 1e268 + f64::from_bits(0x77C83A99C3EC7EB0), // 1e269 + f64::from_bits(0x77FE494034E79E5C), // 1e270 + f64::from_bits(0x7832EDC82110C2F9), // 1e271 + f64::from_bits(0x7867A93A2954F3B8), // 1e272 + f64::from_bits(0x789D9388B3AA30A5), // 1e273 + f64::from_bits(0x78D27C35704A5E67), // 1e274 + f64::from_bits(0x79071B42CC5CF601), // 1e275 + f64::from_bits(0x793CE2137F743382), // 1e276 + f64::from_bits(0x79720D4C2FA8A031), // 1e277 + f64::from_bits(0x79A6909F3B92C83D), // 1e278 + f64::from_bits(0x79DC34C70A777A4D), // 1e279 + f64::from_bits(0x7A11A0FC668AAC70), // 1e280 + f64::from_bits(0x7A46093B802D578C), // 1e281 + f64::from_bits(0x7A7B8B8A6038AD6F), // 1e282 + f64::from_bits(0x7AB137367C236C65), // 1e283 + f64::from_bits(0x7AE585041B2C477F), // 1e284 + f64::from_bits(0x7B1AE64521F7595E), // 1e285 + f64::from_bits(0x7B50CFEB353A97DB), // 1e286 + f64::from_bits(0x7B8503E602893DD2), // 1e287 + f64::from_bits(0x7BBA44DF832B8D46), // 1e288 + f64::from_bits(0x7BF06B0BB1FB384C), // 1e289 + f64::from_bits(0x7C2485CE9E7A065F), // 1e290 + f64::from_bits(0x7C59A742461887F6), // 1e291 + f64::from_bits(0x7C9008896BCF54FA), // 1e292 + f64::from_bits(0x7CC40AABC6C32A38), // 1e293 + f64::from_bits(0x7CF90D56B873F4C7), // 1e294 + f64::from_bits(0x7D2F50AC6690F1F8), // 1e295 + f64::from_bits(0x7D63926BC01A973B), // 1e296 + f64::from_bits(0x7D987706B0213D0A), // 1e297 + f64::from_bits(0x7DCE94C85C298C4C), // 1e298 + f64::from_bits(0x7E031CFD3999F7B0), // 1e299 + f64::from_bits(0x7E37E43C8800759C), // 1e300 + f64::from_bits(0x7E6DDD4BAA009303), // 1e301 + f64::from_bits(0x7EA2AA4F4A405BE2), // 1e302 + f64::from_bits(0x7ED754E31CD072DA), // 1e303 + f64::from_bits(0x7F0D2A1BE4048F90), // 1e304 + f64::from_bits(0x7F423A516E82D9BA), // 1e305 + f64::from_bits(0x7F76C8E5CA239029), // 1e306 + f64::from_bits(0x7FAC7B1F3CAC7433), // 1e307 + f64::from_bits(0x7FE1CCF385EBC8A0), // 1e308 +]; diff --git a/src/specializer/fold.rs b/src/specializer/fold.rs index 5400fbb..71ff635 100644 --- a/src/specializer/fold.rs +++ b/src/specializer/fold.rs @@ -61,6 +61,47 @@ pub fn fold(e: SExpr) -> SExpr { _ => e(SKind::IntToFloat(Box::new(inner))), } } + // Wave-1 math: fold children only — the ops themselves stay + // runtime so constant domain errors trap per row exactly like the + // vectorized path we pin against (no fold/vector divergence). + SKind::Str2 { op, a, b } => { + let a = fold(*a); + let b = fold(*b); + e(SKind::Str2 { + op, + a: Box::new(a), + b: Box::new(b), + }) + } + SKind::SLen { bytes, a } => { + let a = fold(*a); + e(SKind::SLen { + bytes, + a: Box::new(a), + }) + } + SKind::Round2 { trunc, a, n } => { + let a = fold(*a); + let n = fold(*n); + e(SKind::Round2 { + trunc, + a: Box::new(a), + n: Box::new(n), + }) + } + SKind::MathF1 { op, a } => { + let a = fold(*a); + e(SKind::MathF1 { op, a: Box::new(a) }) + } + SKind::MathF2 { op, a, b } => { + let a = fold(*a); + let b = fold(*b); + e(SKind::MathF2 { + op, + a: Box::new(a), + b: Box::new(b), + }) + } SKind::Not(inner) => { let inner = fold(*inner); match as_const(&inner) { diff --git a/src/specializer/frontend.rs b/src/specializer/frontend.rs index fd3442a..7378f67 100644 --- a/src/specializer/frontend.rs +++ b/src/specializer/frontend.rs @@ -28,7 +28,7 @@ use sqlparser::dialect::DuckDbDialect; use sqlparser::parser::Parser; use super::fold::fold; -use super::ir::{CmpPred, Col, Lit, TrimSide, Ty}; +use super::ir::{BinOp, CmpPred, Col, Lit, NumOp1, StrOp2, TrimSide, Ty}; use super::plan::{ArithOp, JoinKind, JoinSpec, Rel, SExpr, SKind, StaticTable}; #[derive(Debug, PartialEq, Eq)] @@ -477,7 +477,82 @@ struct Binder<'a> { select_aliases: Vec, } +fn math1_node(op: NumOp1, inner: SExpr) -> SExpr { + let nullable = inner.nullable; + SExpr { + kind: SKind::MathF1 { + op, + a: Box::new(inner), + }, + ty: Ty::F64, + nullable, + } +} + +/// AST constructors for the BETWEEN/IN desugars. +fn ast_bin(op: BinaryOperator, l: SqlExpr, r: SqlExpr) -> SqlExpr { + SqlExpr::BinaryOp { + left: Box::new(l), + op, + right: Box::new(r), + } +} + +fn ast_not_if(negated: bool, e: SqlExpr) -> SqlExpr { + if negated { + SqlExpr::UnaryOp { + op: UnaryOperator::Not, + expr: Box::new(e), + } + } else { + e + } +} + impl Binder<'_> { + /// DuckDB unifies BETWEEN/IN across the WHOLE construct (wave-1 pins): + /// one common type for the subject and every bound/element, so a single + /// f64 side promotes all sides. Numeric-with-string/bool mixing has + /// exec-time cast semantics we don't model — clean-unsupported. + fn unify_family(&self, exprs: &[&SqlExpr]) -> Result, PrepareError> { + let (mut any_f64, mut any_num, mut any_other) = (false, false, false); + for e in exprs { + if let Some(b) = self.expr_or_null(e)? { + match b.ty { + Ty::F64 => (any_f64, any_num) = (true, true), + Ty::I64 => any_num = true, + Ty::Str | Ty::I1 => any_other = true, + } + } + } + if any_num && any_other { + return Err(unsup( + "BETWEEN/IN mixing strings or booleans with numbers (exec-time cast semantics)", + )); + } + Ok(exprs + .iter() + .map(|e| { + let e = (*e).clone(); + if any_f64 { + // CAST is a no-op on already-f64 sides and types NULL + // literals from context; exactly DuckDB's unification. + SqlExpr::Cast { + kind: CastKind::Cast, + expr: Box::new(e), + data_type: sqlparser::ast::DataType::Double( + sqlparser::ast::ExactNumberInfo::None, + ), + format: None, + array: false, + } + } else { + e + } + }) + .collect()) + } + /// Expand `*` / `tbl.*` per DuckDB's measured semantics (1.5.5): FROM /// order, declared column order within a table, EXCLUDE filtered /// case-insensitively. A star item covering a joined static table is @@ -679,8 +754,66 @@ impl Binder<'_> { substring_for, .. } => self.substr_node(expr, substring_from.as_deref(), substring_for.as_deref()), - SqlExpr::Between { .. } => Err(unsup("BETWEEN")), - SqlExpr::InList { .. } => Err(unsup("IN (...)")), + // SQL-standard position(needle IN haystack) — needle-first, + // same op as instr/strpos (measured). + SqlExpr::Position { expr, r#in } => self.str2("position", StrOp2::Find, r#in, expr), + // sqlparser gives FLOOR/CEIL dedicated AST nodes, not Function + // calls; the datetime `CEIL(x TO field)` form rejects by name. + SqlExpr::Floor { expr, field } => match field { + sqlparser::ast::CeilFloorKind::Scale(_) => Err(unsup("floor with scale argument")), + sqlparser::ast::CeilFloorKind::DateTimeField( + sqlparser::ast::DateTimeField::NoDateTime, + ) => self.math1("floor", NumOp1::Ffloor, expr), + _ => Err(unsup("FLOOR(x TO datetime-field)")), + }, + SqlExpr::Ceil { expr, field } => match field { + sqlparser::ast::CeilFloorKind::Scale(_) => Err(unsup("ceil with scale argument")), + sqlparser::ast::CeilFloorKind::DateTimeField( + sqlparser::ast::DateTimeField::NoDateTime, + ) => self.math1("ceil", NumOp1::Fceil, expr), + _ => Err(unsup("CEIL(x TO datetime-field)")), + }, + // BETWEEN and IN are exact K3 desugars (wave-1 pins): DuckDB's + // truth tables over NULL/NaN fall out of Kleene AND/OR of the + // duck_fcmp comparisons with zero special cases. DuckDB unifies + // types across the WHOLE construct (one common type for the + // subject and every bound/element), so any f64 side promotes + // all sides before the pairwise desugar. + SqlExpr::Between { + expr, + negated, + low, + high, + } => { + let mut u = self.unify_family(&[expr, low, high])?; + let (e, lo, hi) = (u.remove(0), u.remove(0), u.remove(0)); + let both = ast_bin( + BinaryOperator::And, + ast_bin(BinaryOperator::GtEq, e.clone(), lo), + ast_bin(BinaryOperator::LtEq, e, hi), + ); + self.bind(&ast_not_if(*negated, both)) + } + SqlExpr::InList { + expr, + list, + negated, + } => { + let mut family: Vec<&SqlExpr> = vec![expr]; + family.extend(list.iter()); + let mut unified = self.unify_family(&family)?; + let subject = unified.remove(0); + let mut chain: Option = None; + for item in unified { + let eq = ast_bin(BinaryOperator::Eq, subject.clone(), item); + chain = Some(match chain { + None => eq, + Some(prev) => ast_bin(BinaryOperator::Or, prev, eq), + }); + } + let chain = chain.ok_or_else(|| unsup("empty IN list"))?; + self.bind(&ast_not_if(*negated, chain)) + } SqlExpr::Like { .. } => Err(unsup("LIKE")), other => Err(unsup(format!("expression: {other}"))), } @@ -878,6 +1011,14 @@ impl Binder<'_> { nullable, }) } + // DuckDB's ^ IS pow — but sqlparser parses ^ BELOW * while + // DuckDB binds it above (measured: duck 2*x^y = 2*(x^y), + // sqlparser tree = (2*x)^y). Mapping it would silently compute + // the wrong tree, so the operator stays cleanly unsupported; + // pow()/power() cover the semantics. + BinaryOperator::BitwiseXor => Err(unsup( + "operator ^ (sqlparser precedence differs from DuckDB pow)", + )), other => Err(unsup(format!("operator {other}"))), } } @@ -1169,6 +1310,116 @@ impl Binder<'_> { _ => Err(PrepareError::Bind(format!("{name} takes 1 or 2 arguments"))), } } + // Wave-1 string search (pins): instr/strpos/2-arg position are + // one op with (haystack, needle) order; prefix/suffix alias + // starts_with/ends_with; positions are 1-based codepoints. + "instr" | "strpos" | "position" | "contains" | "starts_with" | "prefix" + | "ends_with" | "suffix" => { + let op = match name.as_str() { + "instr" | "strpos" | "position" => StrOp2::Find, + "contains" => StrOp2::Contains, + "starts_with" | "prefix" => StrOp2::Starts, + _ => StrOp2::Ends, + }; + let [h, n] = args[..] else { + return Err(PrepareError::Bind(format!( + "{name} takes exactly 2 arguments" + ))); + }; + self.str2(&name, op, h, n) + } + "length" | "len" | "char_length" | "character_length" | "strlen" => { + let [arg] = args[..] else { + return Err(PrepareError::Bind(format!( + "{name} takes exactly 1 argument" + ))); + }; + let Some(inner) = self.expr_or_null(arg)? else { + return Ok(null_of(Ty::I64)); + }; + if inner.ty != Ty::Str { + // No implicit numeric->VARCHAR casts here (measured). + return Err(PrepareError::Bind(format!( + "no function matches {name}({})", + inner.ty.name() + ))); + } + let nullable = inner.nullable; + Ok(SExpr { + kind: SKind::SLen { + bytes: name == "strlen", + a: Box::new(inner), + }, + ty: Ty::I64, + nullable, + }) + } + // Wave-1 f64 unary math (pins: 2026-07-26-wave1-builtin-pins.md). + // 1-arg log IS base 10 in DuckDB — handled under "log" below. + "ln" | "log2" | "log10" | "exp" | "sqrt" | "cbrt" | "sin" | "cos" | "tan" | "floor" + | "ceil" | "ceiling" => { + let op = match name.as_str() { + "ln" => NumOp1::Ln, + "log2" => NumOp1::Log2, + "log10" => NumOp1::Log10, + "exp" => NumOp1::Fexp, + "sqrt" => NumOp1::Fsqrt, + "cbrt" => NumOp1::Fcbrt, + "sin" => NumOp1::Fsin, + "cos" => NumOp1::Fcos, + "tan" => NumOp1::Ftan, + "floor" => NumOp1::Ffloor, + _ => NumOp1::Fceil, + }; + let [arg] = args[..] else { + return Err(PrepareError::Bind(format!( + "{name} takes exactly 1 argument" + ))); + }; + self.math1(&name, op, arg) + } + "log" => match args[..] { + [x] => self.math1("log", NumOp1::Log10, x), + [b, x] => self.math2("log", BinOp::Flogb, b, x), + _ => Err(PrepareError::Bind("log takes 1 or 2 arguments".to_string())), + }, + "pow" | "power" => match args[..] { + [x, y] => self.math2(&name, BinOp::Fpow, x, y), + _ => Err(PrepareError::Bind(format!( + "{name} takes exactly 2 arguments" + ))), + }, + "pi" => { + if !args.is_empty() { + return Err(PrepareError::Bind("pi takes no arguments".to_string())); + } + // Bit-equal to DuckDB's pi() (measured 0x400921FB54442D18). + Ok(SExpr { + kind: SKind::Lit(Lit::F64(std::f64::consts::PI)), + ty: Ty::F64, + nullable: false, + }) + } + "trunc" => match args[..] { + [arg] => { + let Some(inner) = self.expr_or_null(arg)? else { + return Ok(null_of(Ty::I64)); + }; + match inner.ty { + // Measured: integer trunc is identity, type preserved. + Ty::I64 => Ok(inner), + Ty::F64 => Ok(math1_node(NumOp1::Ftrunc, inner)), + other => Err(PrepareError::Bind(format!( + "no function matches trunc({})", + other.name() + ))), + } + } + [x, n] => self.round2(true, x, n), + _ => Err(PrepareError::Bind( + "trunc takes 1 or 2 arguments".to_string(), + )), + }, "abs" => { let [arg] = args[..] else { return Err(PrepareError::Bind( @@ -1214,7 +1465,7 @@ impl Binder<'_> { ))), } } - [_, _] => Err(unsup("round with digits (scale-then-round algorithm)")), + [x, n] => self.round2(false, x, n), _ => Err(PrepareError::Bind( "round takes 1 or 2 arguments".to_string(), )), @@ -1333,6 +1584,83 @@ impl Binder<'_> { } Ok(acc) } + // least/greatest: NULL-IGNORING (result NULL only when every + // arg is), ties return the FIRST argument, NaN sorts above + // +inf — all of which the CASE + duck-order-cmp composition + // reproduces exactly (wave-1 pins), so no IR op exists. + "least" | "greatest" => { + if args.is_empty() { + return Err(PrepareError::Bind(format!( + "{name} needs at least 1 argument" + ))); + } + let mut bound = Vec::new(); + for arg in &args { + // Literal NULL args contribute nothing (NULL-ignoring). + if let Some(e) = self.expr_or_null(arg)? { + bound.push(e); + } + } + if bound.is_empty() { + return Err(unsup(format!("{name} of only NULL literals"))); + } + let mut unified = bound[0].ty; + for e in &bound[1..] { + unified = match (unified, e.ty) { + (u, t) if u == t => u, + (Ty::I64, Ty::F64) | (Ty::F64, Ty::I64) => Ty::F64, + (u, t) => { + return Err(PrepareError::Bind(format!( + "{name} arguments disagree: {} vs {}", + u.name(), + t.name() + ))) + } + }; + } + let bound: Vec = bound + .into_iter() + .map(|e| { + if e.ty == Ty::I64 && unified == Ty::F64 { + promote_f64(e) + } else { + e + } + }) + .collect(); + let pred = if name == "greatest" { + CmpPred::Ge + } else { + CmpPred::Le + }; + let mut it = bound.into_iter(); + let mut acc = it.next().expect("non-empty"); + for b in it { + let cmp = self.cmp(pred, acc.clone(), b.clone())?; + let is_null = |e: &SExpr| SExpr { + kind: SKind::IsNull { + negated: false, + inner: Box::new(e.clone()), + }, + ty: Ty::I1, + nullable: false, + }; + let nullable = acc.nullable && b.nullable; + acc = SExpr { + kind: SKind::Case { + arms: vec![ + (is_null(&acc), b.clone()), + (is_null(&b), acc.clone()), + (cmp, acc), + ], + default: Some(Box::new(b)), + }, + ty: unified, + nullable, + }; + } + Ok(acc) + } "nullif" => { let [a, b] = args[..] else { return Err(PrepareError::Bind( @@ -1367,6 +1695,148 @@ impl Binder<'_> { } } + /// Wave-1 string search: both args must be Str (no implicit numeric + /// casts — measured binder errors). A literal NULL binds to the typed + /// NULL result for every member EXCEPT contains, where DuckDB's + /// overloads (MAP/LIST) make a bare NULL a binder error — mirrored. + fn str2( + &self, + name: &str, + op: StrOp2, + h: &SqlExpr, + n: &SqlExpr, + ) -> Result { + let (bh, bn) = (self.expr_or_null(h)?, self.expr_or_null(n)?); + // contains has MAP/LIST overloads; a NULL literal NEEDLE binds only + // when a NON-literal Str haystack anchors resolution (measured: + // contains(s, NULL) and contains(NULL, 'o') work, contains('abc', + // NULL) and contains(NULL, NULL) are binder errors — the corpus + // refuted the fleet's blanket-error pin, so this mirrors exactly). + if name == "contains" && bn.is_none() { + let anchored = matches!(&bh, Some(e) if !matches!(e.kind, SKind::Lit(_))); + if !anchored { + return Err(PrepareError::Bind( + "contains with a NULL literal is ambiguous (VARCHAR/MAP/LIST overloads)" + .to_string(), + )); + } + } + let (Some(bh), Some(bn)) = (bh, bn) else { + return Ok(null_of(op.result_ty())); + }; + for e in [&bh, &bn] { + if e.ty != Ty::Str { + return Err(PrepareError::Bind(format!( + "no function matches {name}({})", + e.ty.name() + ))); + } + } + let nullable = bh.nullable || bn.nullable; + Ok(SExpr { + kind: SKind::Str2 { + op, + a: Box::new(bh), + b: Box::new(bn), + }, + ty: op.result_ty(), + nullable, + }) + } + + /// round(x, n) / trunc(x, n): result type == subject type; digits must + /// be integer-typed. Total on both types (i64 wraps — pinned). + fn round2(&self, trunc: bool, x: &SqlExpr, n: &SqlExpr) -> Result { + let name = if trunc { "trunc" } else { "round" }; + let Some(subject) = self.expr_or_null(x)? else { + return Ok(null_of(Ty::I64)); + }; + if !matches!(subject.ty, Ty::I64 | Ty::F64) { + return Err(PrepareError::Bind(format!( + "no function matches {name}({}, digits)", + subject.ty.name() + ))); + } + let ty = subject.ty; + let Some(digits) = self.expr_or_null(n)? else { + return Ok(null_of(ty)); + }; + if digits.ty != Ty::I64 { + return Err(PrepareError::Bind(format!( + "no function matches {name}({}, {})", + ty.name(), + digits.ty.name() + ))); + } + let nullable = subject.nullable || digits.nullable; + Ok(SExpr { + kind: SKind::Round2 { + trunc, + a: Box::new(subject), + n: Box::new(digits), + }, + ty, + nullable, + }) + } + + /// Wave-1 unary f64 math: numeric args promote to DOUBLE, VARCHAR and + /// BOOLEAN columns are binder errors (no implicit cast — measured), a + /// literal NULL binds to the DOUBLE overload. + fn math1(&self, name: &str, op: NumOp1, arg: &SqlExpr) -> Result { + let Some(inner) = self.expr_or_null(arg)? else { + return Ok(null_of(Ty::F64)); + }; + let inner = match inner.ty { + Ty::F64 => inner, + Ty::I64 => promote_f64(inner), + other => { + return Err(PrepareError::Bind(format!( + "no function matches {name}({})", + other.name() + ))) + } + }; + Ok(math1_node(op, inner)) + } + + /// Wave-1 binary f64 math (pow / log(base, x)); same argument typing + /// rules as `math1`. A literal NULL in either slot pre-empts every + /// domain check (measured: log(-2.0, NULL) is NULL, not an error). + fn math2( + &self, + name: &str, + op: BinOp, + a: &SqlExpr, + b: &SqlExpr, + ) -> Result { + let (ba, bb) = (self.expr_or_null(a)?, self.expr_or_null(b)?); + let (Some(ba), Some(bb)) = (ba, bb) else { + return Ok(null_of(Ty::F64)); + }; + let promote = |e: SExpr| -> Result { + match e.ty { + Ty::F64 => Ok(e), + Ty::I64 => Ok(promote_f64(e)), + other => Err(PrepareError::Bind(format!( + "no function matches {name}({})", + other.name() + ))), + } + }; + let (ba, bb) = (promote(ba)?, promote(bb)?); + let nullable = ba.nullable || bb.nullable; + Ok(SExpr { + kind: SKind::MathF2 { + op, + a: Box::new(ba), + b: Box::new(bb), + }, + ty: Ty::F64, + nullable, + }) + } + /// All TRIM forms plus ltrim/rtrim. `chars` is the optional trim-set /// expression; absent means DuckDB's default — the single space (only /// 0x20 is trimmed, never tabs/newlines). diff --git a/src/specializer/ir/gen.rs b/src/specializer/ir/gen.rs index fcb4feb..7c4a4b0 100644 --- a/src/specializer/ir/gen.rs +++ b/src/specializer/ir/gen.rs @@ -10,7 +10,7 @@ use super::{ BinOp, Block, BlockId, Builder, CmpPred, Col, ColTy, Inst, Lit, NumOp1, Program, RoundMode, - StaticTy, StrOp1, Term, TrimSide, Ty, Value, + StaticTy, StrOp1, StrOp2, Term, TrimSide, Ty, Value, }; pub struct Rng(u64); @@ -222,7 +222,7 @@ fn load_all( fn compute(rng: &mut Rng, b: &mut Builder, scope: &mut Scope, insts: &mut Vec) { let n = rng.below(7); for _ in 0..n { - match rng.below(12) { + match rng.below(13) { 0 => { let ops = [ BinOp::Iadd, @@ -233,13 +233,20 @@ fn compute(rng: &mut Rng, b: &mut Builder, scope: &mut Scope, insts: &mut Vec { // iabs excluded: it traps on i64::MIN, and generated programs - // must stay executable for M-interp fuzzing. - let op = if rng.chance(50) { - NumOp1::Fabs + // must stay executable for M-interp fuzzing. The wave-1 + // trapping unaries join at low weight (trap-agreement is + // signal, but programs should mostly run to completion). + let total = [ + NumOp1::Fabs, + NumOp1::Fround, + NumOp1::Ffloor, + NumOp1::Fceil, + NumOp1::Ftrunc, + NumOp1::Fexp, + NumOp1::Fcbrt, + ]; + let trapping = [ + NumOp1::Ln, + NumOp1::Log2, + NumOp1::Log10, + NumOp1::Fsqrt, + NumOp1::Fsin, + NumOp1::Fcos, + NumOp1::Ftan, + ]; + let op = if rng.chance(20) { + trapping[rng.below(trapping.len() as u64) as usize] } else { - NumOp1::Fround + total[rng.below(total.len() as u64) as usize] }; let a = ensure(rng, b, scope, insts, Ty::F64); let dst = b.fresh(); insts.push(Inst::Num1 { op, dst, a }); scope.add(dst, Ty::F64); } + 12 if rng.chance(35) => { + // round/trunc with digits: total on both int and float. + let trunc = rng.chance(50); + let n = ensure(rng, b, scope, insts, Ty::I64); + if rng.chance(50) { + let a = ensure(rng, b, scope, insts, Ty::F64); + let dst = b.fresh(); + insts.push(Inst::Round2f { trunc, dst, a, n }); + scope.add(dst, Ty::F64); + } else { + let a = ensure(rng, b, scope, insts, Ty::I64); + let dst = b.fresh(); + insts.push(Inst::Round2i { trunc, dst, a, n }); + scope.add(dst, Ty::I64); + } + } + 12 => { + let a = ensure(rng, b, scope, insts, Ty::Str); + if rng.chance(30) { + let dst = b.fresh(); + insts.push(Inst::SLen { + bytes: rng.chance(50), + dst, + a, + }); + scope.add(dst, Ty::I64); + } else { + let n = ensure(rng, b, scope, insts, Ty::Str); + let ops = [StrOp2::Find, StrOp2::Contains, StrOp2::Starts, StrOp2::Ends]; + let op = ops[rng.below(4) as usize]; + let dst = b.fresh(); + insts.push(Inst::Str2 { op, dst, a, b: n }); + scope.add(dst, op.result_ty()); + } + } _ => { let (from, mk): (Ty, fn(Value, Value) -> Inst) = if rng.chance(50) { (Ty::I64, |dst, a| Inst::Itos { dst, a }) diff --git a/src/specializer/ir/mod.rs b/src/specializer/ir/mod.rs index 775512f..bdef9db 100644 --- a/src/specializer/ir/mod.rs +++ b/src/specializer/ir/mod.rs @@ -241,6 +241,12 @@ pub enum BinOp { /// f64 `%`, IEEE remainder-of-truncated-division (Rust `%`): sign of the /// dividend, `x % 0.0` is NaN. Never traps (measured DuckDB 1.5.5). Frem, + /// f64 pow — TOTAL, pure IEEE (pow(NaN,0)=1, pow(0,-1)=inf; wave-1 pins). + Fpow, + /// log(base, x) == log10(x)/log10(base) bit-exactly (wave-1 pins). + /// Traps: base checked FIRST (zero / negative / base==1 each with their + /// own DuckDB message), then x (zero / negative). + Flogb, And, Or, Xor, @@ -253,9 +259,13 @@ impl BinOp { BinOp::Iadd | BinOp::Isub | BinOp::Imul | BinOp::Idiv | BinOp::Irem => { (Ty::I64, Ty::I64) } - BinOp::Fadd | BinOp::Fsub | BinOp::Fmul | BinOp::Fdiv | BinOp::Frem => { - (Ty::F64, Ty::F64) - } + BinOp::Fadd + | BinOp::Fsub + | BinOp::Fmul + | BinOp::Fdiv + | BinOp::Frem + | BinOp::Fpow + | BinOp::Flogb => (Ty::F64, Ty::F64), BinOp::And | BinOp::Or | BinOp::Xor => (Ty::I1, Ty::I1), } } @@ -272,6 +282,8 @@ impl BinOp { BinOp::Fmul => "fmul", BinOp::Fdiv => "fdiv", BinOp::Frem => "frem", + BinOp::Fpow => "fpow", + BinOp::Flogb => "flogb", BinOp::And => "and", BinOp::Or => "or", BinOp::Xor => "xor", @@ -325,6 +337,36 @@ impl StrOp1 { } } +/// Wave-1 string search: str × str, TOTAL (no traps), NULL-strict via +/// lanes. Positions are 1-based CODEPOINT indices; an empty needle +/// matches everything (pins spec 2026-07-26). +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum StrOp2 { + /// instr/strpos/position: 1-based codepoint index, 0 = not found. + Find, + Contains, + Starts, + Ends, +} + +impl StrOp2 { + pub fn result_ty(self) -> Ty { + match self { + StrOp2::Find => Ty::I64, + _ => Ty::I1, + } + } + + pub fn name(self) -> &'static str { + match self { + StrOp2::Find => "sfind", + StrOp2::Contains => "scontains", + StrOp2::Starts => "sstarts", + StrOp2::Ends => "sends", + } + } +} + #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum TrimSide { Both, @@ -350,6 +392,22 @@ pub enum NumOp1 { Iabs, Fabs, Fround, + // Wave-1 math unaries (pins: docs/superpowers/specs/2026-07-26-wave1- + // builtin-pins.md). Ln/Log2/Log10 trap on x <= 0; Fsqrt traps on + // negatives; Fsin/Fcos/Ftan trap on +-inf (NaN passes through + // bit-exactly); Fexp/Fcbrt/Ffloor/Fceil/Ftrunc are total. + Ln, + Log2, + Log10, + Fexp, + Fsqrt, + Fcbrt, + Fsin, + Fcos, + Ftan, + Ffloor, + Fceil, + Ftrunc, } impl NumOp1 { @@ -357,7 +415,7 @@ impl NumOp1 { pub fn sig(self) -> Ty { match self { NumOp1::Iabs => Ty::I64, - NumOp1::Fabs | NumOp1::Fround => Ty::F64, + _ => Ty::F64, } } @@ -366,6 +424,18 @@ impl NumOp1 { NumOp1::Iabs => "iabs", NumOp1::Fabs => "fabs", NumOp1::Fround => "fround", + NumOp1::Ln => "ln", + NumOp1::Log2 => "log2", + NumOp1::Log10 => "log10", + NumOp1::Fexp => "fexp", + NumOp1::Fsqrt => "fsqrt", + NumOp1::Fcbrt => "fcbrt", + NumOp1::Fsin => "fsin", + NumOp1::Fcos => "fcos", + NumOp1::Ftan => "ftan", + NumOp1::Ffloor => "ffloor", + NumOp1::Fceil => "fceil", + NumOp1::Ftrunc => "ftrunc", } } } @@ -434,6 +504,36 @@ pub enum Inst { a: Value, b: Value, }, + /// Wave-1 string search — `a` is the haystack, `b` the needle. + Str2 { + op: StrOp2, + dst: Value, + a: Value, + b: Value, + }, + /// String length: codepoints (`slenc`) or UTF-8 bytes (`slenb`). + SLen { + bytes: bool, + dst: Value, + a: Value, + }, + /// round/trunc with digits on f64 — DuckDB's scale-then-round with the + /// oracle-extracted pow10 table; TOTAL (non-finite fallbacks differ + /// between round and trunc by measurement). `n` is an i64 register. + Round2f { + trunc: bool, + dst: Value, + a: Value, + n: Value, + }, + /// Integer round/trunc with digits: identity for n >= 0, WRAPPING + /// half-add at i64 for round with n < 0 (measured — never traps). + Round2i { + trunc: bool, + dst: Value, + a: Value, + n: Value, + }, /// `supper` / `slower` — Str -> Str, simple case mapping. Str1 { op: StrOp1, @@ -563,6 +663,10 @@ impl Inst { | Inst::Ftos { dst, .. } | Inst::Sconcat { dst, .. } | Inst::Str1 { dst, .. } + | Inst::Str2 { dst, .. } + | Inst::SLen { dst, .. } + | Inst::Round2f { dst, .. } + | Inst::Round2i { dst, .. } | Inst::Strim { dst, .. } | Inst::Ssubstr { dst, .. } | Inst::Num1 { dst, .. } @@ -614,6 +718,7 @@ impl Inst { | Inst::Itof { dst, a } | Inst::Ftoi { dst, a, .. } | Inst::Str1 { dst, a, .. } + | Inst::SLen { dst, a, .. } | Inst::Num1 { dst, a, .. } | Inst::Not { dst, a } => { *dst = m(*dst); @@ -624,6 +729,9 @@ impl Inst { | Inst::Strim { dst, a, chars: b, .. } + | Inst::Str2 { dst, a, b, .. } + | Inst::Round2f { dst, a, n: b, .. } + | Inst::Round2i { dst, a, n: b, .. } | Inst::Sconcat { dst, a, b } => { *dst = m(*dst); *a = m(*a); diff --git a/src/specializer/ir/parse.rs b/src/specializer/ir/parse.rs index 544e18b..ae743ef 100644 --- a/src/specializer/ir/parse.rs +++ b/src/specializer/ir/parse.rs @@ -11,7 +11,7 @@ use std::collections::HashMap; use super::{ BinOp, Block, BlockId, CmpPred, Col, ColTy, Inst, Lit, NumOp1, Program, RoundMode, StaticTy, - StrOp1, Term, TrimSide, Ty, Value, + StrOp1, StrOp2, Term, TrimSide, Ty, Value, }; #[derive(Debug)] @@ -857,7 +857,7 @@ impl Parser { Inst::Const { dst: def!(0), lit } } "iadd" | "isub" | "imul" | "idiv" | "irem" | "fadd" | "fsub" | "fmul" | "fdiv" - | "frem" | "and" | "or" | "xor" => { + | "frem" | "fpow" | "flogb" | "and" | "or" | "xor" => { want_dsts(1, self)?; let op = match opcode.as_str() { "iadd" => BinOp::Iadd, @@ -870,6 +870,8 @@ impl Parser { "fmul" => BinOp::Fmul, "fdiv" => BinOp::Fdiv, "frem" => BinOp::Frem, + "fpow" => BinOp::Fpow, + "flogb" => BinOp::Flogb, "and" => BinOp::And, "or" => BinOp::Or, _ => BinOp::Xor, @@ -970,6 +972,55 @@ impl Parser { Inst::StofOpt { flag, dst, a } } } + "sfind" | "scontains" | "sstarts" | "sends" => { + want_dsts(1, self)?; + let op = match opcode.as_str() { + "sfind" => StrOp2::Find, + "scontains" => StrOp2::Contains, + "sstarts" => StrOp2::Starts, + _ => StrOp2::Ends, + }; + let a = self.use_value()?; + self.expect(Tok::Comma)?; + let b = self.use_value()?; + Inst::Str2 { + op, + dst: def!(0), + a, + b, + } + } + "fround2" | "ftrunc2" | "iround2" | "itrunc2" => { + want_dsts(1, self)?; + let a = self.use_value()?; + self.expect(Tok::Comma)?; + let n = self.use_value()?; + let trunc = opcode.contains("trunc"); + if opcode.starts_with('f') { + Inst::Round2f { + trunc, + dst: def!(0), + a, + n, + } + } else { + Inst::Round2i { + trunc, + dst: def!(0), + a, + n, + } + } + } + "slenc" | "slenb" => { + want_dsts(1, self)?; + let a = self.use_value()?; + Inst::SLen { + bytes: opcode == "slenb", + dst: def!(0), + a, + } + } "sconcat" => { want_dsts(1, self)?; let a = self.use_value()?; @@ -1026,12 +1077,25 @@ impl Parser { len, } } - "iabs" | "fabs" | "fround" => { + "iabs" | "fabs" | "fround" | "ln" | "log2" | "log10" | "fexp" | "fsqrt" | "fcbrt" + | "fsin" | "fcos" | "ftan" | "ffloor" | "fceil" | "ftrunc" => { want_dsts(1, self)?; let op = match opcode.as_str() { "iabs" => NumOp1::Iabs, "fabs" => NumOp1::Fabs, - _ => NumOp1::Fround, + "fround" => NumOp1::Fround, + "ln" => NumOp1::Ln, + "log2" => NumOp1::Log2, + "log10" => NumOp1::Log10, + "fexp" => NumOp1::Fexp, + "fsqrt" => NumOp1::Fsqrt, + "fcbrt" => NumOp1::Fcbrt, + "fsin" => NumOp1::Fsin, + "fcos" => NumOp1::Fcos, + "ftan" => NumOp1::Ftan, + "ffloor" => NumOp1::Ffloor, + "fceil" => NumOp1::Fceil, + _ => NumOp1::Ftrunc, }; let a = self.use_value()?; Inst::Num1 { diff --git a/src/specializer/ir/print.rs b/src/specializer/ir/print.rs index 8d93a96..e78d257 100644 --- a/src/specializer/ir/print.rs +++ b/src/specializer/ir/print.rs @@ -124,6 +124,20 @@ fn print_inst(s: &mut String, p: &Program, inst: &Inst) { Inst::Str1 { op, a, .. } => { let _ = write!(s, "{} {}", op.name(), val(*a)); } + Inst::Str2 { op, a, b, .. } => { + let _ = write!(s, "{} {}, {}", op.name(), val(*a), val(*b)); + } + Inst::Round2f { trunc, a, n, .. } => { + let op = if *trunc { "ftrunc2" } else { "fround2" }; + let _ = write!(s, "{} {}, {}", op, val(*a), val(*n)); + } + Inst::Round2i { trunc, a, n, .. } => { + let op = if *trunc { "itrunc2" } else { "iround2" }; + let _ = write!(s, "{} {}, {}", op, val(*a), val(*n)); + } + Inst::SLen { bytes, a, .. } => { + let _ = write!(s, "{} {}", if *bytes { "slenb" } else { "slenc" }, val(*a)); + } Inst::Strim { side, a, chars, .. } => { let _ = write!(s, "strim.{} {}, {}", side.name(), val(*a), val(*chars)); } diff --git a/src/specializer/ir/verify.rs b/src/specializer/ir/verify.rs index fbe44a7..dde51bd 100644 --- a/src/specializer/ir/verify.rs +++ b/src/specializer/ir/verify.rs @@ -185,6 +185,10 @@ fn dst_types(p: &Program, inst: &Inst) -> Vec<(Value, Ty)> { vec![(*dst, Ty::Str)] } Inst::Num1 { op, dst, .. } => vec![(*dst, op.sig())], + Inst::Str2 { op, dst, .. } => vec![(*dst, op.result_ty())], + Inst::SLen { dst, .. } => vec![(*dst, Ty::I64)], + Inst::Round2f { dst, .. } => vec![(*dst, Ty::F64)], + Inst::Round2i { dst, .. } => vec![(*dst, Ty::I64)], Inst::StoiOpt { flag, dst, .. } => vec![(*flag, Ty::I1), (*dst, Ty::I64)], Inst::StofOpt { flag, dst, .. } => vec![(*flag, Ty::I1), (*dst, Ty::F64)], Inst::Load { dst, col } => vec![(*dst, in_col(*col).unwrap_or(Ty::I1))], @@ -353,11 +357,23 @@ fn check_block( Inst::StoiOpt { a, .. } | Inst::StofOpt { a, .. } => { want(&in_scope, def_types, *a, Ty::Str, "operand", bi, i, errs) } - Inst::Sconcat { a, b: rhs, .. } | Inst::Strim { a, chars: rhs, .. } => { + Inst::Sconcat { a, b: rhs, .. } + | Inst::Strim { a, chars: rhs, .. } + | Inst::Str2 { a, b: rhs, .. } => { want(&in_scope, def_types, *a, Ty::Str, "operand", bi, i, errs); want(&in_scope, def_types, *rhs, Ty::Str, "operand", bi, i, errs); } - Inst::Str1 { a, .. } => want(&in_scope, def_types, *a, Ty::Str, "operand", bi, i, errs), + Inst::Str1 { a, .. } | Inst::SLen { a, .. } => { + want(&in_scope, def_types, *a, Ty::Str, "operand", bi, i, errs) + } + Inst::Round2f { a, n, .. } => { + want(&in_scope, def_types, *a, Ty::F64, "operand", bi, i, errs); + want(&in_scope, def_types, *n, Ty::I64, "operand", bi, i, errs); + } + Inst::Round2i { a, n, .. } => { + want(&in_scope, def_types, *a, Ty::I64, "operand", bi, i, errs); + want(&in_scope, def_types, *n, Ty::I64, "operand", bi, i, errs); + } Inst::Ssubstr { a, start, len, .. } => { want(&in_scope, def_types, *a, Ty::Str, "operand", bi, i, errs); want( diff --git a/src/specializer/lower.rs b/src/specializer/lower.rs index a7ced09..4b5d063 100644 --- a/src/specializer/lower.rs +++ b/src/specializer/lower.rs @@ -274,18 +274,30 @@ impl<'a> FB<'a> { None => l.val, Some(f) => { let d = self.default_of(ty); - let dst = self.fresh(); - self.inst(Inst::Select { - dst, - cond: f, - a: l.val, - b: d, - }); - dst + self.select_of(f, l.val, d) + } + } + } + + /// Like `masked`, but to an op-specific SAFE constant: the type default + /// (0.0) is itself in the trap domain of ln/logb, so those mask to a + /// value the op accepts (the result is discarded under the false flag). + fn masked_to(&mut self, l: Lane, safe: Lit) -> Value { + match l.flag { + None => l.val, + Some(f) => { + let d = self.const_lit(safe); + self.select_of(f, l.val, d) } } } + fn select_of(&mut self, cond: Value, a: Value, b: Value) -> Value { + let dst = self.fresh(); + self.inst(Inst::Select { dst, cond, a, b }); + dst + } + fn default_of(&mut self, ty: Ty) -> Value { self.const_lit(match ty { Ty::I1 => Lit::I1(false), @@ -568,6 +580,79 @@ impl<'a> FB<'a> { val: dst, }) } + SKind::Round2 { trunc, a, n } => { + let la = self.emit(a, live)?; + live.push((la, a.ty)); + let ln = self.emit(n, live)?; + let (la, _) = live.pop().expect("pushed above"); + let dst = self.fresh(); + if a.ty == Ty::F64 { + self.inst(Inst::Round2f { + trunc: *trunc, + dst, + a: la.val, + n: ln.val, + }); + } else { + self.inst(Inst::Round2i { + trunc: *trunc, + dst, + a: la.val, + n: ln.val, + }); + } + Ok(Lane { + flag: self.combine_flags(la.flag, ln.flag), + val: dst, + }) + } + SKind::MathF1 { op, a } => { + let l = self.emit(a, live)?; + // Safe mask per op: the value must be OUTSIDE the trap + // domain (1.0 for logs; 0.0 is fine for sqrt/trig; total + // ops need no mask at all). + let av = match op { + NumOp1::Ln | NumOp1::Log2 | NumOp1::Log10 => self.masked_to(l, Lit::F64(1.0)), + NumOp1::Fsqrt | NumOp1::Fsin | NumOp1::Fcos | NumOp1::Ftan => { + self.masked_to(l, Lit::F64(0.0)) + } + _ => l.val, + }; + let dst = self.fresh(); + self.inst(Inst::Num1 { + op: *op, + dst, + a: av, + }); + Ok(Lane { + flag: l.flag, + val: dst, + }) + } + SKind::MathF2 { op, a, b } => { + let la = self.emit(a, live)?; + live.push((la, a.ty)); + let lb = self.emit(b, live)?; + let (la, _) = live.pop().expect("pushed above"); + // Flogb traps on base<=0 / base==1 / x<=0. NULL pre-empts + // EVERY domain check (pinned: log(-2.0, NULL) is NULL, not + // an error) — so both payloads mask under the COMBINED + // flag: either side NULL makes both sides safe. + let flag = self.combine_flags(la.flag, lb.flag); + let (av, bv) = match op { + BinOp::Flogb => { + let a = Lane { flag, val: la.val }; + let b = Lane { flag, val: lb.val }; + ( + self.masked_to(a, Lit::F64(10.0)), + self.masked_to(b, Lit::F64(1.0)), + ) + } + _ => (la.val, lb.val), + }; + let val = self.bin(*op, av, bv); + Ok(Lane { flag, val }) + } SKind::Round(a) => { let l = self.emit(a, live)?; let dst = self.fresh(); @@ -581,6 +666,36 @@ impl<'a> FB<'a> { val: dst, }) } + SKind::Str2 { op, a, b } => { + let la = self.emit(a, live)?; + live.push((la, Ty::Str)); + let lb = self.emit(b, live)?; + let (la, _) = live.pop().expect("pushed above"); + let dst = self.fresh(); + self.inst(Inst::Str2 { + op: *op, + dst, + a: la.val, + b: lb.val, + }); + Ok(Lane { + flag: self.combine_flags(la.flag, lb.flag), + val: dst, + }) + } + SKind::SLen { bytes, a } => { + let l = self.emit(a, live)?; + let dst = self.fresh(); + self.inst(Inst::SLen { + bytes: *bytes, + dst, + a: l.val, + }); + Ok(Lane { + flag: l.flag, + val: dst, + }) + } SKind::Concat { a, b } => { let la = self.emit(a, live)?; live.push((la, Ty::Str)); diff --git a/src/specializer/plan.rs b/src/specializer/plan.rs index 278e2b1..2728c40 100644 --- a/src/specializer/plan.rs +++ b/src/specializer/plan.rs @@ -160,6 +160,37 @@ pub enum SKind { a: Box, b: Box, }, + /// Wave-1 string search (haystack, needle) — total, NULL-propagating. + Str2 { + op: super::ir::StrOp2, + a: Box, + b: Box, + }, + /// String length: codepoints (length) or UTF-8 bytes (strlen). + SLen { + bytes: bool, + a: Box, + }, + /// round/trunc with digits — result type == subject type (I64 or F64); + /// total, NULL-propagating. + Round2 { + trunc: bool, + a: Box, + n: Box, + }, + /// Wave-1 f64 unary math (operand promoted to F64 by the frontend); + /// NULL-propagating; the trapping ops get safe-masked payloads in + /// lowering so a NULL row can never fire the domain trap. + MathF1 { + op: super::ir::NumOp1, + a: Box, + }, + /// Wave-1 f64 binary math: Fpow (total) and Flogb(base, x) (trapping). + MathF2 { + op: super::ir::BinOp, + a: Box, + b: Box, + }, } /// SQL-level arithmetic. `Div` is DuckDB's `/` — ALWAYS float division diff --git a/src/specializer/tests.rs b/src/specializer/tests.rs index 1ae837e..5adeca3 100644 --- a/src/specializer/tests.rs +++ b/src/specializer/tests.rs @@ -481,7 +481,7 @@ fn unsupported_constructs_are_named_cleanly() { // function arm until the catalogue distinguishes aggregation. ("SELECT sum(a) FROM __THIS__", "function sum"), ("SELECT a FROM __THIS__ GROUP BY a", "aggregation"), - ("SELECT length('x') FROM __THIS__", "function"), + ("SELECT levenshtein('x', 'y') FROM __THIS__", "function"), // Star now expands; the still-unsupported star forms reject by name. ("SELECT * REPLACE (a + 1 AS a) FROM __THIS__", "REPLACE"), ("SELECT COLUMNS('a') FROM __THIS__", "function COLUMNS"), diff --git a/tests/test_duckdb_interpreter.py b/tests/test_duckdb_interpreter.py index 6749a59..4679810 100644 --- a/tests/test_duckdb_interpreter.py +++ b/tests/test_duckdb_interpreter.py @@ -8,6 +8,7 @@ from __future__ import annotations +import math from typing import Any import duckdb @@ -582,3 +583,848 @@ def test_star_over_joined_table_rejects_by_name(): row_tables={"__THIS__": _row_model({"k": "int"})}, static_tables={"dim": DIM}, ) + + +# --------------------------------------------------------- TASK-47: +# BETWEEN / IN desugars — measured DuckDB 1.5.5 truth tables (wave-1 pins). +nan, inf = float("nan"), float("inf") + + +def test_between_three_valued_int(): + duck_check( + "SELECT x BETWEEN lo AND hi AS b, x NOT BETWEEN lo AND hi AS nb FROM __THIS__", + {"x": "int?", "lo": "int?", "hi": "int?"}, + [ + {"x": 5, "lo": 1, "hi": 10}, # True + {"x": 5, "lo": 10, "hi": 1}, # empty range -> False + {"x": None, "lo": 1, "hi": 10}, # NULL + {"x": None, "lo": 10, "hi": 1}, # NULL x + empty range -> still NULL + {"x": 5, "lo": None, "hi": 10}, # NULL + {"x": 5, "lo": None, "hi": 4}, # False — failing half decides + {"x": 5, "lo": 1, "hi": None}, # NULL + {"x": 5, "lo": 6, "hi": None}, # False + {"x": None, "lo": None, "hi": None}, # NULL + {"x": 5, "lo": 5, "hi": 5}, # inclusive -> True + ], + ) + + +def test_between_int_x_double_bounds_unifies_to_double(): + duck_check( + "SELECT x BETWEEN lo AND hi AS b FROM __THIS__", + {"x": "int", "lo": "float", "hi": "float"}, + [ + {"x": 5, "lo": 0.5, "hi": 5.5}, + {"x": 5, "lo": 5.1, "hi": 9.9}, + # int->double is lossy at 2^53 — DuckDB does the same cast: True + {"x": 9007199254740993, "lo": 9007199254740992.0, "hi": 9007199254740992.0}, + ], + ) + + +def test_between_duck_double_order_specials(): + duck_check( + "SELECT x BETWEEN lo AND hi AS b, x NOT BETWEEN lo AND hi AS nb FROM __THIS__", + {"x": "float?", "lo": "float?", "hi": "float?"}, + [ + {"x": nan, "lo": 1.0, "hi": 2.0}, # False (nan<=2 False) + {"x": 5.0, "lo": 1.0, "hi": nan}, # True (nan above everything) + {"x": 5.0, "lo": nan, "hi": 10.0}, # False + {"x": nan, "lo": nan, "hi": nan}, # True (nan = nan) + {"x": inf, "lo": 1.0, "hi": nan}, # True (nan above inf) + {"x": nan, "lo": None, "hi": 1.0}, # False (NULL AND False) + {"x": nan, "lo": 1.0, "hi": None}, # NULL (True AND NULL) + {"x": 0.0, "lo": -0.0, "hi": -0.0}, # True (zeros equal) + {"x": -0.0, "lo": 0.0, "hi": 0.0}, # True + {"x": -inf, "lo": -inf, "hi": inf}, # True + ], + ) + + +def test_between_strings_byte_order(): + duck_check( + "SELECT x BETWEEN lo AND hi AS b FROM __THIS__", + {"x": "str", "lo": "str", "hi": "str"}, + [ + {"x": "b", "lo": "a", "hi": "c"}, # True + {"x": "B", "lo": "a", "hi": "c"}, # False (0x42 < 0x61) + {"x": "Z", "lo": "a", "hi": "z"}, # False + {"x": "é", "lo": "a", "hi": "z"}, # False (0xC3A9 > 'z') + {"x": "abc", "lo": "ab", "hi": "abd"}, # True + {"x": "", "lo": "", "hi": "a"}, # True + {"x": "a", "lo": "c", "hi": "a"}, # False + ], + ) + + +def test_in_null_element_truth_table(): + duck_check( + "SELECT x IN (1, NULL) AS i, x NOT IN (1, NULL) AS ni," + " x IN (1, 2) AS i2, x NOT IN (1, 2) AS ni2, x IN (NULL) AS io FROM __THIS__", + {"x": "int?"}, + [{"x": 1}, {"x": 3}, {"x": None}], + # i: T/N/N ni: F/N/N i2: T/F/N ni2: F/T/N io: N/N/N + ) + + +def test_in_column_elements_is_the_or_chain(): + duck_check( + "SELECT x IN (y, z) AS r, x NOT IN (y, z) AS nr FROM __THIS__", + {"x": "int?", "y": "int?", "z": "int?"}, + [ + {"x": 1, "y": 1, "z": None}, # True / False + {"x": 3, "y": 1, "z": None}, # NULL / NULL + {"x": 3, "y": 1, "z": 2}, # False / True + {"x": None, "y": 1, "z": 2}, # NULL / NULL + ], + ) + + +def test_in_int_col_float_literals_small_values(): + duck_check( + "SELECT x IN (1.0) AS a, x IN (0.5, 1.0) AS b, x IN (1.5) AS c FROM __THIS__", + {"x": "int?"}, + [{"x": 1}, {"x": 2}, {"x": 3}, {"x": None}], # 1.5 never rounds to 2 + ) + + +def test_in_nan_and_signed_zero(): + duck_check( + "SELECT x IN ('NaN'::DOUBLE) AS n, x IN (1.0, 'NaN'::DOUBLE) AS m," + " x IN (0.0) AS z, x NOT IN ('NaN'::DOUBLE) AS nn FROM __THIS__", + {"x": "float"}, + [{"x": nan}, {"x": 1.0}, {"x": -0.0}], # NaN=NaN True; -0.0 IN (0.0) True + ) + + +def test_in_strings_and_bools(): + duck_check( + "SELECT s IN ('a', 'b') AS r, s IN ('b', NULL) AS rn FROM __THIS__", + {"s": "str?"}, + [{"s": "a"}, {"s": "A"}, {"s": None}], + ) + # BOOLEAN comparison stays the engine's pre-existing clean limit, so + # bool IN/BETWEEN desugar into it and reject by name. + with pytest.raises(ValueError, match="comparison on BOOLEAN"): + DuckDBInferFn( + "SELECT p IN (true) AS r FROM __THIS__", + row_tables={"__THIS__": _row_model({"p": "bool?"})}, + static_tables={}, + ) + + +# --------------------------------------------------------- TASK-47: +# wave-1 math builtins - measured DuckDB 1.5.5 pins as oracle tests. + + +def test_logexp_basic_values(): + duck_check( + "SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d," + " exp(x) AS e FROM __THIS__", + {"x": "float"}, + [ + {"x": v} + for v in [ + 1.0, + 2.718281828459045, + 2.0, + 10.0, + 0.5, + 1000.0, + 1e308, + 5e-324, + 709.782712893384, + ] + ], + ) + + +def test_logexp_nan_inf_pass_through(): + # NaN and +inf do NOT trip the domain checks; only <=0 does. + duck_check( + "SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d," + " exp(x) AS e FROM __THIS__", + {"x": "float"}, + [{"x": float("nan")}, {"x": float("inf")}], + ) + + +def test_exp_never_errors(): + # overflow -> inf, underflow -> denormal then +0.0; -inf -> +0.0; + # boundary 709.782712893384 -> 1.7976931348622732e+308 + duck_check( + "SELECT exp(x) AS e FROM __THIS__", + {"x": "float"}, + [ + {"x": v} + for v in [ + float("-inf"), + -1000.0, + -746.0, + -745.0, + -744.4400719213812, + 0.0, + -0.0, + 709.782712893384, + 709.7827128933841, + 710.0, + 1000.0, + ] + ], + ) + + +def test_logexp_null_propagation(): + duck_check( + "SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e," + " log(x, 2.0) AS f, log(2.0, x) AS g FROM __THIS__", + {"x": "float?"}, + [{"x": None}, {"x": 4.0}], + ) + + +def test_log_two_arg_is_log10_ratio(): + # log(b,x) = log10(x)/log10(b) bit-exact. Discriminators: (10,1000)->3.0 (ln-ratio + # gives 2.9999999999999996), (e,10)->2.302585092994046 (log2-ratio differs), + # (e,5e-324)->-744.4400719213813 != ln(5e-324). (0.5,1.0)->-0.0 and (inf,0.5)->-0.0 + # pin sign-of-zero; (inf,inf)->nan. + duck_check( + "SELECT log(b, x) AS r FROM __THIS__", + {"b": "float", "x": "float"}, + [ + {"b": b, "x": x} + for b, x in [ + (2.0, 8.0), + (10.0, 1000.0), + (3.0, 7.0), + (0.5, 8.0), + (0.5, 1.0), + (2.0, 1.0), + (2.718281828459045, 5e-324), + (2.718281828459045, 10.0), + (1.0000000000000002, 4.0), + (0.9999999999999999, 4.0), + (5e-324, 8.0), + (1e308, 8.0), + (float("nan"), 8.0), + (2.0, float("nan")), + (float("nan"), float("nan")), + (float("inf"), 8.0), + (float("inf"), 0.5), + (float("inf"), float("inf")), + (float("inf"), 1.0), + (2.0, float("inf")), + (2.0, 5e-324), + (2.0, 1e308), + ] + ], + ) + + +def test_log_two_arg_null_preempts_domain_errors(): + # NULL in either slot wins over every domain error, including base 0 / negative / 1. + duck_check( + "SELECT log(b, x) AS r FROM __THIS__", + {"b": "float?", "x": "float?"}, + [ + {"b": None, "x": 8.0}, + {"b": 2.0, "x": None}, + {"b": None, "x": None}, + {"b": None, "x": -4.0}, + {"b": 0.0, "x": None}, + {"b": -2.0, "x": None}, + {"b": 1.0, "x": None}, + ], + ) + + +def test_logexp_integer_input_yields_double(): + duck_check( + "SELECT ln(x) AS a, log(x) AS b, log2(x) AS c, log10(x) AS d, exp(x) AS e," + " log(x, 8) AS f FROM __THIS__", + {"x": "int"}, + [{"x": 2}, {"x": 9223372036854775807}], + ) + + +def test_logexp_guarded_rows_do_not_trap(): + # A bad row aborts an unguarded query, but WHERE / CASE genuinely + # prevent evaluation. + duck_check( + "SELECT CASE WHEN x > 0 THEN ln(x) ELSE NULL END AS a FROM __THIS__", + {"x": "float"}, + [{"x": 1.0}, {"x": -1.0}, {"x": 4.0}, {"x": 0.0}], + ) + duck_check( + "SELECT ln(x) AS a FROM __THIS__ WHERE x > 0", + {"x": "float"}, + [{"x": 1.0}, {"x": -1.0}, {"x": 4.0}, {"x": 0.0}], + ) + + +def test_pow_bigint_inputs_yield_double(): + # ints promote to DOUBLE before compute: i64::MAX rounds to 2^63, + # 2^53+1 loses the +1, 3^40 is the rounded double not the exact int. + rows = [ + (0, 0), + (2, 10), + (0, -1), + (-8, 2), + (10, 18), + (9223372036854775807, 1), + (-9223372036854775808, 1), + (2, 63), + (2, 64), + (3, 40), + (9007199254740993, 1), + (5, 2), + ] + duck_check( + "SELECT pow(x, y) AS p FROM __THIS__", + {"x": "int", "y": "int"}, + [{"x": x, "y": y} for x, y in rows], + ) + + +def test_sqrt_non_negative_domain(): + # NOTE: any x < 0 (incl -inf) raises OutOfRange; keep those out of duck_check. + xs = [4.0, 2.0, 0.0, -0.0, INF, NAN, None, 5e-324, 1e308] + duck_check( + "SELECT sqrt(x) AS s FROM __THIS__", {"x": "float?"}, [{"x": x} for x in xs] + ) + + +def duck_check_ulp(sql, row_schema, row_rows, max_ulp=1): + """duck_check with a float ulp tolerance, POSITIONAL compare (no joins). + + Exists for exactly one reason so far: DuckDB's own wheels disagree with + each other on cbrt by one ulp across platforms (Windows wheel matches + Rust/ucrt bit-exactly; the Linux wheel's bundled std::cbrt is one ulp + off on e.g. cbrt(27)) — CI-discovered 2026-07-26. The oracle itself is + platform-inconsistent here, so repr-exact parity is unpinnable. + """ + import math + import struct + + model = _row_model(row_schema) + fn = DuckDBInferFn(sql, row_tables={"__THIS__": model}, static_tables={}) + inputs = [model(**r) for r in row_rows] + got = [r.model_dump() for r in fn.infer({"__THIS__": inputs})] + + con = duckdb.connect() + con.register("__arrow_this", static(row_schema, row_rows)) + con.execute("CREATE TABLE __THIS__ AS SELECT * FROM __arrow_this") + want = con.execute(sql).to_arrow_table().to_pylist() + + def close(a, b): + if isinstance(a, float) and isinstance(b, float): + if math.isnan(a) and math.isnan(b): + return True + ia = struct.unpack(" {max_ulp} ulp)" + + +def test_sqrt_cbrt_bigint(): + xs = [4, 0, 27, 9223372036854775807, 9007199254740992, 9007199254740993, None] + duck_check_ulp( + "SELECT sqrt(x) AS s, cbrt(x) AS c FROM __THIS__", + {"x": "int?"}, + [{"x": x} for x in xs], + ) + + +def test_cbrt_total_function(): + # cbrt has NO domain restriction: negatives, -inf, -0.0 all flow through. + # ulp-tolerant: the oracle's own wheels disagree cross-platform on cbrt. + xs = [8.0, -8.0, 27.0, 2.0, 0.0, -0.0, INF, -INF, NAN, None, 5e-324, -1.0] + duck_check_ulp( + "SELECT cbrt(x) AS c FROM __THIS__", {"x": "float?"}, [{"x": x} for x in xs] + ) + + +def test_sqrt_where_guard_is_lazy(): + # Filtered-out negative rows must NOT trap. + duck_check( + "SELECT sqrt(x) AS s FROM __THIS__ WHERE x >= 0", + {"x": "float"}, + [{"x": 4.0}, {"x": -1.0}], + ) + duck_check( + "SELECT CASE WHEN x >= 0 THEN sqrt(x) ELSE NULL END AS s FROM __THIS__", + {"x": "float"}, + [{"x": 4.0}, {"x": -1.0}], + ) + + +def test_sqrt_negative_is_runtime_error(): + # Not expressible as a duck_check equality: both engines must RAISE. + # DuckDB: _duckdb.OutOfRangeException + # 'Out of Range Error: cannot take square root of a negative number' + with pytest.raises(Exception, match="cannot take square root of a negative number"): + duck_check("SELECT sqrt(x) AS s FROM __THIS__", {"x": "float"}, [{"x": -1.0}]) + with pytest.raises(Exception, match="cannot take square root of a negative number"): + duck_check( + "SELECT sqrt(x) AS s FROM __THIS__", {"x": "float"}, [{"x": float("-inf")}] + ) + + +def test_trig_double_column_edges(): + duck_check( + "SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__", + {"x": "float"}, + [{"x": v} for v in TRIG_EDGE_FLOATS], + ) # repr-compare keeps -0.0 vs 0.0 apart: the sin/tan(-0.0) sign pins hold + + +def test_trig_null_propagation(): + duck_check( + "SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__", + {"x": "float?"}, + [{"x": None}, {"x": 1.0}], + ) + + +def test_trig_integer_input_yields_double(): + duck_check( + "SELECT sin(x) AS s, cos(x) AS c, tan(x) AS t FROM __THIS__", + {"x": "int"}, + [ + {"x": 0}, + {"x": 1}, + {"x": -1}, + {"x": 9223372036854775807}, # rounds to 2^63 as f64 BEFORE sin + {"x": -9223372036854775808}, + ], + ) + + +def test_trig_integer_null(): + duck_check( + "SELECT sin(x) AS s FROM __THIS__", + {"x": "int?"}, + [{"x": None}, {"x": 3}], + ) + + +def test_pi_literal_and_vectorized(): + duck_check( + "SELECT pi() AS p, x + pi() AS xp, sin(pi() / 2) AS one, " + "cos(pi()) AS neg_one, tan(pi() / 2) AS big, sin(-pi() / 2) AS neg_one_s " + "FROM __THIS__", + {"x": "float"}, + [{"x": 0.0}, {"x": 1.0}], + ) + + +# NOT expressible via duck_check (it asserts success on both sides) — engine needs +# dedicated tests for these, asserting against a plain duckdb replay: +# 1. sin/cos/tan(+-inf) must raise a per-row runtime error whose message mirrors +# "Out of Range Error: input value inf is out of range for numeric function" +# (and "... value -inf ..."); an inf row excluded by WHERE must NOT raise. +# 2. NaN *payload* preservation (e.g. input bits 0x7ff0deadbeef0001 out unchanged): +# repr collapses every NaN to 'nan', so compare struct.pack bits directly. +def test_round_family_single_arg_double(): + vals = [ + 2.5, + 3.5, + -2.5, + -3.5, + 0.5, + -0.5, + 1.5, + 0.49999999999999994, + -0.49999999999999994, + 2.675, + -0.4, + -0.04, + 0.0, + -0.0, + 5e-324, + -5e-324, + 4503599627370495.5, + -4503599627370495.5, + 9.3e18, + -9.3e18, + 1.7976931348623157e308, + 12345.678901234567, + NAN, + INF, + -INF, + None, + ] + duck_check( + "SELECT floor(x) AS f, ceil(x) AS c, ceiling(x) AS c2," + " trunc(x) AS t, round(x) AS r FROM __THIS__", + {"x": "float?"}, + [{"x": v} for v in vals], + ) + + +def test_floor_ceil_int64_go_through_double(): + rows = [ + {"i": v} + for v in [7, 9007199254740993, 9223372036854775807, -9223372036854775808, None] + ] + duck_check( + "SELECT floor(i) AS f, ceil(i) AS c, ceiling(i) AS c2 FROM __THIS__", + {"i": "int?"}, + rows, + ) + + +# Shared constants for the wave-1 math oracle tests. + +NAN = float("nan") +INF = float("inf") + +POW_SPECIALS = [ + (0.0, 0.0), + (0.0, -1.0), + (-0.0, -1.0), + (0.0, -2.0), + (-0.0, -2.0), + (-0.0, 3.0), + (-0.0, 2.0), + (-0.0, 0.5), + (-8.0, 1.0 / 3.0), + (-2.0, 0.5), + (-2.0, -0.5), + (NAN, 0.0), + (NAN, -0.0), + (1.0, NAN), + (NAN, NAN), + (NAN, 1.0), + (0.0, NAN), + (INF, 0.0), + (-INF, 0.0), + (-1.0, INF), + (-1.0, -INF), + (1.0, INF), + (INF, -1.0), + (-INF, -1.0), + (-INF, -2.0), + (-INF, 3.0), + (-INF, 2.0), + (-INF, 0.5), + (0.0, INF), + (-0.0, INF), + (0.0, -INF), + (-0.0, -INF), + (2.0, INF), + (0.5, INF), + (2.0, -INF), + (0.5, -INF), + (-0.5, INF), + (-2.0, -INF), + (1e308, 2.0), + (2.0, 1024.0), + (2.0, -1075.0), + (-2.0, -1075.0), + (2.0, 53.0), + (10.0, 18.0), + (5e-324, 0.5), + (-2.0, 3.0), + (None, 2.0), + (2.0, None), + (None, None), +] + +TRIG_EDGE_FLOATS = [ + 0.0, + -0.0, + 1.0, + -1.0, + 0.5, + math.pi / 2, + -math.pi / 2, + math.pi, + 2 * math.pi, + 1e15, + float(2**53), + 1e300, + 1.7976931348623157e308, + 5e-324, + 2.2250738585072014e-308, + 9.223372036854776e18, + float("nan"), +] + + +def test_pow_double_specials(): + # pow and power are one function; pin the IEEE specials per row. The ^ + # operator is deliberately NOT mapped: sqlparser parses it below * + # while DuckDB binds it above (measured 2*x^y = 2*(x^y) vs sqlparser's + # (2*x)^y) — mapping would silently compute the wrong tree. ** does + # not parse at all. Both stay clean errors. + duck_check( + "SELECT pow(x, y) AS p, power(x, y) AS q FROM __THIS__", + {"x": "float?", "y": "float?"}, + [{"x": x, "y": y} for x, y in POW_SPECIALS], + ) + + +def test_pow_operator_rejects_cleanly(): + with pytest.raises(ValueError, match="operator .*precedence differs"): + DuckDBInferFn( + "SELECT x ^ 2 AS r FROM __THIS__", + row_tables={"__THIS__": _row_model({"x": "float"})}, + static_tables={}, + ) + + +# --------------------------------------------------------- TASK-47: +# wave-1 string search - measured DuckDB 1.5.5 pins as oracle tests. +# string-search family - contract pins measured against DuckDB 1.5.5 (2026-07-26). +# All values below were probed through the vectorized path (table columns); +# literal-fold agreed on every pair (0 divergences). + +STRSEARCH_SCHEMA = {"s": "str?", "n": "str?"} +STRSEARCH_ROWS = [ + {"s": "abc", "n": "b"}, # simple hit -> 2 + {"s": "abc", "n": "z"}, # miss -> 0 + {"s": "abc", "n": ""}, # empty needle -> 1 / TRUE + {"s": "", "n": "a"}, # empty haystack -> 0 / FALSE + {"s": "", "n": ""}, # empty-in-empty -> 1 / TRUE + {"s": "abc", "n": "abcd"}, # needle longer -> 0 / FALSE + {"s": "abcabc", "n": "bc"}, # first occurrence -> 2 + {"s": "aaa", "n": "aa"}, # overlapping -> 1 + {"s": "abc", "n": "B"}, # case-sensitive -> 0 + {"s": "héllo", "n": "l"}, # CODEPOINT position -> 3 (bytes would be 4) + {"s": "héllo", "n": "é"}, # -> 2 + {"s": "héllo", "n": "él"}, # multibyte needle -> 2 + {"s": "a\U0001f44db", "n": "b"}, # emoji = 1 codepoint -> 3 (bytes would be 6) + {"s": "\U0001f468‍\U0001f469‍\U0001f467", "n": "\U0001f467"}, # ZWJ -> 5, not 1 + {"s": "ééa", "n": "a"}, # -> 3 + {"s": "xabc", "n": "abc"}, # suffix hit -> 2 + {"s": "abc", "n": "abc"}, # exact -> 1 + {"s": "é", "n": "é"}, # NO normalization -> 0 / FALSE + {"s": "é", "n": "́"}, # bare combining mark -> 2 + {"s": "abc", "n": None}, # NULL-strict every argument + {"s": None, "n": "b"}, + {"s": None, "n": None}, + {"s": None, "n": ""}, # NULL haystack + empty needle is still NULL +] + + +def test_strsearch_predicates(): + for fn in ["contains", "starts_with", "prefix", "ends_with", "suffix"]: + duck_check( + f"SELECT {fn}(s, n) AS r FROM __THIS__", + STRSEARCH_SCHEMA, + STRSEARCH_ROWS, + ) + + +def test_strsearch_three_valued_where(): + # NULL predicate rows must be dropped by WHERE, same as FALSE. + duck_check( + "SELECT s FROM __THIS__ WHERE contains(s, n)", + STRSEARCH_SCHEMA, + STRSEARCH_ROWS, + ) + duck_check( + "SELECT s FROM __THIS__ WHERE instr(s, n) > 1", + STRSEARCH_SCHEMA, + STRSEARCH_ROWS, + ) + + +def test_strsearch_positions(): + for fn_expr in ["instr(s, n)", "strpos(s, n)", "position(n IN s)"]: + duck_check( + f"SELECT {fn_expr} AS r FROM __THIS__", + STRSEARCH_SCHEMA, + STRSEARCH_ROWS, + ) + + +def test_strsearch_length_family(): + duck_check( + "SELECT length(s) AS lc, char_length(s) AS cc, strlen(s) AS lb FROM __THIS__", + {"s": "str?"}, + [ + {"s": "abc"}, + {"s": ""}, + {"s": "héllo"}, + {"s": "a\U0001f44db"}, + {"s": "\U0001f468‍\U0001f469‍\U0001f467"}, + {"s": "é"}, + {"s": None}, + ], + ) + + +# --------------------------------------------------------- TASK-47: +# least/greatest - NULL-ignoring, first-arg ties, duck order (pins). + + +def test_least_greatest_null_ignoring_int(): + duck_check( + "SELECT least(a, b) AS l, greatest(a, b) AS g," + " least(a) AS l1, least(a, b, 3, 0) AS ln FROM __THIS__", + {"a": "int?", "b": "int?"}, + [ + {"a": 1, "b": None}, # l=1 g=1 — NULL ignored + {"a": None, "b": 2}, # l=2 g=2 + {"a": None, "b": None}, # NULL only when ALL null + {"a": 1, "b": 2}, + {"a": 9223372036854775807, "b": -9223372036854775808}, # exact i64 + ], + ) + + +def test_least_greatest_double_nan_and_tie_keeps_first(): + duck_check( + "SELECT least(a, b) AS l, greatest(a, b) AS g FROM __THIS__", + {"a": "float?", "b": "float?"}, + [ + {"a": nan, "b": 1.0}, # l=1.0 g=nan (nan above everything) + {"a": 1.0, "b": nan}, + {"a": nan, "b": inf}, # l=inf g=nan + {"a": nan, "b": None}, # nan / nan — NULL ignored + {"a": -inf, "b": inf}, + {"a": -0.0, "b": 0.0}, # BOTH return first arg: l=-0.0 g=-0.0 + {"a": 0.0, "b": -0.0}, # l=0.0 g=0.0 + {"a": nan, "b": nan}, + ], + ) + + +def test_least_greatest_mixed_int_double_unifies_to_double(): + duck_check( + "SELECT least(i, d) AS l, greatest(i, d) AS g FROM __THIS__", + {"i": "int", "d": "float"}, + [ + {"i": 1, "d": 2.5}, + {"i": 3, "d": 2.5}, + # DuckDB casts the i64 to double (lossy): l = 9007199254740992.0 + {"i": 9007199254740993, "d": 9007199254740994.0}, + {"i": 9223372036854775807, "d": 1.0}, # g = 9.223372036854776e+18 + ], + ) + + +def test_least_greatest_strings_and_bools(): + duck_check( + "SELECT least(a, b) AS l, greatest(a, b) AS g FROM __THIS__", + {"a": "str?", "b": "str?"}, + [ + {"a": "b", "b": "a"}, + {"a": "B", "b": "a"}, # least='B' — byte order + {"a": "é", "b": "z"}, # greatest='é' + {"a": "", "b": "a"}, + {"a": "a", "b": None}, # NULL ignored + ], + ) + + +# --------------------------------------------------------- TASK-47: +# round/trunc with digits - oracle pow10 table semantics (pins). + + +def test_round_double_with_digits_constant_n(): + vals = [ + 2.675, + 0.285, + 1.005, + 0.145, + 1.015, + 2.665, + 0.125, + 0.375, + 1.255, + 0.045, + 1.115, + 1234.5, + -1234.5, + 1250.0, + -1250.0, + -44.0, + -0.04, + 1.7e308, + 9.9e307, + 5e-324, + -0.0, + NAN, + INF, + -INF, + None, + ] + duck_check( + "SELECT round(x, 2) AS r2, round(x, 0) AS r0, round(x, -1) AS rn1," + " round(x, -2) AS rn2, round(x, 309) AS rbig, round(x, -308) AS rn308," + " round(x, -400) AS rneg, trunc(x, 2) AS t2, trunc(x, -1) AS tn1," + " trunc(x, -400) AS tneg FROM __THIS__", + {"x": "float?"}, + [{"x": v} for v in vals], + ) + + +def test_round_double_pow_table_ulp_witnesses(): + # Only DuckDB's own pow(10,23) modifier (bits 0x44B52D02C7E14AF7) passes these. + rows = [ + {"x": float.fromhex("0x1.9fc4c8ce7ceb8p+8")}, # -> 415.76868906545445 + {"x": float.fromhex("0x1.bc25ab024d0ffp+8")}, # -> 444.14714064008143 + {"x": float.fromhex("-0x1.a279f7d13a32ep+8")}, + ] # -> -418.4764376418197 + duck_check("SELECT round(x, 23) AS r FROM __THIS__", {"x": "float"}, rows) + + +def test_round_variable_n_from_column(): + # engine 'int' columns are BIGINT: round(DOUBLE, BIGINT) does not bind, so CAST. + rows = [ + {"x": 2.675, "n": 2}, + {"x": 1234.5, "n": -2}, + {"x": NAN, "n": -2}, + {"x": INF, "n": -2}, + {"x": -INF, "n": -2}, + {"x": 2.5, "n": None}, + {"x": None, "n": 2}, + {"x": 1.7e308, "n": -308}, + {"x": 2.675, "n": 400}, + {"x": 2.675, "n": -400}, + {"x": -44.0, "n": -2}, + ] + duck_check( + "SELECT round(x, CAST(n AS INTEGER)) AS r," + " trunc(x, CAST(n AS INTEGER)) AS t FROM __THIS__", + {"x": "float?", "n": "int?"}, + rows, + ) + + +def test_round_family_int64_identity_and_negative_n(): + vals = [ + 1250, + -1250, + 1249, + -1249, + 25, + -25, + 49, + 50, + -49, + -50, + 0, + 15, + 9223372036854775807, + -9223372036854775808, + 9223372036854775757, + 9223372036854775800, + 5000000000000000000, + None, + ] + duck_check( + "SELECT trunc(i) AS t1, round(i) AS r1, round(i, 2) AS rpos," + " round(i, -2) AS rn2, round(i, -18) AS rn18, round(i, -19) AS rcut," + " trunc(i, -2) AS tn2, trunc(i, -19) AS tcut FROM __THIS__", + {"i": "int?"}, + [{"i": v} for v in vals], + )