linq_fold: array _join splice → m3f join_* parity (chunk N+3) - #2913
Merged
Conversation
Mirror of plan_decs_join for arrays. New `register_array_join_rows` pattern + `emit_array_join` produces hashB-collect + srcA-probe inline, preserving the count-no-where bucket-length fast path (per PR #2848's D2-A guarantee on the decs side). Two sources bind as invoke parameters (mirrors Zip's 2-source wrap rather than the 1-source Array helper). Primitive equi-key gate via existing `is_primitive_join_key_type`. Asymmetric "array _join decs-bridge" cascades via new `array_join_srcb_is_array` predicate. Supports `_join + count`, `_join + _where + count`, `_join + _where + to_array`, `_join + _select + to_array`, `_join + _select` (bare-select on array chain). Bare-`_select` typer-order subtlety: `selCall._type.firstType` may stay as unresolved `typedecl(result_selector(type<TT>))` when the chain doesn't end with `to_array()` (array-overload select returns array directly, so no enclosing wrap forces resolution). Extract the resolved type from the select lambda's body via `peel_lambda_single_return` instead. emit_decs_join doesn't hit this because its bench chains universally end with `|> to_array()` (decs returns iterator-typed bridge). 10 splice-exercising tests added to `tests/linq/test_linq_join.das`, including the bare-`_select` shape that exposed the typer-order subtlety. All 198 from_decs tests + 47 join tests + 28 terminal_select tests pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirror of PR #2861's DecsJoin adapter for arrays. Adds `ArrayJoinShape` struct + `SourceAdapter.ArrayJoin` variant arm, extends 5 adapter helpers (`adapter_bind_name`, `adapter_element_type`, `adapter_wrap_source_loop`, `adapter_wrap_invoke`, `plan_group_by_core` allocation prefix), and wires `upstream_join` slot into the `group_by_array` pattern row with new `array_join_invariants` requires predicate. `plan_group_by_core` requires zero further changes — `ArrayJoin` flows through the existing adapter abstraction layer the same way `DecsJoin` does. `adapter_wrap_source_loop`'s new `ArrayJoin` branch performs hashB-collect + srcA-probe via plain `for` loops (no `for_each_archetype` wrapping); `adapter_wrap_invoke`'s new `ArrayJoin` branch produces a 2-source invoke (mirrors Zip's wrap shape, not Decs's zero-arg form). Closes the `_join |> _group_by(_.K) |> _select((K, agg))` array-side gap. Bench m3f now matches m4 (within ±25% INTERP, ±15% JIT) on `join_groupby_count` + `join_groupby_to_array`, completing the chunk N+2 parity target across all 5 `join_*` cells. 4 new cross-arm tests appended to `test_linq_join.das` (count, sum, min/ max, empty-srcb). All 14 existing decs `theme3_decs_join_groupby` tests still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- `doc/source/reference/linq_fold_patterns.rst`: add "Array-array equi-join" section parallel to the existing "Decs-decs equi-join" section. Same 5 row shape covering count / implicit-to-array / trailing _select / trailing _where / cross-arm _group_by. - `tests/linq/test_linq_join.das`: add non-primitive-key cascade test verifying struct-keyed _join still produces correct results via tier-2 cascade (silent splice-bail, no error). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Full INTERP+JIT rerun across benchmarks/sql/*.das. m3f beats m4 on all 5 join_* cells in both lanes: | Cell | m3f INTERP before/after | m3f JIT before/after | |---|---:|---:| | join_count | 128.2 → 51.0 | 34.6 → 11.6 | | join_select | 148.1 → 72.0 | 41.6 → 19.7 | | join_where_count | 148.5 → 60.1 | 41.2 → 20.4 | | join_groupby_count | 186.2 → 78.3 | 47.2 → 19.4 | | join_groupby_to_array | 217.8 → 88.1 | 55.5 → 19.5 | Drift across the non-join matrix is at measurement floor (±5% INTERP, ±10% JIT). The splice is gated on array_source + primitive equi-key + (for cross-arm) upstream_join capture, so non-join chains aren't touched. Header refreshed; Notes-on-missing-lanes section unchanged (no — cells flipped). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an array-side _join splice in linq_fold mirroring the existing decs-side emit_decs_join, plus a cross-arm _join |> _group_by path via a new ArrayJoin SourceAdapter variant. Closes the m3f-vs-m4 parity gap across all five join_* SQL benchmarks (m3f now beats m4 in both INTERP and JIT).
Changes:
- New
ArrayJoinShape+SourceAdapter.ArrayJoinvariant; extendsadapter_bind_name,adapter_element_type,adapter_wrap_source_loop,adapter_wrap_invoke, andplan_group_by_core's name prefix. - New
register_array_join_rows/emit_array_join(Phase 1) witharray_join_srcb_is_arrayguard; newarray_join_invariantsguard added togroup_by_arraypattern with optionalupstream_joincapture (Phase 2). - 11 new tests in
tests/linq/test_linq_join.das, docs row inlinq_fold_patterns.rst, refreshed bench matrix inresults.md.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| daslib/linq_fold.das | Adds ArrayJoinShape/ArrayJoin adapter, array_join_* guard predicates, splice emit + registration, cross-arm group_by wiring. |
| tests/linq/test_linq_join.das | Adds 11 tests exercising Phase 1 splice arms, Phase 2 join+group_by, and non-primitive-key cascade. |
| doc/source/reference/linq_fold_patterns.rst | Documents the array-array equi-join patterns and the cross-arm group_by composition. |
| benchmarks/sql/results.md | Refreshes the INTERP+JIT benchmark matrix to reflect the new m3f wins on join_* rows. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
11 tasks
pull Bot
pushed a commit
to forksnd/daScript
that referenced
this pull request
May 28, 2026
The Helper B refactor (qmacro_block_to_array → qmacro_expr inside the ArrayJoin adapter branch of adapter_wrap_source_loop) happens to drop an extra ExprBlock layer the original inline qmacro_block produced. m3f INTERP improves 88.1 → 78.1 ns/op (-11.4%). Other 4 join_* m3f cells stay at noise floor (±2.5% drift, uniform with m4 — measurement- floor noise, not refactor-induced). Refreshes: - bottom INTERP table cell + ratio (1.04× → 1.17× — m3f beats m4 even more decisively now) - opening "before / after" summary table cell + attribution (chunk N+3 + dedup follow-up) JIT lane not re-measured this round — the JIT bench (`bin/daslang -jit dastest/dastest.das ...`) currently fails with "can't delete locked array" in clargs at startup, a pre-existing regression on master that also affects the unmodified baseline (not a refactor-induced break). The JIT cells reflect PR GaijinEntertainment#2913 values; refactor doesn't touch codegen logic so JIT codegen runs on a functionally-identical final AST. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
join_*bench family. Mirror of PR linq_fold: plan_decs_join + LinqJoin auto-typed result lambda (Session 3) #2848 (plan_decs_join) + PR linq_fold: Theme 3 Phase 1 — cross-arm join+group_by splice (C3) #2861 (_join |> _group_bycross-arm) for arrays. Before this PR, every_fold(arr |> _join(...) ...)chain fell through to tier-2 cascade (join_implatdaslib/linq.das:1674) — 3 intermediate allocations, per-pair invoke overhead, m3f 2-3× slower than m4 across the board. Now m3f beats m4 on all 5 cells in both INTERP and JIT.register_array_join_rowspattern +emit_array_joinproduces hashB-collect + srcA-probe inline (preserves the count-no-where bucket-length fast path; mirrors PR linq_fold: plan_decs_join + LinqJoin auto-typed result lambda (Session 3) #2848's D2-A guarantee). 2-source invoke wrap (likeZip); primitive equi-key gate via existingis_primitive_join_key_type; non-primitive keys + decs-bridge srcb cascade silently.ArrayJoinShapestruct +SourceAdapter.ArrayJoinvariant. Extends 4 adapter helpers (adapter_bind_name,adapter_element_type,adapter_wrap_source_loop,adapter_wrap_invoke) + allocation prefix inplan_group_by_core. Addsupstream_joinslot togroup_by_arrayrow +array_join_invariantsrequires predicate.plan_group_by_coreis fully reused —ArrayJoinflows through the adapter abstraction the same wayDecsJoindoes.Bench (full INTERP + JIT rerun, n = 100 000)
join_countjoin_selectjoin_where_countjoin_groupby_countjoin_groupby_to_arrayDrift across the non-join matrix is at measurement floor (±5% INTERP, ±10% JIT). The splice is gated on
array_source+ primitive equi-key + (for cross-arm)upstream_joincapture, so non-join chains aren't touched.Known typer-order subtlety
Bare-
_selectafter_joinon array sources (no trailingto_array()) leavesselCall._type.firstTypeas an unresolvedtypedecl(result_selector(type<TT>))because nothing forces resolution in the chain —select's array overload returnsarray<...>directly.emit_array_joinextracts the resolved type from the select lambda's body viapeel_lambda_single_returninstead of cloningselCall._type.firstType.emit_decs_joindoesn't hit this because its bench chains universally end with|> to_array()(decs returns iterator-typed bridge). Test coverage includes both bare-select and iterator-chain-with-to_array forms.Follow-up (next PR, agreed upfront with the author)
emit_array_joinandemit_decs_joinare ~70% structural duplicates; same for theDecsJoin/ArrayJoinbranches ofadapter_wrap_source_loop. Deferred to a follow-up refactor PR (extractemit_join_collect_probe_shapehelper across all 4 callsites; ~80-100 LOC dedup) — same staging call PR #2848 made: ship the perf-fix at zero risk to the 4 already-closed decs cells, then refactor with both sides bench-locked.Test plan
mcp__daslang__format_file+mcp__daslang__lintclean on every touched.dasmcp__daslang__run_test tests/linq/test_linq_join.das— 52/52 (10 new splice-exercising tests appended: 7 Phase 1, 4 Phase 2, 1 non-primitive-key cascade)mcp__daslang__run_test tests/linq/test_linq_from_decs.das— 198/198 (decs splice not regressed)mcp__daslang__run_test tests/linq/test_linq_fold_terminal_select.das— 28/28mcp__daslang__run_test tests/linq/test_linq_fold_theme3_decs_join_groupby.das— 14/14 (PR linq_fold: Theme 3 Phase 1 — cross-arm join+group_by splice (C3) #2861 surface intact)benchmarks/sql/*.das; all 5join_*m3f cells beat m4mcp__daslang__compile_check benchmarks/sql/join_*.das— all 5 compile under the new splice🤖 Generated with Claude Code