feat(engine): introduce planned query execution

[azimafroozeh]

What & why

This PR introduces planned query execution for engine v2. It lays the foundation for query optimization, pipelining, vectorized execution and scheduling.

Engine v2 resolves the four issues below. Enable it with set engine = v2; or --set engine=v2; v1 remains the default and does not receive these fixes.

Issue	Planning or execution problem	Fix in v2
#704	A bare-node count requests full rows, including unused vector columns.	Aggregate pushdown replaces eligible unfiltered counts with a metadata count, avoiding the row scan.
#703	Traversal reads full destination rows and duplicates unused properties across edges.	Projection pushdown limits destination scans to required columns before fan-out. Single-hop traversal emits bounded batches.
#723	Traversal materializes the full fan-out before grouped aggregation can consume it.	Bound-edge traversal and hydration run in batches, with spillable sorting feeding DataFusion aggregation.
#750	A search predicate on a traversal destination is lost during execution.	The plan represents the destination read as an explicit scan carrying its predicates. The scan applies search membership together with scalar filters.

Local benchmark rerun: release build, Apple M5 Max, four Tokio workers, 20,000 nodes with 1,024-dimensional vectors and 100,000 edges. The issue_shapes benchmark used a fresh fixture and four fresh processes per engine/query pair, with three warmups and eleven timed executions per process. Engine order alternated between repeats. Complete results were checked against the fixture before timing and after every query. The store used local file:// storage with an uncontrolled OS page cache; these are preliminary diagnostic measurements. The source fingerprint remained unchanged through compilation and measurement. The benchmark records the v2 query plans alongside the correctness checks.

Query	v1 median (ms)	v2 median (ms)	v2 versus v1
Bare-node count	5.684833	0.287125	19.8× faster
Destination projection	104.763584	31.800083	3.3× faster
Grouped fan-out	75.278000	18.043291	4.2× faster
Destination search	Wrong result	29.551208	Correct now

Latency uses the lower middle value of the four process medians. Peak RSS is measured with ru_maxrss and includes store opening, warmups and complete-result checks.

Query	v1 peak RSS (MiB)	v2 peak RSS (MiB)	v2 versus v1
Bare-node count	299.265625	46.156250	6.5× lower
Destination projection	2094.484375	299.515625	7.0× lower
Grouped fan-out	1362.312500	71.859375	19.0× lower
Destination search	Wrong result	173.984375	Correct now

Backing issue / RFC

• Implements an accepted RFC (RFC 0068, query engine v2 planner; acceptance and the canonical RFC link need confirmation).
• Closes performance: count($var) on a bare variable fails open to a full-width scan, materializing Vector columns #704
• Closes performance: Expand hydrates full destination rows and copies them per edge — OOM / Arrow 2 GB offset overflow on wide destinations #703
• Closes bug: grouped aggregation is O(input rows) in memory, runs post-materialisation and ignores limit #723
• Closes A search predicate or ranking on a traversal-introduced target is dropped #750

Checklist

• Change is focused (planned read execution for engine v2).
• Tests added/updated for behavior changes (GQT plan and result assertions, both-engine corpus coverage, resource limits and producer lifecycle tests).
• Public docs updated if user-facing surface changed (docs/user/queries/explain.md, engine settings and search behavior).
• Reviewed against docs/dev/invariants.md: no Hard Invariant weakened, no deny-list item hit (final review pending).

Local verification

• Planner: 56 tests passed, including regression tests confirmed failing before the fix.
• Engine v2: 11 integration tests, 14 memory tests, 22 API guards and the nested-field statistics test passed.
• V2 GQT corpus: 135 cases passed with four test workers.
• Workspace Clippy, formatting, documentation and comment checks passed.
• Release benchmark: 28 fresh child runs and 308 timed results passed complete fixture-derived checks.
• The full workspace suite passed before the final estimation change; the affected tests above were rerun after it.

Notes for reviewers

• Scheduling here covers operator streams and bounded producers. An explicit pipeline dependency graph, parallel pipeline scheduler and session fairness remain future work.
• Memory estimates are heuristic. A hash-build memory refusal retries through ID lookup before probing; failures after probing propagate. Query budgets do not bound shared caches or total process memory.