TASK-35: build the callout struct in feature_names_in_ order

[claude-agent-ahrzb]

Makes the call-order vs fitted-order desync bug class unrepresentable. AmirHossein's insight, measured before filing.

The problem — we scramble, then unscramble twice, in two languages

1. _transformer_ref.py::_named_struct   BUILDS the struct from call.expressions (USER's SQL order)
2. _transformer_udf.py                   reorders back to feature_names_in_        (Python)
3. src/expr.rs                           reorders back AGAIN                        (Rust)

We control step 1, so we chose an order and then spent two independent implementations repairing the mismatch we created. in_schema was derived from the call order too, so the struct and the UDF's Exact struct signature could disagree whenever call order ≠ fitted order.

Three of the four blocking bugs found across TASK-3's review rounds were exactly that desync, and twice a fix for one instance introduced the next. That's a design problem, not a coding one.

The change

# before: struct in call order, in_schema in call order
in_schema, out_schema, y = _probe(obj, cols, table)
call.set("expressions", [_named_struct(call.expressions)])

# after: struct AND in_schema in feature_names_in_ order — one order everywhere
in_schema, out_schema, y = _probe(obj, feat, table)
by_name = {a.name: a for a in call.expressions}
call.set("expressions", [_named_struct([by_name[f] for f in feat])])

With one order used everywhere — struct, in_schema, and what the engines feed sklearn — there is no second order to disagree with. The class stops being fixed-but-fragile and becomes unrepresentable.

Scope caveat (AC#3) — the engine-side reorders are RETAINED

Steps 2 and 3 go no-op for the {sc}(...) authoring path but stay, each with a comment saying why: hand-authored SQL can still supply a named_struct in any order —

"SELECT __tfm_0__(named_struct('income', income, 'age', age)) ..."   # test_diff_transformer_callout.py:70

— so they remain its defense. The win is the authoring path no longer manufacturing mismatches for them to repair, not that the repair disappears. test_struct_field_order_independence_parity covers the hand-authored out-of-order case against both engines.

Not a semantic change (AC#6)

The reorder is at build time, so {sc}(income, age) still equals {sc}(age, income). Whether it should — sklearn refuses a reordered DataFrame — is a separate, deliberately-unresolved question, out of scope here and untouched.

Acceptance criteria

AC	How	Test
#1 struct in fitted order	_named_struct([by_name[f] for f in feat])	test_authored_struct_is_built_in_fitted_order_not_call_order (new)
#2 in_schema from same order	_probe(obj, feat, table)	same
#3 engine reorders retained + commented	_transformer_udf.py, src/expr.rs unchanged, comments present	test_struct_field_order_independence_parity
#4 hand-authored out-of-order works	—	test_struct_field_order_independence_parity (both engines)
#5 parity green both engines	—	full suite
#6 call orders agree	—	test_call_order_does_not_change_the_result (new)

Verification

• Suite 574 passed, 12 xfailed (572 baseline + 2 new tests). No src/*.rs change — the retained Rust reorder was already correct.
• AC#1 test mutation-checked: revert to call-order struct → it fails ('income' != 'age').
• The behavior was measured identical to baseline before this ticket was even filed.

🤖 Generated with Claude Code

[claude-agent-ahrzb]

Closing as obsolete: the SQLTransform line this targets has been reset (#54). The DataFusion native engine, the codegen backend, and the batch transform() path are removed; what remains is the confit-served fit-and-serve path.

Nothing here is lost — the branch and its commits stay reachable if this work is ever revived.