Remove transformer support from SQLTransform and the native interpreter — TASK-40

[claude-agent-ahrzb]

Removes the {transformer}(col) authoring surface and the opaque callout machinery behind it, from both the Python front-end and the native Rust interpreter. Closes TASK-40.

v0, so this goes out clean — no deprecation shim, no compat path.

−1,464 lines net. The Rust diff adds exactly two lines (a trimmed pyo3 signature and a narrowed import); everything else is deletion.

What's gone

Python surface

• Deleted _transformer_ref.py (is_transformer, _bind_names, _probe, resolve_transformer_refs) and _transformer_udf.py (the DataFusion ScalarUDF wrapper).
• _compose.py — dropped Ref.is_transformer, the is_transformer branch, and the transformer-shaped-but-unfitted guard.
• __init__.py — dropped _udf_specs, the sqlt/tfm ref split, and the transformers= kwarg to InferFn.
• _batch.py — dropped the UDF registration loop.
• _codegen/engine.py — dropped the __tfm_ UnsupportedInCodegen branch; an unknown function is now uniformly a ValueError.

Native interpreter

• expr.rs — removed the Expr::Transform variant and its eval arm (GIL attach, feature reorder, .transform(), .tolist() marshalling).
• lib.rs — removed ResolvedTransformer, read_feature_names_in, the resolve_transformers tree rewrite, and the transformers constructor param.
• types.rs / plan.rs — removed the Expr::Transform arms from infer_type and validate_expr.
• schema.rs — removed arrow_schema_to_ordered_fields; its only caller was the transformer path.

Five test files deleted, plus the transformer cases in _compose_test.py and test_diff_errors.py.

What stays

SQLTransform-to-SQLTransform composition is untouched. That is a separate feature sharing the same t-string entry point, and it keeps working — frozen {a.transform} refs, bare {a} re-fit refs, and nesting. The README section that documented sklearn refs now documents composition instead, with an example I ran end to end.

A fitted sklearn transformer now falls through to the pre-existing TypeError, which already names the offending type:

interpolation {sc} must be a SQLTransform or its .transform, got StandardScaler

I deleted the dedicated not-fitted guard rather than repointing it — its message told users to call .fit(), which no longer helps.

Verification

• cargo check — clean, zero warnings.
• uv run pytest — 523 passed, 12 xfailed. The xfails are pre-existing and unrelated.
• ruff check + ruff format --check — clean; pre-commit hooks passed.
• Ran the rewritten README example: transform() → [1.0, 1.0, 1.0, 1.0], infer() → m=1.0 (so the native path is genuinely exercised), a bare {t} re-fit ref → [0.4, 0.8, 1.2, 1.6], and a fitted StandardScaler rejected with the expected TypeError.

One thing I deliberately left alone: cargo fmt --check reports 11 diffs, all pre-existing (master has 13, in the same files). cargo fmt is not part of mise run fmt, so the crate was never fmt-clean. Reformatting here would bury the deletion in noise.

Ticket fallout — worth a look

Both superseded tickets carried standing pre-authorizations to auto-dispatch devs, which is the part that actually mattered: left open, they would send someone at deleted code.

• TASK-3 → closed Done. Its five completed fixes are already on master; ACs feat(codegen): Substrait front-end for the codegen serving engine #1–ci: on-demand PR review via /review comment #6 are void, not deferred, since they describe the removed surface. Retires the dispatch-to-Wren authorization.
• TASK-34 → archived (recoverable, not deleted). It exists to port this feature to codegen, so its premise is gone. Retires the auto-dispatch-to-Ritchie-when-TASK-29-lands authorization.
• TASK-31 → stays Done, with a note; its test file went with the feature. CASE parity itself is still covered by test_diff_case.py and test_case_window_integration.py.

Design specs under docs/superpowers/specs and the backlog docs/decisions/drafts are left as history.

🤖 Generated with Claude Code

[claude-agent-ahrzb]

Closing as obsolete: the SQLTransform line this targets has been reset (#54). The DataFusion native engine, the codegen backend, and the batch transform() path are removed; what remains is the confit-served fit-and-serve path.

Nothing here is lost — the branch and its commits stay reachable if this work is ever revived.