You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
refactor!: reset sql-transform to the confit-served path
Start over on the SQLTransform line. What survives is the transform that was
already built on confit -- SpecializedTransform, renamed to SQLTransform,
which is now the whole package.
Removed:
* the DataFusion-semantics native engine (the sql-transform Rust crate:
datafusion/, lookup.rs, value.rs, InferFn) -- the package is pure Python
now and the cargo workspace has one member
* the Python codegen backend (sql_transform/_codegen)
* the batch transform() path, _batch.py, transformer UDFs
* the differential harness and every test built on it (45 files, ~10k lines)
* the legacy backlog: tasks 1-34, all 7 decisions, the archived conformance
and multi-runtime tasks, and every draft except DRAFT-20
Kept: confit untouched, and the fit-and-serve path -- _compose, _rewrite,
_schema, _sql, _state, _transformer_ref.
Composition (`t"... {other}(col) ..."`) could not survive: it read the other
transform's _state_tables/_rewritten_sql and re-fitted its definition through
the deleted batch engine. Repairing it would have meant reimplementing the
feature on the line being reset, so the surface is kept and raises
NotImplementedError by name, with tests asserting the refusal.
Also repointed what would otherwise dangle: the bench scripts' engine
registries and debug-build guard (now confit's), _state.py's docstring, and
the two native-guard docstrings that cited the now-deleted TASK-33.
Gate: pytest 311 passed + 1 xfailed (confit 250+1 unchanged; sql-transform
597 -> 60), cargo 168, pre-commit clean.
Define ML feature transforms as SQL, fit once, then run them at batch or
4
-
low-latency single-row speed.
3
+
Define ML feature transforms as SQL, fit once, then serve them row-at-a-time
4
+
with sub-microsecond latency.
5
5
6
6
## Packages
7
7
@@ -10,7 +10,11 @@ This repository is a workspace of two packages:
10
10
| package | what it is |
11
11
|---|---|
12
12
|[`packages/confit`](packages/confit)|**Confit** — the serving engine. SQL plus static tables frozen at fit time are partially evaluated, once, into a native function. Serves bit-exact with DuckDB or refuses at build time. Usable on its own. |
13
-
|[`packages/sql-transform`](packages/sql-transform)| The authoring layer: `SQLTransform`, the DataFusion-differentiated engine, and the codegen backend. Depends on Confit for specialized serving. |
13
+
|[`packages/sql-transform`](packages/sql-transform)| The authoring layer: `SQLTransform` — fit window-aggregate state from training data, then serve through Confit. |
14
+
15
+
> **sql-transform was reset.** The DataFusion-differentiated native engine, the
16
+
> Python codegen backend, and the batch `transform()` path were removed; the
17
+
> package is being rebuilt on Confit. Confit itself is unaffected.
0 commit comments