Skip to content

Specializer raw-dict output mode — output='dict' opt-in, python_dict gap closed to 0.74-1.11x (TASK-51) - #37

Merged
ahrzb merged 1 commit into
masterfrom
claude/specializer-raw-dict-output
Jul 26, 2026
Merged

Specializer raw-dict output mode — output='dict' opt-in, python_dict gap closed to 0.74-1.11x (TASK-51)#37
ahrzb merged 1 commit into
masterfrom
claude/specializer-raw-dict-output

Conversation

@claude-agent-ahrzb

Copy link
Copy Markdown

Raw-dict output mode — output="dict" opt-in closes the python_dict gap (TASK-51)

The serving report's standing caveat was that a handcrafted server returning plain dicts beat the specializer's typed output by 1.3–2×, because pydantic model construction dominated our cost. This ships the opt-in lever:

fn = DuckDBInferFn(sql, row_tables=..., static_tables=...)              # typed models (unchanged default)
fn = DuckDBInferFn(sql, row_tables=..., static_tables=..., output="dict")  # plain dicts

Design

  • The generated marshaller already builds the exact per-row dict the model would consume — dict mode returns it and skips construction, so the modes agree field-for-field by construction. The serving parity gate now enforces dict == typed before any timing.
  • Strictly opt-in: the typed default is untouched; output="dict" is mutually exclusive with a supplied output_model; unknown values reject; an output getter exposes the mode.
  • All four emit paths honor it: generated marshaller, generic env-pinned baseline, the reentrancy fallback, and the constant engine (which copies per call so caller mutation can't leak into the next response).

Measured (n=1024 p50, 4 Kaggle scenarios)

  • 25–35% off the typed path.
  • vs the handcrafted python_dict floor: 0.74–1.11×spec_dict outright wins house_prices and sits within ~35% elsewhere, down from the 1.3–2× caveat. The remaining gap is input marshalling, not output.

Gate green (759 passed + 13 xfail); 6 new tests + the parity-gate extension + a spec_dict bench engine row. TASK-50's Done-flip rides this branch.

🤖 Generated with Claude Code

… the python_dict gap (TASK-51)

The marshaller already builds the per-row dict the model would consume;
dict mode returns it and skips model construction. Typed default is
untouched; mutually exclusive with output_model; all four emit paths
honor it (marshaller / generic / reentrant / constant, the latter
copying per call). Parity gate extended: dict == typed field-for-field
before any timing. Bench gains the spec_dict row: 25-35% off the typed
path at n=1024, and the handcrafted python_dict floor shrinks from the
standing 1.3-2x caveat to 0.74-1.11x — spec_dict WINS house_prices;
what remains is input marshalling, not output.

TASK-50 Done-flip and TASK-51 ride this branch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ahrzb
ahrzb merged commit 13a3a7e into master Jul 26, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant