fix(io): format dataframe-level checks in to_script - #2418
Open
chuenchen309 wants to merge 1 commit into
Open
Conversation
to_script(minimal=False) interpolated the raw statistics list for
dataframe-level checks straight into the script template, while the
column path 19 lines above formats its checks with _format_checks.
The generated script runs and looks correct, but the schema it builds
carries plain dicts where Check objects belong. Every validate() call on
it then fails with TypeError("'dict' object is not callable"), including
on data the original schema accepts, so the round-tripped schema is
unusable rather than merely wrong.
Format dataframe-level checks the same way the column path does.
_format_checks(None) already returns "None", so schemas without
dataframe-level checks are unaffected.
The existing test_to_script only covers column checks, so this path was
untested; add a round-trip test with a dataframe-level check.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com>
chuenchen309
force-pushed
the
fix/to-script-dataframe-level-checks
branch
from
July 16, 2026 12:52
0af3e08 to
8056883
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
to_script(minimal=False)produces a script that runs fine and looks correct, but the schema it rebuilds is unusable — everyvalidate()call on it raisesTypeError: 'dict' object is not callable, including on data the original schema accepts:Column-level checks round-trip correctly, so the two paths disagree:
The failure is quiet at the point of corruption: the generated script is valid Python and the
DataFrameSchema(...)call succeeds. It only surfaces later, as a confusingTypeErrorfrom inside validation that gives no hint thatto_scriptwas the culprit.Cause
In
to_scriptthe column path formats its checks, but the dataframe-level path passes the raw statistics list straight into the template — 19 lines apart in the same function:Every other caller of the template data (
_format_index,_format_column_minimal, the column path) goes through_format_checks; this one line is the exception.Fix
Format dataframe-level checks the same way the column path does.
_format_checks(None)already returns"None", so schemas without dataframe-level checks are unaffected.After the fix,
rt.checksis[<Check greater_than: greater_than(0)>],schema == rtisTrue, and the round-tripped schema accepts and rejects exactly what the original does.Testing
test_to_script_dataframe_level_checkstotests/io/test_pandas_io.py. The existingtest_to_scriptasserts the right invariant (schema == schema_to_write) but its_create_schema()fixture defines only column checks, so this path had no coverage. Verified the new test fails before the fix and passes after.tests/io/→ 73 passed. The 5test_frictionless_*failures are pre-existing: they fail identically on an unmodifiedmainin my environment becausefrictionlessisn't installed, and are unrelated to this change.tests/pandas/test_schema_statistics.py+tests/pandas/test_schema_inference.py→ 80 passed, 1 skipped (to_scriptgoes throughget_dataframe_schema_statistics).ruff check(the linter CI runs) andruff format --checkclean on both changed files.Note on #2402
Heads-up that #2402 moves
to_script/SCRIPT_TEMPLATE/_format_checksintopandera/io/common_io.pyand carrieschecks=statistics["checks"]across unchanged, so the bug would survive that refactor. Happy to rebase onto it, or to re-send againstcommon_io.pyinstead if you'd rather land #2402 first — whichever is less work for you.AI disclosure
Prepared with AI assistance (Claude Code). I reproduced the bug against the unmodified library, verified the fix and the test locally, and take responsibility for the change.