test(core-py): document and pin the physical-table visibility contract#2576
Open
ttw225 wants to merge 2 commits into
Open
test(core-py): document and pin the physical-table visibility contract#2576ttw225 wants to merge 2 commits into
ttw225 wants to merge 2 commits into
Conversation
ttw225
marked this pull request as ready for review
July 24, 2026 06:59
Contributor
WalkthroughSessionContext documentation and examples now define physical table registration and MDL loading behavior. New PyArrow-based tests verify catalog visibility, direct querying, dry runs, and semantic resolution for Parquet and CSV tables. ChangesPhysical table visibility
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#2495 made
apply_wren_on_ctxsnapshot top-level catalog membership perderivation.
register_parquet/register_csvwrite into the pre-existingdefault catalog (
datafusion.public), whose internals are live-sharedwith derived contexts — so tables registered after a session context was
created are still visible through
query/dry_run/list_tables. TheDataFusionConnectorconsumer relies on exactly this, but until now thebehavior was implicit: no documentation stated it and no test guarded it.
#2504 asks for both.
What
register_parquet/register_csv(where tables land, why late registration is visible, and that brand-new
top-level catalogs must exist before MDL construction,
load_mdl, or atransform), plus
load_mdl's concurrency contract (its exclusive borrowrejects overlapping calls with
RuntimeError).tableReference↔ registered-name mapping requirement, and fix the QuickStart sample (it passed a manifest object and a non-existent
remote_functionskwarg to the constructor).tests/test_physical_tables.py: register-before-load_mdl,register-after-derivation-without-reload (Python twin of the wren-core
catalog_racevisibility test), and register-then-reload.pyarrowjoins the dev group to write Parquet fixtures and decode the Arrow IPC
bytes
query()returns, enabling row-level assertions.No production code changes; no release bump expected.
Note: fixtures declare/store
c_custkeyas int32 on purpose — writing theIPC stream with the logical plan's schema corrupts results when MDL types
differ from physical file types. That serialization issue was found by
these tests and is fixed separately in a follow-up
fix(core-py)PR.Test Plan
just test-py: 38 passed (35 existing + 3 new), on this branch withoutany production change.
just test-rs: 37 passed.ruff check/ruff format --check: clean on the new test file.Part of #2504.
Summary by CodeRabbit
New Features
Documentation
Tests