Follow-up (wren-core-py): document + test physical-table visibility through exec_ctx after per-call catalog snapshot

[goldmedal]

Context

#2495 makes each apply_wren_on_ctx call use a private top-level catalog-list snapshot so concurrent same-context transforms can't race on catalog registration. That fix is scoped to wren-core.

As noted in the review discussion, this shifts a wren-core-py consumer guarantee onto an implicit invariant. PySessionContext::register_parquet / register_csv document "Tables registered on base_ctx are visible to exec_ctx via shared catalog." After #2495, exec_ctx holds a private snapshot of the top-level catalog list, so that guarantee survives only because those methods register into base_ctx's default catalog — which exists at apply time and stays Arc-shared (Layer-2 shared internals). A physical table registered into a new top-level catalog after exec_ctx is built would silently not be visible.

This issue tracks the wren-core-py-side follow-up so that guarantee is documented and test-covered, and to carry the original same-context concurrency goal.

Follow-up items

• Docs — clarify register_parquet / register_csv: physical tables register into base_ctx's default catalog, and visibility through exec_ctx relies on that catalog already existing at apply time. State the precondition explicitly: complete top-level physical-catalog registration before starting a transform (mirroring the note now on apply_wren_on_ctx).
• Tests — add wren-core-py integration coverage for physical-table visibility through exec_ctx, including registration performed after exec_ctx has been created (register a parquet/csv, then resolve/query it via exec_ctx).
• call_lock removal — remove the per-context call_lock to enable true same-context concurrency; retain the same-context concurrency regression coverage and add throughput validation.

Related

• Builds on fix(core): prevent catalog races in concurrent same-context transforms #2495 (wren-core catalog-list isolation).

[ttw225]

Thanks for putting together this follow-up and clearly outlining the remaining work!

I’m planning to take ownership of this issue and work through it in a few focused steps:

1. Document and test the physical-table visibility contract.
   • test(core-py): document and pin the physical-table visibility contract #2576
   • fix(core-py): serialize query results with the execution stream schema #2577
2. Fix the request-scoped analyzer state that currently prevents safely removing the lock.
3. Remove call_lock, then validate same-context concurrency and throughput.

I’ll share the related PRs here as they’re ready.

[ttw225]

Step 1 is up, in two PRs:

• test(core-py): document and pin the physical-table visibility contract #2576
  documents the register_parquet/register_csv visibility
  contract and pins it with integration tests (register before/after the
  derived contexts exist, with and without a reload).
• fix(core-py): serialize query results with the execution stream schema #2577
  a bug the new tests uncovered: query() serialized the Arrow IPC
  stream with the logical plan's schema while batches carry physical types,
  so results decoded corrupted whenever MDL-declared types differ from the
  file's (e.g. integer over an int64 Parquet column).
  Fixed by taking the execution stream's schema.

#2576 merges independently of#2577 fix.