-
Notifications
You must be signed in to change notification settings - Fork 948
Remove the file name from the output in cudf-polars' explain APIs #18752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove the file name from the output in cudf-polars' explain APIs #18752
Conversation
Thanks for looking into this @Matt711 ! Unfortunately, this doesn't really address the "problem" I am seeing with large multi-file datasets. Even if we print the name of the file, I still see patterns like this:
|
My weakly-held opinion is that we get enough information from @_repr_ir.register
def _(ir: Scan, *, offset: str = "") -> str:
label = f"SCAN {ir.typ.upper()}"
return _repr_header(offset, label, ir.schema) This way, we end up with patterns like:
|
Thanks for print out! I see, yeah I'llremove the file name and suffix. |
Yeah I think for the purposes of pdsh, the schema will be enough to distinguish. Maybe I can follow-up and add a |
Yeah - I agree we may end up with applications that benefit from the path information, but you're right that we don't need this to understand PDSH yet. I have a feeling we can capture this information without any optional arguments, but I'm not entirely sure yet (maybe we go back to special handling for |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You probably need to adjust test_explain_logical_plan_wide_table_with_scan
for CI to pass, but LGTM once CI is green. Thanks!
Thanks! |
…clude-first-path-only
…clude-first-path-only
/merge |
Description
Follow up to #18708 that addresses this comment.
Checklist