Skip to content

[Enh]: DataFrame.with_backend #2193

Description

@matthewwardrop

We would like to learn about your use case. For example, if this feature is needed to adopt Narwhals in an open source project, could you please enter the link to it below?

No response

Please describe the purpose of the new feature or describe the problem to solve.

Hi @MarcoGorelli ,

I've been enjoying narwhals in formulaic and in some internal inference related tooling at Netflix... but one thing I keep coming up against is misalignment in the namespaces of multiple narwhals objects (e.g. when we need to do a merge of some standard dataframe against a user supplied dataframe that may or may not be backed by the same kind of namespace; or when we do feature generation the output type might not match the target type). I currently work around this by explicitly doing the requisite .from_native(.from_native(...).to_<namespace>())... but this becomes cumbersome; and so I've started writing abstract helper functions that do this alignment for me.

Suggest a solution if possible.

I was wondering if you would be open to patches that do these transformations upstream. Something like: [DataFrame | Series].convert_namespace(target: str | narwhals.Implementation). Obviously this isn't possible for all pairs (e.g. pandas to duckdb), but it would also save some annoying workarounds.

If so, perhaps we could also have an argument to join (e.g. auto_align_namespace) that automatically does the conversions for you.

There are lots of places where you probably want to handle conversions manually... but in my experience, and even greater set of circumstances where you really just want it to work.

If you have tried alternatives, please describe them below.

Manually writing a whole bunch of helper functions.

Additional information that may help us understand your needs.

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions