Skip to content

Create a method to return a spark df for pyspark backend #10673

Open
@sonic-x11

Description

Is your feature request related to a problem?

No response

What is the motivation behind your request?

If we return a spark df the user can write to various formats as spark supports in my case it was hudi

Describe the solution you'd like

Create a simple to_spark or similar method which returns a spark df

currently there is hacky workaround pointed out by @gforsyth

#10671 (comment)

What version of ibis are you running?

9.5

What backend(s) are you using, if any?

Pyspark and duckdb

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Assignees

No one assigned

    Labels

    featureFeatures or general enhancements

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions