Delta-rs Write with Ray DataSet

[raviranak]

Description

Currently delta-rs write takes pyarrow.Table, BatchRecord and pandas . ray dataset is a distributed dataset with underlying record is ObjectRef of pa.Table

Use Case
Ability to Write from ray dataset directly to delta lake

I have added my changes in fork repo
https://github.com/raviranak/delta-rs

[roeap]

Hi @raviranak - we would be happy to review a PR, since you seem to have a working solution already :)

[raviranak]

Hi @roeap , PR link #1679

[MrPowers]

Do you think this should go in the deltaray project? https://github.com/delta-incubator/deltaray

[ion-elgreco]

This should live upstream indeed