Skip to content

Empty DataFrame with Nothing column creates String column when written to Arrow #1812

@Jolanrensen

Description

@Jolanrensen

As follow up for #1807

Empty DataFrame with Nothing column creates String column when written to Arrow/Feather (probably others too):

val original = dataFrameOf("emptyCol" to DataColumn.empty())

val featherBytes = original.saveArrowFeatherToByteArray()
val fromFeather = DataFrame.readArrowFeather(featherBytes)
fromFeather shouldBe original // fails, type is String instead of Nothing

This is likely because Nothing matches isSubtypeOf(typeOf<String?>()) in KType.toArrowField()

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingfilesreading/writing from/to files

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions