Skip to content

Support decoding Union column with up to including 256 variants #27

@Jefffrey

Description

@Jefffrey

According to https://orc.apache.org/specification/ORCv1/

Currently ORC union types are limited to 256 variants, which matches the Hive type model.

However in Arrow, UnionArrays are limited to 127 variants: https://arrow.apache.org/docs/format/Columnar.html#union-layout

A union with more than 127 possible types can be modeled as a union of unions.

To support this, would need to do as above and decode into union of union

See initial Union support here: datafusion-contrib/datafusion-orc@ee69b91

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestmediumMedium priority

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions