Skip to content

Support for data decoding functions #820

@vbarua

Description

@vbarua

During discussion around disallowing the any type in function return values (see PR), the example of

  - name: from_avro
    description: FromAvro
    impls:
      - args:
          - value: binary?
            name: input
          - value: string
            name: avro_schema
        return: any?

came up as an example of when it might be wanted.

In this function, the type of the output any? is derived based on the avro_schema string. Being able to decode binary input like this is a valid usecase, however, we should provide a mechanism to allow this that doesn't rely on an any return type.

In this case, especially, the fact that plan producer can provide the avro_schema means that it has enough information to be able to provide an concrete Substrait type for the output.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions