Skip to content

[FEATURE] Support UDFs management in Gravitino #6158

Open
@xloya

Description

@xloya

Describe the feature

In enterprise production practice, if the built-in functions of the computing engine cannot meet user needs, users usually customize UDFs(User-Defined-Functions) through relevant interfaces and reference them in the engine. Especially for scenarios that require a lot of reuse, it is very important to effectively manage these custom functions. In Xiaomi's internal practice, there is also a UDF management system that basically meets user needs (limited to Spark/Flink, and there is no good support for AI + Python scenarios).
As a unified Data / AI metadata center, I think Gravitino can also support UDFs management in Data / AI scenarios.
The following are some commercial product practices:

  1. Aliyun: https://help.aliyun.com/zh/flink/user-guide/manage-udfs
  2. UnityCatalog: https://docs.databricks.com/en/udf/unity-catalog.html

Motivation

No response

Describe the solution

No response

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions