[FLINK-37586][udf] Add support for options in user-defined functions and update related documentation

[Hisoka-X]

This close https://issues.apache.org/jira/browse/FLINK-37586

This pull request adds support for passing configuration options to user-defined functions (UDFs) in pipeline definitions. It updates both the YAML schema and the internal Java representations to allow specifying an options block for each UDF, making these options available at runtime. The documentation is updated to explain how to use this feature, and new tests are added to ensure correct parsing and handling.

[Hisoka-X]

cc @yuxiqian

[yuxiqian]

LGTM overall.

+@lvyanquan as it is related to public API change.

[Copilot]

Pull request overview

This PR adds support for passing configuration options to user-defined functions (UDFs) in Flink CDC pipeline definitions. Previously, UDFs could be registered but had no way to receive external configuration at startup.

Changes:

• Extended UdfDef class with an options field, new constructor, and updated equals/hashCode/toString
• Updated the YAML parser to read the new options block for UDFs, and fixed TransformTranslator to pass these options through (previously it always passed an empty HashMap)
• Added example UDF implementations (Java and Scala), new test YAML fixture and unit/integration tests, and documentation in both English and Chinese

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
flink-cdc-composer/src/main/java/.../definition/UdfDef.java	Adds options map field with constructor overloads and updated equals/hashCode/toString
flink-cdc-composer/src/main/java/.../translator/TransformTranslator.java	Passes actual UDF options instead of an empty HashMap
flink-cdc-cli/src/main/java/.../parser/YamlPipelineDefinitionParser.java	Parses optional options block from YAML UDF definitions
flink-cdc-cli/src/test/java/.../YamlPipelineDefinitionParserTest.java	Adds test for parsing UDF definitions with options
flink-cdc-cli/src/test/resources/definitions/pipeline-definition-with-udf-options.yaml	Test YAML fixture with options block for a UDF
flink-cdc-composer/src/test/java/.../FlinkPipelineUdfITCase.java	Integration test for configurable UDFs using options
flink-cdc-pipeline-udf-examples/.../java/ConfigurableFunctionClass.java	Java example UDF that reads options via ConfigOptions
flink-cdc-pipeline-udf-examples/.../scala/ConfigurableFunctionClass.scala	Scala example UDF that reads options via ConfigOptions
flink-cdc-e2e-tests/.../rules/unexpected.yaml	Updates the allowed-keys error message to include options
docs/content/docs/core-concept/transform.md	English documentation for the new options feature
docs/content.zh/docs/core-concept/transform.md	Chinese documentation for the new options feature

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[lvyanquan]

LGTM.