[FLINK-38906] Pass transform parser in context and polish built-in functions

[yuxiqian]

This closes FLINK-38906...

...by allowing decimal arithmetic expressions and extended previously missing temporal functions.

Also organized TransformParser context into named structs, and added a utility function to infer physical type of calcite nodes.

[Copilot]

Pull request overview

This pull request refactors the TransformParser context handling and extends built-in function support for FLINK-38906. The PR introduces a structured Context class to organize transformation parameters, adds support for decimal arithmetic operations, and extends temporal functions with TO_TIMESTAMP_LTZ, DATE_FORMAT_TZ, and DATE_ADD. It also defers script expression generation from TransformFilter initialization to TransformFilterProcessor construction for better architecture.

Changes:

• Introduced JaninoCompiler.Context class to encapsulate columns, column name mappings, UDF descriptors, and metadata columns
• Added decimal arithmetic handling that generates method invocations for BigDecimal operations
• Extended temporal functions with TO_TIMESTAMP_LTZ, DATE_FORMAT_TZ, DATE_ADD, and added DATE/TIME support to DATE_FORMAT
• Moved script expression generation from TransformFilter to TransformFilterProcessor for proper column access
• Added Byte type support to comparison functions and refactored IN/NOT IN operations

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
JaninoCompiler.java	Introduced Context class, added decimal arithmetic handling, updated all method signatures to use Context
TransformParser.java	Updated to create and pass Context objects, added deduceSubExpressionType utility method
TransformFilter.java	Removed scriptExpression field and moved generation responsibility to processor
TransformFilterProcessor.java	Now generates script expression at construction time with proper column access
TransformExpressionKey.java	Moved loadSystemFunction call from construction to getExpression() method
ProjectionColumnProcessor.java	Removed redundant loadSystemFunction call
PreTransformOperator.java	Updated TransformFilter.of() call to not pass udfDescriptors
PostTransformOperator.java	Updated TransformFilter.of() call to not pass udfDescriptors
TransformSqlOperatorTable.java	Added TO_TIMESTAMP_LTZ, DATE_FORMAT_TZ, DATE_ADD functions and extended DATE_FORMAT operand types
ComparisonFunctions.java	Added Byte support for BETWEEN/IN operations, simplified IN/NOT IN implementations
TransformParserTest.java	Updated test calls to include new Context parameters
FlinkPipelineTransformITCase.java	Updated error message assertions to include LOAD_MODULES_EXPRESSION prefix
temporal.yaml	Removed ignore flags for tests now passing with new functionality
decimal.yaml	Removed ignore flags for tests now passing with decimal arithmetic support
comparison.yaml	Removed ignore flags for tests now passing with Byte support
meta.yaml	Removed unrelated test case for FIELD_NAME_LOWER_CASE converter
transform.md (English)	Added documentation for new temporal functions with formatting issues
transform.md (Chinese)	Added documentation for new temporal functions

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yuxiqian]

Hi @Hisoka-X, I've optimized transform error message style by trimming "import ..." prefix as it's not useful for debugging, and added a utility function to deduce physical type of Calcite SqlNodes (might be useful in #4241). Looking forward to your comments :)

[Hisoka-X]

Thanks for ping!

[Hisoka-X]

LGTM

[lvyanquan]

+1.