[FLINK-38996][transform] Enhance error messages for projection and filtering expressions#4243
Merged
Merged
Conversation
…ltering expressions
Contributor
|
Hi @Hisoka-X, you may need to rebase master to fix confilct. |
# Conflicts: # flink-cdc-composer/src/test/java/org/apache/flink/cdc/composer/flink/FlinkPipelineTransformITCase.java # flink-cdc-runtime/src/test/java/org/apache/flink/cdc/runtime/operators/transform/PostTransformOperatorTest.java
Member
Author
done |
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request enhances error messages for transformation expressions in Flink CDC by displaying the original user-provided SQL expressions alongside the compiled internal expressions. This makes it significantly easier for users to debug transformation errors by showing them the exact expressions they wrote rather than the compiled Janino code.
Changes:
- Added
originalExpressionfield toTransformExpressionKeyto store user-provided expressions for error reporting - Updated error messages in
ProjectionColumnProcessorandTransformFilterProcessorto display both original and compiled expressions - Added
getExpression()method toProjectionColumnto expose the original expression - Updated all tests and test fixtures to match the new error message formats
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| TransformExpressionKey.java | Added nullable originalExpression field and updated equals/hashCode/toString to include it for improved error reporting |
| ProjectionColumn.java | Added getExpression() getter method to expose the original user expression |
| ProjectionColumnProcessor.java | Updated error messages to show original expression with compiled expression and column name mapping on separate lines |
| TransformFilterProcessor.java | Updated error messages to show both original and compiled expressions with clearer formatting |
| TransformExpressionCompiler.java | Enhanced compilation error messages to include both original and compiled expressions |
| PostTransformOperatorTest.java | Added assertions to validate the new error message format includes both expressions |
| FlinkPipelineTransformITCase.java | Updated test expectations to match new error message formats |
| casting.yaml | Updated expected error message to show original SQL expression instead of compiled code |
| logical.yaml | Updated expected error message to show original SQL expression instead of compiled code |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
|
Hi @yuxiqian, maybe you can help to review this. |
yuxiqian
reviewed
Jan 30, 2026
Member
yuxiqian
left a comment
yuxiqian
left a comment
There was a problem hiding this comment.
Looks good. Thanks Hisoka for the improvement!
yuxiqian
approved these changes
Jan 30, 2026
ThorneANN
pushed a commit
to ThorneANN/flink-cdc
that referenced
this pull request
Mar 19, 2026
…ltering expressions (apache#4243)
Mrart
pushed a commit
to Mrart/flink-cdc
that referenced
this pull request
Mar 26, 2026
…ltering expressions (apache#4243)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This close https://issues.apache.org/jira/browse/FLINK-38996
This pull request improves error reporting and debugging for transformation expressions in the Flink CDC runtime.
Currently, when a transformation used in
transformencounters an error, we don't include the user-provided expression in the error message, making it difficult for users to directly determine the source of the error.Before:
After: