[FLINK-38413] Adding support for append only tables to MySQL Pipeline connector #4137
Conversation
|
@lvyanquan Can I get a review on the above PR? Thanks. |
|
Please create a jira issue for this. |
|
|
||
| @Experimental | ||
| public static final ConfigOption<Boolean> SCAN_READ_CHANGELOG_AS_APPEND_ONLY_ENABLED = | ||
| ConfigOptions.key("scan.read-changelog-as-append-only.enabled") |
There was a problem hiding this comment.
I think this change would affect too many tables. Could we set a regular expression (e.g., scan.read-changelog-as-append-only.tables: adb.*) to modify the primary key only for tables that match this pattern?
There was a problem hiding this comment.
That's a good suggestion, I will try incorporating it.
| LinkedList<SchemaChangeEvent> changes, | ||
| boolean tinyInt1isBit, | ||
| boolean isTableIdCaseInsensitive) { | ||
| this(parser, listeners, changes, tinyInt1isBit, isTableIdCaseInsensitive, false); |
There was a problem hiding this comment.
Perhaps we can make this change in the MySqlPipelineRecordEmitter instead of modifying the CustomAlterTableParserListener(since this logic is not related to parsing).
There was a problem hiding this comment.
Sure, I can do that. I was curious around the schema evolution side of things if there are any workflows from where primary key might get added accidentally.
There is already a ticket created mentioned in the description. FLINK-38413 for reference. |
lvyanquan
changed the title
2 participants
Description
FLINK-38413: Enables append only table creation from mysql pipeline connector. I have an added an integration test as well that creates out an Append only Paimon table from MySQL.