[FLINK-36165][source-connector/mysql] Support capturing snapshot data with conditions by uicosp · Pull Request #3776 · apache/flink-cdc

uicosp · 2024-12-04T03:56:41Z

Hey, this is an implementation designed to capture snapshot data with filtering conditions.

For example, by specifying scan.snapshot.filters: db.user_table:id > 200;, we can synchronize only the user data where the id is greater than 200.

issue link: https://issues.apache.org/jira/browse/FLINK-36165

… with conditions

…ySQL table source

yuxiqian · 2024-12-06T09:42:08Z

Thanks for @uicosp's great work! It's indeed a long awaited feature.

Seems Debezium has a similar option called snapshot.select.statement.overrides, which allows users to project out unwanted columns and filter rows based on custom predicates.

As these options aren't available in incremental framework, it would be nice if we could support both row and column level pushdown with similar syntax, since they're both related to tweaking snapshot querying SQL. WDYT?

ThorneANN · 2024-12-12T05:23:09Z

Thanks for @uicosp's great work! It's indeed a long awaited feature.

Seems Debezium has a similar option called snapshot.select.statement.overrides, which allows users to project out unwanted columns and filter rows based on custom predicates.

As these options aren't available in incremental framework, it would be nice if we could support both row and column level pushdown with similar syntax, since they're both related to tweaking snapshot querying SQL. WDYT?

it is also support table api and datastream api with incremental framework

ThorneANN · 2024-12-12T07:46:05Z

Should we consider this pr with only snapshot startoption and back fill ?

uicosp · 2024-12-12T15:37:08Z

Column pruning is certainly a feature worth considering. In theory, it would only require minor adjustments to the current design, such as implementing a syntax like scan.snapshot.filters: db.user:id>200:id,name,age.

Taking this a step further, since the pipeline config already includes a transform option, we could leverage it to automatically extract filter conditions and projection fields, dynamically generating scan.snapshot.filters and thereby simplifying the config.

That said, I’m currently focused on other tasks and don't have time to work on this. Once I have time, I’d be happy to revisit and enhance this feature.

mageniao2 · 2025-04-15T01:34:03Z

Does this function also apply to oracle.cdc ?

uicosp · 2025-04-15T02:11:54Z

Does this function also apply to oracle.cdc ?

no, this function only works for mysql connector.

mageniao2 · 2025-04-15T02:18:03Z

Does this function also apply to oracle.cdc ?

no, this function only works for mysql connector.

Thanks for the reply, does oracle connector have similar capabilities? I tried to configure debezium.snapshot.select.statement.overrides but it doesn't work.

uicosp · 2025-04-15T02:43:05Z

Does this function also apply to oracle.cdc ?

no, this function only works for mysql connector.

Thanks for the reply, does oracle connector have similar capabilities? I tried to configure debezium.snapshot.select.statement.overrides but it doesn't work.

I haven't used oracle, so I'm not sure. You can try to find the answer through the offical flink cdc docs or the oracle connector source code.

ThorneANN · 2025-04-15T03:47:05Z

debezium.snapshot.select.statement.overrides
debezium.snapshot.select.statement.overrides option only applicable to read data by lock algorithms，this is incremental snapshot for capturing data.
You can completely implement other incremental snapshot connectors methods by following the snapshot filtering of MySQL connector

mageniao2 · 2025-04-15T05:44:01Z

debezium.snapshot.select.statement.overrides
debezium.snapshot.select.statement.overrides option only applicable to read data by lock algorithms，this is incremental snapshot for capturing data.
You can completely implement other incremental snapshot connectors methods by following the snapshot filtering of MySQL connector

Thank you. I will follow your suggestion to attempt implementing a filtering feature based on an incremental snapshot connector

ThorneANN · 2025-04-16T02:01:02Z

@yuxiqian @loserwang1024 PTAL

lvyanquan · 2025-05-19T08:14:15Z

+        if (snapshotFilters != null) {
+            Map<String, String> snapshotFilterMap = new HashMap<>();
+            for (String snapshotFilter : snapshotFilters.split(";")) {
+                String[] splits = snapshotFilter.trim().split(":");


What if the filter in sql contains ; or :, like city != 'China:beijing'?

Sorry I’m late, I was busy with something else. I have modified the snapshotFilters parsing logic in MySqlDataSourceFactory to correctly handle semicolons and colons within filter conditions.

Key improvements:

parseSnapshotFilters method:

Intelligently parses semicolon delimiters while recognizing quoted content

Supports both single and double quotes

Semicolons within quotes are not treated as delimiters

parseTableAndCondition method:

Splits only on the first colon to avoid misinterpreting colons in condition values

Ensures proper separation of table names and conditions

I think a better solution to this problem should allow passing nested YAML structures (LIST and MAP should be enough) as config option values instead of inventing more customized syntaxes, as they're hard to write and parse.

I think a better solution to this problem should allow passing nested YAML structures (LIST and MAP should be enough) as config option values instead of inventing more customized syntaxes, as they're hard to write and parse.

I agree with u , but i think that we should consider the difference with snapshot filters or cdc filters

You are right. Actually, at the beginning I tried using nested YAML structures (LIST and MAP). However, I found that the YamlPipelineDefinitionParser.toSourceDef() method doesn’t support that; it only supports STRING values (because it uses Map<String, String> in Java).

I’m not sure whether this was intentional or simply a limitation of the current implementation. If I could use Map<String, Object>, I could switch to YAML structures, which is clearly more reasonable than parsing everything from strings.

…filter

# Conflicts: # docs/content/docs/connectors/flink-sources/mysql-cdc.md # docs/content/docs/connectors/pipeline-connectors/mysql.md # flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/main/java/org/apache/flink/cdc/connectors/mysql/factory/MySqlDataSourceFactory.java # flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/main/java/org/apache/flink/cdc/connectors/mysql/source/MySqlDataSourceOptions.java # flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/test/java/org/apache/flink/cdc/connectors/mysql/source/MySqlDataSourceFactoryTest.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/assigners/MySqlChunkSplitter.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceConfig.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceConfigFactory.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceOptions.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSource.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSourceFactory.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/source/MySqlSourceITCase.java # flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSourceFactoryTest.java

…pshot filter

uicosp · 2025-09-05T15:59:42Z

This version adds support for configuring scan.snapshot.filters in the pipeline using a YAML list format.
Example:

scan.snapshot.filters:
  - table: db1.user_table_[0-9]+
    filter: id > 100
  - table: db[1-2].[app|web]_order_\.* 
    filter: city != 'China:beijing'

Note:
The parameter type of Configuration.fromMap has been changed from Map<String, String> map to Map<String, ?> map.
@yuxiqian Please confirm that this change does not break existing design conventions.

yuxiqian

It appears that modifying config methods to support nested types should be safe, since all existing connectors will only declare ConfigOptions with primitive types.

Would @lvyanquan like to take a look?

ThorneANN · 2025-12-19T03:04:48Z

@lvyanquan Hi,Can you push this PR

nickdelnano · 2026-01-23T21:54:28Z

This functionality would be really helpful. What is the status of this PR? Can it be merged?

Copilot

Pull request overview

Implements snapshot-row filtering for MySQL CDC snapshot reads, allowing users to restrict which rows are captured during the snapshot phase via configurable SQL WHERE-like predicates.

Changes:

Add snapshot filter configuration for both the Table API MySQL CDC source (scan.snapshot.filter) and the pipeline MySQL connector (scan.snapshot.filters).
Apply snapshot filters throughout snapshot chunk analysis and snapshot split reading (min/max, row counting, split scan query generation).
Add/extend unit + IT coverage and update YAML/examples/docs.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSourceFactoryTest.java	Extends table source factory tests to include the new snapshot filter option in expected source construction.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtilsTest.java	Adds unit coverage for table-pattern-to-filter matching.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/source/MySqlSourceITCase.java	Adds an integration test validating snapshot filtering behavior under failover.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSourceFactory.java	Wires `scan.snapshot.filter` option into `MySqlTableSource` creation and optional options set.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/table/MySqlTableSource.java	Stores and forwards the snapshot filter into the parallel `MySqlSource` builder path.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/StatementUtils.java	Extends snapshot SQL building utilities to incorporate optional filters in min/max, row count, chunk boundary, and split scan queries.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtils.java	Introduces utility to resolve an applicable snapshot filter for a given table based on configured patterns.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceOptions.java	Adds `scan.snapshot.filter` config option for MySQL CDC source connector.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceConfigFactory.java	Adds snapshot filters storage + builder methods and passes them into `MySqlSourceConfig`.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/config/MySqlSourceConfig.java	Stores snapshot filters map in source config and exposes getter.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/assigners/MySqlChunkSplitter.java	Applies snapshot filter to min/max, row counting, and chunk boundary computation.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/MySqlSourceBuilder.java	Adds builder method for setting snapshot filters.
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/debezium/task/MySqlSnapshotSplitReadTask.java	Applies per-table snapshot filter when generating the split scan SQL.
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/test/java/org/apache/flink/cdc/connectors/mysql/source/MySqlDataSourceFactoryTest.java	Adds pipeline connector factory test coverage for snapshot filters parsing/propagation.
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/main/java/org/apache/flink/cdc/connectors/mysql/source/MySqlDataSourceOptions.java	Adds `scan.snapshot.filters` option as a list of `{table, filter}` maps.
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-mysql/src/main/java/org/apache/flink/cdc/connectors/mysql/factory/MySqlDataSourceFactory.java	Parses `scan.snapshot.filters` and forwards into `MySqlSourceConfigFactory`.
flink-cdc-common/src/main/java/org/apache/flink/cdc/common/configuration/Configuration.java	Broadens `fromMap` to accept non-string values for YAML parsing use-cases.
flink-cdc-cli/src/test/resources/definitions/pipeline-definition-full.yaml	Adds example `scan.snapshot.filters` YAML structure to the full pipeline definition fixture.
flink-cdc-cli/src/test/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParserTest.java	Updates parser tests to use non-string typed values and includes snapshot filters list in expected config.
flink-cdc-cli/src/main/java/org/apache/flink/cdc/cli/parser/YamlPipelineDefinitionParser.java	Parses source config as `Map<String,Object>` to support structured/non-string YAML values.
docs/content/docs/connectors/pipeline-connectors/mysql.md	Documents the new pipeline connector snapshot filters option (but currently mismatched vs implementation).
docs/content/docs/connectors/flink-sources/mysql-cdc.md	Documents the new source connector `scan.snapshot.filter` option.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-27T04:29:53Z

                            .assignUnboundedChunkFirst(assignUnboundedChunkFirst)
+                            .snapshotFilters(
+                                    escapeDot(database) + "." + escapeDot(tableName),
+                                    snapshotFilter)
                            .build();


snapshotFilter is only forwarded into .snapshotFilters(...) when enableParallelRead is true. If a user sets scan.snapshot.filter while parallel snapshot is disabled, the option will currently be silently ignored. Consider either (1) validating and failing fast when the option is set but parallel read is disabled, or (2) implementing equivalent filtering for the non-parallel Debezium source path, and documenting the supported modes.

fixed 3d360c9

Copilot · 2026-01-27T04:29:53Z

+        List<Map<String, String>> snapshotFilters = config.get(SCAN_SNAPSHOT_FILTERS);
+        if (snapshotFilters != null && !snapshotFilters.isEmpty()) {
+            Map<String, String> snapshotFiltersMap =
+                    snapshotFilters.stream()
+                            .collect(
+                                    Collectors.toMap(
+                                            it -> it.get(SNAPSHOT_FILTER_TABLE_KEY),
+                                            it -> it.get(SNAPSHOT_FILTER_FILTER_KEY)));
+            LOG.info("Add snapshotFilters {}.", snapshotFiltersMap);
+            configFactory.snapshotFilters(snapshotFiltersMap);
+        }


scan.snapshot.filters is parsed via Collectors.toMap(...) without validating required keys (table/filter) and without handling duplicates. This can throw a generic NullPointerException/IllegalStateException on malformed or duplicate entries and also drops the original list order (which matters if patterns overlap). Consider validating each entry and building a LinkedHashMap with explicit duplicate handling (e.g., reject with a clear message).

fixed 67fae6d

Copilot · 2026-01-27T04:29:55Z

+        String filter = null;
+        for (Selectors selector : snapshotFilters.keySet()) {
+            if (selector.isMatch(
+                    org.apache.flink.cdc.common.event.TableId.tableId(
+                            tableId.catalog(), tableId.table()))) {
+                filter = snapshotFilters.get(selector);
+                break;
+            }


getSnapshotFilter picks the first matching selector from a HashMap keySet iteration, which is non-deterministic when multiple patterns match the same table. This can cause different filters to be applied across runs/JVMs. Please make matching deterministic (e.g., preserve user-defined order via LinkedHashMap/list, or define precedence like exact match > regex, or fail fast on multiple matches).

fixed b7f13fd

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtilsTest.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/StatementUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…ut parallel read is disabled

…kedHashMap with explicit duplicate handling

…s match

…(queryMin / queryNextChunkMax)

uicosp · 2026-01-29T08:13:10Z

@lvyanquan I've addressed the review comments and pushed the updates.
Could you please take another look when you have time? Thanks!

ThorneANN · 2026-02-03T03:13:40Z

@leonardBang PTAL,TKS.

ruanhang1993

@uicosp Hi, the MySQL changes are significant, and it looks like there are still some issues that need further discussion. It may not be feasible to complete this in version 3.6. We can schedule this feature for the next release.

ruanhang1993 · 2026-03-19T08:56:01Z

    }

+    @Test
+    void testAddSnapshotFilters() {


We need more tests for this feature(IT, E2E test).

Added two tests in commit 85647b2:

MySqlSourceITCase#testSnapshotFiltersMultipleTables — IT test
verifying per-table filters work independently across two tables
via the DataStream API.

MysqlE2eITCase#testSnapshotFilters — E2E test verifying the
full pipeline path using the YAML-based scan.snapshot.filters
configuration.

ruanhang1993 · 2026-03-19T08:57:27Z

        options.add(MySqlSourceOptions.USE_LEGACY_JSON_FORMAT);
        options.add(MySqlSourceOptions.SCAN_INCREMENTAL_SNAPSHOT_UNBOUNDED_CHUNK_FIRST);
        options.add(MySqlSourceOptions.SCAN_READ_CHANGELOG_AS_APPEND_ONLY_ENABLED);
+        options.add(MySqlSourceOptions.SCAN_SNAPSHOT_FILTER);


I am considering whether we should implement SupportsFilterPushDown for the sql connector instead of a new config.

Thanks for the suggestion!
We actually discussed this with @ThorneANN earlier — the conclusion was that SupportsFilterPushDown is not suitable here because of the Incremental Snapshot framework's design.
The current config-based approach (scan.snapshot.filter) is intentional.

ruanhang1993 · 2026-03-19T08:58:15Z

+      <td style="word-wrap: break-word;">(none)</td>
+      <td>String</td>
+      <td>When reading a table snapshot, the rows of captured tables will be filtered using the specified filter expression (AKA a SQL WHERE clause). By default, no filter is applied, meaning the entire table will be synchronized. e.g. `id > 100`.</td> 
+    </tr>


It is better to add a new use example for this feature.

Added usage examples in commit cc477d5:

docs/connectors/pipeline-connectors/mysql.md — Added "Snapshot
Filters" section with a YAML pipeline example demonstrating the
scan.snapshot.filters list syntax.

docs/connectors/flink-sources/mysql-cdc.md — Added "Snapshot
Filter" section with both SQL DDL and DataStream API examples
showing the scan.snapshot.filter option usage.

ruanhang1993 · 2026-03-19T09:09:10Z


+        String filter =
+                SnapshotFilterUtils.getSnapshotFilter(
+                        sourceConfig.getSnapshotFilters(), table.id());


Could we store the filters as a class member variable instead of constructing it on every invocation?

Optimized filter lookup in commit 9e8b5f5:

Moved snapshot filter resolution from
createDataEventsForTable() to the constructor of
MySqlSnapshotSplitReadTask.

The filter is now computed once and stored as a class member
variable (snapshotFilter), avoiding repeated lookups on every
method invocation.

sd4324530 · 2026-03-25T09:13:45Z

Really looking forward to this functionality.

ThorneANN · 2026-03-25T09:31:08Z

@uicosp Hi, the MySQL changes are significant, and it looks like there are still some issues that need further discussion. It may not be feasible to complete this in version 3.6. We can schedule this feature for the next release.

Hi, I think that we do not to supple the SupportsFilterPushDown because of the Incremental snapshot framework.

…it on every invocation

[FLINK-36165][source-connector/mysql] Support capturing snapshot data…

8a8d65c

… with conditions

github-actions Bot added mysql-cdc-connector mysql-pipeline-connector labels Dec 4, 2024

[FLINK-36165][source-connector/mysql] add docs

f1b8776

github-actions Bot added the docs Improvements or additions to documentation label Dec 4, 2024

xiayuxiao added 2 commits December 5, 2024 12:09

[FLINK-36165][source-connector/mysql] Implement snapshot filter for M…

ec83fbe

…ySQL table source

[FLINK-36165][source-connector/mysql] Escape dot

96b885a

lvyanquan reviewed May 19, 2025

View reviewed changes

Mrart mentioned this pull request Aug 21, 2025

[FLINK-36165][source-connector/mysql] Support capturing snapshot data with conditions #4104

Closed

xiayuxiao added 6 commits August 29, 2025 13:56

[FLINK-36165][source-connector/mysql] support colon and semicolon in …

36ae871

…filter

[FLINK-36165][source-connector/mysql] fix test

8ee4c92

[FLINK-36165][source-connector/mysql] fix spotless

5f913f4

[FLINK-36165][source-connector/mysql] fix spotless

e50667d

[FLINK-36165][source-connector/mysql] use yaml list/map to define sna…

e6dd29e

…pshot filter

github-actions Bot added common cli labels Sep 5, 2025

yuxiqian reviewed Sep 8, 2025

View reviewed changes

lvyanquan requested a review from Copilot January 27, 2026 04:18

Copilot started reviewing on behalf of lvyanquan January 27, 2026 04:19 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

uicosp and others added 11 commits January 29, 2026 13:54

Update docs/content/docs/connectors/pipeline-connectors/mysql.md

ed9ddcc

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update flink-cdc-connect/flink-cdc-source-connectors/flink-connector-…

794513b

…mysql-cdc/src/test/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtilsTest.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update flink-cdc-connect/flink-cdc-source-connectors/flink-connector-…

3af4044

…mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/StatementUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update flink-cdc-connect/flink-cdc-source-connectors/flink-connector-…

ca12a3f

…mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/StatementUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update flink-cdc-connect/flink-cdc-source-connectors/flink-connector-…

c025e29

…mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/SnapshotFilterUtils.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

validating and failing fast when scan.snapshot.filter option is set b…

3d360c9

…ut parallel read is disabled

validating each entry in scan.snapshot.filters and and building a Lin…

67fae6d

…kedHashMap with explicit duplicate handling

fix non-deterministic snapshot filter selection when multiple pattern…

b7f13fd

…s match

wrap user filter in parentheses when combining with split conditions …

72724f1

…(queryMin / queryNextChunkMax)

fix MySqlTableSourceFactoryTest

8ae2003

fix SnaphostFilterUtilsTest

2a0bdf2

ruanhang1993 reviewed Mar 19, 2026

View reviewed changes

xiayuxiao and others added 4 commits April 9, 2026 13:47

add more tests for this feature (IT, E2E test)

85647b2

add a new use example for this feature

cc477d5

store the filters as a class member variable instead of constructing …

9e8b5f5

…it on every invocation

Merge branch 'master' into FLINK-36165

0d14a1b

github-actions Bot added the e2e-tests label Apr 9, 2026

resolve merge conflicts

2d8a8ca

lvyanquan added this to the V3.7.0 milestone Apr 10, 2026

lvyanquan self-assigned this Apr 10, 2026

Conversation

uicosp commented Dec 4, 2024

Uh oh!

yuxiqian commented Dec 6, 2024

Uh oh!

ThorneANN commented Dec 12, 2024

Uh oh!

ThorneANN commented Dec 12, 2024

Uh oh!

uicosp commented Dec 12, 2024

Uh oh!

mageniao2 commented Apr 15, 2025

Uh oh!

uicosp commented Apr 15, 2025

Uh oh!

mageniao2 commented Apr 15, 2025

Uh oh!

uicosp commented Apr 15, 2025

Uh oh!

ThorneANN commented Apr 15, 2025

Uh oh!

mageniao2 commented Apr 15, 2025

Uh oh!

ThorneANN commented Apr 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uicosp Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uicosp commented Sep 5, 2025

Uh oh!

yuxiqian left a comment

Choose a reason for hiding this comment

Uh oh!

ThorneANN commented Dec 19, 2025

Uh oh!

nickdelnano commented Jan 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uicosp commented Jan 29, 2026

Uh oh!

ThorneANN commented Feb 3, 2026

Uh oh!

ruanhang1993 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uicosp Sep 5, 2025 •

edited

Loading

uicosp Apr 9, 2026 •

edited

Loading