Skip to content

[FLINK-38930][checkpoint] Filtering record before processing without spilling strategy#27783

Open
1996fanrui wants to merge 4 commits intoapache:masterfrom
1996fanrui:38930/filtering-record
Open

[FLINK-38930][checkpoint] Filtering record before processing without spilling strategy#27783
1996fanrui wants to merge 4 commits intoapache:masterfrom
1996fanrui:38930/filtering-record

Conversation

@1996fanrui
Copy link
Member

@1996fanrui 1996fanrui commented Mar 18, 2026

This PR depends on #27782

What is the purpose of the change

[FLINK-38930][checkpoint] Filtering record before processing without spilling strategy

Brief change log

Core filtering mechanism for recovered channel state buffers:

  • ChannelStateFilteringHandler with per-gate GateFilterHandler
  • RecordFilterContext with VirtualChannelRecordFilterFactory
  • Partial data check in SequentialChannelStateReaderImpl
  • Fix RecordFilterContext for Union downscale scenario

Verifying this change

  • Tons of unit tests

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: no
  • The runtime per-record code paths (performance sensitive):no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
  • The S3 file system connector:no

Documentation

  • Does this pull request introduce a new feature? no

@flinkbot
Copy link
Collaborator

flinkbot commented Mar 18, 2026

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

…spilling strategy

Core filtering mechanism for recovered channel state buffers:
- ChannelStateFilteringHandler with per-gate GateFilterHandler
- RecordFilterContext with VirtualChannelRecordFilterFactory
- Partial data check in SequentialChannelStateReaderImpl
- Fix RecordFilterContext for Union downscale scenario
@1996fanrui 1996fanrui force-pushed the 38930/filtering-record branch from 2b06750 to 997e3a3 Compare March 20, 2026 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants