Support for exactly-once delivery guarantee #106
Description
Use case
Currently, the sink may produce duplicate rows (inherently limitation of Flink AsyncSinkBase).
Describe the solution you'd like
Support for exactly-once delivery
Describe the alternatives you've considered
- record/batch deduplication in ClickHouse (similar to Kafka Connect)
- ReplacingMergeTree
- Flink state (ie, using KeyedState and a custom flatmap function to filter duplicate rows by a deterministic row id)