Skip to content

overview.md

Latest commit

 

History

History
71 lines (58 loc) · 23.1 KB

overview.md

File metadata and controls

71 lines (58 loc) · 23.1 KB
title Overview
weight 1
type docs
aliases
/connectors/pipeline-connectors/overview

Connectors

Flink CDC provides several source and sink connectors to interact with external systems. You can use these connectors out-of-box, by adding released JARs to your Flink CDC environment, and specifying the connector in your YAML pipeline definition.

Supported Connectors

Connector Supported Type External System Download Page
[Apache Doris]({{< ref "docs/connectors/pipeline-connectors/doris" >}}) Sink
  • Apache Doris: 1.2.x, 2.x.x, 3.x.x
    		•  3.5.0, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0
    [Elasticsearch]({{< ref "docs/connectors/pipeline-connectors/elasticsearch" >}}) Sink
  • Elasticsearch: 6.x, 7.x, 8.x
    		•  3.5.0, 3.4.0, 3.3.0
    [Iceberg]({{< ref "docs/connectors/pipeline-connectors/iceberg" >}}) Sink
  • Apache Iceberg
    		•  3.5.0, 3.4.0
    [Fluss]({{< ref "docs/connectors/pipeline-connectors/fluss" >}}) Sink
  • Fluss
    		•  3.5.0
    [Kafka]({{< ref "docs/connectors/pipeline-connectors/kafka" >}}) Sink
  • Kafka
    		•  3.5.0, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0
    [MySQL]({{< ref "docs/connectors/pipeline-connectors/mysql" >}}) Source
  • MySQL: 5.6, 5.7, 8.0.x
  • RDS MySQL: 5.6, 5.7, 8.0.x
  • PolarDB MySQL: 5.6, 5.7, 8.0.x
  • Aurora MySQL: 5.6, 5.7, 8.0.x
  • MariaDB: 10.x
  • PolarDB X: 2.0.1
    		•  3.5.0, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0
    [Postgres]({{< ref "docs/connectors/pipeline-connectors/postgres" >}}) Source
  • PostgreSQL
    		•  3.5.0
    [OceanBase]({{< ref "docs/connectors/pipeline-connectors/oceanbase" >}}) Sink
  • OceanBase: 3.x, 4.x
    		•  3.5.0, 3.4.0, 3.3.0
    [Paimon]({{< ref "docs/connectors/pipeline-connectors/paimon" >}}) Sink
  • Paimon: 0.6, 0.7, 0.8, 0.9, 1.0
    		•  3.5.0, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0
    [StarRocks]({{< ref "docs/connectors/pipeline-connectors/starrocks" >}}) Sink
  • StarRocks: 2.x, 3.x
    		•  3.5.0, 3.4.0, 3.3.0, 3.2.1, 3.2.0, 3.1.1, 3.1.0
    [MaxCompute]({{< ref "docs/connectors/pipeline-connectors/maxcompute" >}}) Sink
  • MaxCompute
    		•  3.5.0, 3.4.0, 3.3.0

    Supported Flink Versions

    The following table shows the version mapping between Flink® CDC Pipeline Connectors and Flink®

    Flink® CDC Version Flink® Version Pipeline Source Pipeline Sink Notes
    3.0.x 1.17.*, 1.18.* Mysql StarRocks,Doris
    3.1.x 1.17.*, 1.18.*, 1.19.* Mysql StarRocks,Doris,Paimon,Kafka only flink-cdc 3.1.1 support flink 1.19.*
    3.2.x 1.17.*, 1.18.*, 1.19.* Mysql StarRocks,Doris,Paimon,Kafka,ElasticSearch
    3.3.x 1.19.*, 1.20.* Mysql StarRocks,Doris,Paimon,Kafka,ElasticSearch,OceanBase,MaxCompute
    3.4.x 1.19.*, 1.20.* Mysql StarRocks,Doris,Paimon,Kafka,ElasticSearch,OceanBase,MaxCompute,Iceberg
    3.5.x 1.19.*, 1.20.* Mysql, Postgres StarRocks,Doris,Paimon,Kafka,ElasticSearch,OceanBase,MaxCompute,Iceberg,Fluss

    Develop Your Own Connector

    If provided connectors cannot fulfill your requirement, you can always develop your own connector to get your external system involved in Flink CDC pipelines. Check out [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}}) to learn how to develop your own connectors.

    {{< top >}}