You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrations/data-ingestion/kafka/kafka-clickhouse-connect-sink.md
+20-8Lines changed: 20 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -201,6 +201,10 @@ The most basic configuration to get you started - it assumes you're running Kafk
201
201
}
202
202
```
203
203
204
+
:::note
205
+
The above connector config requires that you enable client overrides in your worker configuration via `connector.client.config.override.policy=All`. See the [Kafka Connect documentation](https://docs.confluent.io/platform/current/connect/references/allconfigs.html#override-the-worker-configuration) for more information.
206
+
:::
207
+
204
208
#### Basic configuration with multiple topics {#basic-configuration-with-multiple-topics}
205
209
206
210
The connector can consume data from multiple topics
@@ -571,18 +575,22 @@ For optimal performance with ClickHouse, aim for larger batches:
# Optional: Increase minimum fetch size to wait for more data (1 MB)
580
-
consumer.fetch.min.bytes=1048576
584
+
consumer.override.fetch.min.bytes=1048576
581
585
582
586
# Optional: Reduce wait time if latency is critical
583
-
consumer.fetch.max.wait.ms=300
587
+
consumer.override.fetch.max.wait.ms=300
584
588
```
585
589
590
+
:::note
591
+
The above properties require that you enable client overrides in your worker configuration via `connector.client.config.override.policy=All`. See the [Kafka Connect documentation](https://docs.confluent.io/platform/current/connect/references/allconfigs.html#override-the-worker-configuration) for more information.
592
+
:::
593
+
586
594
**Important**: Kafka Connect fetch settings represent compressed data, while ClickHouse receives uncompressed data. Balance these settings based on your compression ratio.
587
595
588
596
**Trade-offs**:
@@ -821,16 +829,20 @@ Here's a complete example optimized for high throughput:
The above connector config requires that you enable client overrides in your worker configuration via `connector.client.config.override.policy=All`. See the [Kafka Connect documentation](https://docs.confluent.io/platform/current/connect/references/allconfigs.html#override-the-worker-configuration) for more information.
0 commit comments