kafka-connect.adoc

Connect Redpanda Console to Kafka Connect Clusters

shared:partial$community-supported-kc.adoc

Redpanda Console provides a user interface that lets you manage multiple Kafka Connect clusters. You can inspect or patch connectors; restart, pause, and resume connector tasks; and delete connectors. Redpanda Console queries all configured Kafka Connect clusters for their status, so you have an overview of all your Kafka Connect clusters.

Prerequisites

You must deploy a Kafka Connect cluster separately before configuring Redpanda Console to connect to it.

Configure a connection to Kafka Connect

For each cluster, provide a unique name, the HTTP address of the cluster, and the authentication settings, if required. The name can be any unique string that helps you to identify the Kafka Connect cluster. See all available configuration options in the Redpanda Console Configuration.

Standalone

kafkaConnect:
  enabled: true
  clusters:
    - name: datawarehouse # Required field, will be used as identifier in the frontend
      url: http://dwh-connect.mycompany.com:8083
      tls:
        enabled: false # Trusted certs are still allowed by default
      username: admin
      # password: # Set using flag --kafkaConnect.clusters.0.password=secret
    - name: analytics # Required field, will be used as identifier in the frontend
      url: http://analytics.mycompany.com:8083
      # No auth configured on that cluster, hence no username/password set

Kubernetes embedded

When Redpanda Console is part of the Redpanda Helm chart or Operator:

Operator

redpanda-console.yaml

apiVersion: cluster.redpanda.com/v1alpha2
kind: Console
metadata:
  name: redpanda-console
spec:
  clusterRef:
    name: redpanda
  config:
    kafkaConnect:
      enabled: true
      clusters:
        - name: datawarehouse
          url: http://dwh-connect.mycompany.com:8083
          tls:
            enabled: false
          username: admin
          # password: (add if needed)
        - name: analytics
          url: http://analytics.mycompany.com:8083
          # No auth configured on this cluster

Helm

redpanda-values.yaml

console:
  enabled: true
  console:
    config:
      kafkaConnect:
        enabled: true
        clusters:
          - name: datawarehouse # Required field, will be used as identifier in the frontend
            url: http://dwh-connect.mycompany.com:8083
            tls:
              enabled: false # Trusted certs are still allowed by default
            username: admin
            # password:
          - name: analytics # Required field, will be used as identifier in the frontend
            url: http://analytics.mycompany.com:8083
            # No auth configured on that cluster, hence no username/password set

Kubernetes standalone

When using the standalone Redpanda Console Helm chart:

console-values.yaml

config:
  kafkaConnect:
    enabled: true
    clusters:
      - name: datawarehouse # Required field, will be used as identifier in the frontend
        url: http://dwh-connect.mycompany.com:8083
        tls:
          enabled: false # Trusted certs are still allowed by default
        username: admin
        # password:
      - name: analytics # Required field, will be used as identifier in the frontend
        url: http://analytics.mycompany.com:8083
        # No auth configured on that cluster, hence no username/password set