Skip to content

Commit 29b5e45

Browse files
lmjclaude
lmj
and
claude
committed
feat: implement Debezium Source Connector Integration (Phase 4)
This commit adds comprehensive Debezium connector integration support including: ## Kafka Connect REST API Client - Full CRUD operations for connector management - Status monitoring and health checks - Pause/Resume/Restart capabilities with retry logic - Custom exceptions for error handling ## Source Connector Configuration Generators - MySqlSourceConnectorConfig: Complete MySQL binlog CDC configuration - OracleSourceConnectorConfig: LogMiner-based Oracle CDC configuration - PostgresSourceConnectorConfig: Logical replication PostgreSQL CDC configuration - Builder patterns with validation and sensible defaults ## Connector Lifecycle Management - Automated deployment and deletion - Health monitoring with automatic recovery - State tracking and status updates - Configurable retry policies with exponential backoff ## PostgreSQL-based Storage Implementations - JdbcOffsetBackingStore: Store Kafka Connect offsets in PostgreSQL - JdbcSchemaHistory: Store Debezium schema history in PostgreSQL - HikariCP connection pooling for performance ## Testing - 38 unit and integration tests passing - Testcontainers-based PostgreSQL integration tests - Configuration validation test coverage - Connector status behavior tests Technical changes: - Added HTTP client, HikariCP, and testing dependencies to connectors module - Temporarily disabled Oracle connector dependency due to mirror issues - Updated PROJECT_STATUS.md to reflect Phase 4 completion 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent c7338d5 commit 29b5e45

22 files changed

Lines changed: 3361 additions & 37 deletions

‎connectors/pom.xml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,12 @@
3636
<groupId>io.debezium</groupId>
3737
<artifactId>debezium-connector-postgres</artifactId>
3838
</dependency>
39+
<!-- Oracle connector temporarily disabled due to dependency resolution issues
3940
<dependency>
4041
<groupId>io.debezium</groupId>
4142
<artifactId>debezium-connector-oracle</artifactId>
4243
</dependency>
44+
-->
4345
<dependency>
4446
<groupId>io.debezium</groupId>
4547
<artifactId>debezium-connector-jdbc</artifactId>
@@ -50,6 +52,10 @@
5052
<groupId>org.apache.kafka</groupId>
5153
<artifactId>connect-api</artifactId>
5254
</dependency>
55+
<dependency>
56+
<groupId>org.apache.kafka</groupId>
57+
<artifactId>connect-runtime</artifactId>
58+
</dependency>
5359

5460
<!-- Lombok -->
5561
<dependency>
@@ -62,13 +68,97 @@
6268
<groupId>org.slf4j</groupId>
6369
<artifactId>slf4j-api</artifactId>
6470
</dependency>
71+
<dependency>
72+
<groupId>ch.qos.logback</groupId>
73+
<artifactId>logback-classic</artifactId>
74+
</dependency>
75+
76+
<!-- JSON Processing -->
77+
<dependency>
78+
<groupId>com.fasterxml.jackson.core</groupId>
79+
<artifactId>jackson-databind</artifactId>
80+
</dependency>
81+
<dependency>
82+
<groupId>com.fasterxml.jackson.datatype</groupId>
83+
<artifactId>jackson-datatype-jsr310</artifactId>
84+
</dependency>
85+
86+
<!-- HTTP Client -->
87+
<dependency>
88+
<groupId>org.apache.httpcomponents.client5</groupId>
89+
<artifactId>httpclient5</artifactId>
90+
<version>5.3.1</version>
91+
</dependency>
92+
93+
<!-- Database -->
94+
<dependency>
95+
<groupId>org.postgresql</groupId>
96+
<artifactId>postgresql</artifactId>
97+
</dependency>
98+
<dependency>
99+
<groupId>com.zaxxer</groupId>
100+
<artifactId>HikariCP</artifactId>
101+
<version>5.1.0</version>
102+
</dependency>
103+
104+
<!-- Validation -->
105+
<dependency>
106+
<groupId>jakarta.validation</groupId>
107+
<artifactId>jakarta.validation-api</artifactId>
108+
<version>3.0.2</version>
109+
</dependency>
65110

66111
<!-- Testing -->
67112
<dependency>
68113
<groupId>org.junit.jupiter</groupId>
69114
<artifactId>junit-jupiter</artifactId>
70115
<scope>test</scope>
71116
</dependency>
117+
<dependency>
118+
<groupId>org.mockito</groupId>
119+
<artifactId>mockito-core</artifactId>
120+
<scope>test</scope>
121+
</dependency>
122+
<dependency>
123+
<groupId>org.mockito</groupId>
124+
<artifactId>mockito-junit-jupiter</artifactId>
125+
<version>${mockito.version}</version>
126+
<scope>test</scope>
127+
</dependency>
128+
<dependency>
129+
<groupId>org.assertj</groupId>
130+
<artifactId>assertj-core</artifactId>
131+
<scope>test</scope>
132+
</dependency>
133+
<dependency>
134+
<groupId>org.testcontainers</groupId>
135+
<artifactId>testcontainers</artifactId>
136+
<scope>test</scope>
137+
</dependency>
138+
<dependency>
139+
<groupId>org.testcontainers</groupId>
140+
<artifactId>junit-jupiter</artifactId>
141+
<version>${testcontainers.version}</version>
142+
<scope>test</scope>
143+
</dependency>
144+
<dependency>
145+
<groupId>org.testcontainers</groupId>
146+
<artifactId>kafka</artifactId>
147+
<version>${testcontainers.version}</version>
148+
<scope>test</scope>
149+
</dependency>
150+
<dependency>
151+
<groupId>org.testcontainers</groupId>
152+
<artifactId>postgresql</artifactId>
153+
<version>${testcontainers.version}</version>
154+
<scope>test</scope>
155+
</dependency>
156+
<dependency>
157+
<groupId>org.testcontainers</groupId>
158+
<artifactId>mysql</artifactId>
159+
<version>${testcontainers.version}</version>
160+
<scope>test</scope>
161+
</dependency>
72162
</dependencies>
73163

74164
</project>

0 commit comments

Comments
 (0)