[major] add support for elasticsearch 9.x.x by istrate · Pull Request #31 · ibm-messaging/kafka-connect-elastic-sink

istrate · 2026-05-18T05:37:46Z

Major Updates

Elasticsearch 9.x Compatibility: Updated connector to be compatible with Elasticsearch 9.4.9
Kafka 4.2.0 Support: Updated to latest Kafka version 4.2.0
Dependency Updates: Updated all dependencies to latest stable versions
TLS 1.3 Support: Added TLS 1.3 protocol support alongside TLS 1.2 for enhanced security

Dependency Changes

Kafka: Updated from 3.6.1 to 4.2.0 (latest stable release)
Jetty HTTP Client: Updated to 9.4.57.v20241219 (latest stable 9.4.x release)
JUnit Jupiter: Updated from 5.10.2 to 5.11.4
SLF4J: Updated from 1.7.36 to 2.0.16
JSON Library: Kept at 20240303 (stable release)

Code Changes

Updated ElasticWriter.java to support both TLS 1.2 and TLS 1.3 protocols
Enhanced SSL/TLS configuration for better compatibility with Elasticsearch 9.x
Improved logging for IBM JRE cipher suite exclusions

Build Configuration

Updated Java target from 17 to 21
Updated Maven Surefire plugin from 3.2.5 to 3.5.2
Updated connector version from 1.0.3 to 2.0.0
Added Maven Shade Plugin: Relocates Jetty classes to avoid conflicts with Kafka 4.2.0's bundled Jetty 12
Jetty dependencies are now shaded under com.ibm.eventstreams.shaded.jetty package

Testing

Connector tested and verified compatible with Elasticsearch 9.4.1 via quickstart

/kafka-connect-elastic-sink/quickstart$ ./RUNME.sh
[INFO] Scanning for projects...
[INFO]
[INFO] ------< com.ibm.eventstreams.connect:kafka-connect-elastic-sink >-------
[INFO] Building kafka-connect-elastic-sink 2.0.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ kafka-connect-elastic-sink ---
[INFO] Deleting /mnt/c/masworkspace/projects/eventstreams/eks/es-sink/kafka-connect-elastic-sink/target
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ kafka-connect-elastic-sink ---
[INFO] skip non existing resourceDirectory /mnt/c/masworkspace/projects/eventstreams/eks/es-sink/kafka-connect-elastic-sink/src/main/resources
[INFO]
[INFO] --- compiler:3.13.0:compile (default-compile) @ kafka-connect-elastic-sink ---
[INFO] Recompiling the module because of changed source code.
[INFO] Compiling 11 source files with javac [debug target 21] to target/classes
[INFO] /mnt/c/masworkspace/projects/eventstreams/eks/es-sink/kafka-connect-elastic-sink/src/main/java/com/ibm/eventstreams/connect/elasticsink/ElasticSinkConnector.java: Some input files use or override a deprecated API.
[INFO] /mnt/c/masworkspace/projects/eventstreams/eks/es-sink/kafka-connect-elastic-sink/src/main/java/com/ibm/eventstreams/connect/elasticsink/ElasticSinkConnector.java: Recompile with -Xlint:deprecation for details.
[INFO]
[INFO] --- resources:3.3.1:testResources (default-testResources) @ kafka-connect-elastic-sink ---
[INFO] Copying 1 resource from src/test/resources to target/test-classes
[INFO]
[INFO] --- compiler:3.13.0:testCompile (default-testCompile) @ kafka-connect-elastic-sink ---
[INFO] Recompiling the module because of changed dependency.
[INFO] Compiling 1 source file with javac [debug target 21] to target/test-classes
[INFO]
[INFO] --- surefire:3.5.2:test (default-test) @ kafka-connect-elastic-sink ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.ibm.eventstreams.connect.elasticsink.ElasticSinkConnectorTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.181 s -- in com.ibm.eventstreams.connect.elasticsink.ElasticSinkConnectorTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- jar:3.3.0:jar (default-jar) @ kafka-connect-elastic-sink ---
[INFO] Building jar: /mnt/c/masworkspace/projects/eventstreams/eks/es-sink/kafka-connect-elastic-sink/target/kafka-connect-elastic-sink-2.0.0.jar
[INFO]
[INFO] --- shade:3.6.0:shade (default) @ kafka-connect-elastic-sink ---
[INFO] Including org.eclipse.jetty:jetty-client:jar:9.4.57.v20241219 in the shaded jar.
[INFO] Including org.eclipse.jetty:jetty-http:jar:9.4.57.v20241219 in the shaded jar.
[INFO] Including org.eclipse.jetty:jetty-util:jar:9.4.57.v20241219 in the shaded jar.
[INFO] Including org.eclipse.jetty:jetty-io:jar:9.4.57.v20241219 in the shaded jar.
[INFO] Including org.json:json:jar:20240303 in the shaded jar.
[INFO] Attaching shaded artifact.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  11.993 s
[INFO] Finished at: 2026-05-18T08:31:32+03:00
[INFO] ------------------------------------------------------------------------
[+] Building 2.5s (17/17) FINISHED                                                                                                          docker:default
 => [internal] load build definition from Dockerfile                                                                                                  0.0s
 => => transferring dockerfile: 1.17kB                                                                                                                0.0s
 => [internal] load metadata for quay.io/strimzi/kafka:latest-kafka-4.2.0                                                                             0.0s
 => [internal] load metadata for docker.io/library/ibm-semeru-runtimes:open-21-jre                                                                    1.1s
 => [internal] load .dockerignore                                                                                                                     0.0s
 => => transferring context: 2B                                                                                                                       0.0s
 => [kafka 1/1] FROM quay.io/strimzi/kafka:latest-kafka-4.2.0                                                                                         0.0s
 => [stage-1  1/10] FROM docker.io/library/ibm-semeru-runtimes:open-21-jre@sha256:515e4059f6ebc09ff06e38198d905ff40e1ee2be29dfb327e8a20f630db06c33    0.0s
 => [internal] load build context                                                                                                                     1.0s
 => => transferring context: 1.47MB                                                                                                                   1.0s
 => CACHED [stage-1  2/10] RUN groupadd --gid 5000 --system esgroup &&     useradd --uid 5000 -g esgroup --system esuser                              0.0s
 => CACHED [stage-1  3/10] COPY --chown=esuser:esgroup --from=kafka /opt/kafka/bin/ /opt/kafka/bin/                                                   0.0s
 => CACHED [stage-1  4/10] COPY --chown=esuser:esgroup --from=kafka /opt/kafka/libs/ /opt/kafka/libs/                                                 0.0s
 => CACHED [stage-1  5/10] COPY --chown=esuser:esgroup --from=kafka /opt/kafka/config/ /opt/kafka/config/                                             0.0s
 => CACHED [stage-1  6/10] RUN mkdir /opt/kafka/logs && chown esuser:esgroup /opt/kafka/logs                                                          0.0s
 => [stage-1  7/10] COPY --chown=esuser:esgroup target/kafka-connect-elastic-sink-*-jar-with-dependencies.jar /opt/kafka/libs/                        0.0s
 => [stage-1  8/10] COPY --chown=esuser:esgroup quickstart/elastic-sink-qs.properties /opt/kafka/config                                               0.0s
 => [stage-1  9/10] COPY --chown=esuser:esgroup quickstart/connect-standalone-qs.properties /opt/kafka/config/                                        0.0s
 => [stage-1 10/10] WORKDIR /opt/kafka                                                                                                                0.0s
 => exporting to image                                                                                                                                0.1s
 => => exporting layers                                                                                                                               0.1s
 => => writing image sha256:d183e5acd86d5bc2a7741a78ede1af33e1d6fa2636f3a55117cd1b8308583545                                                          0.0s
 => => naming to docker.io/library/kafka-connect-elastic-sink:latest                                                                                  0.0s
[+] Running 1/1
 ✔ Network qs_default  Removed                                                                                                                        0.3s
[+] Running 4/4
 ✔ Network qs_default        Created                                                                                                                  0.0s
 ✔ Container qs-kafka-1      Started                                                                                                                  0.3s
 ✔ Container qs-elastic-1    Started                                                                                                                  0.3s
 ✔ Container qs-connector-1  Started                                                                                                                  0.5s
Waiting for 60 seconds...
Creating Kafka topic
Publishing to topic

Waiting for 60 seconds...
Transferred documents = 10
OK

Snap from test:

Maintains backward compatibility with Elasticsearch 7.x and 8.x

Breaking Changes

Requires Java 21 or later (previously Java 17)
Requires Kafka 4.2.0 or later (previously 3.6.1)

Signed-off-by: Daniel Istrate <daniel.istrate@ymail.com>

Joel-hanson · 2026-05-19T04:03:47Z

 ## License

-Copyright 2020, 2023 IBM Corporation
+Copyright 2020, 2026 IBM Corporation


The copyright should be appended not a amended

2020, 2023, 2026

Joel-hanson · 2026-05-19T04:06:01Z

I don't think this file is required

Joel-hanson · 2026-05-19T04:06:33Z

-* Apache Kafka 2.0.0 or later, either standalone or included as part of an offering such as IBM Event Streams
-* Elasticsearch 7.0.0 or later
+* Apache Kafka 4.2.0 or later, either standalone or included as part of an offering such as IBM Event Streams
+* Elasticsearch 7.0.0 or later (tested and compatible with Elasticsearch 9.4.1)


I don't think we should specify which version we tested on, it would give an impression that it was tested only with 9.4.1

Joel-hanson · 2026-05-19T04:07:16Z


 public class Version {
-    public static final String VERSION = "1.0.3"; // Must match the value in pom.xml
+    public static final String VERSION = "2.0.0"; // Must match the value in pom.xml - Updated for Elasticsearch 9.x compatibility


can you please remove - Updated for Elasticsearch 9.x compatibility

Joel-hanson · 2026-05-19T04:08:29Z

@@ -506,10 +506,11 @@ private HttpClient setupConnection() {
            throw new ConnectException(e);
        }



please remove Elasticsearch 9.x compatibility

Joel-hanson · 2026-05-19T04:22:42Z

We don’t keep changelogs as a separate file; we usually add this information to the release description. It’s totally fine if we add one, but the dates might be misaligned, as the release date may not match what’s mentioned in the file. I think it’s best to remove this file, as we don’t want to duplicate information or add the burden of maintaining it for each release.

Joel-hanson · 2026-05-19T04:27:51Z

We don't keep changelogs as a seperate file, we usually add this information to the release description. its ok if we add it but the dates might miss align as the date we release won't the match what is there in the file

Joel-hanson · 2026-05-19T04:31:18Z

Joel-hanson · 2026-05-19T04:31:55Z

Daniel Istrate added 2 commits May 18, 2026 08:18

add support for eks9xx

9134a6e

last updates

32e6c8e

Signed-off-by: Daniel Istrate <daniel.istrate@ymail.com>

Joel-hanson reviewed May 19, 2026

View reviewed changes

Joel-hanson requested changes May 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[major] add support for elasticsearch 9.x.x#31

[major] add support for elasticsearch 9.x.x#31
istrate wants to merge 2 commits into
ibm-messaging:mainfrom
istrate:eks94

istrate commented May 18, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Joel-hanson May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		@@ -506,10 +506,11 @@ private HttpClient setupConnection() {
		throw new ConnectException(e);
		}

Conversation

istrate commented May 18, 2026

Major Updates

Dependency Changes

Code Changes

Build Configuration

Testing

Breaking Changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants