|
Note
|
This repository contains the guide documentation source. To view the guide in published form, view it on the Open Liberty website. |
Learn how to test reactive Java microservices in true-to-production environments using Testcontainers.
You will learn how to write integration tests for reactive Java microservices and to run the tests in true-to-production environments by using containers with Testcontainers and JUnit. Testcontainers tests your containerized application from outside the container so that you are testing the exact same image that runs in production. The reactive application in this guide sends and receives messages between services by using an external message broker, Apache Kafka. Using an external message broker enables asynchronous communications between services so that requests are non-blocking and decoupled from responses. You can learn more about reactive Java services that use an external message broker to manage communications in the Creating reactive Java microservices guide.
True-to-production integration testing with Testcontainers
Tests sometimes pass during the development and testing stages of an application’s lifecycle but then fail in production because of differences between your development and production environments. While you can create mock objects and custom setups to minimize differences between environments, it is difficult to mimic a production system for an application that uses an external messaging system. Testcontainers addresses this problem by enabling the testing of applications in the same Docker containers that you’ll use in production. As a result, your environment remains the same throughout the application’s lifecycle – from development, through testing, and into production. You can learn more about Testcontainers in the Building true-to-production integration tests with Testcontainers guide.
You need to have Docker installed. For installation instructions, refer to the official Docker documentation. You will build and run the microservices in Docker containers. An installation of Apache Kafka is provided in another Docker container.
The finish directory in the root of this guide contains the finished application. Give it a try before you proceed.
To try out the tests, go to the finish directory and run the following Maven goal to install the models artifact to the local Maven repository:
mvnw.cmd -pl models install./mvnw -pl models install./mvnw -pl models installNext, navigate to the finish directory and run the following Maven goal to build the system microservice and run the integration tests on an Open Liberty server in a container:
mvnw.cmd -pl system verifyexport TESTCONTAINERS_RYUK_DISABLED=true
./mvnw -pl system verifyexport TESTCONTAINERS_RYUK_DISABLED=true
./mvnw -pl system verifyYou will see the following output:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.46 s - in it.io.openliberty.guides.system.SystemServiceIT
Results:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
--- failsafe:3.2.5:verify (verify) @ system ---
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 57.710 s
Finished at: 2024-02-01T08:48:15-08:00
------------------------------------------------------------------------This command might take some time to run the first time because the dependencies and the Docker image for Open Liberty must download. If you run the same command again, it will be faster.
You can also try out the inventory integration tests by repeating the same command but replacing system with inventory.
system/pom.xml
link:finish/system/pom.xml[role=include]inventory/pom.xml
link:finish/inventory/pom.xml[role=include]system/microprofile-config.properties
link:finish/system/src/main/resources/META-INF/microprofile-config.properties[role=include]startKafka.sh
link:finish/scripts/startKafka.sh[role=include]SystemServiceIT.java
link:finish/system/src/test/java/it/io/openliberty/guides/system/SystemServiceIT.java[role=include]Navigate to the start directory to begin.
The example reactive application consists of the system and inventory microservices. The system microservice produces messages to the Kafka message broker, and the inventory microservice consumes messages from the Kafka message broker. You will write integration tests to see how you can use the Kafka consumer and producer client APIs to test each service. Kafka test containers, Testcontainers, and JUnit are already included as required test dependencies in your Maven pom.xml files for the system and inventory microservices.
The start directory contains three directories: the system microservice directory, the inventory microservice directory, and the models directory. The models directory contains the model class that defines the structure of the system load data that is used in the application. Run the following Maven goal to install the packaged models artifact to the local Maven repository so it can be used later by the system and inventory microservices:
mvnw.cmd -pl models install./mvnw -pl models install./mvnw -pl models installStart the microservices in dev mode by running the following command to launch a Kafka instance that replicates the production environment. The startKafka script launches a local Kafka container. It also establishes a reactive-app network that allows the system and inventory microservices to connect to the Kafka message broker.
.\scripts\startKafka.bat./scripts/startKafka.sh./scripts/startKafka.shTo launch the system microservice in dev mode with container support, configure the container by specifying the options within the <containerRunOpts> element to connect to the reactive-app network and expose the container port.
Run the following goal to start the system microservice in dev mode with container support:
mvnw.cmd -pl system liberty:devcexport TESTCONTAINERS_RYUK_DISABLED=true
./mvnw -pl system liberty:devcFor more information about disabling Ryuk, see the Testcontainers custom configuration document.
export TESTCONTAINERS_RYUK_DISABLED=true
./mvnw -pl system liberty:devcFor more information about disabling Ryuk, see the Testcontainers custom configuration document.
After you see the following message, your Liberty instance is ready in dev mode:
************************************************************** * Liberty is running in dev mode. * ... * Liberty container port information: * Internal container HTTP port [ 9083 ] is mapped to container host port [ 9083 ] < * ...
Dev mode holds your command-line session to listen for file changes. Open another command-line session to continue, or open the project in your editor.
The system microservice actively seeks a Kafka topic for message push operations. After the Kafka service starts, the system microservice connects to the Kafka message broker by using the mp.messaging.connector.liberty-kafka.bootstrap.servers property. When you run your application in dev mode with container support, the running system container exposes its service on the 9083 port for testing purposes.
Now you can start writing the test by using Testcontainers.
Create theSystemServiceITclass.system/src/test/java/it/io/openliberty/guides/system/SystemServiceIT.java
SystemServiceIT.java
link:finish/system/src/test/java/it/io/openliberty/guides/system/SystemServiceIT.java[role=include]SystemLoad.java
link:finish/models/src/main/java/io/openliberty/guides/models/SystemLoad.java[role=include]SystemService.java
link:finish/system/src/main/java/io/openliberty/guides/system/SystemService.java[role=include]Dockerfile
link:finish/system/Dockerfile[role=include]Construct the systemImage by using the ImageFromDockerfile class, which allows Testcontainers to build the Docker image from a Dockerfile during the test run time. For instance, the provided Dockerfile at the specified ./Dockerfile paths is used to generate the system:1.0-SNAPSHOT image.
Use the kafkaContainer class to instantiate the kafkaContainer test container, initiating the apache/kafka:latest Docker image. Similarly, use the GenericContainer class to create the systemContainer test container, starting the system:1.0-SNAPSHOT Docker image.
The withListener() is configured to kafka:19092, as the containerized system microservice functions as an additional producer. Therefore, the Kafka container needs to set up a listener to accommodate this requirement. For more information about using an additional consumer or producer with a Kafka container, see the Testcontainers Kafka documentation
Because containers are isolated by default, facilitating communication between the kafkaContainer and the systemContainer requires placing them on the same network. The dependsOn() method is used to indicate that the system microservice container starts only after ensuring the readiness of the Kafka container.
Before you start the systemContainer, you must override the mp.messaging.connector.liberty-kafka.bootstrap.servers property with kafka:19092 by using the withEnv() method. This step creates a listener in the Kafka container that is configured to handle an additional producer.
The test uses the KafkaConsumer client API, configuring the consumer to use the BOOTSTRAP_SERVERS_CONFIG property with the Kafka broker address if a local system microservice container is present. In the absence of a local service container, it uses the getBootstrapServers() method to obtain the broker address from the Kafka test container. Then, the consumer is set up to consume messages from the system.load topic within the Kafka container.
To consume messages from a stream, the messages need to be deserialized from bytes. Kafka has its own default deserializer, but a custom deserializer is provided for you. The deserializer is configured by the VALUE_DESERIALIZER_CLASS_CONFIG property and is implemented in the SystemLoad class. To learn more about Kafka APIs and their usage, see the official Kafka Documentation.
The running system microservice container produces messages to the systemLoad Kafka topic, as denoted by the @Outgoing annotation. The testCpuStatus() test method uses the consumer.poll() method from the KafkaConsumer client API to retrieve a record from Kafka every 3 seconds within a specified timeout limit. This record is produced by the system service. Then, the method uses Assertions to verify that the polled record aligns with the expected record.
You will see the following output:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.674 s - in it.io.openliberty.guides.system.SystemServiceIT
Results:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Integration tests finished.After you are finished running tests, stop the Open Liberty server by pressing CTRL+C in the command-line session where you ran the server.
If you aren’t running in dev mode, you can run the tests by running the following command:
mvnw.cmd -pl system clean verify./mvnw -pl system clean verify./mvnw -pl system clean verifyYou will see the following output:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.63 s - in it.io.openliberty.guides.system.SystemServiceIT
Results:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
--- failsafe:3.2.5:verify (verify) @ system ---
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 55.636 s
Finished at: 2024-01-31T11:33:40-08:00
------------------------------------------------------------------------The inventory microservice is tested in the same way as the system microservice. The only difference is that the inventory microservice consumes messages, which means that tests are written to use the Kafka producer client.
Run the following goal to start the inventory microservice in dev mode with container support:
mvnw.cmd -pl inventory liberty:devc./mvnw -pl inventory liberty:devc./mvnw -pl inventory liberty:devcCreate a REST client interface to access the inventory microservice.
Create theInventoryResourceClientclass.inventory/src/test/java/it/io/openliberty/guides/inventory/InventoryResourceClient.java
InventoryResourceClient.java
link:finish/inventory/src/test/java/it/io/openliberty/guides/inventory/InventoryResourceClient.java[role=include]The InventoryResourceClient interface declares the getSystems() and resetSystems() methods for accessing the corresponding endpoints within the inventory microservice.
Now you can start writing the test by using Testcontainers.
Create theInventoryServiceITclass.inventory/src/test/java/it/io/openliberty/guides/inventory/InventoryServiceIT.java
InventoryServiceIT.java
link:finish/inventory/src/test/java/it/io/openliberty/guides/inventory/InventoryServiceIT.java[role=include]SystemLoad.java
link:finish/models/src/main/java/io/openliberty/guides/models/SystemLoad.java[role=include]InventoryResource.java
link:finish/inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.java[role=include]The InventoryServiceIT class uses the KafkaProducer client API to generate messages in the test environment, which are then consumed by the inventory microservice container.
Similar to system microservice testing, the configuration of the producer BOOTSTRAP_SERVERS_CONFIG property depends on whether a local inventory microservice container is detected. In addition, the producer is configured with a custom serializer provided in the SystemLoad class.
The testCpuUsage test method uses the producer.send() method, using the KafkaProducer client API, to generate the Systemload message. Then, it uses Assertions to verify that the response from the inventory microservice aligns with the expected outcome.
You will see the following output:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 32.564 s - in it.io.openliberty.guides.inventory.InventoryServiceIT
Results:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Integration tests finished.After you are finished running tests, stop the Open Liberty server by pressing CTRL+C in the command-line session where you ran the server.
If you aren’t running in dev mode, you can run the tests by running the following command:
mvnw.cmd -pl inventory clean verify./mvnw -pl inventory clean verify./mvnw -pl inventory clean verifyYou will see the following output:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 53.22 s - in it.io.openliberty.guides.inventory.InventoryServiceIT
Results:
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
--- failsafe:3.2.5:verify (verify) @ inventory ---
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
Total time: 58.789 s
Finished at: 2024-01-31T11:40:43-08:00
------------------------------------------------------------------------When you’re finished trying out the microservice, you can stop the local Kafka container by running the following command from the start directory:
.\scripts\stopKafka.bat./scripts/stopKafka.shLearn more about Testcontainers.