ProducerRMQ

Sends data from and to a PostgreSQL database using RabbitMQ.
Explore the docs »

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. License

System Architecture Overview

This application automates the process of retrieving and transmitting database records to a RabbitMQ message queue. It connects to a PostgreSQL database, extracts relevant data from multiple tables (such as weather, connection, and AIS data), transforms the data into JSON format, and publishes it to a message queue for further processing.

Key Features:

• Database Connectivity: Establishes a connection with a PostgreSQL database and retrieves records from configurable tables.
• Configurable Data Processing: Supports configuration-based control over which data sets (weather, connection, AIS) should be retrieved and published.
• JSON Data Transformation: Converts SQL query results into structured JSON objects for easy integration with other systems.
• Message Queue Integration: Uses RabbitMQ to efficiently distribute data for downstream applications or analysis.

(back to top)

Built With

This section lists any major frameworks/libraries used while developing this tool.

(back to top)

Getting Started

Prerequisites

RabbitMQ and PostgreSQL Setup:

• RabbitMQ should be running on the host and ip specified in your config.properties. This application will connect to it and sends the messages to the RabbitMQ server.
• PostgreSQL should also be running with the mirrored version of the Consumers database on the host and ip + port specified in your config.properties.
• Ensure the correct credentials are provided in the config.properties.

Java Runtime Environment (JRE):

Make sure you have a compatible JRE (Java Runtime Environment) installed, JRE 8 or higher is required.

java -version

Installation

1. Download the latest release from the Releases Page and extract the ZIP file.
2. Configure database and rabbitmq server parameters by updating the configuration file with your credentials:

    PostgreSQL Configuratie
    db.url=jdbc:postgresql://<ip>:<port>/<database_name>
    db.username=<username>
    db.password=<password>
   
    RabbitMQ Configuratie
    rabbitmq.host=<ip>
    rabbitmq.username=<username>
    rabbitmq.password=<password>
    
    SQL Limit
    sql.limit=<amount of rows>
    
    Tables to Send
    send.weather=<true/false>
    send.conn=<true/false>
    send.ais=<true/false>

3. Run the script:

   java -jar produceRMQ.jar

After running the program, it will send the data from the specified database and tables to the RabbitMQ server queue, if a Consumer it listening to it, it will forward the messages.

Usage

This application is designed for sending data to and from a PostgreSQL database using RabbitMQ and JSON for batch processing. It retrieves records from the database, converts them into JSON format, and publishes them to a message queue for further processing. The combination of PostgreSQL, RabbitMQ, and JSON enables seamless integration with various data-driven applications.

License

This project is licensed under the GPL-3.0 License. See LICENSE.txt for more information.

(back to top)