Pinned Loading

1. Apache-Airflow-Kafka-Spark-DeltaLake-Real-Time-Stream-Pipeline Apache-Airflow-Kafka-Spark-DeltaLake-Real-Time-Stream-Pipeline Public

   This project implements a real-time data pipeline using Apache Airflow, Kafka, Apache Spark, and Delta Lake. It supports both batch (Coldpath) and real-time (Hotpath) data ingestion, processing, an…

   Python

2. FMA-A-Dataset-For-Music-Analysis FMA-A-Dataset-For-Music-Analysis Public

   Scripts for music feature analysis, model training, and real-time recommendation using Apache Kafka. Extract features, store them in MongoDB, and process the data with Apache Spark. A web interfac…

   Python 3 1

3. Apache-Flink-Spark-Data-Streaming Apache-Flink-Spark-Data-Streaming Public

   This project showcases a real-time data streaming pipeline using Apache Flink, Apache Spark, and Grafana. It streams data, stores it in Parquet format, and performs aggregations for insights, with …

   Python

4. Apache-Kafka-and-Frequent-Item-sets Apache-Kafka-and-Frequent-Item-sets Public

   This Bash script automates the setup and execution of a data processing pipeline using Apache Kafka and Python scripts, ensuring fault tolerance and streamlined management of Kafka-based data pipel…

   Python 3

5. MeshJoin-Streaming-ETL-Data-Warehouse MeshJoin-Streaming-ETL-Data-Warehouse Public

   MeshJoin-Streaming-ETL-Data-Warehouse integrates real-time transactional data with master data using the Mesh Join algorithm. It processes and enriches data, then loads it into a data warehouse for…

   Java

6. Deep-Learning-Grid-World-Q-Learning Deep-Learning-Grid-World-Q-Learning Public

   Deep Learning Grid World Q-Learning . Implement Q-learning in a 5x5 grid where an agent navigates obstacles and rewards. Train the agent with varying learning rates, visualize its progress, and see…

   Python