Pinned Loading

1. airflow-postgres-bigquery-pipeline airflow-postgres-bigquery-pipeline Public

   Automates ETL pipelines using Apache Airflow. Ingests data to PostgreSQL hourly and loads daily data to BigQuery with incremental and partitioned processing. Implements core data engineering practi…

   Python

2. realtime-purchase-streaming realtime-purchase-streaming Public

   Real-time data pipeline using Google Cloud Pub/Sub and Apache Beam (Dataflow) to stream and transform dummy purchase data into BigQuery. Focuses on cloud-native streaming architecture, reliable pip…

   Python

3. nyc-taxi-airflow-dbt-gcp nyc-taxi-airflow-dbt-gcp Public

   A modern ELT data pipeline built with Apache Airflow (orchestration), dbt (incremental transformations), and Pandas to process NYC Taxi data from raw Parquet files into a scalable data warehouse on…

   Python

4. ramadiansyahIM ramadiansyahIM Public

   This repository contains my Final Project from 2009, where I developed a Yahoo! Messenger application using J2ME and jYMSG (which I stripped down). At the time, Yahoo! Messenger was a widely popula…

   Java 1

5. asetku-api-scraper-to-bigquery asetku-api-scraper-to-bigquery Public

   Automated ETL pipeline to scrape data from AsetKu API, process it using Python/Pandas, and load the results into Google BigQuery, orchestrated by Apache Airflow.

   Python

6. ma-web-scrapping-to-bigquery ma-web-scrapping-to-bigquery Public

   ma-web-scrapping-to is an Airflow-orchestrated Python project for scraping and processing public court decision data from the Indonesian Supreme Court (MA) website. It uses Beautiful Soup for data …

   Python