Logistics Analytics Portal

A data analytics project focused on analyzing air logistics performance using real-world U.S. transportation and weather data. The goal is to study delays, cancellations, route efficiency, and the impact of weather conditions on logistics operations.

---

Project Objectives

• Analyze flight-level logistics performance
• Identify delay and cancellation patterns
• Evaluate weather impact on operations
• Build analytics-ready datasets
• Create interactive Power BI dashboards

---

Prerequisites

Before running any scripts in this project, ensure the following are installed on your system:

• Python 3.9 or higher
• pip (Python package manager)

Verify your Python installation:

python --version

---

Install Python Dependencies

All required Python libraries are listed in requirements.txt.

From the project root, install dependencies using:

pip install -r requirements.txt

This installs stable versions of the following libraries:

• requests
• pandas
• numpy
• pyarrow
• sqlalchemy
• python-dotenv

---

Download Raw Data

This project includes a utility script to download raw BTS flight data and NOAA weather data.

Script Location

scripts/download_data.py

How to Run

From the project root directory:

python scripts/download_data.py

Script Behavior

• Creates required folders under data/raw/
• Downloads data only if:
  • the file does not exist, or
  • the file exists but is empty, or
  • the remote file is larger than the local file
• Skips download if the local file is already up to date
• Does not unzip or rename any files
• Safe to re-run multiple times

---

Data Sources

This project uses free, publicly available U.S. government datasets.

BTS – On-Time Performance (Flights)

• Source: U.S. Department of Transportation (BTS)
• Link: https://www.transtats.bts.gov/PREZIP/
• Years used: 2023–2025

NOAA – Daily Weather (GHCN-Daily)

• Source: National Oceanic and Atmospheric Administration (NOAA)
• Link: https://www.ncei.noaa.gov/pub/data/ghcn/daily/by_year/
• Years used: 2023–2025

Large data files are not committed to GitHub.
See docs/DATA.md or run the Python script located at scripts/download_data.py to fetch the data.

---

High-Level Architecture

Public Data (BTS, NOAA)
        ↓
Local Raw Data (not versioned)
        ↓
ETL Processing (Python)
        ↓
Analytics Tables
        ↓
Power BI Dashboards

---

Repository Structure

logistics-analytics-portal/
│
├── .github/                           # GitHub Actions, templates
│
├── backend/                           # Python backend (API / services)
│   ├── app/
│   │   ├── main.py
│   │   ├── api/                       # route handlers
│   │   ├── core/                      # config, logging, settings
│   │   ├── db/                        # DB connections/models (future)
│   │   └── services/                 # business logic
│   └── tests/
│
├── frontend/                          # React/Vite frontend (UI)
│   ├── src/
│   ├── public/
│   ├── package.json
│   └── vite.config.*
│
├── etl/                               # ETL pipeline (Python)
│   ├── extract/
│   ├── transform/
│   ├── load/
│   └── validation/
│
├── scripts/                           # One-off utilities
│   └── download_data.py               # downloads BTS + NOAA into data/raw/
│
├── data/                              # ❌ NOT tracked (gitignored)
│   ├── raw/
│   │   ├── bts_air_on_time/
│   │   │   ├── 2023/
│   │   │   ├── 2024/
│   │   │   └── 2025/
│   │   └── noaa_weather_data/
│   │       ├── 2023.csv.gz
│   │       ├── 2024.csv.gz
│   │       ├── 2025.csv.gz
│   │       ├── ghcnd-stations.txt
│   │       └── readme.txt
│   ├── processed/
│   └── curated/
│
├── bi/                                # Business Intelligence artifacts
│   └── powerbi/
│       ├── pbix/
│       ├── exports/
│       ├── model/
│       └── deployment/
│
├── docs/                              # 📘 ALL documentation lives here
│   ├── README.md                      # main project documentation
│   ├── DATA.md                       # data sources & download steps
│   ├── ARCHITECTURE.md               # high-level system design
│   ├── SCHEMA.md                     # star schema & tables
│   ├── JOIN_LOGIC.md                 # BTS ↔ NOAA join logic
│   └── ETL_DESIGN.md                 # ETL design (no code)
│
├── docker/                            # Containerization (optional)
│   ├── Dockerfile
│   └── docker-compose.yml
│
├── .env.example
├── .gitignore
├── LICENSE
└── requirements.txt                   # shared Python deps (scripts / etl / backend)

---

Design Principles

• Clear separation of concerns
• Raw data excluded from version control
• Documentation-first approach
• Scalable and reproducible structure
• Analytics-focused design

---

Business Intelligence

Power BI dashboards and related files are stored under:

bi/powerbi/

This includes dashboard files, exports, and model documentation.

---

Important Notes

• The data/ directory is excluded from Git version control
• Raw data files should never be committed to GitHub
• All transformations and analytics are performed in later ETL stages (etl/)
• The data/ directory is intentionally excluded from Git
• Large datasets are stored locally or via GitHub Releases
• This project is intended for learning, demonstration, and academic evaluation