Skip to content

README.md

Latest commit

 

History

History
122 lines (91 loc) · 4.93 KB

README.md

File metadata and controls

122 lines (91 loc) · 4.93 KB

TaskFlow: High-Concurrency Microservice Architecture 🚀

Python 3.13 FastAPI Docker Redis Architecture

A high-performance asynchronous task management system designed to handle concurrent workloads using Event-Driven Architecture.

📖 Executive Summary

TaskFlow is a backend microservice built to solve the scalability issues inherent in traditional synchronous web applications.

  • ⚡ Non-blocking Performance: Leverages FastAPI’s asynchronous capabilities combined with Redis Pub/Sub to handle high-concurrency workloads without thread blocking.
  • 🔄 Real-time Synchronization: Implements WebSocket-based event broadcasting to ensure immediate data consistency across distributed clients.
  • 🏭 Architecture: Adopts a Layered Architecture (Service-Repository pattern) and Background Processing to simulate a robust, enterprise-ready environment.

🏗 System Architecture

The system follows a Service-Repository Pattern to decouple business logic from the data access layer, ensuring maintainability and testability.

graph TD
    Client["Client (Web/Mobile)"] -->|HTTP/WebSocket| LB["API Gateway / Load Balancer"]
    LB --> API["FastAPI Server"]
    
    subgraph "Application Layer"
        API --> Auth["Auth Service (JWT)"]
        API --> Task["Task Service"]
        API --> Socket["WebSocket Manager"]
    end
    
    subgraph "Data & Event Layer"
        Task -->|Read/Write| DB[("MySQL 8.0")]
        Task -->|Cache Hit/Miss| Cache[("Redis Cache")]
        Task -->|Publish Event| PubSub[("Redis Pub/Sub")]
        PubSub -->|Subscribe| Socket
        Socket -->|Broadcast Updates| Client
    end
Loading

🔧 Tech Stack & Rationale

Component Technology Engineering Decision
Core Framework FastAPI Chosen for its native async/await support (ASGI) and high performance comparable to Node.js and Go.
Database MySQL 8.0 Utilized for robust relational data mapping and ACID compliance for critical user data.
ORM SQLAlchemy (Async) Implemented Asynchronous Sessions to prevent thread blocking during database transactions, maximizing throughput.
Message Broker Redis Pub/Sub Selected to handle real-time event distribution and decouple the notification service from the main request lifecycle.
Containerization Docker Compose Ensuring environment consistency across development and production.

📂 Project Structure

I adopted a modular directory structure to adhere to Domain-Driven Design (DDD) principles.

TaskFlow/
├── app/
│   ├── api/            # API Routers (Controllers)
│   ├── core/           # Global Configs & Security (JWT, CORS)
│   ├── db/             # Async Database Sessions & Connection Pools
│   ├── models/         # SQLAlchemy ORM Models
│   ├── schemas/        # Pydantic DTOs (Data Transfer Objects)
│   ├── services/       # Business Logic Layer
│   └── repositories/   # Data Access Layer (CRUD)
├── tests/              # PyTest Suites (Unit & Integration)
├── docker-compose.yml  # Infrastructure Orchestration
└── requirements.txt    # Dependencies

🛠 Getting Started

Prerequisites

  • Docker & Docker Compose
  • Python 3.13+ (for local env)

Installation & Run

  1. Clone the Repository

    git clone https://github.com/your-username/TaskFlow.git
cd TaskFlow

  2. Environment Setup Create a .env file based on the example.

    cp .env.example .env

  3. Run with Docker (Recommended) Spins up FastAPI, MySQL, and Redis containers.

    docker-compose up -d --build

  4. Explore the API

🧪 Testing Strategy

Reliability is ensured through rigorous testing using PyTest.

  • Unit Tests: Validate individual business logic functions.
  • Integration Tests: Verify interactions between the API, Database, and Redis using TestClient.
  • Fixture Management: utilized conftest.py to manage async DB sessions and test data lifecycles.
# Run full test suite
docker-compose exec app pytest -v