Production-Ready MLOps Workflow 🚀

An industrialized ML pipeline that transforms a ML model into a scalable, tested, and containerized microservice.

📑 Table of Contents

• ⚡ Quick Start
• 🎯 Project Purpose
• 📁 Project Structure
• 🛠️ Technical Documentation
  • Setup
  • Running Tests
  • Training & Execution
• 🔄 CI/CD & Quality Control
• 🚀 Future & Tech Stack

Quick Start (30 seconds)

If you have Docker installed, you can spin up the entire ecosystem with a single command:

docker compose -f config/docker-compose.yml up --build

• API: http://localhost:5000
• UI: http://localhost:8501

Project Purpose

This project demonstrates production-ready MLOps practices rather than focusing solely on achieving state-of-the-art model performance. The Wisconsin Breast Cancer dataset is used as a proof-of-concept to validate the MLOps infrastructure.

The goal is to showcase best practices in:

• Reproducible ML Pipelines: Using scikit-learn pipelines for consistent preprocessing and inference
• API Design: Building robust REST APIs with proper validation and error handling
• Containerization: Multi-service architecture with Docker Compose
• CI/CD: Automated quality gates, testing, and deployment pipelines
• Code Quality: Type checking, linting, and comprehensive testing

The real question this project answers:

"How do I ensure my ML model works the same way in production as it does in development?"

This project solves this through container immutability and environment parity.

Project Structure

production-ready-mlops-workflow/
├── .github/workflows/         # 🔄 CI/CD: Quality gates & automated deployment
├── config/                    # 🐳 Docker: Multi-container orchestration
├── data/                      # 📊 Dataset storage (raw & processed)
├── models/                    # 🧠 Trained model artifacts (joblib)
├── notebooks/                 # 📓 EDA and experimentation
├── reports/                   # 📈 Generated metrics and figures
├── src/                       # 🛠️ Source code
│   ├── app.py                 # 🌐 Inference API (Flask)
│   ├── schemas.py             # ✅ Data validation (Pydantic)
│   └── model/                 # 🚂 Training and inference logic
├── tests/                     # 🧪 Unit & Integration test suite
└── pyproject.toml             # 📦 Dependency management (uv)

Technical Documentation

Setup

Prerequisites

• Python 3.12+
• Docker and Docker Compose (for containerized deployment)
• uv (recommended) or pip for dependency management

Using uv (Recommended)

uv is a fast Python package installer written in Rust, offering faster dependency resolution and installation.

1. Clone the repository:

   git clone https://github.com/anibalrojosan/production-ready-mlops-workflow
   cd production-ready-mlops-workflow

2. Install uv globally (if needed):

   # On macOS/Linux:
   curl -LsSf https://astral.sh/uv/install.sh | sh
   # Or via pip:
   pip install uv

3. Install dependencies:

   uv sync --all-groups

4. Activate the virtual env (optional):

   source .venv/bin/activate

   Note: you can run commands using uv run if you don't want to activate the virtual env.

Using pip (Alternative)

1. Create and activate virtual environment:

   python -m venv .venv
   # On Windows:
   .\.venv\Scripts\Activate.ps1
   # On Linux/macOS:
   source .venv/bin/activate

2. Upgrade pip:

   pip install --upgrade pip

3. Install dependencies:

   Option A: using the requirements.txt (recommended for production).

   pip install -r requirements.txt

   Option B: using the pyproject.toml (recommended for development).

   pip install .

Running Tests

The project includes comprehensive tests with a coverage requirement of 80%+.

Run all tests:

uv run pytest

Run with verbose output:

uv run pytest -v

Run with coverage report:

uv run pytest --cov=src --cov-report=term-missing

Training the Model

Train the ML pipeline and save the model artifact:

uv run python -m src.model.model_training

This will:

1. Load and preprocess the data
2. Train a Random Forest classifier within a scikit-learn pipeline
3. Evaluate the pipeline
4. Save the complete trained pipeline to models/model.joblib

Note: The model must be trained before running the API.

Running the API

Start the Flask API locally:

uv run python -m src.app

The API will be accessible at http://127.0.0.1:5000/

API Endpoints

The API exposes a POST /predict endpoint that accepts features as JSON and returns the prediction with probabilities. It also includes a GET / health check endpoint to verify service and model status.

For full validation details and data structures, refer to the Pydantic schemas in src/schemas.py.

Example using test scripts:

• Linux/macOS: ./tests/integration/bash_test.sh
• Windows PowerShell: .\tests\integration\powershell_test.ps1

Streamlit UI

The Streamlit application provides an interactive web interface for making predictions:

streamlit run src/streamlit_app.py

Ensure the Flask API is running first. The UI will open at http://localhost:8501.

Docker Deployment

The project uses Docker Compose to orchestrate both the Flask API and Streamlit UI services.

Build and Run

docker compose -f config/docker-compose.yml up --build -d

This will:

• Build optimized images using multi-stage Docker builds
• Start both API and Streamlit services
• Make API available at http://localhost:5000/
• Make Streamlit UI available at http://localhost:8501/

Stop Services

docker compose -f config/docker-compose.yml down

CI/CD & Quality Control

The project implements a continuous integration pipeline that acts as a quality filter (Quality Gates):

1. Static Analysis: ruff for linting and mypy for strict typing.
2. Automated Testing: pytest with a minimum coverage requirement of 80%.
3. Container Security: Multi-stage Docker builds for lightweight and secure images.
4. Integration Tests: Endpoint validation in isolated containers before deployment.

Pipeline Details (GitHub Actions)

The workflow defined in .github/workflows/main.yml includes:

Quality Gates Job

1. Linting: ruff for code style and quality
2. Type Checking: mypy for static type analysis
3. Testing: pytest with coverage reporting
4. Coverage Requirement: 80% minimum (pipeline fails if below)

Build & Deploy Job (runs after quality gates pass)

1. Train Model: Train and save model artifact
2. Build Docker Images: Create optimized container images
3. Push to Docker Hub: Store images in registry
4. Integration Testing: Test services in isolated containers
5. Health Checks: Verify API and UI endpoints
6. Cleanup: Remove test containers

This ensures that only tested and validated code reaches production.

Future & Tech Stack

🔮 Future Improvements

Potential enhancements to further strengthen the MLOps workflow:

• Model Versioning: Implement MLFlow for experiment tracking and model registry.
• Monitoring: Add model performance monitoring and drift detection.
• A/B Testing: Framework for comparing model versions in production.
• Feature Store: Centralized feature management for multiple models.
• Automated Retraining: Scheduled retraining based on data drift or performance degradation.

📚 Technologies Used

• ML Stack: scikit-learn (Pipeline & Models), pandas, joblib.
• Backend & UI: Flask (Inference API), Streamlit (Interactive Dashboard).
• Modern Tooling: uv (Package Manager), ruff (Linter), mypy (Type Checking), pydantic (Validation).
• Testing: pytest, pytest-mock, pytest-cov.
• Infrastructure: Docker, Docker Compose, GitHub Actions.

This project was developed with ❤️ by Anibal Rojo as a proof of concept for a real-world MLOps pipeline.

Remember: The value of this project is in the engineering practices, not the model metrics. These practices ensure your ML models work reliably in production, regardless of the problem domain or dataset complexity.