Skip to content

knownbymanoj/agentic_fleetops_copilot

Repository files navigation

Scania-Inspired Agentic FleetOps Copilot

A conversational AI system for fleet managers that intelligently monitors truck health, predicts component failures using real Scania data, and recommends maintenance + route optimization actions. Built with pydantic-ai, langfuse, XGBoost, FastAPI, and a realistic fleet simulator fed by Component X degradation profiles.


🎯 What You Get

For Fleet Managers

  • "Which trucks are at risk this week?" → Real-time risk ranking from a trained predictive maintenance model
  • "Why is Truck T17 high risk?" → SHAP-based explanations of top contributing features
  • "Can we reschedule T23?" → Route and maintenance recommendations
  • Web chat interface to ask questions in natural language

For Data Scientists / Portfolio

  • Complete ML pipeline: EDA → Feature Engineering → Model Training (cost-sensitive XGBoost)
    • Trained on 1.2GB Scania Component X dataset (23,550 trucks, 107 sensor columns)
    • 99.7% cost reduction vs. baseline (65,396 → 169 on Scania cost matrix)
    • SHAP explainability for model transparency
  • Production-ready deployment:
    • Async FastAPI backend
    • SQLAlchemy + Alembic migrations
    • Pydantic-AI agent framework w/ typed tools
    • Real-time fleet simulator using actual Component X profiles

📊 High-Level Architecture

Phase 1: Training (Notebooks) → Champion XGBoost (99.7% cost reduction)
                ↓
Phase 2: Simulator (Component X profiles + Real-time risk scores)
                ↓
Phase 3: Agents & Services (Fleet data access + PdM inference)
                ↓
Phase 4: FastAPI Endpoints (/api/simulator/state, /api/pdm/shap/vehicle/...)
                ↓
Phase 5: Web UI (pydantic-ai chat interface w/ tool calling)

🚀 Quick Start (2 Commands — ~5 Minutes)

1. Install & Setup

uv sync
pip install 'pydantic-ai-slim[web]'
alembic upgrade head
python -m apps.api.db.seed

2. Start Backend API (Port 8000)

uvicorn apps.api.main:app --reload --port 8000

3. Start Web UI (Port 7932)

dotenv -f .env run uvicorn apps.api.agents.pydantic_web:app --reload --port 7932

Open browser: http://127.0.0.1:7932

Try these immediately:

  • Ask: "Which trucks are at risk?" → Agent calls rank_vehicles_by_risk(top_n=5)
  • Ask: "Tell me about T17" → Agent calls inspect_vehicle("T17") → Get risk + SHAP explanation + chart

📚 Complete Data Flow

Phase 1: ML Training (Notebooks)

01_eda.ipynb — Scania Component X Exploration

  • 23,550 trucks, 107 sensors, 1.2GB raw data
  • Key insight: 9.65% failure rate (9—91 class imbalance)
  • Top sensors: 167_0, 167_8, 459_18 (highest mutual information)

02_feature_engineering.ipynb — Feature Pipeline

  • Last readout + deltas (rate of change) + aggregates (mean/std/min/max)
  • Spec encoding (8 categorical → one-hot encoded)
  • Output: test_features.parquet, train_features.parquet (352+ features)

03_model_training.ipynb → Champion XGBoost

  • Scania cost matrix: Penalizes missed repairs 50x more than false alarms
  • Baseline LR cost: 65,396
  • Champion XGB cost (validation): 169 (99.7% improvement)
  • SHAP analysis: Per-vehicle top features saved as PNG images

Phase 2: Real-Time Simulator

apps/api/simulator/profiles.py — Load Component X Test Data

  • Real truck degradation trajectories from .data/component_x/proc/test_features.parquet
  • Each truck gets a unique history; simulator replays it

apps/api/simulator/simulator.py — FleetSimulator Engine

  • 30 synthetic trucks, each with a Component-X profile
  • Each tick: advance time_step → get feature vector → run PdM model → write snapshot to DB
  • Fallback to heuristic if model missing (robust)

Phase 3: Data Services & Tools

apps/api/services/fleet_service.py — Fleet Queries

  • get_fleet_snapshot() — all vehicles + latest risk
  • detect_delayed_vehicles() — missed ETAs

apps/api/services/pdm_service.py — PdM Inference

  • get_failure_risk(vehicle_code) — risk score + class + probabilities
  • rank_vehicles_by_risk(top_n) — sorted by risk descending
  • explain_risk(vehicle_code) — top features

Phase 4: Agent Orchestration

apps/api/agents/copilot.py — Rule-Based Orchestrator (for non-LLM API calls)

  • Routes queries to appropriate services
  • Ready for LLM integration

apps/api/agents/pydantic_web.py — Pydantic-AI Web Tools (3 focused tools)

  • get_fleet_snapshot() — All vehicles with live risks
  • rank_vehicles_by_risk(top_n) — Top N at-risk trucks
  • inspect_vehicle(vehicle_code) — Comprehensive inspection: risk + SHAP explanation + chart
  • Graceful Langfuse integration: works with or without credentials

Phase 5: FastAPI Backend

apps/api/main.py — HTTP Endpoints

Fleet Status:

  • GET /api/fleet/snapshot — all vehicles (DB-backed)
  • GET /api/fleet/risks?top_n=10 — ranked by risk (DB-backed)
  • GET /api/vehicle/{code} — full history + snapshots

Simulator (Real-Time):

  • GET /api/simulator/state — live fleet state (Component X profiles)
  • GET /api/simulator/vehicle/{code} — single truck live state
  • POST /api/simulator/tick?n=1 — advance simulator, write snapshots

PdM & Explainability:

  • GET /api/pdm/shap — global model metadata + artifacts
  • GET /api/pdm/shap/vehicle/{code}?as_image=trueon-demand SHAP: compute top features + generate PNG chart

Chat:

  • POST /api/chat — copilot query endpoint

🎮 Web UI Demo — 3 Simple Tools Walkthrough

URL: http://127.0.0.1:7932

Available Tools (optimized for clarity; one tool = one action):

  1. get_fleet_snapshot() → All vehicles with live risk scores

    • Returns: Full fleet state from simulator
    • Use when: "Which trucks do we have?"
  2. rank_vehicles_by_risk(top_n=10) → Top N at-risk trucks (sorted by risk descending)

    • Returns: Vehicle codes + risk scores + risk classes
    • Use when: "Which trucks need maintenance?" or "Show top 5 risks"
  3. inspect_vehicle(vehicle_code)Deep inspection of ONE truck (the star tool!)

    • Returns:
      • Live status (fuel %, route progress)
      • Risk score + class + probabilities
      • Top 10 features contributing to risk
      • SHAP explanation with top features + bar chart image URL
    • Use when: "Tell me about truck T17" or "Why is T42 so risky?"
    • Perfect for video demo: Shows risk + explanation in one call

Example Interactive Flow

Question: "Which trucks need maintenance?"
→ Tool rank_vehicles_by_risk(top_n=5) is called.
→ Agent returns: T17 (risk=0.78), T42 (risk=0.65), T08 (risk=0.59), ...

Question: "Why is T17 at 78% risk?"
→ Tool inspect_vehicle("T17") is called.
→ Tool returns:
   {
     "vehicle_code": "T17",
     "status": "in_transit",
     "fuel_level_pct": 0.65,
     "prediction": {"risk_score": 0.78, "risk_class": 4, "probabilities": {...}},
     "shap_explanation": {
       "top_shap_features": [
         {"feature": "time_step", "shap_value": 0.156, ...},
         {"feature": "397_3", "shap_value": 0.089, ...},
         ...
       ],
       "shap_image_url": "/artifacts/api_models/shap_vehicle_T17.png"
     }
   }
→ Agent summarises with all the relevant information needed.

📋 Project Structure

agentic_fleetops_copilot/
├── notebooks/
│   ├── 01_eda.ipynb                   ← Load 10% Scania sample (23.5k trucks)
│   ├── 02_feature_engineering.ipynb   ← Engineer 352+ features
│   └── 03_model_training.ipynb        ← Train champion XGBoost + SHAP
│
├── apps/api/
│   ├── main.py                        ← FastAPI (8 endpoints)
│   ├── db/
│   │   ├── engine.py                  ← SQLAlchemy async
│   │   ├── models.py                  ← ORM (Hub, Vehicle, Route, VehicleRiskSnapshot)
│   │   └── seed.py                    ← Populate 5 hubs, 30 trucks, 8 routes
│   ├── agents/
│   │   ├── copilot.py                 ← Rule-based orchestrator
│   │   └── pydantic_web.py            ← Pydantic-AI web tools (3 focused tools)
│   ├── services/
│   │   ├── fleet_service.py           ← Fleet queries (DB-backed)
│   │   └── pdm_service.py             ← PdM DB lookups (no inline inference)
│   └── simulator/
│       ├── simulator.py               ← FleetSimulator engine (with refactored helpers)
│       └── profiles.py                ← Load Component X test features
│
├── models/                            ← Model artifacts
│   ├── champion_pdm_v1.joblib         ← Trained XGBoost
│   ├── champion_pdm_v1_metadata.json  ← Config + metrics
│   └── shap_class4_*.png              ← Global SHAP charts
│
├── .data/component_x/                 ← Scania dataset (1.2GB, not in repo)
│   └── proc/
│       ├── test_features.parquet      ← Engineered test set (simulator uses this)
│       └── train_features.parquet     ← Engineered training set
│
└── alembic/                           ← Database migrations
    └── versions/
        └── 0eaf77011b27_init_models.py

📊 Key Metrics

Metric Value
Training dataset 23,550 trucks
Sensor columns 107
Failure rate 9.65%
Features after engineering 352+
Baseline LR cost 65,396
Champion XGB val. cost 169
Cost reduction (validation) 99.7% (65,396 → 169)
Test cost (held-out, unbiased) 44,388
Synthetic fleet trucks 30
Hubs 5
Routes 8

Model Performance Note: Validation vs. Production

  • Validation gap: 169 (validation) → 44,388 (test) indicates overfitting
  • Design decision: Despite the gap, the champion model was shipped because the goal is simulator accuracy and integration, not production deployment
  • Rationale:
    • The simulator needs a working PdM model to generate realistic degradation patterns
    • A model that learns detailed patterns from training data is better for realistic simulation than a conservative underfit model
    • The test cost (44,388) is still a 33% improvement over baseline (65,396)
    • Future iterations can apply regularization/L1-L2 tuning, ensemble methods, or data augmentation to close the gap
  • Next steps: If moving to production, focus on reducing overfitting through cross-validation ensembles, simpler models, or additional data collection

🛠️ Development

Verify Setup

# Check liveness
curl http://127.0.0.1:8000/api/health

# Get live fleet state
curl http://127.0.0.1:8000/api/simulator/state | jq .

# Generate SHAP for truck T02
curl "http://127.0.0.1:8000/api/pdm/shap/vehicle/T02?as_image=true" | jq .

# Run simulator 3 steps
curl -X POST "http://127.0.0.1:8000/api/simulator/tick?n=3" | jq .

Troubleshooting

"Model not found": Copy your champion joblib to apps/api/models/champion_pdm_v1.joblib

"DB connection error": Check DATABASE_URL in .env

"Tools not firing": Ensure LLM API key is set (or click tool button directly in web UI)


❓ FAQ

Q: Why only 3 tools? What happened to the 6? A: Simplified for interview clarity. The new inspect_vehicle("T17") tool does everything: returns risk score + probabilities + top 10 features + SHAP explanation + bar chart URL. One call, one story. Perfect for a video walkthrough.

Q: Can I use this without a database? A: Yes! The demo runs with the simulator in-memory. Just run the API endpoints — no need for Supabase. Database is only needed if you want to persist snapshots.

Q: What if Langfuse credentials aren't set? A: The app logs a warning and proceeds without tracing. No crash, no dependency on external services.

Q: How does SHAP work per-vehicle? A: When you call inspect_vehicle("T01"), internally:

  1. Gets T01's current simulated feature vector from the simulator
  2. Loads the champion XGBoost model from disk
  3. Computes SHAP values for those features (top 10 by absolute impact)
  4. Generates a bar chart PNG and returns the URL
  5. Returns everything in one JSON response

Q: Can I swap the LLM? A: Yes. Edit apps/api/agents/pydantic_web.py:

DEFAULT_MODEL = 'google-gla:gemini-3.1-flash-lite-preview'  # or openai:gpt-5.5, etc.

Q: How realistic is the fleet simulation? A: Very realistic for a simulator. Each truck gets a sampled Component X degradation trajectory. As time ticks, the champion model predicts risk based on real sensor patterns. The model uses learned patterns from 23,550 real trucks—even if it overfits on the test set, it captures genuine degradation signatures. Perfect for demos and development; for production risk scoring, use a cross-validated or ensemble model.

Q: Can I use this without the Scania dataset? A: Yes, with fallbacks. The simulator has a heuristic risk scorer that works without the model. If Component X profiles aren't available, it uses deterministic fallback features. Everything degrades gracefully.


⚠️ Model Performance & Design Philosophy

Q: Why is there a 26x gap between validation (169) and test (44,388) cost?

This reveals overfitting: the model learned specific patterns from the training/validation split that don't generalize to the held-out test set. Normally this would be unacceptable for production.

Q: Why ship a model with known overfitting?

Simulator-First Design: The primary goal is not production deployment—it's building a realistic fleet simulator. The overfitted model is actually better for this because:

  1. Realistic degradation patterns: A model that learns fine-grained patterns from training data produces more varied and interesting risk trajectories in the simulator
  2. Diverse fleet behavior: Each simulated truck gets different risk dynamics, making the demo less monotonous
  3. ML playground: Shows the full pipeline: data → training → integration → deployed system (even if the model itself could be improved)
  4. Testable architecture: The system is modular—you can swap in a better model (regularized, ensemble, or production-hardened) without changing the simulator or API

Production Improvements (future work):

  • Reduce overfitting: L1/L2 regularization, reduce feature count via feature selection
  • Ensemble methods: Combine XGBoost with LogisticRegression or LightGBM
  • Data augmentation: More training samples or synthetic features
  • Cross-validation: Use 5-fold CV results instead of single validation split

Bottom line: The system is designed to be upgradeable. The model is a placeholder that works; when you need production-grade accuracy, replace it without changing any agent or API code.


Modular: Each layer (notebooks, simulator, services, agents, API, UI) is independent

Robust: Fallbacks for missing models, DB, features (demo mode always works)

Transparent: SHAP explainability for every prediction

Real Data: Trained on actual Scania Component X; simulator uses test trajectories


Screenshots of chatbot in action

image image image image image image

Made with ❤️ | FleetOps Copilot | 2026

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages