Skip to content

QUICKSTART.md

Latest commit

 

History

History
266 lines (199 loc) · 6.47 KB

QUICKSTART.md

File metadata and controls

266 lines (199 loc) · 6.47 KB

🚀 Quick Start Guide

This guide will get you up and running with the Real-time Anomaly Detection system in minutes.

Prerequisites Check

# Check Python version (need 3.9+)
python --version

# Check if CUDA is available (optional, for GPU training)
python -c "import torch; print(f'CUDA Available: {torch.cuda.is_available()}')"

Installation (5 minutes)

# 1. Clone repository (or navigate to project)
cd "d:\Upload Projects\AI projects\Real-time Anomaly Detection on Edge via Quantized DL"

# 2. Create virtual environment
python -m venv venv

# 3. Activate virtual environment
# Windows:
venv\Scripts\activate
# Linux/Mac:
# source venv/bin/activate

# 4. Install dependencies
pip install --upgrade pip
pip install -r requirements.txt

# 5. Install package
pip install -e .

Quick Training (10-30 minutes)

Option A: Synthetic Data (Quick Start)

# 1. Generate synthetic training data (2-3 minutes)
python scripts\generate_synthetic_data.py

# 2. Train model (10-20 minutes on GPU, longer on CPU)
python train.py

# 3. Evaluate model
python evaluate.py --checkpoint checkpoints\best_model.ckpt

Option B: Real CWRU Data (Recommended for Production)

# 1. Download CWRU dataset (automatic on first run)
python scripts\download_dataset.py

# 2. Train model
python train.py --config config\config.yaml

# 3. Calculate optimal threshold
python scripts\calculate_threshold.py --checkpoint checkpoints\best_model.ckpt

# 4. Evaluate
python evaluate.py --checkpoint checkpoints\best_model.ckpt

Deployment (5 minutes)

Export to ONNX (Cross-Platform)

# 1. Export model
python export_onnx.py --checkpoint checkpoints\best_model.ckpt --output exports\model.onnx

# 2. Test inference
python inference.py --model exports\model.onnx --backend onnx --input data\test_sample.npy

Run API Server

# Start API server
$env:MODEL_PATH="exports\model.onnx"
$env:BACKEND="onnx"
$env:THRESHOLD="0.1"
python api\app.py

# In another terminal, test API:
curl -X POST http://localhost:8000/predict -H "Content-Type: application/json" -d "{\"features\": [0.1, ...]}"

# Or visit http://localhost:8000/docs for interactive API documentation

Docker Deployment (Production)

# Build image
docker build -t edge-anomaly-detection:latest .

# Run container
docker run -d --name anomaly-api -p 8000:8000 edge-anomaly-detection:latest

# Or use docker-compose (includes monitoring)
docker-compose up -d

# Access services:
# - API: http://localhost:8000
# - Prometheus: http://localhost:9090
# - Grafana: http://localhost:3000 (admin/admin)

Common Commands

Training

# Basic training
python train.py

# Resume from checkpoint
python train.py --resume checkpoints\last.ckpt

# Custom configuration
python train.py --config config\custom_config.yaml

# Enable quantization-aware training
python train.py --qat

Quantization

# Post-Training Quantization (PTQ)
python quantize_ptq.py --checkpoint checkpoints\best_model.ckpt

# Export quantized model to ONNX
python export_onnx.py --checkpoint checkpoints\quantized_ptq.ckpt

Inference

# PyTorch inference
python inference.py --model checkpoints\best_model.ckpt --backend pytorch --input data\test.npy

# ONNX inference (faster)
python inference.py --model exports\model.onnx --backend onnx --input data\test.npy

# TensorRT inference (fastest, NVIDIA only)
python export_tensorrt.py --onnx exports\model.onnx --fp16
python inference.py --model exports\model.engine --backend tensorrt --input data\test.npy

Monitoring

# View training logs
tensorboard --logdir logs\tensorboard

# Check Prometheus metrics
curl http://localhost:8000/metrics

# View Grafana dashboards
# Open http://localhost:3000
# Default credentials: admin/admin

Troubleshooting

CUDA Out of Memory

# Reduce batch size in config/config.yaml
training:
  batch_size: 128  # Default is 256

Slow Training

# Enable mixed precision training (already enabled by default)
training:
  use_amp: true
  amp_dtype: "float16"

Module Import Errors

# Reinstall package
pip install -e .

API Not Responding

# Check if model path is correct
echo $env:MODEL_PATH  # Windows
# echo $MODEL_PATH  # Linux/Mac

# Check logs
docker logs anomaly-api

Project Structure Quick Reference

📁 Key Files:
├── train.py                    # Main training script
├── inference.py                # Inference script (all backends)
├── evaluate.py                 # Model evaluation
├── export_onnx.py              # ONNX export
├── export_tensorrt.py          # TensorRT export
├── quantize_ptq.py             # Post-training quantization
├── docker-compose.yml          # Multi-service deployment
│
📁 Configuration:
├── config/config.yaml          # Main configuration
│
📁 Source Code:
├── src/
│   ├── models/                 # Model architectures
│   ├── data/                   # Data loading & preprocessing
│   ├── training/               # Training utilities
│   ├── quantization/           # PTQ & QAT
│   ├── signal_processing/      # FFT, wavelets, entropy
│   └── utils/                  # Helper utilities
│
📁 Scripts:
├── scripts/
│   ├── generate_synthetic_data.py
│   └── calculate_threshold.py
│
📁 API:
└── api/
    └── app.py                  # FastAPI server

Next Steps

  1. Customize Configuration: Edit config/config.yaml for your use case
  2. Experiment Tracking: Enable MLflow or Weights & Biases in config
  3. Hyperparameter Tuning: Adjust model architecture, learning rate, etc.
  4. Edge Deployment: Deploy to Jetson Nano or Raspberry Pi
  5. Real Data: Replace synthetic data with real sensor data
  6. Production Monitoring: Set up Grafana dashboards for your metrics

Getting Help

  • 📧 Email: mmmonani747@gmail.com
  • 🐛 Issues: Create an issue on GitHub
  • 📚 Documentation: See /docs folder for detailed guides
  • 💬 Discussions: Use GitHub Discussions for questions

Performance Expectations

Platform Model Latency Throughput
NVIDIA A4000 INT8 TensorRT ~0.5ms 2000 samples/s
Jetson Nano INT8 TensorRT ~12ms 83 samples/s
Raspberry Pi 4 ONNX CPU ~78ms 13 samples/s
Intel i7 CPU ONNX ~15ms 67 samples/s

Happy Coding! 🚀

For detailed documentation, see README.md