EV Drone Inspection — Control Center

Autonomous warehouse drone inspection, simulated end-to-end: a PX4 quadcopter flies through a Gazebo warehouse of EV battery shelving, detects defects with YOLO in real time, and generates an inspection report — all driven from an interactive browser dashboard.

What it does

• Warehouse Layout Editor — build and edit the warehouse from a 2D top-down canvas. Spawn, move, and delete shelves/objects live in the running Gazebo sim, or bake the layout into a .world file for a cold restart.
• Mission Planner — edit inspection waypoints on the map and run the autonomous inspection route.
• Live Flight Control — connect, take off, click anywhere on the map to fly there, land, RTL — with a live MJPEG camera feed from the drone.
• Vision pipeline — YOLO defect detection runs in real time during flight; deeper image analysis with a local VLM (Qwen2.5-VL via Ollama) runs after landing to avoid GPU contention with the sim. A PDF inspection report is generated at the end.

Architecture

PX4 SITL + Gazebo Classic 11 (warehouse world, iris + FPV camera)
        │  MAVSDK (udp://:14540)                │  ROS2 Humble
        ▼                                        ▼
control/drone_controller.py            ros2_ws/camera_bridge
  persistent 20 Hz offboard              camera frames → disk/MJPEG
  setpoint streamer                      live scene spawn/delete
        │                                        │
        └────────────► dashboard/hub.py ◄────────┘
                       (ControlHub: drone + layout + mission state)
                                │ FastAPI + REST + MJPEG
                                ▼
                dashboard/static/index.html  (vanilla JS, 3 tabs)
                        http://localhost:8765

Key components:

Path	Role
control/drone_controller.py	MAVSDK flight controller. A single persistent offboard setpoint streamer (20 Hz) with a smooth "carrot" trajectory (~1.5 m/s, yaw into travel direction) and a divergence guard (altitude/distance limits → hold and land).
world/layout.py	JSON warehouse layout ⇄ Gazebo SDF. Generates worlds with gazebo_ros plugins baked in for live editing.
world/scene_client.py	Live world editing client — spawn/move/delete models in the running sim via the ROS2 bridge.
dashboard/	FastAPI server (server.py), central state hub (hub.py), and the single-page frontend.
vision/	YOLO defect detector (runs in an executor so it never blocks the flight control loop) + VLM visual analyst.
agent/	LLM mission agent and prompts (local model via Ollama's OpenAI-compatible API).
report/	PDF inspection report generator (ReportLab).
simulator/worlds/	Pre-built warehouse worlds (full + lite).
ros2_ws/	ROS2 workspace with the camera_bridge package (camera + scene nodes).

Prerequisites

• Ubuntu 22.04
• PX4-Autopilot built for SITL with Gazebo Classic 11
• ROS2 Humble (ros-humble-desktop, ros-humble-gazebo-ros-pkgs, ros-humble-cv-bridge)
• Python 3.10+
• Optional: Ollama running Qwen2.5-VL for post-flight visual analysis

pip install -r requirements.txt
pip install fastapi uvicorn openai

# build the ROS2 workspace
cd ros2_ws
colcon build --symlink-install
source install/setup.bash

Note: run_sim.sh, start_inspection.sh, and the .desktop launcher contain machine-specific absolute paths (project root, PX4 checkout, Python interpreter). Edit those variables at the top of each script for your setup. config.py points YOLO_MODEL at a custom-trained defect model; it falls back to standard YOLO weights (auto-downloaded by ultralytics) if you don't have one.

Running

One-click: run start_inspection.sh (or install the included .desktop launcher). It brings up the full stack in order and tears everything down when you close the terminal.

Manual (3 terminals):

# 1. PX4 SITL + Gazebo warehouse
./run_sim.sh

# 2. ROS2 bridge (camera feed + live world editing)
source /opt/ros/humble/setup.bash && source ros2_ws/install/setup.bash
ros2 run camera_bridge bridge_all

# 3. Control Center (opens browser at http://localhost:8765)
python run_control_center.py

Nothing auto-flies — connect and drive it from the dashboard.

Design notes

• Flight stability: PX4 offboard mode fails over if the setpoint stream ever starves. The controller therefore runs one persistent asyncio setpoint loop for the whole session; waypoint changes are non-blocking pointer swaps, and all heavy work (YOLO inference) runs in a thread executor off the event loop.
• GPU budget: designed for a 4 GB laptop GPU — the Gazebo GUI client is killed shortly after launch (headless sim keeps running), camera streams at 10 Hz, and the VLM analysis is batched after landing instead of competing with the sim in real time.
• Safety rails: minimum safe altitude clamp on all waypoints, divergence guard (altitude > 12 m or distance > 25 m from target → hold, then land), and the LLM agent never sits in the control loop — it only plans and analyzes.