Industrial Security Demo

Edge AI-Powered Industrial Security Monitoring on NVIDIA Jetson

🌐 Language: English | 中文 (Chinese)

Industrial security monitoring demo for Seeed reComputer Industrial series Jetson edge devices: multi-camera RTSP/USB integration, TensorRT FP16 person detection, centroid tracking, interactive zone intrusion/line crossing/loitering rules, SQLite event persistence, and browser-based real-time monitoring dashboard.

Why Edge AI? Project Highlights

Advantage	Description
Data Security, Privacy Compliance	Full-stack local inference, video streams and event data stay within the facility/campus, meeting industrial security and privacy compliance requirements. No need to upload sensitive video to the cloud
TensorRT FP16 Acceleration	Leverages Jetson GPU + TensorRT for FP16 quantized inference. YOLO26n latency is only ~3.7ms (268 QPS), far exceeding cloud solutions in real-time performance
NMS-Free End-to-End Inference	Supports latest Ultralytics YOLO26, natively NMS-free post-processing, further reducing latency, optimized for edge scenarios
GStreamer Hardware Decoding	Jetson NVDEC hardware decoding of RTSP video streams with near-zero CPU overhead
Multi-Camera Support	Simultaneous access to multiple RTSP cameras with independent processing pipelines, shared detection models, and adaptive grid layout on web
Offline Deployment, Low Bandwidth	No internet dependency, suitable for mines, factories, warehouses, construction sites with no/weak network
Flexible Secondary Development	Supports custom-trained models (YOLOv5/v8/v11/v26), custom rules, REST API integration, ready-to-use yet deeply customizable
Interactive Zone Configuration	Draw detection zones directly on video feed in browser, each camera independently configured without modifying config files
Event Persistence	SQLite database stores events, supports historical queries and date filtering, no data loss on restart

Current Test Device

Item	Details
Device	Seeed reComputer Industrial J4012
SoM	NVIDIA Jetson Orin NX 16GB (p3767-0000-super)
JetPack	6.2 (L4T R36.4.3, Ubuntu 22.04)
GPU	Ampere, 1024 CUDA cores, TensorRT 10.3.0

Compatibility: This project adapts to the entire Seeed reComputer Industrial Jetson device series (Orin NX / Orin Nano, etc.), as well as other Jetson platforms running JetPack 6.x.

Feature Overview
System Architecture
Environment & Dependencies
Quick Start
Deployment Guide
Configuration
Multi-Camera Management
HDMI Display & Fullscreen Toggle
Web Zone Drawing
Event Persistence & Query
Models & TensorRT Engine
Custom Models & Development
API & Event Output
Web & Optimized Mode
Command Line Arguments
Troubleshooting
Docker on Jetson

Feature Overview

Capability	Description
Multi-Camera Integration	Supports simultaneous access to multiple RTSP/USB cameras with independent processing pipelines and shared detection models
Camera Auto-Discovery	Automatically scans RTSP cameras in subnet, supports manual add/remove via web
Jetson NVDEC Hardware Decode	GStreamer pipeline decoding reduces CPU usage (can be disabled for software fallback)
Person Detection	Default YOLO26n → TensorRT FP16 (NMS-free end-to-end inference); compatible with YOLOv5/v8/v11 ONNX
Object Tracking	Centroid tracking (`CentroidTracker`), configurable distance and timeout
Behavior Rules	Zone intrusion, line crossing, loitering (supports interactive zone drawing in browser, independently configured per camera)
Web Dashboard	Static page + WebSocket low-latency video stream + real-time config push, adaptive grid layout
Event Persistence	SQLite database stores events, supports date queries, no data loss on restart, automatic cleanup of expired data
Event Logging	`events.jsonl` + `output/<cam-id>/events/` screenshots, supports date filtering
HDMI Fullscreen Display	Auto-detects HDMI and displays fullscreen on startup, press F to toggle fullscreen/window
Camera Health Monitoring	Automatically detects offline cameras and reconnects

System Architecture

RTSP-1 ──► GStreamer NVDEC ──► AsyncCapture ──┐
RTSP-2 ──► GStreamer NVDEC ──► AsyncCapture ──┤
  ...                                          │
                                               ▼
                              YOLO26 TensorRT FP16 Inference
                               (Shared Model, Serial Inference)
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
              Pipeline-1      Pipeline-2       Pipeline-N
              (Track+Rules)   (Track+Rules)    (Track+Rules)
                    │               │               │
                    ▼               ▼               ▼
              EventStore       EventStore       EventStore
              (SQLite)         (SQLite)         (SQLite)
                    │
        ┌───────────┴───────────┐
        ▼                       ▼
  OpenCV Local Display    Web: HTTP + WS
 (HDMI Auto-Fullscreen,  (Video Stream + Config +
  F-Key Toggle)          Camera Management Panel)

Environment & Dependencies

Hardware

Recommended Device: Seeed reComputer Industrial series (Jetson Orin NX / Orin Nano)
Supports any NVIDIA Jetson device running JetPack 6.x
Optional: PoE ethernet port for IP camera connection

Software

JetPack 6.x (Ubuntu 22.04)
Python 3.10+
OpenCV with GStreamer backend
TensorRT 10.x (included with JetPack)
CUDA 12.x (included with JetPack)

Python Additional Packages

pip3 install --user numpy websockets

TensorRT / CUDA / cuDNN are provided by JetPack system, no pip installation needed.

Repository Structure

Industrial-security-demo/
├── app/
│   ├── behavior_demo.py         # Main program: capture, detection, tracking, rules, HDMI display
│   ├── multi_camera_manager.py  # Multi-camera pipeline management, frame buffer, event storage
│   ├── camera_discovery.py      # Camera auto-discovery and manual addition
│   ├── event_store_db.py        # SQLite event persistence storage
│   ├── yolo_trt_detector.py     # TensorRT / DNN detector (v5/v8/v11/v26 auto-detection)
│   ├── web_server.py            # Lightweight HTTP + MJPEG (stdlib)
│   └── web_server_optimized.py  # Optimized WebSocket video + config + camera management API
├── config/
│   └── demo_config.json         # Camera, detector, rules, display, web configuration
├── models/
│   ├── yolo26n.onnx             # Default YOLO26n ONNX (NMS-free)
│   ├── yolo26n_fp16.engine      # TensorRT FP16 engine (built on device)
│   ├── yolov5n.onnx             # Optional YOLOv5n
│   └── yolov8n.onnx             # Optional YOLOv8n
├── web/
│   └── index.html               # Web panel (multi-camera, zone drawing, WS video stream)
├── output/
│   ├── cam-0/                   # Camera 0 event data
│   │   ├── events.jsonl         # Event log
│   │   ├── events.db            # SQLite event database
│   │   └── events/              # Event screenshots
│   └── cam-1/                   # Camera 1 event data
├── scripts/
│   └── probe_camera.py          # RTSP path probe
├── build_yolov8_engine.py       # Optional trtexec build script
├── run_demo.sh                  # One-click startup
└── README.md

Quick Start

cd Industrial-security-demo

# 1. Install dependencies
pip3 install --user numpy websockets

# 2. Verify models exist
ls -la models/

# 3. Start (using config file)
bash run_demo.sh

# Or background + Web only (no local window, suitable for SSH)
python3 app/behavior_demo.py --no-window

Open browser: http://<Jetson-IP>:8080

Quick Self-Check

python3 app/behavior_demo.py --no-window --no-web --max-frames 100

Deployment Guide

1. Network & Camera

Ensure Jetson and camera are on the same network:

ping -c 3 <Camera-IP>

2. Probe RTSP Address

python3 scripts/probe_camera.py --ip <Camera-IP> --user admin --password ""

Write the output rtsp://... to camera.source in config/demo_config.json.

3. Build TensorRT Engine (First Time/Device Change)

/usr/src/tensorrt/bin/trtexec \
  --onnx=models/yolo26n.onnx \
  --saveEngine=models/yolo26n_fp16.engine \
  --fp16

Engine is hardware-bound, cannot be shared between different Jetson devices, must be built on target device.

4. Firewall

sudo ufw allow 8080/tcp
sudo ufw allow 8081/tcp
sudo ufw allow 8082/tcp

5. Long-Term Running

Can register as service using systemd:

python3 app/behavior_demo.py --no-window

Configuration

Configuration file: config/demo_config.json

Cameras `cameras`

Field	Description
`mode`	`manual`: manual configuration; `auto`: auto-scan subnet
`manual`	Manual camera list, each item contains `id`, `name`, `source`, `use_gstreamer`, `enabled`
`auto_discover.subnet`	Subnet to auto-scan, e.g., `192.168.3.0/24`
`auto_discover.username`	RTSP username
`auto_discover.password`	RTSP password
`auto_discover.rtsp_paths`	List of RTSP paths to try
`auto_discover.scan_interval_seconds`	Scan interval (seconds)

Example configuration:

"cameras": {
  "mode": "manual",
  "manual": [
    {
      "id": "cam-0",
      "name": "poe-camera-1",
      "source": "rtsp://admin:@192.168.3.10/Streaming/Channels/101",
      "use_gstreamer": true,
      "enabled": true
    },
    {
      "id": "cam-1",
      "name": "poe-camera-2",
      "source": "rtsp://admin:@192.168.3.20/Streaming/Channels/101",
      "use_gstreamer": true,
      "enabled": true
    }
  ]
}

Can also dynamically add cameras via web "Add Camera" feature without modifying config file.

Detector `detector`

Field	Description
`backend`	`yolov5_trt`: TensorRT inference
`onnx_file`	ONNX file name under `models/`, e.g., `yolo26n.onnx`
`conf_threshold`	Confidence threshold
`iou_threshold`	NMS IoU threshold (used by v5/v8, ignored by v26 NMS-free)
`fp16`	TensorRT FP16 inference
`infer_interval`	Inference every N frames

Rules `rules`

zones: Polygon vertices are normalized coordinates [0,1], can be interactively drawn in browser
lines: start / end are normalized coordinates
event_cooldown_seconds: Cooldown time for same type of event

Features `features`

"features": {
  "human_detect": true,
  "tracking": true,
  "zone_detection": true,
  "line_crossing": true,
  "loitering": false
}

Display `display`

Field	Description
`show_window`	Whether to display OpenCV local window
`window_name`	Window title
`resize_width`	Video resize width (pixels)
`web_jpeg_quality`	Web JPEG encoding quality (1-100, default 50, lower = lower latency)

Multi-Camera Management

Architecture Design

Each camera runs an independent processing pipeline (CameraPipeline), containing:

AsyncCapture: Independent thread for reading video frames
CentroidTracker: Independent tracker
EventStore: Independent event storage (SQLite)
FrameBuffer: Thread-safe frame buffer

All cameras share one TensorRT detection model (SharedDetector), serial inference avoids GPU competition.

Web Management

Camera List: Top tabs switch between different cameras
Adaptive Layout: 1 camera fullscreen, 2 cameras left-right split, 3-4 cameras 2×2 grid
Add Camera: Enter IP, username, password, RTSP path in control panel, click "Probe & Add"
Remove Camera: Remove specified camera via API
Independent Zone Configuration: Each camera has independent detection zones, no interference

Auto-Discovery

After configuring cameras.mode = "auto", system periodically scans RTSP cameras in subnet and automatically adds newly discovered cameras.

Health Monitoring

Each camera pipeline has built-in health check:

No frame for 10 seconds → marked as offline
Automatic reconnection attempt
Web displays camera online/offline status

Statistics Smoothing

System applies dual smoothing optimization to key metrics like detected person count, tracked targets, FPS, etc.:

Backend Sliding Window (StatsCollector):

Maintains queue of last 5 update history records
Adds new value and removes oldest value on each update
Returns queue average (FPS) or average rounded (detection count, track count)
Effectively filters instantaneous fluctuations, data more stable

Frontend Animation Transition (pollStats):

Each update moves only 30% of difference (SMOOTH_FACTOR = 0.3)
Directly displays target value when numerical change < 0.5
Resets smoothing state when switching cameras to avoid displaying old data

Effect: Numerical values change smoothly from sudden jumps to gradual transitions, benefiting both Web and HDMI displays.

HDMI Display & Fullscreen Toggle

Auto-Fullscreen

Automatically detects HDMI display connection status on startup (reads /sys/class/drm/card0-HDMI-A-1/status), auto-fullscreen when HDMI detected.

Fullscreen/Window Toggle

Press F key in HDMI display window to toggle between fullscreen and window mode:

Fullscreen Mode: Suitable for monitoring large screen deployment
Window Mode: Suitable for development debugging

Multi-Camera Layout

HDMI display automatically adapts to camera count:

1 camera: Fullscreen display
2 cameras: Left-right split screen
3-4 cameras: 2×2 grid
More cameras: 3-column grid

Real-time HUD Information

Each camera view overlays real-time status information:

Position	Content	Description
Top-Left	🟢/🔴 Status Indicator	Green=Online, Red=Offline
Top-Right	Camera Name	e.g., `poe-camera-1` (green)
Left Side	FPS	Real-time frame rate (sliding window smoothed)
Left Side	Tracks	Current tracked target count (smoothed)
Left Side	Detections	Current detected person count (smoothed)
Left Side	Events	Cumulative event count

All statistics are dual-smoothed with backend sliding window (last 5 updates) and frontend animation transition (30% gradient), ensuring stable numerical changes without jumps.

Font Adaptation

All HUD text sizes dynamically calculate based on screen width, clear and readable in both fullscreen and window modes, with minimum font protection to avoid negative value errors.

Web Zone Drawing

Drawing Detection Zones

Open browser and visit http://<Jetson-IP>:8080
Select the camera to configure in top tabs
Find "Detection Zone" section in right control panel
Click "Enable Zone Drawing" toggle
Click left mouse button on video feed to draw polygon vertices (at least 3 points)
Automatically saves after drawing completes, or click "Finish Drawing"/"Cancel" button

Important: Each camera's detection zone is independent. When switching cameras, that camera's zone configuration is automatically loaded.

Managing Detection Zones

View Zone List: Drawn zones display in control panel
Delete Single Zone: Click ✕ button next to zone
Clear All Zones: Click "Clear All Zones" button (clears only current camera's zones)

Zone Parameters

Loitering Time: Set loitering detection time threshold (seconds) in "Detection Rules"
Cooldown Time: Minimum interval between same type of events (seconds)

Event Persistence & Query

SQLite Storage

All events automatically written to SQLite database (output/<cam-id>/events.db), supporting:

Persistence: Events not lost on application restart
Date Query: Filter events by date via API or Web
Automatic Cleanup: Events older than 30 days automatically cleaned
Dual-Write Mechanism: Simultaneously writes to memory ring buffer and SQLite, queries prioritize SQLite

Event Data Structure

Each event contains:

Field	Description
`timestamp`	Event timestamp (`YYYYMMDD-HHMMSS` format)
`camera_id`	Camera ID (e.g., `cam-0`)
`camera_name`	Camera name (e.g., `poe-camera-1`)
`event_type`	Event type: `zone_enter`, `loitering`, `line_cross`
`track_id`	Target tracking ID
`zone_name` / `line_name`	Triggered zone/line name
`dwell_seconds`	Loitering duration (loitering events only)
`bbox`	Target bounding box
`centroid`	Target centroid coordinates

Event Screenshots

Each event automatically saves screenshot to output/<cam-id>/events/ directory, maximum 200 retained, automatically cleans oldest when exceeded.

Models & TensorRT Engine

Supported Models

Model	Output Format	NMS	Description
YOLO26n (Default)	`(1, 300, 6)`	Built-in (NMS-free)	Latest architecture, optimal for edge
YOLOv5n	`(1, 25200, 85)`	Post-processing	Classic lightweight
YOLOv8n	`(1, 84, 8400)`	Post-processing	Accuracy/speed balance
YOLO11n	Same as v8	Post-processing	v8 architecture upgrade

Performance Comparison (Jetson Orin NX 16G, FP16)

Model	GPU Latency	Throughput
YOLO26n	3.72ms	268 QPS
YOLOv5n	2.96ms	337 QPS
YOLOv8n	3.89ms	256 QPS

Engine Build

Detector automatically builds engine on first run (calls trtexec), or manually:

/usr/src/tensorrt/bin/trtexec --onnx=models/<model>.onnx --saveEngine=models/<model>_fp16.engine --fp16

Custom Models & Development

Using Custom-Trained YOLO Models

Train YOLO model on any machine and export ONNX:

from ultralytics import YOLO
model = YOLO("yolo26n.pt")
model.train(data="your_dataset.yaml", epochs=100)
model.export(format="onnx", imgsz=640)

Copy exported .onnx to models/ directory
Modify detector.onnx_file in config/demo_config.json
TensorRT engine automatically builds on first run

Extension Development

New Detection Backend: Extend in create_detector() in app/behavior_demo.py
New Rules: Add event types in BehaviorDemo._apply_rules
Frontend Customization: Modify HTML/CSS/JS under web/, refresh browser
API Integration: Use REST API to get real-time data, integrate with upper-level platforms

API & Event Output

Endpoint	Method	Description
`/api/cameras`	GET	Camera list (ID, name, status)
`/api/cameras/<id>/stream`	GET	MJPEG video stream for specified camera
`/api/cameras/<id>/stats`	GET	Real-time statistics for specified camera
`/api/cameras/<id>/events`	GET	Event list for specified camera
`/api/cameras/add`	POST	Add camera (requires IP, username, password, etc.)
`/api/cameras/remove`	POST	Remove camera (requires `camera_id`)
`/api/cameras/discover`	GET	Trigger camera auto-discovery
`/api/cameras/probe`	POST	Probe single camera reachability
`/api/stats`	GET	Global statistics (aggregates all cameras)
`/api/events`	GET	Event list, supports `?date=YYYYMMDD` filter
`/api/events/images`	GET	Event screenshot list, supports `?date=YYYYMMDD` filter
`/api/events/img/<cam>/<name>`	GET	Event screenshot image
`/api/events/clear`	POST	Clear event log
`/api/config`	GET/POST	Read/update runtime configuration
`/api/rules`	POST	Update detection rules (zones, lines, etc.), supports per-camera configuration
`/api/models`	GET	Available model list
`/api/model/switch`	POST	Switch model at runtime

WebSocket ports:

:8081 — Video stream (binary JPEG frames)
:8082 — Config channel (JSON bidirectional)

Web & Optimized Mode

After installing websockets, optimized mode automatically enables:

Multi-Camera Grid: Adaptive 1/2/3 column layout, tab switching
Dual WebSocket: Video stream and config channel separated, no blocking
Binary Frame Protocol: Video frames transmitted via binary WebSocket, containing camera ID and timestamp
Dynamic JPEG Quality: Configure web_jpeg_quality to adjust web quality
Interactive Zone Drawing: Draw polygon detection zones directly on video, independently configured per camera
Feature Toggles: Real-time toggle detection/tracking/zone/line-crossing/loitering in browser
Real-time Event Stream: Event list with camera tags, filter by date
Camera Management: Add/remove cameras via web, probe reachability
Auto-Reconnect: Automatic reconnection after WebSocket disconnection (exponential backoff)

If websockets unavailable, automatically falls back to HTTP MJPEG mode.

Command Line Arguments

Argument	Description
`--config`	Configuration file path, default `config/demo_config.json`
`--source`	Override video source in configuration
`--max-frames`	Exit after running N frames
`--no-window`	Do not display OpenCV window
`--no-web`	Do not start Web service
`--web-port`	Override HTTP port

Troubleshooting

Symptom	Action
`Cannot open video source`	Check RTSP URL, ping camera, use `probe_camera.py`
TensorRT initialization failed	Verify engine file was built on current device
`Address already in use`	Use `--web-port` to change port or `ss -ltnp \| grep 8080`
YOLO26 DNN fallback failed	YOLO26 NMS-free requires TensorRT, does not support OpenCV DNN fallback
No display / DISPLAY	Use `--no-window` when SSH, use Web only
Zone/line not triggering	Check `zone_detection`/`line_crossing` toggle in `features`
HDMI window not fullscreen	Check DISPLAY environment variable, ensure X11 service running normally; press F key to manually toggle fullscreen
Web lag	Reduce `web_jpeg_quality` (default 50), check network bandwidth
Events not displaying	Check if detection zones are drawn and `zone_detection`/`loitering` features enabled
Camera offline	Check network connection, system will automatically reconnect; check camera status on web
Zone drawing affects other cameras	Each camera zone stored independently, corresponding configuration automatically loaded when switching cameras

Docker on Jetson

Image Features

No NGC Login Required: Based on ubuntu:22.04
Minimal Size: ~333 MB compressed
High Adaptability: Not bound to specific L4T version

Quick Start

# Build
docker build --network=host -t industrial-security-demo:latest .

# Run
docker compose up -d

# Access
http://<Jetson-IP>:8080

Offline Deployment

# Export
bash scripts/docker-export.sh industrial-security-demo:latest ./industrial-security-demo.tar.gz

# Import
gunzip -c industrial-security-demo.tar.gz | docker load

Name		Name	Last commit message	Last commit date
Latest commit History 19 Commits
app		app
config		config
gallery		gallery
models		models
output		output
scripts		scripts
web		web
.dockerignore		.dockerignore
.gitignore		.gitignore
DOCKER.md		DOCKER.md
DOCKER.zh.md		DOCKER.zh.md
Dockerfile		Dockerfile
README.md		README.md
README.zh.md		README.zh.md
build_yolov8_engine.py		build_yolov8_engine.py
docker-compose.yml		docker-compose.yml
install-docker-jetson.sh		install-docker-jetson.sh
requirements-docker.txt		requirements-docker.txt
requirements.txt		requirements.txt
run_demo.sh		run_demo.sh

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

Industrial Security Demo

Why Edge AI? Project Highlights

Current Test Device

Table of Contents

Feature Overview

System Architecture

Environment & Dependencies

Hardware

Software

Python Additional Packages

Repository Structure

Quick Start

Quick Self-Check

Deployment Guide

1. Network & Camera

2. Probe RTSP Address

3. Build TensorRT Engine (First Time/Device Change)

4. Firewall

5. Long-Term Running

Configuration

Cameras cameras

Detector detector

Rules rules

Features features

Display display

Multi-Camera Management

Architecture Design

Web Management

Auto-Discovery

Health Monitoring

Statistics Smoothing

HDMI Display & Fullscreen Toggle

Auto-Fullscreen

Fullscreen/Window Toggle

Multi-Camera Layout

Real-time HUD Information

Font Adaptation

Web Zone Drawing

Drawing Detection Zones

Managing Detection Zones

Zone Parameters

Event Persistence & Query

SQLite Storage

Event Data Structure

Event Screenshots

Models & TensorRT Engine

Supported Models

Performance Comparison (Jetson Orin NX 16G, FP16)

Engine Build

Custom Models & Development

Using Custom-Trained YOLO Models

Extension Development

API & Event Output

Web & Optimized Mode

Command Line Arguments

Troubleshooting

Docker on Jetson

Image Features

Quick Start

Offline Deployment

License & Disclaimer

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Cameras `cameras`

Detector `detector`

Rules `rules`

Features `features`

Display `display`

Packages