Multi-Sensor Multi-Target Tracking for Harbour Surveillance

DTU course 34763 — Autonomous Marine Robotics, spring 2026.

A Python implementation of a real-time multi-sensor multi-target tracking system for harbour surveillance. The system fuses data from four sensors (mm-wave radar, stereo camera, AIS receiver, GNSS) using an Extended Kalman Filter with Hungarian data association.

---

Setup

pip install -r requirements.txt

Requirements: numpy, matplotlib, scipy, pytest.

---

Project structure

src/
  cfm/          Coordinate Frame Manager — sensor geometry, h(x), H, R
  ekf/          EKF, CV motion model, track lifecycle, tracker, metrics
  real_data/    Experimental data loader and coordinate conversions

examples/
  validate_scenario_A.py    Radar-only single-target EKF
  validate_scenario_B.py    Radar + camera fusion comparison
  validate_scenario_C.py    AIS asynchronous fusion and dropout
  validate_scenario_D.py    Multi-target crossing trajectories (MOTP/CE)
  validate_scenario_E.py    Mixed AIS/non-AIS harbour traffic (MOTP/CE)
  run_real_data.py          Phase 4 — real Copenhagen harbour data

data/
  simulated/    scenario_A.json … scenario_E.json
  experimental/ mm_wave_radar.csv, camera.csv, ais.csv, gnss.csv

notebooks/
  harbour_simulation.ipynb  Simulation environment (DT_TRUE = 1/30 s)

tests/          83 unit tests (pytest)
out/            Validation figures for all scenarios and real data
docs/           Project brief PDF

---

Running the validators

# Simulated scenarios
python -m examples.validate_scenario_A
python -m examples.validate_scenario_B
python -m examples.validate_scenario_C
python -m examples.validate_scenario_D
python -m examples.validate_scenario_E

# Save figures to out/
python -m examples.validate_scenario_D --save out/
python -m examples.validate_scenario_E --save out/

# Real data (Phase 4)
python -m examples.run_real_data --save out/

# Tests
python -m pytest tests/

---

Simulation results

All five scenarios use pre-generated JSON files in data/simulated/. Default sensor parameters: radar σ_r = 5 m, σ_φ = 0.3°; camera σ_r = 8 m, σ_φ = 0.15°; AIS σ = 4 m.

Scenario	Description	Key metric	Result
A	Single target, radar only	RMSE < 12 m, NIS ≥ 90%	RMSE 5.3 m, NIS 97% ✅
B	Single target, radar + camera	RMSE comparison seq vs joint	Both architectures consistent ✅
C	AIS target, 30 s dropout	Track survives dropout	Smooth re-acquisition ✅
D	4 crossing targets	MOTP < 15 m, CE < 0.5	MOTP 3.3 m, CE 0.27 ✅
E	6 targets, mixed AIS/non-AIS	MOTP < 20 m, CE < 1.0	MOTP 2.82 m, CE 0.18 ✅

---

Key design choices

Data association — Hungarian / GNN: fusion_mode="gnn" in TrackerConfig. Replaces greedy nearest-neighbour with globally optimal assignment using scipy.optimize.linear_sum_assignment. Prevents false alarms from stealing associations during crossing scenarios.

Track lifecycle — M-of-N: default M = 3, N = 5. Deferred two-point initiation (velocity estimated from first two detections). Coasting on missed detections, deletion after K_del = 5 consecutive misses, duplicate merging via Mahalanobis distance.

Multi-sensor fusion: sequential update (radar → camera → AIS) or joint centralised update, selectable per run via TrackerConfig.

---

Real data (Phase 4)

Dataset: Copenhagen harbour, departure of Dana IV, 5 March 2026. Sensor data in data/experimental/.

Coordinate conversions applied in src/real_data/loader.py:

• Radar: bearing in degrees, rotated 16° from NED → bearing_ned = radians(bearing_csv − 16°)
• Camera: Cartesian (X, Z) in camera frame → range = hypot(X, Z), bearing_ned = atan2(X, Z) + radians(28°)
• AIS: own-ship (Dana IV, MMSI 219384000) filtered by distance to GNSS vessel position

Result: 2 confirmed tracks (AIS-equipped ships at 1.7 km and 5.6 km). Radar-only tracks not confirmed due to harbour clutter and sensor noise higher than simulation assumptions.

---

Sensor specifications

Sensor	Range	FOV	Rate	Platform	Output
mm-wave radar	1 km	360°	0.3 Hz	Land (static)	range + bearing
Stereo camera	0.5 km	180°	0.5 Hz	Land (static)	range + bearing
AIS receiver	5 km	360°	~0.3 Hz	Vessel (moving)	NED position
GNSS receiver	—	—	1 Hz	Vessel (moving)	NED position