Skip to content

Commit a606c28

Browse files
authored
Merge branch 'main' into sbel/scene_circular_dependency
2 parents d37994a + f176b5a commit a606c28

File tree

9 files changed

+994
-1
lines changed

9 files changed

+994
-1
lines changed

.github/ISSUE_TEMPLATE/bug_report.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ name: 🐞 Bug Report
33
about: Report a bug to help us improve the project
44
title: "[Bug] <short description>"
55
labels: bug
6+
type: "Bug"
67
assignees: ""
78
---
89

.github/ISSUE_TEMPLATE/feature_request.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ name: ✨ Feature Request
33
about: Suggest an idea or improvement for this project
44
title: "[Feature Request] <short description>"
55
labels: feature
6+
type: "Feature"
67
assignees: ""
78
---
89

tools/tracker/evaluation/evaluators/README.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,56 @@ print(f"IDF1: {metrics['IDF1']:.3f}")
9696

9797
**Tests**: See [tests/test_trackeval_evaluator.py](tests/test_trackeval_evaluator.py) for comprehensive test suite with 16 test cases covering configuration, processing, evaluation, and integration workflows.
9898

99+
### DiagnosticEvaluator
100+
101+
**Purpose**: Per-frame location comparison and error analysis between matched output tracks and ground-truth tracks.
102+
103+
**Status**: **FULLY IMPLEMENTED** - Bipartite track matching with per-frame location and distance CSV/plot outputs.
104+
105+
**Supported Metrics**:
106+
107+
- **LOC_T_X**: Per-frame X position of each matched (output, GT) track pair
108+
- **LOC_T_Y**: Per-frame Y position of each matched (output, GT) track pair
109+
- **DIST_T**: Per-frame Euclidean distance error between each matched pair
110+
111+
**Key Features**:
112+
113+
- **Track Matching**: Bipartite assignment (Hungarian algorithm) minimizing mean Euclidean distance over overlapping frames. Requires a minimum of 10 overlapping frames (`MIN_OVERLAP_FRAMES`).
114+
- **Missing Frame Handling**: Frames where only one side (output or GT) has data produce `NaN` in CSV output, preserving full temporal context.
115+
- **CSV Output**: Per-metric CSV files with headers:
116+
- LOC_T_X / LOC_T_Y: `[frame_id, track_id, gt_id, value_track, value_gt]`
117+
- DIST_T: `[frame_id, track_id, gt_id, distance]`
118+
- **Plot Output**: One matplotlib figure per metric with all matched pairs overlaid.
119+
- **Summary Scalars**: `evaluate_metrics()` returns `DIST_T_mean`, `LOC_T_X_mae`, `LOC_T_Y_mae`, and `num_matches`.
120+
121+
**Usage Example**:
122+
123+
```python
124+
from evaluators.diagnostic_evaluator import DiagnosticEvaluator
125+
from pathlib import Path
126+
127+
evaluator = DiagnosticEvaluator()
128+
metrics = (evaluator
129+
.configure_metrics(['LOC_T_X', 'LOC_T_Y', 'DIST_T'])
130+
.set_output_folder(Path('/path/to/results'))
131+
.process_tracker_outputs(tracker_outputs, gt_file_path)
132+
.evaluate_metrics())
133+
print(f"Mean distance: {metrics['DIST_T_mean']:.3f}")
134+
print(f"X MAE: {metrics['LOC_T_X_mae']:.3f}")
135+
print(f"Y MAE: {metrics['LOC_T_Y_mae']:.3f}")
136+
print(f"Matched pairs: {int(metrics['num_matches'])}")
137+
```
138+
139+
**Current Limitations**:
140+
141+
- Uses only X and Y coordinates (Z ignored)
142+
- Single-sequence evaluation only
143+
- No configurable overlap threshold (fixed at 10 frames)
144+
145+
**Implementation**: [diagnostic_evaluator.py](diagnostic_evaluator.py)
146+
147+
**Tests**: See [tests/test_diagnostic_evaluator.py](tests/test_diagnostic_evaluator.py) for unit tests covering track matching, scalar metrics, CSV output, and reset workflows.
148+
99149
## Adding New Evaluators
100150

101151
To add support for a new metric computation library:

tools/tracker/evaluation/evaluators/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,6 @@
33

44
"""Evaluator implementations for tracker evaluation."""
55
from .trackeval_evaluator import TrackEvalEvaluator
6+
from .diagnostic_evaluator import DiagnosticEvaluator
67

7-
__all__ = ['TrackEvalEvaluator']
8+
__all__ = ['TrackEvalEvaluator', 'DiagnosticEvaluator']

0 commit comments

Comments
 (0)