[ITEP-89467] Tracking Evaluation Pipeline: Add jitter metrics (RMS jerk and acceleration variance)

[dmytroye]

📝 Description

Adds a new tracking-evaluation “jitter” evaluator to compute smoothness metrics (RMS jerk and acceleration variance) and wires it into the evaluation pipeline.

Changes:

• Introduces JitterEvaluator with RMS jerk and acceleration variance computations based on numerical differentiation.
• Adds unit tests for configuration, processing, metric computation, and result-file output.
• Updates pipeline configs and docs to include/describe the new evaluator.

Metrics:

Metric	Source	Description
rms_jerk	Tracker output	RMS jerk across all tracker output tracks (m/s³)
acceleration_variance	Tracker output	Variance of acceleration magnitudes across all tracker output tracks (m/s²)²
rms_jerk_gt	Ground truth	Same as rms_jerk computed on ground-truth tracks
acceleration_variance_gt	Ground truth	Same as acceleration_variance computed on ground-truth tracks
rms_jerk_ratio	Tracker / GT	rms_jerk / rms_jerk_gt — tracker jitter relative to GT (1.0 = equal)
acceleration_variance_ratio	Tracker / GT	acceleration_variance / acceleration_variance_gt

Comparing rms_jerk with rms_jerk_gt shows how much jitter the tracker
adds on top of any jitter already present in the test data.

Algorithm:

All metrics are derived by applying three sequential layers of forward finite differences to 3D positions, accounting for variable time steps between frames:

$$v_i = \frac{p_{i+1} - p_i}{\Delta t_i}, \quad a_i = \frac{v_{i+1} - v_i}{\Delta t_{v,i}}, \quad j_i = \frac{a_{i+1} - a_i}{\Delta t_{a,i}}$$

• rms_jerk / rms_jerk_gt: $\sqrt{\frac{1}{N}\sum |j_i|^2}$ over all jerk samples from all tracks.
• acceleration_variance / acceleration_variance_gt: $\text{Var}(|a_i|)$ over all acceleration magnitude samples from all tracks.
• rms_jerk_ratio / acceleration_variance_ratio: tracker metric divided by the corresponding GT metric. Returns 0.0 when the GT denominator is zero. Values >1.0 indicate the tracker adds more jitter than is inherent in the ground truth.

Minimum track length: 3 points for acceleration, 4 points for jerk. Shorter tracks are skipped; if no eligible tracks exist, the metric returns 0.0.

For GT metrics, ground-truth frame numbers are converted to relative timestamps using the FPS derived from the tracker output.

Key Features:

• Builds per-track position histories from canonical tracker output format.
• Parses MOTChallenge 3D CSV ground-truth file for GT metric computation.
• Supports variable frame rates — time deltas are computed from actual timestamps.
• Deduplicates frames with identical timestamps (mirrors TrackEvalEvaluator behaviour).
• Sorts each track's positions by timestamp before metric computation.
• Saves a plain-text jitter_results.txt summary to the configured output folder.

✨ Type of Change

Select the type of change your PR introduces:

• 🐞 Bug fix – Non-breaking change which fixes an issue
• 🚀 New feature – Non-breaking change which adds functionality
• 🔨 Refactor – Non-breaking change which refactors the code base
• 💥 Breaking change – Changes that break existing functionality
• 📚 Documentation update
• 🔒 Security update
• 🧪 Tests
• 🚂 CI

🧪 Testing Scenarios

Describe how the changes were tested and how reviewers can test them too:

Unit tests (no Docker required):

cd tools/tracker/evaluation
source .venv/bin/activate
pip install -r requirements.txt
pytest evaluators/tests/test_jitter_evaluator.py -v

Full unit test suite (verify no regressions):

pytest . -v -m "not integration"

End-to-end pipeline run (requires Docker + scenescape-controller image):

python pipeline_engine.py pipeline_configs/controller_evaluation.yaml

• ✅ Tested manually
• 🤖 Ran automated end-to-end tests

✅ Checklist

Before submitting the PR, ensure the following:

• 🔍 PR title is clear and descriptive
• 📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
• 💬 I have commented my code, especially in hard-to-understand areas
• 📄 I have made corresponding changes to the documentation
• ✅ I have added tests that prove my fix is effective or my feature works

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new tracking-evaluation “jitter” evaluator to compute smoothness metrics (RMS jerk and acceleration variance) and wires it into the evaluation pipeline.

Changes:

• Introduces JitterEvaluator with RMS jerk and acceleration variance computations based on numerical differentiation.
• Adds unit tests for configuration, processing, metric computation, and result-file output.
• Updates pipeline configs and docs to include/describe the new evaluator.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
tools/tracker/evaluation/pipeline_configs/metric_test_evaluation.yaml	Adds JitterEvaluator to the metric test pipeline config.
tools/tracker/evaluation/evaluators/jitter_evaluator.py	Implements jitter metric processing, computation, and result persistence.
tools/tracker/evaluation/evaluators/tests/test_jitter_evaluator.py	Adds unit tests covering JitterEvaluator behavior.
tools/tracker/evaluation/evaluators/init.py	Exposes JitterEvaluator from the evaluators package.
tools/tracker/evaluation/evaluators/README.md	Documents JitterEvaluator usage, metrics, and algorithm.
tools/tracker/evaluation/README.md	Documents pipeline configuration and lists JitterEvaluator as available.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

[tdorauintc]

LGTM, minor comments added.

Please update evaluators in the Agents.md