Jbrodovsky/issue108

[jbrodovsky]

Summary: Comprehensive Integration Tests for INS Filters (Issue #108)

Changes Made

New Files Created

1. core/tests/integration_tests.rs (603 lines)

   • Comprehensive integration test suite for INS filters
   • 4 test functions covering dead reckoning and UKF closed-loop filtering
   • Error metric computation using haversine distance and component-wise differences
   • Real data loading and processing utilities

2. core/tests/INTEGRATION_TESTS.md

   • Complete documentation of the integration test suite
   • Error metric descriptions and formulas
   • Test expectations and typical results
   • Usage instructions and runtime estimates

Modified Files

1. core/src/sim.rs (lines 751, 768)
   • Bug fix: Corrected dead_reckoning function to use proper From implementation
   • Changed from From<(&DateTime, &DVector, &DMatrix)> to From<(&DateTime, &StrapdownState)>
   • This fix ensures dead reckoning produces correct NavigationResult instances

Test Suite Overview

Tests Implemented

1. test_dead_reckoning_on_real_data

   • Validates pure INS dead reckoning without GNSS
   • Establishes baseline drift metrics
   • Ensures solution remains finite

2. test_ukf_closed_loop_on_real_data

   • Tests UKF with full-rate GNSS measurements
   • Validates position error < 30m RMS
   • Confirms velocity and attitude stability

3. test_ukf_with_degraded_gnss

   • Tests UKF with 5-second GNSS update intervals
   • Validates degraded performance bounds (< 600m max error)
   • Demonstrates filter resilience to reduced update rates

4. test_ukf_outperforms_dead_reckoning

   • Compares UKF to dead reckoning performance
   • Demonstrates >99.99% error reduction with GNSS
   • Validates value of sensor fusion

Error Metrics (as specified in issue)

✅ Haversine distance for planar position error (latitude and longitude)
✅ Simple difference for altitude error
✅ Component-wise differences for velocity (N, E, D)
✅ Component-wise differences for orientation (roll, pitch, yaw)

Test Results

All tests pass successfully:

running 4 tests
test test_dead_reckoning_on_real_data ... ok
test test_ukf_closed_loop_on_real_data ... ok
test test_ukf_outperforms_dead_reckoning ... ok
test test_ukf_with_degraded_gnss ... ok

test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Typical Performance

UKF with full-rate GNSS:

• RMS horizontal error: 23.59 m
• RMS altitude error: 3.74 m
• Mean velocity errors: <1 m/s

UKF with degraded GNSS (5s updates):

• RMS horizontal error: 27.54 m
• Max horizontal error: 553.12 m

Dead reckoning (for comparison):

• RMS horizontal error: 7,099,537 m (expected drift)

Usage

Run all integration tests:

cd core
cargo test --test integration_tests

Run with visible output:

cd core
cargo test --test integration_tests -- --nocapture

Runtime: ~4-5 minutes (processing 90 minutes of real sensor data)

Data Source

Tests use real sensor data from:

• Location: test_data.csv
• Source: Sensor Logger mobile app
• Duration: ~90 minutes
• Samples: ~5366 records
• IMU rate: ~100 Hz
• GNSS rate: ~1 Hz

Benefits

1. Comprehensive validation: Tests entire INS pipeline with real data
2. Performance baselines: Establishes expected error bounds for MEMS IMU
3. Regression detection: Catches bugs in filter implementation or data processing
4. Documentation: Provides examples of realistic filter performance
5. Bug discovery: Found and fixed dead_reckoning conversion bug

Recommendations

1. ✅ Merge these tests into the codebase
2. ✅ Include in CI/CD pipeline (note: tests take ~5 minutes)
3. Consider creating shorter test datasets for faster CI runs
4. Extend to particle filter when ready
5. Add more scenarios (outages, corruption, fault injection)

References

• Issue Create comprehensive practical test for INS filters using the provided real data in sim/data #108: "Create comprehensive practical test for INS filters using the provided real data in sim/data"
• Groves textbook (Chapters 5.4-5.5 for strapdown mechanization)
• Project documentation in CONTRIBUTING.md and README.md