Deep Learning-based Trajectory Prediction

[Khushi0512]

Algorithm/Conceptual basis

LSTM-GRU Encoder-Decoder Network for Multi-Vehicle Trajectory Prediction

Description & Application

This deep learning model forecasts the future behaviors of surrounding vehicles by utilizing a recurrent neural network, more specifically LSTM (Long Short-Term Memory) and GRU (Gated Recurrent Unit) networks. Taking the input as a historical state trajectory data (position, velocity, acceleration), the model is then required to predict future positions over a prediction horizon (typically 3-5 seconds).
The encoder-decoder architecture deals with temporal sequences: the encoder that extracts the features out of previous trajectories and a decoder which produces predictions on future ones. Further, an attention mechanism can be used to capture the vehicle's interactions and determine which other vehicles affect the target vehicle's trajectory more. This is essential for self-driving cars to predict and respond to the action of traffic participants in their vicinity.

Plan for Implementation

1. Data Generation/Collection:

• Simulate or use real highway trajectory datasets (NGSIM, HighD)

• Generate synthetic multi-vehicle scenarios with various maneuvers

2. Data Preprocessing:

• Extract features: position (x, y), velocity (vx, vy), acceleration (ax, ay), heading

• Normalize/standardize trajectory data

• Create sliding window sequences (observation window: 3s, prediction horizon: 5s)

3. Model Architecture:

• LSTM Encoder: Process historical trajectory sequences

• GRU Decoder: Generate future trajectory predictions

• Attention Layer: Model vehicle interactions (optional)

• Fully Connected Output: Map to predicted coordinates

4. Training:

• Loss function: Mean Squared Error (MSE) or Mean Absolute Error (MAE)

• Optimizer: Adam with learning rate scheduling

• Validation: K-fold cross-validation

5. Comparative Analysis: Implement and compare vanilla RNN, LSTM, GRU, Bi-LSTM models
6. Visualization: Plot predicted vs. actual trajectories, attention weights, error distributions

References in Consideration

1. Kaouther Messaoud, Itheri Yahiaoui, Anne Verroust-Blondet, Fawzi Nashashibi, "Attention Based
   Vehicle Trajectory Prediction. IEEE Transactions on Intelligent Vehicles, Institute of Electrical and
   Electronics Engineers", In press, 10.1109/TIV.2020.2991952. hal-02543967
   https://www.researchgate.net/publication/341055264_Attention_Based_Vehicle_Trajectory_Prediction
2. Jiaqin Wang, Kai Liu, Hantao Li, "LSTM-based Graph attention network for vehicle trajectory prediction", Computer Networks, volume 248, June 2024, 110477
   https://www.sciencedirect.com/science/article/abs/pii/S1389128624003098
3. Zhao, Tianyang & Xu, Yifei & Monfort, Mathew & Choi, Wongun & Baker, Chris & Zhao, Yibiao & Wang, Yizhou & Wu, Yingnian. (2019). Multi-Agent Tensor Fusion for Contextual Trajectory Prediction. 12118-12126. 10.1109/CVPR.2019.01240.
   https://www.researchgate.net/publication/338509582_Multi-Agent_Tensor_Fusion_for_Contextual_Trajectory_Prediction

Expected Outcomes

• Trained models achieving <1.5m prediction error at 3s horizon, <3m at 5s horizon

• Comparison showing GRU achieving similar/better accuracy than LSTM with fewer parameters

• Visualization of predicted trajectories overlaid on ground truth

• Attention heatmaps showing which vehicles influence predictions

• Performance metrics: RMSE, MAE, ADE (Average Displacement Error), FDE (Final Displacement Error)

• Ablation study showing impact of attention mechanism and interaction modeling

• Real-time inference capability demonstration (<100ms per prediction)

• Unit tests for data preprocessing, model architecture, and prediction accuracy

Execution structure and Pre-requisite

1. Pre-requisite: PyTorch/TensorFlow, NumPy, Pandas, Matplotlib, scikit-learn
2. File Structure:
   (a) Main Model
   (b) Dataset Handling
   (c) LSTM, GRU, attention Modules
   (d) Training Loop
   (e) Testing and Metrics
3. Dataset: NGSIM I-80/US-101
4. Hyperparameters: Hidden Size, Number of Layers, Dropout rate, learning rate, Batch Size.
5. Training Details: Epochs, Batch Size, Early Stopping Criteria
6. Applications: Collision avoidance, Lane Change Prediction, Motion Planning
7. Limitations: Assumes constant road structure, may struggle with rare maneuvers
8. Future Work: Graph Neural Networks for better interaction modeling, multi-modal predictions, uncertainty quantification
9. Computational Requirements: GPU recommended for training, CPU sufficient for inference.

[ShisatoYano]

Hi @Khushi0512 , thank you for creating this issue. It looks so great but can not be accepted if GPU is required for computation because I hope my project can be used by people who don't have GPU. So, Please consider another easier and simpler algorithms and implementations which GPU is not necessary for beginners.

And same as the issue #52, this issue can also be separated into multiple smaller issues.