Autonomous Spacecraft Proximity Operations in HEO

Integrated DRL-MPC Framework for Rendezvous & Docking

This repository contains the implementation of a hybrid Guidance and Control (G&C) framework that integrates Deep Reinforcement Learning (DRL) with Model Predictive Control (MPC).

The DRL agent acts as an "expert tuner," dynamically optimizing MPC parameters in real-time to balance fuel efficiency and computational load during proximity operations in Highly Elliptical Orbits (HEO).

---

🚀 Key Performance Results

The hybrid approach demonstrates significant improvements over baseline controllers:

Metric	Improvement
Fuel Savings	10% to 40% improvement over static MPC
Efficiency	~65% reduction in relative computational time
Safety	100% mission success rate via MPC constraint enforcement

---

📂 Repository Structure

The project is organized into four main stages, following the research workflow:

[01] Classical MPC Controller

A standard linear MPC implementation used as a benchmark to evaluate the performance gains of the RL-tuned versions.

[02] RL Horizon & Timestep Tuning ($T_h$ & $dt$)

Focuses on training a DRL agent to dynamically adjust the Prediction Horizon ($T_h$) and Control Timestep ($dt$) to optimize computational resources without sacrificing stability.

[03] RL Weighting Matrix Tuning ($Q$ & $R$)

Dedicated to training the DRL agent to tune the State ($Q$) and Control ($R$) weighting matrices to optimize trajectory tracking and fuel consumption.

[04] Testing and Validation

Final evaluation via Monte Carlo simulations across various orbital conditions. Includes:

• Comparative analysis: Classical MPC vs. Tuned RL-MPC.
• Performance bar charts and statistical validation.

Trajectory Analysis & Performance Benchmarking

In the figures below, the RL-Tuned MPC shows a smoother approach and better constraint satisfaction in HEO compared to the Classical MPC.

Baseline Trajectory	RL-Tuned Trajectory

Quantitative Comparison

📖 Documentation

Detailed theoretical background and mission analysis can be found in the Docs/ folder:

• Executive Summary: A high-level overview of the methodology and key results.
• Thesis (Full Text): "Autonomous spacecraft proximity operations in Highly Elliptical Orbits"
• Author: Pietro Cavalletti (2025)

---

🛠 Setup & Requirements

• MATLAB / Simulink (R2024a or later recommended)
• Model Predictive Control Toolbox
• Reinforcement Learning Toolbox
• Deep Learning Toolbox