ENH: Implement Multivariate Rejection Sampling (MRS)

[Lucas-Prates]

Pull request type

• Code changes (bugfix, features)

Checklist

• Unit tests for the changes have been added
• Integration tests for the changes have been added
• Docs have been reviewed and added / updated
• Lint (black rocketpy/ tests/) has passed locally
• All tests (pytest tests -m slow --runslow) have passed locally
• CHANGELOG.md has been updated (if relevant)
• RST documentation
• Monte Carlo comparison Feature

New behavior

This PR implements the MRS requested in #162 and described in RocketPy paper.

Breaking change

• Yes (perhaps)

Additional information

[codecov]

Codecov Report

Attention: Patch coverage is 82.80543% with 38 lines in your changes missing coverage. Please review.

Project coverage is 79.76%. Comparing base (4df0b38) to head (3530eed).
Report is 4 commits behind head on develop.

Files with missing lines	Patch %	Lines
rocketpy/plots/monte_carlo_plots.py	80.00%	16 Missing ⚠️
rocketpy/prints/monte_carlo_prints.py	57.69%	11 Missing ⚠️
...ketpy/simulation/multivariate_rejection_sampler.py	89.81%	11 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #738      +/-   ##
===========================================
+ Coverage    79.11%   79.76%   +0.64%     
===========================================
  Files           96       97       +1     
  Lines        11575    11877     +302     
===========================================
+ Hits          9158     9474     +316     
+ Misses        2417     2403      -14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Lucas-Prates]

This PR is ready for a first "Design Review." I would like to get your opinion if this implementation provides what you think on how the user should use the MRS.

Albeit a implementation as a function seems natural, I implemented as a class because:

1. the function would be humongous;
2. if the user wants to resample several times, the data monte carlo data is only read once from the harddrive.

It currently works as follows:

0. Input: monte carlo filepath prefix, mrs filepath prefix, distribution dictionary;
1. Load input and output data from a monte carlo simulation into memory (python objects - lists of jsons);
2. To avoid having to read data twice, while loading, precompute some important properties required in the sampler algorithm;
3. Select and save iteratively accepted samples;
4. Output: files are saved in the same "scheme" as the MonteCarlo simulation.

I provided a quick and dirty notebook, which will be removed, just to show how the class is being used at the moment.

[Gui-FernandesBR]

I don't have much time so I'll be short

• The class dies after you sample the data once, this makes it pointless to have a class
• Instead I'd remove the sample dictionary from arguments.
• First read the data during initialization. Then you use a function to set the variables you are going to allow be varied. This way u can already anticipate which variables may be varied.
• We want almost instant results for a MonteCarlo simulation after using MRS.
• Other thing is that the user must supply the original pdf. Could we possile estimate this from data? (imagine 70k)
• Finally, plotting is crucial for MRS, or even tables. We'd love to see more on that later.

[Lucas-Prates]

This PR is ready for review again. Changes from last time:

1. addressed some of the previous suggestions of @phmbressan;
2. added a comparison_info method that provides a statistical summary print comparison of the results of two monte carlo simulations, similar to the method used in the regular monte carlo class;
3. added a comparison_plots method that plots boxplots and histograms comparison of the results of two monte carlo simulations;
4. added a compare_ellipses method that plots the ellipses for the apogee and landing point comparison of the results of two monte carlo simulations;
5. added the mrs.rst file to the documentation explaining what is the MRS and how to use it;
6. added the MultivariateRejectionSampler class .rst documentation;
7. removed the test_mrs.ipynb notebook since the .rst file does the same in a much clearer way;
8. on my pc, the MRS is 500x faster than running a monte carlo simulation of the equivalent sample size.

For review, I recommend mostly checking the .rst documentation. Here are some previews of the comparisons if you do not have the time to compile the html:

Print comparison:

Plot comparison:

Ellipses comparison:

[Lucas-Prates]

A second comment: I made the design choice to implement the comparison methods in the MonteCarlo class instead of the MultivariateRejectionSampler. Here are the most relevant arguments:

1. It makes sense to compare monte carlo simulations even outside of MRS, i.e. even one is not a sub-sample of the other. Hence, implementing this inside the MRS would make this usage extremely odd;
2. It keeps the MultivariateRejectionSampler class implementation really simple and with only one job: to sample.

One option is, of course, to create another class for that job, but it did not seem to be the best option.

[Gui-FernandesBR]

Also, please update the CHANGELOG file!

[Lucas-Prates]

I have addressed the suggestions made by @Gui-FernandesBR, implemented unit and integration tests for the MRS, and updated the CHANGELOG.

[Gui-FernandesBR]

LGTM

[Gui-FernandesBR]

@Lucas-Prates good work. Please squash and merge at your earliest convinience

[MateusStano]

This is really good overall! Great job

[MateusStano]

Improve colors of the points/ellipses. Using oposite colors is probably best

[Gui-FernandesBR]

There's no such thing as "opposite colors", but matplotlib does offer a fair good guide to selecting colormaps: https://matplotlib.org/stable/users/explain/colors/colormaps.html

I suggest blue/orange or blue/red.

Ideally the user should be able to select their specific color... But I understand the current limitations.

[MateusStano]

I guess the right term in complementary not "oposite". Anyway, I meant something like this:

[Lucas-Prates]

I will use then a "tetradic" color combination I found in this site. Here is the pallete I got, just for reference:

[Lucas-Prates]

This is the tetradic colors for the ellipses plot:

[Lucas-Prates]

This PR is ready again for review. Since last time:

1 - addressed the points in Stano's review about documentation and color;
2 - removed some input checks which I think were not that useful but were a bit trick to do correctly;
3 - modified the flatten_dict function to better handle variables in inner levels.

The most important point is 3 since it might introduce a breaking change.

[Gui-FernandesBR]

All looking good to me, great implementation.
I don't see the changes to flatten_dict as a problem.
Nobody is using that function right now.

Waiting for @MateusStano 's final comments so we can proceed.

[Copilot]

Copilot reviewed 14 out of 21 changed files in this pull request and generated 1 comment.

Files not reviewed (7)

• .vscode/settings.json: Language not supported
• docs/notebooks/monte_carlo_analysis/monte_carlo_analysis_outputs/mrs.outputs.txt: Language not supported
• docs/reference/classes/MultivariateRejectionSampler.rst: Language not supported
• docs/reference/index.rst: Language not supported
• docs/user/index.rst: Language not supported
• docs/user/mrs.rst: Language not supported
• docs/user/sensitivity.rst: Language not supported

Comments suppressed due to low confidence (1)

rocketpy/tools.py:595

• [nitpick] Consider renaming 'flatted_dict' to 'flattened_dict' for clarity and consistency with common terminology.

flatted_dict = {}