Skip to content

Support for parallelising ranking computations #530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
MoAly98 wants to merge 44 commits into
base: master
Choose a base branch
from
Draft

Support for parallelising ranking computations #530

MoAly98 wants to merge 44 commits into from

Conversation

@MoAly98
Copy link
Collaborator

@MoAly98 MoAly98 commented Mar 31, 2025 •
edited
Loading

Closes #442 and related to #421.

This PR is intended to introduce the infrastructure to be able to easily parallelise the production of ranking plots:

  • allow a list of parameters to be passed to the ranking function for which the ranking is computed
  • provide a utility to combine the ranking results from a list of ranking function calls
  • [maybe] provide simple support for utilising a user-provided dask client
    • write tests
    • improve docstrings and comments
  • pass parameters via CLI

Blocked by #514

MoAly98 added 30 commits March 12, 2025 17:02
@MoAly98
rough implementation of covariance-based impacts
693f928
@MoAly98
towards modularising more
c4ef0c8
@MoAly98
modularise impacts computation
651729c
@MoAly98
harmonise code for impact calculation with np shifting and covariance…
083abe7 
… method
@MoAly98
fix tests, add covariance test in test_integration
d8111cc
@MoAly98
add tests
f004b8f
@MoAly98
improve coverage
b69a1b2
@MoAly98
fix tests
22d57d4
@MoAly98
add impacts method to RankingResult object
9647cd3
@MoAly98
covariance method plotting added
49d486e
@MoAly98
add tests
79da2a3
@MoAly98
add CLI support
035a714
@MoAly98
attempt fix for faliling python 3.8 CI
9b18d39
@MoAly98
clear up and improved docstring
b94c7fa
@MoAly98
fix missing sqrt in impacts summary calculation and prepare for compu…
b5b2cdf 
…ting statistical uncertainty in non-np-shift methods
@MoAly98
statstical uncertainty for non-np-shift methods
45ea309
@MoAly98
fix stat unc calculation
3cf1662
@MoAly98
move datastat impact evaluation into function
7b82529
@MoAly98
fixes to code and tests adjusting for additional fit in np_shift rank…
af475f0 
…ings
@MoAly98
docstrings
b568bb4
@MoAly98
docstrings fix typing
24ba0ae
@MoAly98
first implementation of auxdata_shift method
b451272
@MoAly98
add auxdata shifting plot
5e5972e
@MoAly98
visualize_plot_result test
081c75c
@MoAly98
add ranking tests for auxdata shifting ranking method
9523550
@MoAly98
add integration test to auxdata shifting method
903f857
@MoAly98
add missing reference figure
396c1ff
@MoAly98
fix for typing of fit kwargs in impact calculations
d37dac9
@MoAly98
missing annotation
dbfcdf2
@MoAly98
keep impacts_summary as defaultdict
cd449eb
@codecov
Copy link

codecov bot commented Mar 31, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 93.71981% with 13 lines in your changes missing coverage. Please review.

Project coverage is 99.42%. Comparing base (1576a4d) to head (3a663d4).
Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
src/cabinetry/fit/__init__.py 92.07% 9 Missing and 4 partials ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##            master     #530      +/-   ##
===========================================
- Coverage   100.00%   99.42%   -0.58%     
===========================================
  Files           22       23       +1     
  Lines         2096     2264     +168     
  Branches       291      320      +29     
===========================================
+ Hits          2096     2251     +155     
- Misses           0        9       +9     
- Partials         0        4       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MoAly98 MoAly98 marked this pull request as draft March 31, 2025 09:03
@MoAly98 MoAly98 self-assigned this Mar 31, 2025
@MoAly98
Copy link
Collaborator Author

MoAly98 commented Mar 31, 2025 •
edited
Loading

@alexander-held what do you think of this implementation to parallelise with a user-fed client? Tests currently fail because dask is not a dependency yet. I would diff this with #514 to see the change

@MoAly98 MoAly98 added enhancement New feature or request performance Related to performance labels Apr 1, 2025
alexander-held
alexander-held reviewed Apr 1, 2025
View reviewed changes
@github-actions
Copy link

github-actions bot commented Apr 2, 2025

Status: Blocked ❌

Issues blocking this PR:

This comment was automatically written by the Blocking Issues bot, and this PR will be monitored for further progress.

@github-actions github-actions bot added the blocked Waiting for another PR/issue to be merged/closed. label Apr 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexander-held alexander-held alexander-held left review comments

Assignees

@MoAly98 MoAly98

Labels

blocked Waiting for another PR/issue to be merged/closed. enhancement New feature or request performance Related to performance

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Parallelization of nuisance parameter ranking

2 participants

@MoAly98 @alexander-held