Skip to content

Support for parallelising ranking computations #530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 44 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
693f928
rough implementation of covariance-based impacts
MoAly98 Mar 6, 2025
c4ef0c8
towards modularising more
MoAly98 Mar 6, 2025
651729c
modularise impacts computation
MoAly98 Mar 6, 2025
083abe7
harmonise code for impact calculation with np shifting and covariance…
MoAly98 Mar 7, 2025
d8111cc
fix tests, add covariance test in test_integration
MoAly98 Mar 7, 2025
f004b8f
add tests
MoAly98 Mar 7, 2025
b69a1b2
improve coverage
MoAly98 Mar 7, 2025
22d57d4
fix tests
MoAly98 Mar 7, 2025
9647cd3
add impacts method to RankingResult object
MoAly98 Mar 7, 2025
49d486e
covariance method plotting added
MoAly98 Mar 7, 2025
79da2a3
add tests
MoAly98 Mar 7, 2025
035a714
add CLI support
MoAly98 Mar 7, 2025
9b18d39
attempt fix for faliling python 3.8 CI
MoAly98 Mar 8, 2025
b94c7fa
clear up and improved docstring
MoAly98 Mar 8, 2025
b5b2cdf
fix missing sqrt in impacts summary calculation and prepare for compu…
MoAly98 Mar 8, 2025
45ea309
statstical uncertainty for non-np-shift methods
MoAly98 Mar 8, 2025
3cf1662
fix stat unc calculation
MoAly98 Mar 8, 2025
7b82529
move datastat impact evaluation into function
MoAly98 Mar 8, 2025
af475f0
fixes to code and tests adjusting for additional fit in np_shift rank…
MoAly98 Mar 8, 2025
b568bb4
docstrings
MoAly98 Mar 9, 2025
24ba0ae
docstrings fix typing
MoAly98 Mar 9, 2025
b451272
first implementation of auxdata_shift method
MoAly98 Mar 10, 2025
5e5972e
add auxdata shifting plot
MoAly98 Mar 11, 2025
081c75c
visualize_plot_result test
MoAly98 Mar 11, 2025
9523550
add ranking tests for auxdata shifting ranking method
MoAly98 Mar 11, 2025
903f857
add integration test to auxdata shifting method
MoAly98 Mar 11, 2025
396c1ff
add missing reference figure
MoAly98 Mar 11, 2025
d37dac9
fix for typing of fit kwargs in impact calculations
MoAly98 Mar 12, 2025
dbfcdf2
missing annotation
MoAly98 Mar 12, 2025
cd449eb
keep impacts_summary as defaultdict
MoAly98 Mar 12, 2025
5d441aa
remove boilerplate
MoAly98 Mar 30, 2025
222b7c7
allow passing parameter list to ranking to allow for parallelisation
MoAly98 Mar 31, 2025
86993a7
add test
MoAly98 Mar 31, 2025
81ff9de
fsspec version for python 3.8
MoAly98 Mar 31, 2025
852c9d2
add utility to combine ranking results from multiple calls to ranking…
MoAly98 Mar 31, 2025
1c7be4c
allow maximal parallelisation of ranking fits if a dask client is passed
MoAly98 Mar 31, 2025
1658367
remove fsspec hack after upstream fix
MoAly98 Apr 1, 2025
c3a2efc
add dask as dependency
MoAly98 Apr 1, 2025
bf95e3b
add dask distributed as dependency
MoAly98 Apr 1, 2025
727fb3d
make dask dependency optional
MoAly98 Apr 1, 2025
36271ae
re-enable mypy and fix typing issues with untyped dask methods
MoAly98 Apr 1, 2025
bfdec8c
add docstring to impact fit task function
MoAly98 Apr 1, 2025
dc61970
add comments
MoAly98 Apr 1, 2025
3a663d4
fix missing arg to np_impacts method
MoAly98 Apr 2, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ repos:
rev: 25.1.0
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.15.0
hooks:
- id: mypy
name: mypy with Python 3.12
files: src/cabinetry
additional_dependencies: ["numpy>=1.22", "boost-histogram>=1.0.1", "click>=8", "types-tabulate", "types-PyYAML", "hist>=2.3.0"]
args: ["--python-version=3.12"]
#- repo: https://github.com/pre-commit/mirrors-mypy
# rev: v1.15.0
# hooks:
# - id: mypy
# name: mypy with Python 3.12
# files: src/cabinetry
# additional_dependencies: ["numpy>=1.22", "boost-histogram>=1.0.1", "click>=8", "types-tabulate", "types-PyYAML", "hist>=2.3.0"]
# args: ["--python-version=3.12"]
- repo: https://github.com/pycqa/flake8
rev: 7.1.2
hooks:
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ dependencies = [
"click",
"scipy",
"packaging", # for version parsing
"dask[distributed]",
]

[project.scripts]
Expand Down
19 changes: 16 additions & 3 deletions src/cabinetry/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,8 +154,17 @@ def fit(
default="figures",
help='folder to save figures to (default: "figures")',
)
@click.option(
"--impacts_method",
default="covariance",
help="The method to be used for computing impacts",
)
def ranking(
ws_spec: io.TextIOWrapper, asimov: bool, max_pars: int, figfolder: str
ws_spec: io.TextIOWrapper,
asimov: bool,
max_pars: int,
figfolder: str,
impacts_method: str,
) -> None:
"""Ranks nuisance parameters and visualizes the result.

Expand All @@ -165,9 +174,13 @@ def ranking(
ws = json.load(ws_spec)
model, data = cabinetry_model_utils.model_and_data(ws, asimov=asimov)
fit_results = cabinetry_fit.fit(model, data)
ranking_results = cabinetry_fit.ranking(model, data, fit_results=fit_results)
ranking_results = cabinetry_fit.ranking(
model, data, fit_results=fit_results, impacts_method=impacts_method
)
cabinetry_visualize.ranking(
ranking_results, figure_folder=figfolder, max_pars=max_pars
ranking_results,
figure_folder=figfolder,
max_pars=max_pars,
)


Expand Down
Loading