feat: Improve handling of parameter bounds when computing impacts in ranking

[MoAly98]

This is based on #490. Closes #511.

Summary

When building parameter rankings based on impact, 5 fits must be evaluated for each NP. The fits are evaluated at:

• The best-fit value of the NP
• NP + up uncertainty (pre-fit)
• NP + down uncertainty (pre-fit)
• NP + up uncertainty (post-fit)
• NP + down uncertainty (post-fit)

The uncertainties so far have been extracted from the Hessian. If the bounds of the parameter, which specify valid range of values of this parameter, are not specified by the user, one of the NP values above may step out of the appropriate bound. For example, for a poisson-constrained NP (shapesys). If the best-fit value is 1.0 and pre/post-fit uncertainty is >1, the NP value will be negative for some of the above fits. Negative parameter values are not allowed in the poisson terms, and hence the corresponding fits fails.

This can indicate a defective model, or it can be the result of the Hessian uncertainty being symmetrised without respecting parameter bounds. If the user is sure their model is good, some options are added in this PR to help the fits converge:

1- Use suggested bounds from pyhf to force the ranking to cap the value of an NP at its physical limit, independently of the uncertainty extracted from the fit.
2- Use the uncertainty from MINOS on the NP which will respect the parameter boundaries by construction.

In this MR, an option to the fit.ranking function is added to specify that suggested bounds should be used to set limits on the parameters bounds are not set by user. In addition, the ranking function can now read MINOS uncertainties for relevant parameters if the fit results are provided to it. If the fit results are not provided, the ranking function can now perform the fit itself while allowing the use of MINOS for model parameters.

Example spec

The following spec fails even for pre-fit uncertainties.

spec = {
    "channels": [
        {
            "name": "Signal_region",
            "samples": [
                {
                    "data": [50.0],
                    "modifiers": [
                        {
                            "data": None,
                            "name": "mu",
                            "type": "normfactor"
                        },
                        {
                            "data": [80.0],
                            "name": "test_shapesys",
                            "type": "shapesys"
                        }
                    ],
                    "name": "Signal"
                },
                {
                    "data": [200.0],
                    "modifiers": [],
                    "name": "Background"
                }
            ]
        }
    ],
    "measurements": [
        {
            "config": {
                "parameters": [],
                "poi": "mu"
            },
            "name": "minimal_example"
        }
    ],
    "observations": [
        {
            "data": [250.0],
            "name": "Signal_region"
        }
    ],
    "version": "1.0.0"

With this PR the following setups are possible, utilising suggested bounds:

model, data = model_utils.model_and_data(spec)
fit_results = fit.fit(model, data, minos=["test_shapesys[0]"])
rank = fit.ranking(model, data, fit_results=fit_results) # fails
rank = fit.ranking(model, data, fit_results=fit_results, par_bounds=[(0,10),(1e-10,10.0)]) # works and already supported
rank = fit.ranking(model, data, fit_results=fit_results, use_suggested_bounds=True) # works and new 

To utilise MINOS, consider the following setup:

# We want the fit results to be problematic to check use of minos
bestfit = np.asarray([1.0, 1.0])
uncertainty = np.asarray([0.85, 1.1])
labels = ["POI", "SHAPESYS"]
fit_results = FitResults(bestfit, uncertainty, labels, np.empty(0), 0.0)
fit_results_with_minos = FitResults(bestfit, uncertainty, labels, np.empty(0), 0.0, minos_uncertainty={'test_shapesys[0]': (-0.89, 2.55)})

rank = fit.ranking(model, data, fit_results=fit_results) # fails
rank = fit.ranking(model, data, fit_results=fit_results_with_minos) # works because MINOS is limiting the bounds

* Added `bool` option for user to specify if suggested bounds should be used when user does not set bounds
* Ranking now accepts minos arguments when `fit_results` object is not provided
* Ranking now uses post-fit MINOS uncertainties from fit results if they are available for a parameter

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (131a667) to head (1f4213c).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##            master      #512   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        22           
  Lines         2096      2117   +21     
  Branches       291       297    +6     
=========================================
+ Hits          2096      2117   +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[MoAly98]

@alexander-held this is ready for review now with tests. If I fix the partially covered line (by switching elif to else) mypy complains because it cannot figure out that par_bounds is definitely not None.

[alexander-held]

I am wondering if there is a case where users would want use_suggested_bounds=False and where bounds would be applied if the setting was turned on instead. For cases where bounds are not applied regardless, this setting does not do anything. All cases I can think of would cause problems in terms of crashes or unreasonable results if bounding would be needed but is not applied. This situation means that parameter uncertainties are already large when compared to the distance of the best-fit point to the boundary, so they cannot necessarily be trusted to begin with, so it's not clear to me that the unbounded calculation is producing anything meaningful.

[MoAly98]

@alexander-held this is ready for review now with tests. If I fix the partially covered line (by switching elif to else) mypy complains because it cannot figure out that par_bounds is definitely not None.

@alexander-held --- thoughts on this?

Otherwise, think this is ready.

[alexander-held]

The partial coverage question is no longer relevant now, right? With the restructuring this looks fine.