Question in the pull plot

[zjkjsd]

Aloha Alex,

Is the pull defined as (bestfit - suggested_init) / uncertainty? I see the pull plot function directly plots the bestfit. Could you explain it?

Mahalo,
Boyang

[alexander-held]

Hi Boyang, the central value plotted are the bestfit values of each parameter as you say. For parameters that control histosys and normsys modifiers, which come with a Gaussian constraint term, the suggested_init defaults to 0 and the pre-fit parameter uncertainty as defined by the constraint term is 1 (by convention essentially). This means that (when considering pre-fit uncertainty), bestfit is equal to (bestfit - suggested_init) / uncertainty.

[zjkjsd]

Thank you Alex! What about other modifiers like shapesys or staterror, their defaults are 1. And also for some cases, suggested_init of some modifiers might be set to some user defined values. Do you think it would make the tool more general if we plot the central value as bestfit - suggested_init and errorbar as uncertainty?

[alexander-held]

I'm open to making this more flexible for other use cases. For staterror in particular a redefinition could be useful to standardize the width and make it easier to see constraints. At the moment those parameters are not included in the plot. I am not sure if suggested_init is the best way to go: by default that will be at the values maximizing the auxiliary measurements for constrained parameters, but users may set it to other values for technical reasons, e.g. to help with convergence. It is not clear to me whether users would want that reflected in the plot. Additionally I am not sure how to label such a plot on the axis. Another option might be subtracting model_utils.asimov_parameters, which are the values maximizing the constraint terms and they default to the inits for free-floating parameters (the labeling issue remains). The uncertainty for standardizing results by division could be model_utils.prefit_uncertainties, that is non-zero for constrained and non-fixed parameters.

if we plot the central value as bestfit - suggested_init and errorbar as uncertainty

If you mean the width of the bar in the pull plot, that is already taken from the uncertainty property in the fit results.

[zjkjsd]

Aloha Alex,

Thank you for the detailed answer! I think the current label ($(\hat{\theta} - \theta_0)/\delta\theta$) is already good. For $\theta_0$, I personally would choose suggested_init as we usually start a new project with suggested_init=model_utils.asimov_parameters. Even if some technical perturbations are required later on, one would hope the user-defined values are not too different, and I kind of want to see this being reflected in the plot. However, this is my personal opinion, it also makes sense to use model_utils.asimov_parameters. I agree that model_utils.prefit_uncertainties is a great idea for $\delta\theta$.