Skip to content

Poor error reporting from AsymptoticLimits with bad fits #925

@andrzejnovak

Description

@andrzejnovak

I have a workspace for which the expected limits fits are currently failing kind of silently by just not printing the expected results

╰─ff combine -M AsymptoticLimits -d model_combined.root
 <<< Combine >>>
>>> random number generator seed is 123456
>>> method used is AsymptoticLimits
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,

 -- AsymptoticLimits ( CLs ) --
Observed Limit: r < 0.6503

Done in 16.02 min (cpu), 16.02 min (real)

In the meantime, while looking at available args to debug, and came across --picky Abort on fit failures which does not actually abort, but instead prints the garbage expected limit along with extra errors

╰─ff combine -M AsymptoticLimits -d model_combined.root --picky
 <<< Combine >>>
>>> random number generator seed is 123456
>>> method used is AsymptoticLimits
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
SimNLL created with 15 channels, 0 generic constraints, 181 fast gaussian constraints, 0 fast poisson constraints, 0 fast group constraints,
Minimization failed in an unrecoverable way
Minimization failed in an unrecoverable way

 -- AsymptoticLimits ( CLs ) --
Observed Limit: r < 1.2718
Expected  2.5%: r < 0.0078
Expected 16.0%: r < 0.0087
Expected 50.0%: r < 0.0625
Expected 84.0%: r < 0.0665
Expected 97.5%: r < 0.0670

Done in 18.35 min (cpu), 18.37 min (real)

This would ideally report more meaningful errors.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions