Wrong poi_upper_limit computation with pyhf backend about spey HOT 5 CLOSED

Apparently it worked up to the commit 74a8f1c (Wed Feb 22 16:33:26).

from spey.

Hi @APMDSLHC, I did some major changes in the code structure which fixed the issue on my end but please do all your checks again because these changes are big. So you can modify your code as follows

statModel = spey.get_multi_region_statistical_model("test", patch, bkg)
config = statModel.backend.model.config()
bounds = config.suggested_bounds
bounds[config.poi_index] = (config.minimum_poi, 100)
mu_ul_spey = statModel.poi_upper_limit(
    expected=spey.ExpectationType.observed, allow_negative_signal=False, par_bounds=bounds
)
print("muUL:", mu_ul_spey)
# Out: muUL: 38.38686452333036

Note that from now on, it won't do the adaptive bound adjusting; it seems like that is the thing that was giving you weird results. If you check the CLs distribution with respect to POI in the previous commit, the distribution has a discontinuity which caused by this adaptive bound rescaling. So I think we might need to come up with smarter ways to do the bound adjustment.

Please let me know if you find anything related to this.

from spey.

Thanks for the fixes.
Now I have an error when calling poi_upper_limit with SL backend:

File "/home/pascal/SModelS/spey/src/spey/optimizer/core.py", line 21, in fit
    print("model_configuration.fixed_poi_bounds():",model_configuration.fixed_poi_bounds())
AttributeError: 'list' object has no attribute 'fixed_poi_bounds'

The problem seems to arise because model_configuration is a list of zeros.
Tell me if it's not a straightforward fix, I'll share you my code.

from spey.

Thanks @APMDSLHC, I totally missed this one, I updated the necessary bits. It works on my side now:

stat_model_sl = spey.get_multi_region_statistical_model(
     "simple_sl_test",
     signal=[12.0, 11.0],
     observed=[51.0, 48.0],
     covariance=[[3.,0.5], [0.6,7.]],
     nb=[50.0, 52.0],
     delta_sys=0.,
     third_moment=[0.2, 0.1],
     xsection=0.5
 )
print(stat_model_sl.poi_upper_limit())
# Out: 0.8704245319083141

from spey.

Thank you, it works well now, I am closing the issue.

from spey.