plot_pvalue in multiple data sets fitted simultaneously about sherpa HOT 5 OPEN

Just to point out that the "as you know" part is actually wriong!

from sherpa.

Need to check if this has been fixed in 4.12

from sherpa.

A quick note: most of our functions that allow running multiple datasets have id=None as their default value (e.g. fit, conf, ...) which means that they default to using all datasets by default.

I note that plot_pvalue defaults to id=1, which means that it only uses the default dataset by default. I think it should be changed to id=None to match (and is an easy fix), but if there are difficulties in using with multiple datasets maybe we shouldn't make this change until #112 is resolved.

from sherpa.

I should add, I have no idea about how plot_pvalue is meant to be used, what its arguments are, and what happens for multiple datasets. So I am unlikely to work on this until someone can provide guidance on these issues.

from sherpa.

@DougBurke as you know this function performs simulations assuming a null model and fits the simulated data with a null model and a more complex model to calculate the likelihood ratio between the two at each simulation step. The summary is printed to the screen and the plot shows the histogram of the likelihood ratio from N simulations and the vertical line showing the observed ratio. Also, the simulation results can be obtained with get_pvalue_results and get_pvalue_plot.

The simulations make sense for single data set, or simultaneous fit to multiple data set, so the null and the complex models are well defined. In the case of fitting different data sets with different models plot_pvalue as defined would only make sense for running simulation for each data set separately, which is what one could do with the current implementation.

from sherpa.