Add paired Cohen's d about dabest-python HOT 4 CLOSED

Joses, I think this problem is just my inexperience with DABEST. Looking at running the Cohen’s d function s_control.cohens_d DABEST v0.3.0 ============= Good morning! The current time is Thu Apr 16 11:45:23 2020. The unpaired Cohen's d between OMEGA and MOE is 0.182 [95%CI -0.0108, 0.367]. The p-value of the two-sided permutation t-test is 0.0704. The unpaired Cohen's d between OMEGA and Macromodel is 0.0295 [95%CI -0.168, 0.215]. The p-value of the two-sided permutation t-test is 0.766. The unpaired Cohen's d between OMEGA and Desmond is 0.37 [95%CI 0.162, 0.572]. The p-value of the two-sided permutation t-test is 0.0002. The unpaired Cohen's d between OMEGA and RDKit is 0.593 [95%CI 0.36, 0.805]. The p-value of the two-sided permutation t-test is 0.0. The unpaired Cohen's d between OMEGA and Prime is -0.0323 [95%CI -0.224, 0.162]. The p-value of the two-sided permutation t-test is 0.747. 5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated. The p-value(s) reported are the likelihood(s) of observing the effect size(s), if the null hypothesis of zero difference is true. For each p-value, 5000 reshuffles of the control and test labels were performed. To get the results of all valid statistical tests, use `.cohens_d.statistical_tests` DABEST says it is reporting unpaired Cohen’s d. When looking at the results of s_control.cohens_d.statistical_tests There is a column ‘is_paired’ which is set to False, so I thought that being able to set it to True would solve my problem, but I could not find a way to do that. However the values returned by the ‘cohens_d’ function are more or less identical to the values I get for paired d, so this appears to be a problem that has solved itself. However, if I run s_control = db.load(new_df,idx=('OMEGA','MOE','Macromodel','Desmond','RDKit','Prime'),paired=True) DABEST returns an error ValueError: `is_paired` is True, but some idx in ('OMEGA', 'MOE', 'Macromodel', 'Desmond', 'RDKit', 'Prime') does not consist only of two groups. It seems like DABEST only allows paired tests between two sets of data, while paired comparisons can be carried out on three or more sets of data. As you can see this is with DABSET 0.3.0. Paul. From: Joses W. Ho <[email protected]> Sent: Thursday, April 23, 2020 4:49 AM To: ACCLAB/DABEST-python <[email protected]> Cc: Paul Hawkins <[email protected]>; Mention <[email protected]> Subject: Re: [ACCLAB/DABEST-python] Add paired Cohen's d (#99) Hi @paul-hawkins<https://github.com/paul-hawkins>, Are you asking how to compute paired Cohen's d? Or are you saying that the paired Cohen's d returned by DABEST is not actually paired? If you question is the first one, simply load an experiment as a paired experiment: import pandas as pd import dabest # Load the iris dataset. Requires internet access. iris = pd.read_csv("https://github.com/mwaskom/seaborn-data/raw/master/iris.csv") iris.reset_index(inplace=True) virginica = iris[iris.species=="virginica"].copy() virginica_melted = pd.melt(virginica, id_vars="index", value_vars=["sepal_length", "petal_length"], var_name="flower_part", value_name="width") virginica_paired = dabest.load(data=virginica_melted, x="flower_part", y="width", paired=True, id_col="index", idx=("sepal_length", "petal_length")) then produce the Cohen's d: virginica_paired.cohens_d DABEST v0.3.0 ============= Good evening! The current time is Thu Apr 23 18:44:08 2020. The paired Cohen's d between sepal_length and petal_length is -1.74 [95%CI -2.1, -1.37]. The p-value of the two-sided permutation t-test is 0.0. 5000 bootstrap samples were taken; the confidence interval is bias-corrected and accelerated. The p-value(s) reported are the likelihood(s) of observing the effect size(s), if the null hypothesis of zero difference is true. For each p-value, 5000 reshuffles of the control and test labels were performed. To get the results of all valid statistical tests, use `.cohens_d.statistical_tests` If you are saying the latter (ie the paired Cohen's d returned by DABEST is not actually paired), could you provide a dummy dataset with the expected accurate values, vis-a-vis what DABEST produces? Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#99 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AE6VZPUBJQEXUNRJWCHCENLROAMJXANCNFSM4MMXABCQ>.