gasparl / possa Goto Github PK

Same as local alphas, the futility bounds could automatically be adjusted to achieve any given level of desired power (beta). This is not a priority, but would be nice to have. (As for the mechanism, the same staircase procedure could be used as for adjusting local alphas for T1ER.)

Perhaps the sim function could allow the user to create the sample for each look via a custom user-function (instead of subsampling internally from the maximum sample). However, this is not a priority; I can't actually think of a scenario where this would be needed, but it might be nice to provide this option, to be sure.

Your repo contains two LICENSE files, one plain-text (as required by JOSS) and one in Markdown (.md). Do you really need both? If so, could you make sure that the plain-text file is identical to the other?

This relates to JOSS submission openjournals/joss-reviews#4643

Instead of any specific number, the number of iterations (i.e., how many times the simulated experiment is run) should be decided on in an objective manner – preferably based on when the values of interest are "sufficiently accurate" (e.g., narrow enough CI, etc.).

More practical examples should be added to a dedicated vignette (e.g., unequal samples and SD; ranked data; DeLong's test; AOV).

Hi, @gasparl, you package looks nice! I see there are many package contains a code of conduct. I suggest you may consider to have that:)

Please add a reference for Pocock's correction in the JOSS paper.

This relates to the JOSS submission openjournals/joss-reviews#4643.

Although the POSSA internals are very fast and there seems to be little else to help speed, it would be great to somehow make the simulations faster.

For example, one way might be to have an option to provide total population once to the function (for each given sample), so that, instead of each time calling a sampling function, the population can be simply sampled in base R. Then again, the speed gain is probably not much with this, because the most time-consuming part is the statistical testing, which however seems inevitable.

There could be a function to plot the distribution (density) of p values, with vertical line(s) indicating the given alpha(s) and corresponding Type I and Type II errors highlighted under the density curves with color fills. It could also be interactive somewhat like this, perhaps using Shiny.

Then again, all this seems pretty complicated to implement with sequential testing, let alone with multiple hypotheses. Also it's not crucial. Still, it would be insightful and instructive, so would be good to have eventually.

An obvious strength of the package is its flexibility to perform power analyses for almost any number of pretty much any statistical test in R. However, this requires that the user code their own test function such as:

customTest = function(sample1) {
  c(
    a = t.test(...)$p.value,
    b = aov(...)[[1]][[1]][["Pr(>F)"]][2]
  )
}

As you also point out in your vignettes, this can be somewhat tricky for ANOVA or regression fits but even adding the unusual $p.value after the t-test call might be somewhat difficult to understand for the average R user. Would there perhaps be a way of supplying some wrapper function that automatically extracts the desired p-value from common (not all) test objects so that the user can just write something like the following?

customTest = function(sample1) {
  c(
    a = extract(t.test(...)),
    b = extract(aov(...), "FactorA:FactorB")
  )
}

This relates to the JOSS submission openjournals/joss-reviews#4643.

The outcomes of the actual experiment in case of a sequential design should be adjusted; see the related SO question for details and ideas.