Right now all functions work on rows by default and column versions transpose the input data and call the corresponding row function.

However in R column-wise functions should be about 2x faster due to how the matrices are stored in memory. Therefore it's worth investigating if making the test functions work on columns instead of rows could add additional boost to performance.

Is there anyway to include a padj for multiple test correction using Bonferroni correction or Benjamini-Hochberg procedure?

Double-check and synchronize the way different tests handle Inf values.

Right now some of the non-parametric tests treat the Inf/-Inf values as highest/lowest ranks. However the behaviour for parametric tests are not fully defined. In particular when both Inf and -Inf values are present within the same group.

• t_equalvar
• t_paired
• t_onesample
• t_welch
• cor_pearson
• f_var
• oneway_equalvar
• oneway_welch
• kruskalwallis
• waerden
• bartlett
• jarquebera
• flignerkilleen
• levene
• brownforsythe
• wilcoxon_onesample
• wilcoxon_paired
• wilcoxon_twosample
• cosinor
• andersondarling

Consider changing the behaviour of conf.level=NA from throwing an error to suppressing the result of confidence interval. In most cases this should provide a considerable improvement in speed.

• t_equalvar
• t_paired
• t_onesample
• t_welch
• cor_pearson
• f_var

Right now the name for this argument is copied from base. In turn this can change from on test to another. As an example: it is called mu in t.test() but ratio in var.test().

Probably should always be named null.

Unit test coverage now is 97%. Only column-wise function variants seem to be missing.

Add a KS test to compare distribution of two samples (ks.test() in base R stats package).

For now focus on two sample variant and name it row_kolmogorovsmirnov_twosample().

Later, if necessary, a version that compares a single sample against a named distribution can be added.

Currently row_ievora calls row_bartlett and row_t_welch. Both these functions do some preprocessing on their own.

Rewriting row_ievora to have all the needed code to run the tests within itself would improve the speeed and handling of warnings.

Currently row_wilcoxon_twosample does not produce a warning when all the values are constant and equal to the NULL:

row_wilcoxon_twosample(c(1,1,1), c(2,2,2), exact=FALSE, null=1)
# NaN p-value but no warning

In a similar situation row_wilcoxon_onesample() can end up producing three warnings for the same row:

row_wilcoxon_onesample(c(0,0,0))
# 3 separate warnings are produced

And the above issue is also present in a paired version.

Solution: add the warning for "twosample" case and adjust the warnings so that only one is produced at the time.

After R 4.0.0. wilcox.test() will change it's behaviour with Infinite values. Once R 4.0.0. is out we need to make sure row_wilcox_test() will handle Inf values in the same way wilcox.test() does.

Travis is failing for a while now, instead of spending time fixing it we can take this opportunity to try out GitHub actions.

Right now some of the warning messages are a bit different between tests. For example:

row_oneway_equalvar: 1 of the rows had essentially constant values.\nFirst occurrence at row 1
row_kruskalwallis: 1 of the rows were essentially constant.\nFirst occurrence at row 1

This is a bug reported by @Close-your-eyes via email.

m1 <- matrix(rnorm(100000), nrow=4)                                                                                                                                                                                                 
m2 <- matrix(rnorm(1000000), nrow=4)

row_wilcoxon_twosample(m1, m2)

obs.x  obs.y obs.tot  statistic pvalue location.null alternative exact corrected
1 25000 250000  275000 3130434539     NA             0   two.sided FALSE      TRUE
2 25000 250000  275000 3118218180     NA             0   two.sided FALSE      TRUE
3 25000 250000  275000 3141448608     NA             0   two.sided FALSE      TRUE
4 25000 250000  275000 3117295315     NA             0   two.sided FALSE      TRUE
   
Warning messages:
1: In nx * ny : NAs produced by integer overflow
2: In nx * ny : NAs produced by integer overflow

This is caused by storing matrix dimensions as integers - then wilcox test multiplies number of observations from both samples together which leads to integer overflow. Solution is to store those values as numeric.

Fisher's exact G test is used to test the null hypothesis of Gaussian white noise against the alternative of an added deterministic periodic component in a timeseries.

In R the test is implemented by fisher.g.test function in a GeneTS package. It should be useful for detecting hidden periodicities.

https://en.wikipedia.org/wiki/Boschloo%27s_test

Thanks for this amazing package! I have used it in so many projects already!

I just wanted to check if there are plans to support sparse matrices as well any time soon?

"Cucconi test" is an interesting test for both location and scale.

Seems like NULL distribution is based on permutations but that shouldn't be an issue.

• Remove testthat dependency
• Try to automate the parameter testing based on input argument types

A test for location difference between multiple groups. Seems to be somewhat in the middle between ANOVA and Kruskal-Wallis. Wiki.

Need to come up with a name and an interface. Current proposal:

• Name: row_lm_f()
• Arguments: 1) data matrix 2) alternative model matrix 3) null model matrix

Have to do a lot of testing in order to make sure it will produce the same output as lm(). Keep track of pivoting.

For testing could be tuned with ad.test() from the nortest package.

Right now parameters are expanded only in a few special cases. For example when x has 10 rows and y has 1 row - y will be repeated 10 times to match x.

We can add this for other parameters as well, like conf.level. This would allow running same test on the same input with different confidence levels in parallel.

Hi,
First thanks a lot for developing such an awesome package! It gives me a lot of help!
I noticed that only pearson correlation is allowed now, is it possible for you to add spearman correlation in cor_test?

Thanks!

Yang

https://en.wikipedia.org/wiki/Barnard%27s_test

For now just a place holder as a reminder that such a test might be useful in this package.

O'Brien's test is yet another test to for homogeneity of variances.

Currently row_wilcoxon_* tests do not return the pseudo-median estimates nor confidence intervals. The tests in base R can return confidence intervals when asked: wilcox.test(..., conf.int=TRUE).

The problem is that it's hard to speed up confidence interval calculations for this tests so for now they are not implemented. But would be a nice addition in the future.

Would be nice to try improving the speed further. However cannot reliably do this without having benchmarks first.

To reproduce:

mat <- matrix(c(0,0,1,1,0,0,1,1),nrow=2)
grps <- c(0,1,0,1)
res <- row_oneway_welch(mat,grps)
res <- matrixTests::row_oneway_welch(mat,grps)

Thanks to @matthewcarlucci for pointing this out.

Implement Fisher's exact test for a 2x2 case.

Input can be a pair of matrices. For a row-wise case each row of both matrices will have 2 unique levels (logical, factor, character or even numeric).

Implement dip test. The original R implementation is available in the diptest library.

Add Cuzick test, possibly using PMCMRplus::cuzickTest implementation as reference.

Currently the col_ versions of all tests transpose the inputs and call row_ functions.

This behaviour is undesirable because of at least three reasons:

1. Transposing inputs takes time.
2. col-wise functions (like colMeans) are typically faster than their row equivalents
3. When inputs have different lengths the functions might throw a warning about rows.

• row_t_onesample(x)
• row_t_welch(x, y)
• row_t_equalvar(x, y)
• row_t_paired(x, y)
• row_wilcoxon_onesample(x)
• row_wilcoxon_twosample(x, y)
• row_wilcoxon_paired(x, y)
• row_cor_pearson(x, y)
• row_oneway_welch(x, g)
• row_oneway_equalvar(x, g)
• row_kruskalwallis(x, g)
• row_f_var(x, y)
• row_bartlett(x, g)
• row_flignerkilleen(x, g)
• row_jarquebera(x)

Currently, when a test for a specific row cannot be executed because of special corner cases like constant values, only test statistic, p-value, and confidence intervals are turned to NA. As a result, in some situations, degrees of freedom can be still returned as 0 or even -1, and standard-error often is returned as NaN.

Better - turn all outputs computed by the test to NA, except for descriptive statistics like mean, number of observations, etc.