Some of the compressed sparse matrix method implementations seem to have been translated directly from CSparse. E.g:

https://github.com/rwl/ParallelColt/blob/master/src/main/java/cern/colt/matrix/tdouble/impl/SparseRCDoubleMatrix2D.java#L710

https://github.com/rwl/CSparseJ/blob/master/src/main/java/edu/emory/mathcs/csparsej/tdouble/Dcs_dupl.java#L44

CSparse is distributed under the LGPL. The LGPL is not so permissive as the CERN license under which the cern.colt.matrix.* packages are provided.

Hey there,

In sparse complex matricies you get an ArrayIndexOutOfBounds error, when an element (although existing) is removed, because it turns 0.0.

This line is causing the problem:

It should look like this line:

Because it is complex the matrix contains twice as many elements.

If you prefer a PR please let me know.

Ciao
Andi

sincere thank for you

Could you please update the release on maven central that the latest changes are included?

If not, I can do it but with another group ID: de.biomedical-imaging.parallelcolt

The Colt license states that its bundled copy of FreeHEP AIDA is licensed as follows: "Permission to use and/or redistribute this work is granted under the terms of the LGPL License, with the exception that any usage related to military applications is expressly forbidden."

The upstream AIDA interfaces are licensed under the actual LGPL. If the version that Colt incorporates was originally licensed under the LGPL, then the additional usage restrictions are invalid, since the LGPL does not permit adding such restrictions when redistributing LGPL software.

If the version that Colt incorporates was originally licensed under the LGPL with those added usage restrictions, then it would make it much easier for downstream packagers (like Fedora and Debian) to incorporate Colt and software depending on Colt if you were to move to depend instead on a more recent version of AIDA that is available under the actual LGPL.

(The ideal solution would be to not bundle AIDA at all and add it as a maven dependency instead; then Colt could benefit from improvements to AIDA itself as well as to its license.)

Here is a failing test from parallel colt 0.10.0

  @Test
  public void testLargeSparseSum() {
    int size = 110000;
    DoubleMatrix2D m = new SparseDoubleMatrix2D(size, size);
    int i = 100000;
    assertEquals(0, m.viewRow(i).zSum(), 0.0001);
    m.set(i, i, 5);
    assertEquals(5, m.viewRow(i).zSum(), 0.0001);
  }

expected:<5.0> but was:<0.0>

m.viewRow(i).toString() :
1 x 110000 sparse matrix, nnz = 0

Strangely enough, columns work:
m.viewColumn(i).toString()

1 x 110000 sparse matrix, nnz = 1
(100000) 5.0
.get(i,i) does work as well

At the moment, the latest version published to Maven Central is v0.10.1:
https://mvnrepository.com/artifact/net.sourceforge.parallelcolt/parallelcolt

v0.11.4 contains an important fix for #6.

@rwl @amalagaura @thorstenwagner, could you pls also address the remaining 'duplicate class' issues reported in
#6 (comment) ?

Hi,

Is there a more efficient to store a symmetric DoubleMatrixDataset2D?
I mean when a matrix is an exact equal to its transposed self, storing values twice for large matrices is not efficient. Is there already an implementation of a matrix that can more efficiently store this?

Cheers,
Marc Jan Bonder

As reported by Laurent Condamin:

"The meaning of the parameter for the Gamma distribution are different for the sampler cern.jet.random.tdouble.Gamma.nextDouble() and for the statistical distribution cern.jet.stat.tdouble.Probability.gamma(). alpha and beta parameters are mingled in the
cern.jet.stat.tdouble.Probability.gamma(). As a consequence, when you invoke
cern.jet.random.tdouble.Gamma.cdf(x), it invokes cern.jet.stat.tdouble.Probability.gamma(alpha,beta,x) whereas gamma() mingles alpha and beta.

To get the good result, cern.jet.random.tdouble.Gamma.cdf(x) should
invoke cern.jet.stat.tdouble.Probability.gamma(beta,alpha,x)."

Is it possible to make parallelcolt to work with larger than 2^31 cell matrices?
I want to create a matrix of 7.700 columns by 350.000 rows and it crashes with the error that the matrix is to large.

Thanks,
Marc Jan

Hi Richard,

The class is:
https://github.com/rwl/ParallelColt/blob/master/src/main/java/edu/emory/mathcs/utils/ConcurrencyUtils.java
The dependency to jplasma adds a modified version of this class to the class path.
This leads to problems when attempting to package with sbt assembly. One way to solve it would be to remove the class and use the one from the dependency. Alternatively, if it really does something differently, it could get a new name/package to get rid of the ambiguity.

Thanks and kind regards,
Philip

I am doing dot products of large sparse matrixes. I pass in C to zmult as a new sparse matrix. Works fine at 30k x 30k. At approx 100k x 100k (well under max int), it runs for 80min. After it finishes I run forEachNonZero and it gives negative row and column indices.

My guess is that it is having issues at values greater than square root of max int. There is some int overflow. An int which should be a long?

Hi I am having an issue where I get an java.lang.ArrayIndexOutOfBoundsException in edu.emory.mathcs.utils.pc.ConcurrencyUtils from ParallelColt v0.11.4:

java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 8988800
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at edu.emory.mathcs.utils.pc.ConcurrencyUtils.waitForCompletion(ConcurrencyUtils.java:176)
	at edu.emory.mathcs.restoretools.iterative.wpl.WPLFloatIterativeDeconvolver3D.convolveFD(WPLFloatIterativeDeconvolver3D.java:443)
	at edu.emory.mathcs.restoretools.iterative.wpl.WPLFloatIterativeDeconvolver3D.deconvolve(WPLFloatIterativeDeconvolver3D.java:313)

Does anyone have any idea why this problem may be occurring? It is occurring on the waitForCompletion() method:

    /**
     * Waits for all threads to complete computation.
     * 
     * @param futures
     *            handles to running threads
     */
    public static void waitForCompletion(Future<?>[] futures) {
        int size = futures.length;
        try {
            for (int j = 0; j < size; j++) {
                futures[j].get();
            }
        } catch (ExecutionException ex) {
            ex.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

The exception is thrown on the futures[j].get(); line, however I wonder if the problem is to do with JTransforms? As in ImageJ the current build uses version 2.4, but I believe this copy of ConcurrencyUtils is designed to work with Transforms v3.1?

Although I get the error, the deconvolution I am running still seems to continue.

As you can see, I am using the ParallelIterativeDeconvolution code designed by Piotr Wendykier - I have changed its dependency from its internal edu.emory.mathcs.utils.ConcurrencyUtils to the edu.emory.mathcs.utils.pc.ConcurrencyUtils found in ParallelColt v0.11.4. You can see this in a fork repo I made here - https://github.com/stevenjwest/iterativedeconv

• I changed this dependency because the code is run in imagej, and imagej contains other JARs that also contain the edu.emory.mathcs.utils.ConcurrencyUtils package/class! Specifically:

  • net.sourceforge.jplasma:jplasma:jar:1.2.0

  • net.sourceforge.jtransforms:jtransforms:jar:2.4.0