stefanengblom / stenglib Goto Github PK

Greetings, I am new to this and having hard time running fsparse.m in MATLAB. Can you please guide on how to use it or how to fully install the Fast package?

When generating large matrices it can happen that the number of elements is larger then the max value that can be stored in "int" format. If this happens I use the workaround below to generate the matrix in multiple steps. However this is not ideal. Can the "int" limitation be changed into "long" or would this result in other problems?

if length(MyRowIdx) > intmax('int32')
    
    Idx = false(length(MyRowIdx),1);
    Idx( 1 : intmax('int32') ) = true;
    H = fsparse(MyRowIdx(Idx), MyColIdx(Idx), Val(Idx), [numElem, numElem]);
    
    MyRowIdx(Idx) = [];
    MyColIdx(Idx) = [];
    Val(Idx) = [];
    
    while any(MyRowIdx)
        
        Idx = false(length(MyRowIdx),1);
        Idx( 1 : min( length(MyRowIdx) , intmax('int32') ) ) = true;
        
        H = H + fsparse(MyRowIdx(Idx), MyColIdx(Idx), Val(Idx), [numElem, numElem]);
        
        MyRowIdx(Idx) = [];
        MyColIdx(Idx) = [];
        Val(Idx) = [];
        
    end
    
else
    % we can generate the matrix in a single fsparse call
    H = fsparse(MyRowIdx, MyColIdx, Val, [numElem, numElem]);
end

Dear Stefan,
Just to let you know I had to move make.m up a directory from Fast/source to Fast/ since it expects to look into pwd / source. Then it built fine. I'm on ubuntu 16.04 linux, Matlab R2017a.

But on 4-core i7 it doesn't beat the native matlab by much (here openmp=true):

% test speed of forming sparse mats, including stenglib. Barnett 10/10/18
addpath ~/matlab/stenglib/Fast
clear
n = 1e5;
nnz = 3e7;
i=randi(n,nnz,1);
j=randi(n,nnz,1);
a = rand(nnz,1);
tic; A = sparse(i,j,a,n,n); toc % single thread
tic; A = fsparse(i,j,a,[n,n,nnz]); toc % is multi thread
tic; A = fsparse(i,j,a,[n,n,nnz],'nosort'); toc % seems single thread
tic; [i,x]=sort(i); j=j(x);a=a(x); A = fsparse(i,j,a,[n,n,nnz]); toc % slowest
tic; A = fsparse(i,j,a,[n,n,nnz]); toc % after sort, fast

Elapsed time is 4.991984 seconds.
Elapsed time is 3.538113 seconds.
Elapsed time is 3.828368 seconds.
Elapsed time is 5.219050 seconds.
Elapsed time is 1.020627 seconds.

With openmp off the 'nosort' case is still 4.4 s, ie faster than matlab.

I imagine this is because it's memory-bandwidth limited. I'll try xeon next. Anyway, thanks for writing it!
Alex

Hi Stefan,

thanks a lot for you awesome library! I'm using it a lot.
Just a minor question: Did you compare your repeat function with MATLAB's repelem? If I'm not mistaken they are doing something very similar. repelem also seems to be native code.