Crashed building contigs about soapdenovo2 HOT 10 CLOSED

With max_rd_len set to 150 and nothing else changed, the pregraph completed fine and contig building is now running without issue.

from soapdenovo2.

The "Ran out of memory while applying" error came from the ckalloc function in the file standardPregraph/check.c. The ckalloc function takes a single parameter (size) with type "unsigned long long". So I don't think ckalloc is the problem, instead, some code elsewhere calling ckalloc should be the culprit, but I don't have the stack trace from you. Would it be possible for you to trace down who has called ckalloc, and probably, the problem could be solved as easy as changing the type of the variable that stores the size of memory to be allocated to 64bit.

from soapdenovo2.

from soapdenovo2.

I did two checks to be sure. The size_t is indeed 8 bytes just like unsigned long long so that's not the problem. It turns out that the issue is two fold. The error us displaying a signed long long (%lld) at line 132 of check.c so changing that to an unsigned long long (%llu) results in the true number which is 18446744060029749504 bytes, in other words 18446744 TB so my pathetic 6TB isn't going to do it. I've obviously got a lot of input data here but my suspicion is the use of 250bp for max_rd_len because I've run this assembly previously with just 150bp max_rd_len and about 20% less input data and it completed within 2TB of RAM. I'm running again now having set max_rd_len to 150bp again and I'll see what happens. It is clearly loading the data more quickly as I only started it yesterday and it has loaded half the data already.

from soapdenovo2.

I'm still a bit confused of how the problem was caused by changing %lld to %llu if it's a cast, not a pointer reference but please let me know if you want to propose a fix to the code. And please let me know how your new run using 150bp as max_rd_len goes. Thank you.

from soapdenovo2.

from soapdenovo2.

Regarding using sparse_pregraph, its memory efficiency depends very much on the complexity of the genome you are assembling. At this point I strongly suggest you to stick to pregraph. If it doesn't work with 150bp either, I would suggest you to use Megahit to create contigs first. Megahit uses about 4 times less memory than SOAPdenovo, and the contigs could be further assembled into scaffolds using the finalfusion module in SOAPdenovo.

from soapdenovo2.

The genome is highly repetitive (around 80%) but also very large. I've previously assembled it using a subset of the data I have using 150bp PE reads plus the jumping libraries but it was quite fragmented producing 31 million scaffolds. I had less memory at the time and with more I thought I could be more ambitious but I think I'll dial it back to closer to the successful run and build up from there. I'll have a look at Megahit and finalfusion if this current run doesn't get past the contigs. Thanks!

from soapdenovo2.

With max_rd_len set to 150 and nothing else changed, the pregraph completed fine and contig building is now running without issue.

Thank you very much, this solved my problem

from soapdenovo2.

I set max_rd_len to 150, but I still have this problem with K= 29, but no error with K=127, what is the possible problem?

from soapdenovo2.