Early crash in multithreaded mode about octopus HOT 4 CLOSED

Thanks for the bug report. The problem is that the operating system maximum open file limit is being exceeded (i.e. ulimit -n). When calling with multiple threads, Octopus creates a temporary BCF file for each unique contig in the calling region set. Since you're not explicitly specifying any input regions in your command, the default behaviour is it to call all contigs in the reference genome. That's all 3366 contigs in GRCh38_full_analysis_set_plus_decoy_hla.fa, even though reads are only mapped to chr1.

A robust solution to this is not going to be simple. However, it's fairly easy for you to avoid this problem by either:

• Increasing the maximum open file limit (e.g. set ulimit -n 10000).
• Manually specify input regions (e.g. add -T chr1 to your command).

from octopus.

For my own information, some ideas to help - but not solve - this problem:

• Only open temporary contig BCFs when the first calls from the contig are in.
• Close temporary contig BCFs once all calls for the contig are written.
• Improve exception handling/reporting when file opening fails.

from octopus.

Thanks; adding -T chr1 does work around the problem.

from octopus.

From 42fa364, Octopus will open and close temporary BCFs as needed, which should reduce the occurrence of this type of error. There is also better error reporting. You should now be able to run your command without the -T chr1 (although I'd still recommend including it if you're only calling chromosome 1).

from octopus.