Error running this command: /bioinf-tools/cortex/scripts/calling/run_calls.pl about clockwork HOT 11 OPEN

Hi there, could you attach these logs?

Var_call_ERR4822319/cortex/cortex.log

from clockwork.

Thank you very much for your quick response! And the log message is as follows

_Start time: Mon Sep 4 12:29:49 CST 2023

*********** Command-line used was : *********************
perl /bioinf-tools/cortex/scripts/calling/run_calls.pl --fastaq_index /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.in.index --auto_cleaning yes --first_kmer 31 --bc yes --pd no --outdir /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out --outvcf cortex --ploidy 2 --minimap2_bin /bioinf-tools/minimap2 --list_ref_fasta /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.in.index_ref.fofn --refbindir /public/home/bdpstu/cenhui2018/work/MTB/drug_resistant/call_vcf/clockwork/script/Ref.H37Rv --genome_size 4411532 --qthresh 5 --mem_height 22 --mem_width 100 --vcftools_dir /bioinf-tools/vcftools-0.1.15 --do_union yes --ref CoordinatesAndInCalling --workflow independent --logfile /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.log

mkdir -p /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/binaries/
mkdir -p /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/calls/
mkdir -p /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/vcfs/
mkdir -p /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/vcfs/tmp_working/

Build uncleaned binaries:

Build graph using quality threshold 5
/bioinf-tools/cortex//bin/cortex_var_31_c1 --sample_id sample --kmer_size 31 --mem_height 22 --mem_width 100 --dump_binary /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/binaries/uncleaned/31/sample.unclean.kmer31.q5.ctx --dump_covg_distribution /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/binaries/uncleaned/31/sample.unclean.kmer31.q5.ctx.covg --se_list /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.in.fofn --quality_score_threshold 5 > /public/home/bdpstu/cenhui2018/work/MTB/Public/data/20230506/data/Var_call_ERR4822319/cortex/cortex.out/binaries/uncleaned/31/sample.unclean.kmer31.q5.ctx.build_log 2>&1_

from clockwork.

One possible cause is not having enough RAM

from clockwork.

How can I solve this problem? How to set?
Thanks again for your reply！

from clockwork.

how much RAM does your machine have?

from clockwork.

This is my memory information.

from clockwork.

righto, RAM is not the problem then! It peaks at something like 10GB. Must be something else (@iqbal-lab is the cortex expert).

I'll download the reads and see if I can reproduce

from clockwork.

I've reproduced this error.

There's a cortex option that defines how much memory it can use. The clockwork defaults usually work 99% of the time. This is a rare case where cortex needs more memory. There's the log message:

tail -n1 cortex/cortex.out/binaries/uncleaned/31/sample.unclean.kmer31.q5.ctx.build_log
Error: Dear user - you have not allocated enough memory to contain your sequence data. Either allocate more memory (have you done your calculations right? have you allowed for sequencing errors?), or threshold more harshly on quality score, and try again. Aborting mission.

If you rerun reference_prepare but with --cortex_mem_height 23, and then run variant_call_one_sample again with --cortex_mem_height 23 and pointing at the newly made reference_prepare output directory, it should work.

from clockwork.

I used --cortex_mem_height 23 parameter and reran reference_prepare. The command is as follows.

singularity exec clockwork_v0.11.3.img clockwork reference_prepare --cortex_mem_height 23 --contam_tsv Ref.download/remove_contam.tsv --outdir Ref.remove_contam.0905 Ref.download/remove_contam.fa.gz

singularity exec clockwork_v0.11.3.img clockwork reference_prepare --cortex_mem_height 23 --outdir Ref.H37Rv.0905 Ref.download/NC_000962.3.fa

ERR4822319 was analyzed next with the following command.

singularity exec clockwork_v0.11.3.img clockwork map_reads --threads 4 --unsorted_sam ERR4822319 Ref.remove_contam.0905/ref.fa ERR4822319.sam ERR4822319/ERR4822319_1.fastq.gz ERR4822319/ERR4822319_2.fastq.gz

singularity exec clockwork_v0.11.3.img clockwork remove_contam Ref.remove_contam.0905/remove_contam_metadata.tsv ERR4822319.sam ERR4822319.decontam.counts.tsv ERR4822319.decontam_1.fq.gz ERR4822319.decontam_2.fq.gz

The above two steps can run smoothly.

But when I did variant_call_one_sample, (command below) appeared:

clockwork: error: unrecognized arguments: --cortex_mem_height

so I removed --cortex_mem_height 23, then the previous error occurred again.

Error running this command: /bioinf-tools/cortex/scripts/calling/run_calls.pl

Is there something I did wrong please, thanks for replying again!

PS: See picture for my memory info.

from clockwork.

Sorry, I typo'd the option. It's clockwork variant_call_one_sample --mem_height 23 ...

from clockwork.

The program ran smoothly, thank you very much for your help！

from clockwork.