Giter VIP home page Giter VIP logo

Comments (7)

martinghunt avatar martinghunt commented on June 30, 2024

Sorry but there's no option to have the script (I presume you're using) clockwork variant_call_one_sample use more than 1 core/thread. It uses several tools, most of which don't support >1 thread.

I recommend you run the script on multiple samples in parallel to maximise use of your computer.

from clockwork.

dhcvn avatar dhcvn commented on June 30, 2024

I am running a batch of sample normally 5 samples in a .sh file. Is that the way running multiple samples in parallel which you mention?

from clockwork.

martinghunt avatar martinghunt commented on June 30, 2024

Depends what's in your .sh file? There are various ways to get multiple jobs running at the same time. One way would be to use parallel. eg put the commands you want to run in one file called commands.txt:

clockwork variant_call_one_sample ref_dir out1 reads1_1.fq reads1_2.fq
clockwork variant_call_one_sample ref_dir out2 reads2_1.fq reads2_2.fq
clockwork variant_call_one_sample ref_dir out3 reads3_1.fq reads3_2.fq
... etc

and then run 3 jobs in parallel with:

cat commands.txt | parallel -j3 {}

from clockwork.

dhcvn avatar dhcvn commented on June 30, 2024

I structure my data folder and then create command ready-to-run sh file using python, eg:

singularity exec clockwork.img clockwork map_reads --unsorted_sam SAMPLENAME Ref.remove_contam/ref.fa SAMPLENAME.sam read1.fastq.gz read2.fastq.gz
singularity exec clockwork.img clockwork remove_contam Ref.remove_contam/remove_contam_metadata.tsv SAMPLENAME.sam SAMPLENAME.decontam.counts.tsv SAMPLENAME.decontam_1.fq.gz SAMPLENAME.decontam_2.fq.gz
singularity exec clockwork.img clockwork variant_call_one_sample --sample_name SAMPLENAME Ref.H37Rv Var_call_SAMPLENAME SAMPLENAME.decontam_1.fq.gz SAMPLENAME.decontam_2.fq.gz

singularity exec clockwork.img clockwork map_reads --unsorted_sam SAMPLENAME2 Ref.remove_contam/ref.fa SAMPLENAME2.sam read1.fastq.gz read2.fastq.gz
singularity exec clockwork.img clockwork remove_contam Ref.remove_contam/remove_contam_metadata.tsv SAMPLENAME2.sam SAMPLENAME2.decontam.counts.tsv SAMPLENAME2.decontam_1.fq.gz SAMPLENAME2.decontam_2.fq.gz
singularity exec clockwork.img clockwork variant_call_one_sample --sample_name SAMPLENAME2 Ref.H37Rv Var_call_SAMPLENAME2 SAMPLENAME2.decontam_1.fq.gz SAMPLENAME2.decontam_2.fq.gz

singularity exec clockwork.img clockwork map_reads --unsorted_sam SAMPLENAME3 Ref.remove_contam/ref.fa SAMPLENAME3.sam read1.fastq.gz read2.fastq.gz
singularity exec clockwork.img clockwork remove_contam Ref.remove_contam/remove_contam_metadata.tsv SAMPLENAME3.sam SAMPLENAME3.decontam.counts.tsv SAMPLENAME3.decontam_1.fq.gz SAMPLENAME3.decontam_2.fq.gz
singularity exec clockwork.img clockwork variant_call_one_sample --sample_name SAMPLENAME3 Ref.H37Rv Var_call_SAMPLENAME3 SAMPLENAME3.decontam_1.fq.gz SAMPLENAME3.decontam_2.fq.gz

Any advise to improve my approach?

from clockwork.

martinghunt avatar martinghunt commented on June 30, 2024

You could make one script that runs map reads, remove contam, variant call for one sample. So you'd call it like this:

process_sample.sh sample_name reads1.fq reads2.fq outdir

The contents of process_sample.sh would be along these lines:

name=$1
reads1=$2
reads2=$3
out=$4

singularity exec clockwork.img clockwork map_reads --unsorted_sam $name Ref.remove_contam/ref.fa $out.sam $reads1 $reads2
... etc for the other 2 commands ...

Then make a file of commands commands.txt:

process_sample.sh sample1 reads1_1.fq reads1_2.fq out1
process_sample.sh sample2 reads2_1.fq reads2_2.fq out2
process_sample.sh sample3 reads3_1.fq reads3_2.fq out3
... etc

then use parallel:

cat commands.txt | parallel -j3 {}

from clockwork.

dhcvn avatar dhcvn commented on June 30, 2024

Hi,
I have another question about /bioinf-tools/cortex/scripts/calling/run_calls.pl.
When I run this, an error occur perl: warning: Setting locale failed.
Can this error effect result of cortex?

from clockwork.

martinghunt avatar martinghunt commented on June 30, 2024

Don't worry about it, it's just a warning you can ignore. It won't affect results

from clockwork.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.