pengni / ccsmethphase Goto Github PK
View Code? Open in Web Editor NEWMethylatino phasing using PacBio CCS reads
License: MIT License
Methylatino phasing using PacBio CCS reads
License: MIT License
Run command๏ผnextflow run main.nf --dsname test -profile singularity,test
Error information๏ผ
[process] DSS_callDMR
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (value) bound ; channel: ch_utils
port 2: (cntrl) - ; channel: $
Mar-01 17:31:40.300 [main] DEBUG nextflow.Session - Session await > all processes finished
Mar-01 17:31:40.301 [main] DEBUG nextflow.Session - Session await > all barriers passed
Mar-01 17:31:40.332 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (3)
Mar-01 17:31:40.382 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill CheckCMModel (CheckCMModel) -- command: kill -TERM 215509; exit: 1
bash: line 0: kill: (215509) - No such process
Mar-01 17:31:40.411 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: SAMTOOLS_index_bam (hg002.chr20_demo.hifi); status: COMPLETED; exit: 255; error: -; workDir: /mnt/Data_disk/zhoubiaofeng/opt/ccsmethphase/ccsmethphase/work/2b/7203a95eebc6f145e92847bfb52178]
Mar-01 17:31:40.412 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=SAMTOOLS_index_bam (hg002.chr20_demo.hifi); work-dir=/mnt/Data_disk/zhoubiaofeng/opt/ccsmethphase/ccsmethphase/work/2b/7203a95eebc6f145e92847bfb52178
error [nextflow.exception.ProcessFailedException]: Process SAMTOOLS_index_bam (hg002.chr20_demo.hifi)
terminated with an error exit status (255)
Mar-01 17:31:40.415 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Mar-01 17:31:40.452 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=2; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=2; succeedDuration=0ms; failedDuration=1m 44s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=4; peakCpus=20; peakMemory=48 GB; ]
Mar-01 17:31:40.452 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Mar-01 17:31:40.455 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Mar-01 17:31:42.313 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Mar-01 19:20:03.349 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Mar-01 19:20:03.386 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye
Any suggestions to fix this problem?
Hello from a new users
It seems that the output path is not set, How should I set the output path for my file?
And another question about Will you consider directly performing DMR analysis on data from different groups within the program?
Hi Peng,
I am trying to test run ccsmethphase on the demo dataset but am running into an issue that is causing the program to error out when trying to run WhatsHap. See below:
xecutor > slurm (7)
[15/09d90d] process > CheckGenome (CheckGenome) [100%] 1 of 1, cached: 1 โ
[61/d51248] process > CheckCMModel (CheckCMModel) [100%] 1 of 1, cached: 1 โ
[72/e313a6] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, cached: 1 โ
[- ] process > CCSMETH_pbccs_call_hifi -
[2e/bef2a4] process > SAMTOOLS_index_bam (hg002.c... [100%] 1 of 1, cached: 1 โ
[03/683597] process > CCSMETH_call_mods_denovo (h... [100%] 1 of 1, cached: 1 โ
[7b/797f14] process > CCSMETH_align_hifi (hg002.c... [100%] 1 of 1, cached: 1 โ
[69/00f338] process > SAMTOOLS_merge_sortedbams (... [100%] 1 of 1, cached: 1 โ
[f9/57dc73] process > CLAIR3_hifi (G1.HG002_demo.... [100%] 1 of 1, cached: 1 โ
[15/c50c48] process > WHATSHAP_snv_phase_haplotag... [100%] 3 of 3, failed: 2...
[35/c32f06] process > CCSMETH_call_freq_bam (G1.H... [100%] 1 of 1 โ
[9f/f804ca] process > DSS_callDMR (G1.HG002_demo) [ 80%] 4 of 5, failed: 4...
[b8/2f54b0] NOTE: Process DSS_callDMR (G1.HG002_demo)
terminated with an error exit status (2) -- Execution is retried (3)
[3f/0ae8d3] NOTE: Process DSS_callDMR (G1.HG002_demo)
terminated with an error exit status (2) -- Execution is retried (4)
hej,
I am trying to run the demo data but it throws me an error in the last steps:
[71/07e145] process > CheckGenome (CheckGenome) [100%] 1 of 1, cached: 1 โ
[2e/b2d050] process > CheckCMModel (CheckCMModel) [100%] 1 of 1, cached: 1 โ
[0e/b4443f] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, cached: 1 โ
[- ] process > CCSMETH_pbccs_call_hifi -
[0a/680851] process > SAMTOOLS_index_bam (hg002.chr20_demo.hifi) [100%] 1 of 1, cached: 1 โ
[f5/7081a8] process > CCSMETH_call_mods_denovo (hg002.chr20_demo.hifi) [100%] 1 of 1, cached: 1 โ
[f6/cab01f] process > CCSMETH_align_hifi (hg002.chr20_demo.hifi.ccsmeth.modbam) [100%] 1 of 1, cached: 1 โ
[13/245444] process > SAMTOOLS_merge_sortedbams (G1.HG002_demo) [100%] 1 of 1, cached: 1 โ
[d3/b24dee] process > CLAIR3_hifi (G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1) [100%] 1 of 1, cached: 1 โ
[59/a3fc4f] process > WHATSHAP_snv_phase_haplotag (G1.HG002_demo) [100%] 1 of 1, cached: 1 โ
[6e/dc1920] process > CCSMETH_call_freq_bam (G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap) [100%] 1 of 1, cached: 1 โ
[02/3aaa5b] process > DSS_callDMR (G1.HG002_demo) [100%] 1 of 1, failed: 1 โ
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'DSS_callDMR (G1.HG002_demo)'
Caused by:
Process DSS_callDMR (G1.HG002_demo)
terminated with an error exit status (2)
Command executed:
date; hostname; pwd
if [[ 3 -gt 3 ]] ; then
bed_hp1=$(find -L ./ -type f -regex '..aggregate.hp1.bed')
bed_hp2=$(find -L ./ -type f -regex '..aggregate.hp2.bed')
else
bed_hp1=$(find -L ./ -type f -regex '..hp1.bed')
bed_hp2=$(find -L ./ -type f -regex '..hp2.bed')
fi
echo ${bed_hp1}
echo ${bed_hp2}
file_prefix=$(echo ${bed_hp1%1.bed})
python ~/tools/pacbio_5mc/scripts/hap/call_dmr_dss_nanomethphase.py -c 1,2,6,10,11 -ca ${bed_hp1} -co ${bed_hp2} -o ./ -op ${file_prefix} --overwrite > ${file_prefix}.logdmr.log
rm ${file_prefix}ReadyForDSS*.tsv
dmr_txt=${file_prefix}callDMR.txt
if [[ ! -f ${dmr_txt} ]] ; then
echo "### ERROR: no expected output file"
exit -1
fi
awk 'BEGIN{ FS=OFS=" " } NR!=1 && $1 !~ /X/ && $1 !~ /Y/ && $1 !~ // && $1 !~ /M/ && sqrt($8 *
Command exit status:
2
Command output:
Thu Nov 2 16:28:51 CET 2023
r484.uppmax.uu.se
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp1.bed
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp2.bed
Command error:
WARNING: Could not find any nv files on this host!
INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
Thu Nov 2 16:28:51 CET 2023
r484.uppmax.uu.se
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp1.bed
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp2.bed
python: can't open file '/home/myonema/tools/pacbio_5mc/scripts/hap/call_dmr_dss_nanomethphase.py': [Errno 2] No such file or directory
Work dir:
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff
Tip: when you have fixed the problem you can continue the execution adding the option -resume
to the run command line
-- Check '.nextflow.log' file for details
I found the file it is complaning about in utils/call_dmr_dss_nanomethphase.py andI just copied it to the location where it is looking for it, but the error remains. Iam new to nextflow and I do not quite yet understand how to quickly fix issues I encounter, sry.
thanks,
myo
Hi Peng,
I am trying to run the pipeline with the following command -
CUDA_VISIBLE_DEVICES=0 nextflow run /crex/proj/snic2021-6-151/nobackup/Suvi/tools/ccsmethphase \ --dsname vv19 \ --genome /crex/proj/snic2021-6-151/private/Suvi/ref/Vv19/ccs/vv19.fa \ --input /crex/proj/snic2021-6-151/private/Suvi/samples/Viatica19_stuff/ccsmeth/input_sheet.tsv \ --chrSet scaffold_1, scaffold_2, scaffold_3, scaffold_4, scaffold_5, scaffold_6, scaffold_7, scaffold_8, scaffold_9, scaffold_10 \ --max_cpus 15 \ --max_memory "20.GB" \ --max_time "48.h" \ -profile singularity
i keep getting this error -
ERROR ~ Error executing process > 'CheckAGModel (CheckAGModel)'
Caused by:
ProcessCheckAGModel (CheckAGModel)
terminated with an error exit status (255)Command executed:
date; hostname; pwd
ccsmeth aggregate model
if [[ true == true && true == true ]]; then
if [ "aggregate" != "aggregate" ]; then
echo "### INFO: params.cf_mode != aggregate"
elif [ null3 == null3 ] ; then
echo "### INFO: gonna use the default ccsmeth_ag_model=tools/ccsmeth/models/model_ccsmeth_5mCpG_aggregate_attbigru_b11.v2p.ckpt"
elif [[ null3 != *.ckpt && null3 != *.checkpoint ]]; then
echo "### ERROR: not recognized ccsmeth_ag_model=null3"
exit -1
fi
cp -a null3 ccsmeth_aggregate_model.ckpt
ls -lh ccsmeth_aggregate_model.ckpt
fiecho "### Check ag_model DONE"
Command exit status:
255Command output:
(empty)Command error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting /Viatica19_stuff/ccsmeth/local_singularity_cache/nipengcsu-ccsmethphase-latest.img: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1Work dir:
Viatica19_stuff/ccsmeth/work/35/43c8a171fa7797ff93679faa5cf006Tip: when you have fixed the problem you can continue the execution adding the option
-resume
to the run command line
-- Check '.nextflow.log' file for details
WARN: Killing running tasks (1)executor > local (2)
[- ] process > CheckGenome [ 0%] 0 of 1
[- ] process > CheckCMModel (CheckCMModel) -
[35/43c8a1] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, failed: 1 โ
[- ] process > CCSMETH_pbccs_call_hifi -
[- ] process > SAMTOOLS_index_bam [ 0%] 0 of 1
[- ] process > CCSMETH_call_mods_denovo -
[- ] process > CCSMETH_align_hifi -
[- ] process > SAMTOOLS_merge_sortedbams -
[- ] process > CLAIR3_hifi -
[- ] process > WHATSHAP_snv_phase_haplotag -
[- ] process > CCSMETH_call_freq_bam -
[- ] process > DSS_callDMR -
ERROR ~ Error executing process > 'CheckAGModel (CheckAGModel)'Caused by:
ProcessCheckAGModel (CheckAGModel)
terminated with an error exit status (255)Command executed:
date; hostname; pwd
ccsmeth aggregate model
if [[ true == true && true == true ]]; then
if [ "aggregate" != "aggregate" ]; then
echo "### INFO: params.cf_mode != aggregate"
elif [ null3 == null3 ] ; then
echo "### INFO: gonna use the default ccsmeth_ag_model=tools/ccsmeth/models/model_ccsmeth_5mCpG_aggregate_attbigru_b11.v2p.ckpt"
elif [[ null3 != *.ckpt && null3 != *.checkpoint ]]; then
echo "### ERROR: not recognized ccsmeth_ag_model=null3"
exit -1
fi
cp -a null3 ccsmeth_aggregate_model.ckpt
ls -lh ccsmeth_aggregate_model.ckpt
fiecho "### Check ag_model DONE"
Command exit status:
255Command output:
(empty)Command error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting Viatica19_stuff/ccsmeth/local_singularity_cache/nipengcsu-ccsmethphase-latest.img: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1Work dir:
Viatica19_stuff/ccsmeth/work/35/43c8a171fa7797ff93679faa5cf006Tip: when you have fixed the problem you can continue the execution adding the option
-resume
to the run command line-- Check '.nextflow.log' file for details
i did change the DEFAULT_CCSMETH_CM_MODEL
and DEFAULT_CCSMETH_AG_MODEL
variable in the nextflow.config file with the absolute path to the model files in the ccsmeth directory. apart from this, no other changes have been made.
i still keep getting CheckAGMiodel or CheckCMModel error and don't get why. could you please help me out?
Thanks,
Suvi
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.