Run command：nextflow run main.nf --dsname test -profile singularity,test

Error information：

[process] DSS_callDMR
status=ACTIVE
port 0: (queue) OPEN ; channel: -
port 1: (value) bound ; channel: ch_utils
port 2: (cntrl) - ; channel: $

Mar-01 17:31:40.300 [main] DEBUG nextflow.Session - Session await > all processes finished
Mar-01 17:31:40.301 [main] DEBUG nextflow.Session - Session await > all barriers passed
Mar-01 17:31:40.332 [main] WARN n.processor.TaskPollingMonitor - Killing running tasks (3)
Mar-01 17:31:40.382 [main] DEBUG n.executor.local.LocalTaskHandler - Unable to kill CheckCMModel (CheckCMModel) -- command: kill -TERM 215509; exit: 1
bash: line 0: kill: (215509) - No such process

Mar-01 17:31:40.411 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 4; name: SAMTOOLS_index_bam (hg002.chr20_demo.hifi); status: COMPLETED; exit: 255; error: -; workDir: /mnt/Data_disk/zhoubiaofeng/opt/ccsmethphase/ccsmethphase/work/2b/7203a95eebc6f145e92847bfb52178]
Mar-01 17:31:40.412 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
task: name=SAMTOOLS_index_bam (hg002.chr20_demo.hifi); work-dir=/mnt/Data_disk/zhoubiaofeng/opt/ccsmethphase/ccsmethphase/work/2b/7203a95eebc6f145e92847bfb52178
error [nextflow.exception.ProcessFailedException]: Process SAMTOOLS_index_bam (hg002.chr20_demo.hifi) terminated with an error exit status (255)
Mar-01 17:31:40.415 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Mar-01 17:31:40.452 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=2; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=2; succeedDuration=0ms; failedDuration=1m 44s; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=4; peakCpus=20; peakMemory=48 GB; ]
Mar-01 17:31:40.452 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file
Mar-01 17:31:40.455 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report
Mar-01 17:31:42.313 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline
Mar-01 19:20:03.349 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Mar-01 19:20:03.386 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Any suggestions to fix this problem?

Hello from a new users
It seems that the output path is not set, How should I set the output path for my file?
And another question about Will you consider directly performing DMR analysis on data from different groups within the program?

Hi Peng,

I am trying to test run ccsmethphase on the demo dataset but am running into an issue that is causing the program to error out when trying to run WhatsHap. See below:

xecutor > slurm (7)
[15/09d90d] process > CheckGenome (CheckGenome) [100%] 1 of 1, cached: 1 ✔
[61/d51248] process > CheckCMModel (CheckCMModel) [100%] 1 of 1, cached: 1 ✔
[72/e313a6] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, cached: 1 ✔
[- ] process > CCSMETH_pbccs_call_hifi -
[2e/bef2a4] process > SAMTOOLS_index_bam (hg002.c... [100%] 1 of 1, cached: 1 ✔
[03/683597] process > CCSMETH_call_mods_denovo (h... [100%] 1 of 1, cached: 1 ✔
[7b/797f14] process > CCSMETH_align_hifi (hg002.c... [100%] 1 of 1, cached: 1 ✔
[69/00f338] process > SAMTOOLS_merge_sortedbams (... [100%] 1 of 1, cached: 1 ✔
[f9/57dc73] process > CLAIR3_hifi (G1.HG002_demo.... [100%] 1 of 1, cached: 1 ✔
[15/c50c48] process > WHATSHAP_snv_phase_haplotag... [100%] 3 of 3, failed: 2...
[35/c32f06] process > CCSMETH_call_freq_bam (G1.H... [100%] 1 of 1 ✔
[9f/f804ca] process > DSS_callDMR (G1.HG002_demo) [ 80%] 4 of 5, failed: 4...
[b8/2f54b0] NOTE: Process DSS_callDMR (G1.HG002_demo) terminated with an error exit status (2) -- Execution is retried (3)
[3f/0ae8d3] NOTE: Process DSS_callDMR (G1.HG002_demo) terminated with an error exit status (2) -- Execution is retried (4)

hej,

I am trying to run the demo data but it throws me an error in the last steps:

[71/07e145] process > CheckGenome (CheckGenome) [100%] 1 of 1, cached: 1 ✔
[2e/b2d050] process > CheckCMModel (CheckCMModel) [100%] 1 of 1, cached: 1 ✔
[0e/b4443f] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, cached: 1 ✔
[- ] process > CCSMETH_pbccs_call_hifi -
[0a/680851] process > SAMTOOLS_index_bam (hg002.chr20_demo.hifi) [100%] 1 of 1, cached: 1 ✔
[f5/7081a8] process > CCSMETH_call_mods_denovo (hg002.chr20_demo.hifi) [100%] 1 of 1, cached: 1 ✔
[f6/cab01f] process > CCSMETH_align_hifi (hg002.chr20_demo.hifi.ccsmeth.modbam) [100%] 1 of 1, cached: 1 ✔
[13/245444] process > SAMTOOLS_merge_sortedbams (G1.HG002_demo) [100%] 1 of 1, cached: 1 ✔
[d3/b24dee] process > CLAIR3_hifi (G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1) [100%] 1 of 1, cached: 1 ✔
[59/a3fc4f] process > WHATSHAP_snv_phase_haplotag (G1.HG002_demo) [100%] 1 of 1, cached: 1 ✔
[6e/dc1920] process > CCSMETH_call_freq_bam (G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap) [100%] 1 of 1, cached: 1 ✔
[02/3aaa5b] process > DSS_callDMR (G1.HG002_demo) [100%] 1 of 1, failed: 1 ✘
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'DSS_callDMR (G1.HG002_demo)'

Caused by:
Process DSS_callDMR (G1.HG002_demo) terminated with an error exit status (2)

Command executed:

date; hostname; pwd

find the bed files used for dss

if [[ 3 -gt 3 ]] ; then
bed_hp1=$(find -L ./ -type f -regex '..aggregate.hp1.bed')
bed_hp2=$(find -L ./ -type f -regex '..aggregate.hp2.bed')
else
bed_hp1=$(find -L ./ -type f -regex '..hp1.bed')
bed_hp2=$(find -L ./ -type f -regex '..hp2.bed')
fi

echo ${bed_hp1}
echo ${bed_hp2}

file_prefix=$(echo ${bed_hp1%1.bed})

python ~/tools/pacbio_5mc/scripts/hap/call_dmr_dss_nanomethphase.py -c 1,2,6,10,11 -ca ${bed_hp1} -co ${bed_hp2} -o ./ -op ${file_prefix} --overwrite > ${file_prefix}.logdmr.log

rm ${file_prefix}ReadyForDSS*.tsv
dmr_txt=${file_prefix}callDMR.txt
if [[ ! -f ${dmr_txt} ]] ; then
echo "### ERROR: no expected output file"
exit -1
fi
awk 'BEGIN{ FS=OFS=" " } NR!=1 && $1 !~ /X/ && $1 !~ /Y/ && $1 !~ // && $1 !~ /M/ && sqrt($8 * $8) &gt; 0.2' ${dmr_txt} > $(echo ${dmr_txt%.txt}).autosomes_cf0.2.bed

Command exit status:
2

Command output:
Thu Nov 2 16:28:51 CET 2023
r484.uppmax.uu.se
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp1.bed
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp2.bed

Command error:
WARNING: Could not find any nv files on this host!
INFO: Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
INFO: Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
Thu Nov 2 16:28:51 CET 2023
r484.uppmax.uu.se
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp1.bed
./G1.HG002_demo.hifi.ccsmeth.modbam.pbmm2.merged_size1.SNV_PASS_whatshap.freq.aggregate.hp2.bed
python: can't open file '/home/myonema/tools/pacbio_5mc/scripts/hap/call_dmr_dss_nanomethphase.py': [Errno 2] No such file or directory

Work dir:
/crex/proj/snic2020-2-19/private/herring/users/sabine/2023-11-02_FishMeth/ccsmethphase/work/02/3aaa5b251edba59c71e4705fb58aff

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line

-- Check '.nextflow.log' file for details

I found the file it is complaning about in utils/call_dmr_dss_nanomethphase.py andI just copied it to the location where it is looking for it, but the error remains. Iam new to nextflow and I do not quite yet understand how to quickly fix issues I encounter, sry.

thanks,
myo

Hi Peng,

I am trying to run the pipeline with the following command -

CUDA_VISIBLE_DEVICES=0 nextflow run /crex/proj/snic2021-6-151/nobackup/Suvi/tools/ccsmethphase \ --dsname vv19 \ --genome /crex/proj/snic2021-6-151/private/Suvi/ref/Vv19/ccs/vv19.fa \ --input /crex/proj/snic2021-6-151/private/Suvi/samples/Viatica19_stuff/ccsmeth/input_sheet.tsv \ --chrSet scaffold_1, scaffold_2, scaffold_3, scaffold_4, scaffold_5, scaffold_6, scaffold_7, scaffold_8, scaffold_9, scaffold_10 \ --max_cpus 15 \ --max_memory "20.GB" \ --max_time "48.h" \ -profile singularity

i keep getting this error -

ERROR ~ Error executing process > 'CheckAGModel (CheckAGModel)'

Caused by:
Process CheckAGModel (CheckAGModel) terminated with an error exit status (255)

Command executed:

date; hostname; pwd

ccsmeth aggregate model
if [[ true == true && true == true ]]; then
if [ "aggregate" != "aggregate" ]; then
echo "### INFO: params.cf_mode != aggregate"
elif [ null3 == null3 ] ; then
echo "### INFO: gonna use the default ccsmeth_ag_model=tools/ccsmeth/models/model_ccsmeth_5mCpG_aggregate_attbigru_b11.v2p.ckpt"
elif [[ null3 != *.ckpt && null3 != *.checkpoint ]]; then
echo "### ERROR: not recognized ccsmeth_ag_model=null3"
exit -1
fi
cp -a null3 ccsmeth_aggregate_model.ckpt
ls -lh ccsmeth_aggregate_model.ckpt
fi

echo "### Check ag_model DONE"

Command exit status:
255

Command output:
(empty)

Command error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting /Viatica19_stuff/ccsmeth/local_singularity_cache/nipengcsu-ccsmethphase-latest.img: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1

Work dir:
Viatica19_stuff/ccsmeth/work/35/43c8a171fa7797ff93679faa5cf006

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line
-- Check '.nextflow.log' file for details
WARN: Killing running tasks (1)

executor > local (2)
[- ] process > CheckGenome [ 0%] 0 of 1
[- ] process > CheckCMModel (CheckCMModel) -
[35/43c8a1] process > CheckAGModel (CheckAGModel) [100%] 1 of 1, failed: 1 ✘
[- ] process > CCSMETH_pbccs_call_hifi -
[- ] process > SAMTOOLS_index_bam [ 0%] 0 of 1
[- ] process > CCSMETH_call_mods_denovo -
[- ] process > CCSMETH_align_hifi -
[- ] process > SAMTOOLS_merge_sortedbams -
[- ] process > CLAIR3_hifi -
[- ] process > WHATSHAP_snv_phase_haplotag -
[- ] process > CCSMETH_call_freq_bam -
[- ] process > DSS_callDMR -
ERROR ~ Error executing process > 'CheckAGModel (CheckAGModel)'

Caused by:
Process CheckAGModel (CheckAGModel) terminated with an error exit status (255)

Command executed:

date; hostname; pwd

ccsmeth aggregate model

if [[ true == true && true == true ]]; then
if [ "aggregate" != "aggregate" ]; then
echo "### INFO: params.cf_mode != aggregate"
elif [ null3 == null3 ] ; then
echo "### INFO: gonna use the default ccsmeth_ag_model=tools/ccsmeth/models/model_ccsmeth_5mCpG_aggregate_attbigru_b11.v2p.ckpt"
elif [[ null3 != *.ckpt && null3 != *.checkpoint ]]; then
echo "### ERROR: not recognized ccsmeth_ag_model=null3"
exit -1
fi
cp -a null3 ccsmeth_aggregate_model.ckpt
ls -lh ccsmeth_aggregate_model.ckpt
fi

echo "### Check ag_model DONE"

Command exit status:
255

Command output:
(empty)

Command error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting Viatica19_stuff/ccsmeth/local_singularity_cache/nipengcsu-ccsmethphase-latest.img: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1

Work dir:
Viatica19_stuff/ccsmeth/work/35/43c8a171fa7797ff93679faa5cf006

Tip: when you have fixed the problem you can continue the execution adding the option -resume to the run command line

-- Check '.nextflow.log' file for details

i did change the DEFAULT_CCSMETH_CM_MODEL and DEFAULT_CCSMETH_AG_MODEL variable in the nextflow.config file with the absolute path to the model files in the ccsmeth directory. apart from this, no other changes have been made.

i still keep getting CheckAGMiodel or CheckCMModel error and don't get why. could you please help me out?

Thanks,
Suvi