Comments (8)
Hi, @yuukiiwa
I did re-do all steps with ENSEMBL reference. Now i got m6anet inference done without any warning or other msgs.
Many thanks!
So far, I don't have a computer efficient to run all process, so i get to try later (dependent on if i get my computer)
Andrea
Hi @AndreaYCT,
If ENSEMBL reference works, then we would suggest yo use it instead.
I was asking you to run m6anet dataprep locally to see whether you can reproduce the error that halt in the HPC, so you would have to turn on your computer and let it run.
But just checking, are you using the same
transcriptome.fa
file running minimap2 and eventalign? If not, we would recommend you to re-run both using the ENSEMBLtranscriptome.fa
that worked with minimap2 and eventalign.Thanks!
Best wishes, Yuk Kei
from m6anet.
Hi @AndreaYCT,
I supposed your HPC run log didn't echo out "m6Anet dataprep done: STM2457_2uM"
.
As you run m6anet with 28 cores, it will take 28 errors to have the run entirely halt, which explains why it was "running" after 12 hours. You can try to see which transcript ids are missing in your data.log
and subset your eventalign.txt
file on those transcript ids and run m6anet on those locally to troubleshoot.
Thanks!
Best wishes,
Yuk Kei
from m6anet.
Hi, Yuk kei,
where to find the "error information"? Since I don't get any error msg form the error.txt.
Or how to skip?
Many thanks! I'm stilling learning how to use HPC.
from m6anet.
Hi @AndreaYCT,
You can use the interactive mode of your HPC to test out running a subset of transcript ids that weren't included in the data.info.
There's no way to skip those if there're problems with the eventalign.txt, you will have to troubleshoot that in the interactive mode of your HPC.
Thanks!
Best wishes,
Yuk Kei
from m6anet.
Hi, @yuukiiwa
I tried and i interrupted by crtl+C. Here are some msg:
Process Consumer-9: Process Consumer-2: Process Consumer-7: Process Consumer-12: Process Consumer-3: Process Consumer-5: Process Consumer-11: Process Consumer-13: Process Consumer-6: Process Consumer-8: Process Consumer-1: Process Consumer-4: Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/bin/m6anet", line 8, in <module> Process Consumer-10: sys.exit(main()) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/__init__.py", line 30, in main args.func(args) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/scripts/dataprep.py", line 63, in main args.out_dir, args.n_processes) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 250, in parallel_index Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() KeyboardInterrupt File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 93, in get with self._rlock: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() KeyboardInterrupt KeyboardInterrupt KeyboardInterrupt KeyboardInterrupt KeyboardInterrupt KeyboardInterrupt KeyboardInterrupt lines = [len(eventalign_file.readline()) for i in range(chunk_concat_size)] File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 250, in <listcomp> lines = [len(eventalign_file.readline()) for i in range(chunk_concat_size)] KeyboardInterrupt Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 85, in run result = self.task_function(*next_task_args,self.locks) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 202, in index with locks['index'], open(out_paths['index'],'a', encoding='utf-8') as f_index: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() Traceback (most recent call last): KeyboardInterrupt File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 85, in run result = self.task_function(*next_task_args,self.locks) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 202, in index with locks['index'], open(out_paths['index'],'a', encoding='utf-8') as f_index: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() KeyboardInterrupt File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 85, in run result = self.task_function(*next_task_args,self.locks) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 202, in index with locks['index'], open(out_paths['index'],'a', encoding='utf-8') as f_index: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() KeyboardInterrupt Traceback (most recent call last): Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 85, in run result = self.task_function(*next_task_args,self.locks) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 81, in run next_task_args = self.task_queue.get() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 202, in index with locks['index'], open(out_paths['index'],'a', encoding='utf-8') as f_index: File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/queues.py", line 94, in get res = self._recv_bytes() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/synchronize.py", line 95, in __enter__ return self._semlock.__enter__() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/connection.py", line 216, in recv_bytes buf = self._recv_bytes(maxlength) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) KeyboardInterrupt File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/connection.py", line 379, in _recv chunk = read(handle, remaining) KeyboardInterrupt Traceback (most recent call last): File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap self.run() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/helper.py", line 85, in run result = self.task_function(*next_task_args,self.locks) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/m6anet/utils/dataprep_utils.py", line 205, in index pos_end += eventalign_result.loc[_index]['line_length'].sum() File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexing.py", line 873, in __getitem__ return self._getitem_tuple(key) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexing.py", line 1044, in _getitem_tuple return self._getitem_lowerdim(tup) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexing.py", line 774, in _getitem_lowerdim result = self._handle_lowerdim_multi_index_axis0(tup) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexing.py", line 1066, in _handle_lowerdim_multi_index_axis0 return self._get_label(tup, axis=axis) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexing.py", line 1059, in _get_label return self.obj.xs(label, axis=axis) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/generic.py", line 3491, in xs loc, new_index = self.index.get_loc_level(key, drop_level=drop_level) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 2857, in get_loc_level return partial_selection(key) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 2844, in partial_selection indexer = self.get_loc(key) File "/opt/ohpc/Taiwania3/pkg/biology/Python/Python_v3.7.10/lib/python3.7/site-packages/pandas/core/indexes/multi.py", line 2741, in get_loc mask = self.codes[i][loc] == self._get_loc_single_level_index( KeyboardInterrupt
any suggestion will be really appreciated!
Hi @AndreaYCT,
You can use the interactive mode of your HPC to test out running a subset of transcript ids that weren't included in the data.info.
There's no way to skip those if there're problems with the eventalign.txt, you will have to troubleshoot that in the interactive mode of your HPC.
Thanks!
Best wishes, Yuk Kei
from m6anet.
Hi @AndreaYCT,
I have skimmed through the error message, which is specific to your KeyboardInterrupt
instead of a potential error message outputted by m6anet dataprep
.
Can you try letting m6anet dataprep
run without interrupting it with your keyboard, please?
Thanks!
Best wishes,
Yuk Kei
from m6anet.
Hi, @yuukiiwa ,
How to run it without interrupting it with your keyboard? I sent codes in a bash file and I also can try again.
But this will not happen when i used ENSEMBL resource.
Thanks!
Andrea
Hi @AndreaYCT,
I have skimmed through the error message, which is specific to your
KeyboardInterrupt
instead of a potential error message outputted bym6anet dataprep
.Can you try letting
m6anet dataprep
run without interrupting it with your keyboard, please?Thanks!
Best wishes, Yuk Kei
from m6anet.
Hi @AndreaYCT,
If ENSEMBL reference works, then we would suggest yo use it instead.
I was asking you to run m6anet dataprep locally to see whether you can reproduce the error that halt in the HPC, so you would have to turn on your computer and let it run.
But just checking, are you using the same transcriptome.fa
file running minimap2 and eventalign? If not, we would recommend you to re-run both using the ENSEMBL transcriptome.fa
that worked with minimap2 and eventalign.
Thanks!
Best wishes,
Yuk Kei
from m6anet.
Related Issues (20)
- Memory usage exploding HOT 3
- Empty Eventalign.txt for Negative Sense Viral RNA HOT 1
- Questions about the m6A motif HOT 1
- Genome vs transcriptome alignment HOT 4
- m6anet, eventalign, galaxy HOT 3
- error with m6anet dataprep HOT 1
- yet another error with m6anet dataprep HOT 1
- running with test eventalign test file file form github produces another error HOT 8
- installation on linux failed HOT 3
- TypeError: argument should be integer or None, not 'numpy.float64' HOT 2
- Interpreting 'inference' results HOT 1
- m6anet dataprep fails to complete: cannot convert float NaN to integer HOT 3
- Issues with Read Selection HOT 2
- The position output by m6anet seems to be 1 less than the actual position HOT 1
- empty dataprep data.json and data.log HOT 5
- Arabidopsis VIRC instead of col0 (wild type) HOT 3
- probability_modified and mod_ratio are 0 HOT 2
- f5c or nanopolish HOT 3
- Disconnect between site and individual read mod probability RNA004
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from m6anet.