benoitmorel / covid19_cme_analysis Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU Affero General Public License v3.0
License: GNU Affero General Public License v3.0
Hi @idaios
I prepare my dataset from scratch having high quality 10 sars-cov2 genome with 2 outgruop so total sequences in my data are 12 for which I again ran all script, this time I am stuck at 5th script
When first time ran 5th script the ERROR was
:
ERROR: Must run iqtree_tests stage of pipeline first
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/5_epa_outgroup_rooting.py", line 23, in <module>
raise e
File "/home/vinita/covid19_cme_analysis/./pipeline/5_epa_outgroup_rooting.py", line 20, in <module>
util.expect_file_exists( paths.raxml_credible_ml_trees )
File "/home/vinita/covid19_cme_analysis/scripts/util.py", line 99, in expect_file_exists
raise RuntimeError( "File doesn't exist: " + file_path )
RuntimeError: File doesn't exist: /home/vinita/covid19_cme_analysis/work_dir/2021-11-17_05/smsao/results/trees/credible_ml_trees.newick
Then I ran 7th script first which is 7_iqtree_tests.py
and again ran 5th script which is giving following error
./pipeline/5_epa_outgroup_rooting.py work_dir/2021-11-17_05/smsao
0 / 13
No protocol specified
No protocol specified
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode -1.
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/5_epa_outgroup_rooting.py", line 71, in <module>
cur_modelfile = raxml_launcher.evaluate(tree_file, ref_msa, cur_outdir)
File "/home/vinita/covid19_cme_analysis/scripts/raxml_launcher.py", line 75, in evaluate
sub.check_call(cmd, cwd=out_dir, stdout=sub.DEVNULL)
File "/home/vinita/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/vinita/covid19_cme_analysis/software/raxml-ng/bin/raxml-ng-mpi', '--evaluate', '--msa', '/home/vinita/covid19_cme_analysis/work_dir/2021-11-17_05/smsao/data/covid_edited.fasta', '--model', 'GTR+FO+G4', '--tree', '/home/vinita/covid19_cme_analysis/work_dir/2021-11-17_05/smsao/runs/epa_runs/0/tree.newick', '--prefix', 'eval', '--threads', '4', '--blopt', 'nr_safe', '--redo', '--blmin', '0.000000001']' returned non-zero exit status 255.
Further I tried 6th script
./pipeline/6_root_digger_rooting.py work_dir/2021-11-17_05/fmsan/
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/6_root_digger_rooting.py", line 84, in <module>
writer = csv.DictWriter(csv_file, fieldnames=results[0].keys())
IndexError: list index out of range
(name_of_my_env) ./pipeline/6_root_digger_rooting.py work_dir/2021-11-17_05/smsao/
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/6_root_digger_rooting.py", line 84, in <module>
writer = csv.DictWriter(csv_file, fieldnames=results[0].keys())
IndexError: list index out of range
But somehow 8_tree_thinning.py, 9_mptp_on_all_trees.py, compare_llhs, and extract_thinned_dataset.py worked on 4 dataset which are FMSAO, SMSAO, FMSAN & SMSAN
For wuhan_placement.py
also get some erro
./pipeline/wuhan_placement.py work_dir/2021-11-17_05/smsao/
ERROR: Must run placement stage of pipeline first
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/wuhan_placement.py", line 18, in <module>
raise e
File "/home/vinita/covid19_cme_analysis/./pipeline/wuhan_placement.py", line 15, in <module>
util.expect_dir_exists( paths.epa_rooting_dir )
File "/home/vinita/covid19_cme_analysis/scripts/util.py", line 95, in expect_dir_exists
raise RuntimeError( "Directory doesn't exist: " + dir_path )
RuntimeError: Directory doesn't exist: /home/vinita/covid19_cme_analysis/work_dir/2021-11-17_05/smsao/results/epa_rooting
KIndly help me to understand these issue wether they are interlinked with my data or something which is not present in my data thats why root_digger_lwr.csv is empty in smsao/results/rootdigger_rooting
It would be very great if you could look at these errors and suggest me how I should solve these.
Thank you very much
Vinita
Hi
I clone this repo in my Ubuntu
system and run the setup.sh
and everything ran perfectcly. I ran first script ./pipeline/0_get_data.py path to my fasta file
and it created a folder with current date in work_dir, But when I tried to run the ./pipeline/1_preprocess_data.py work_dir/2021-10-13_00/fmsan
it gives following error :
/home/vinita/covid19_cme_analysis-master/scripts/preanalysis1.sh /home/vinita/covid19_cme_analysis-master/work_dir/2021-10-13_00/covid_raw_unaligned.fasta fmsan /home/vinita/covid19_cme_analysis-master/scripts /home/vinita/covid19_cme_analysis-master/software/mafft/mafft.bat /home/vinita/covid19_cme_analysis-master/config/outgroups.txt /home/vinita/covid19_cme_analysis-master/work_dir/2021-10-13_00 48
Totally 0 sequences pass the filter of less than 10 Ns
/home/vinita/covid19_cme_analysis-master/scripts/preanalysis1.sh: line 92: /home/vinita/covid19_cme_analysis-master/scripts/../software/genesis/bin/apps/remove_sequences: No such file or directory
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis-master/./pipeline/1_preprocess_data.py", line 14, in <module>
preprocessing.trim_separate_align(paths.raw_sequences,
File "scripts/preprocessing.py", line 24, in trim_separate_align
subprocess.check_call(cmd, cwd=runsdir)
File "/home/vinita/.local/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/vinita/covid19_cme_analysis-master/scripts/preanalysis1.sh', '/home/vinita/covid19_cme_analysis-master/work_dir/2021-10-13_00/covid_raw_unaligned.fasta', 'fmsan', '/home/vinita/covid19_cme_analysis-master/scripts', '/home/vinita/covid19_cme_analysis-master/software/mafft/mafft.bat', '/home/vinita/covid19_cme_analysis-master/config/outgroups.txt', '/home/vinita/covid19_cme_analysis-master/work_dir/2021-10-13_00', '48']' returned non-zero exit status 127
I have checked this genesis older for bin/apps/remove_sequences but this remove_sequences are present in build folder
I tried to install this repo in Mac
as well as in Linux mint
but the setup.sh is giving error and not able to installed on these two OS
Kindly have a look and suggest me how to run these script to get better results for my Sars-Cov2 data
Hi @idaios
Could you please suggest any tool/script in your repo which I can use to ommit internal N's to get less than 10N
from my 30,000 sequences for SARS-CoV2 data.
I would be grateful for any suggestions you may have.
Thank you
Vinita
Hi @Pbdas
I ran ./pipeline/9_mptp_on_all_trees.py work_dir/2021-11-19_00/fmsan/
and I got following error on my terminal
WARNING: delimit was not compiled with libgsl. Likelihood ratio test disabled.
WARNING: A speciation edge is smaller than the specified minimum branch length.
mptp 0.2.4_linux_x86_64, 1008GB RAM, 96 cores
https://github.com/Pas-Kapli/mptp
WARNING: delimit was not compiled with libgsl. Likelihood ratio test disabled.
WARNING: A speciation edge is smaller than the specified minimum branch length.
WARNING: delimit was not compiled with libgsl. Likelihood ratio test disabled.
Traceback (most recent call last):
File "/home/vinita/covid19_cme_analysis/./pipeline/9_mptp_on_all_trees.py", line 191, in
run_mptp_all_rootings_on_trees_one_histogram(paths.raxml_credible_ml_trees, paths.mptp_output, paths.mptp_output_summary_all_rootings_one_histogram)
File "/home/vinita/covid19_cme_analysis/./pipeline/9_mptp_on_all_trees.py", line 181, in run_mptp_all_rootings_on_trees_one_histogram
species = get_all_rootings_counts_one_histogram(results_all_rootings_ml + "/output.txt", species)
File "/home/vinita/covid19_cme_analysis/./pipeline/9_mptp_on_all_trees.py", line 105, in get_all_rootings_counts_one_histogram
for i in range(4, len(lines)):
TypeError: object of type '_io.TextIOWrapper' has no len()
I also found so many tempOutFooBar_.svg
something around 3000 in number , Please have a look at this error .
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.