Comments (14)
Hi Igor,
My pleasure, thanks for trying it out!
I haven't seen this error before, but I think it may have something to do with your study sequence ids. Would you mind sending me your tree (and other input files if possible) to my email used for the google group?
Also, do all of the test commands complete successfully when you run pytest
in the main picrust2 directory?
Thanks,
Gavin
from picrust2.
Hi Gavin
all testt went well after installing, and even now, again. but stil the command does not.
I'm sending you the sequs.
One other question, is it imperative that I use more than one sample data or can I run picrust2 in one single data (despite the fact that until metagenome prediction part where the biom enters that is not relevant.
thanks for the help..
Best
(picrust2-dev) igor@ubuntu:~/picrust2-master$ pytest
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: /home/igor/picrust2-master, inifile:
plugins: cov-2.5.1
collected 27 items
tests/test_hsp.py ....... [ 25%]
tests/test_metagenome_pipeline.py ...... [ 48%]
tests/test_place_seqs.py ...... [ 70%]
tests/test_run_minpath.py .. [ 77%]
tests/test_util.py .... [ 92%]
tests/test_workflow.py .. [100%]
========================== 27 passed in 15.72 seconds ==========================
(picrust2-dev) igor@ubuntu:~/picrust2-master$ hsp.py -i 16S -t /home/igor/Desktop/Alfaguara/Picrust2/alfaguara.tre -o /home/igor/Desktop/Alfaguara/Picrust2/16S_Alfaguara_predicted -p 10 -n
*** caught segfault ***
address 0x556a3, cause 'memory not mapped'
Traceback:
1: .Call(treeBuild, x)
2: FUN(X[[i]], ...)
3: lapply(STRING, .treeBuild)
4: read.tree(Args[1])
An irrecoverable exception occurred. R is aborting now ...
Error running this command:
Rscript /home/igor/picrust2-master/picrust2/Rscripts/castor_hsp.R /home/igor/Desktop/Alfaguara/Picrust2/alfaguara.tre /home/igor/picrust2-master/precalculated/prokaryotic/16S_counts_mean_round_var.txt mp TRUE FALSE FALSE 10 /tmp/wrmm6mzo /tmp/1_418els None None FALSE
from picrust2.
What is you email on google?
sorry...
from picrust2.
Hi Igor,
I have received your files, thanks. The first thing I noticed was that your study sequences have very long names and seem to be already aligned - I'm not sure if this would have caused issues in the produced tree or not. I am re-running your data now after removing gap characters, but it's taking a while since there are so many (>60,000). How were these sequences clustered? In general your data analysis will likely be improved if you can filter out sequences that are extremely rare, which I think could make a difference for your data.
I'll get back to you when I have a chance to troubleshoot your data more.
Best,
Gavin
from picrust2.
Hi Gavin
Thanks for that.
These are already clustered and filtered seqs in mothur. All singletons and doubletons were removed from the data set. But Yes I could more, say below 2%?
I'll wait for your analyses results,
Best
Igor
from picrust2.
Hi Igor,
I think I've made the required fix. The "read.tree" function from the ape R package was crashing when reading in your study tree. I have switched to instead read in the tree using "read_tree" from the castor R package. You can download the modified picrust2 version at this branch: https://github.com/picrust/picrust2/tree/read_tree
Please let me know if you run into any other problems and if this works for you!
Gavin
from picrust2.
from picrust2.
Hi Igor,
How much memory do you have on your machine? If you use fewer processes the job will run slower, but take less memory. Also, does the job die right away or after a while. If it takes a while can you see the memory usage creep up if you watch the activity with the top
command?
from picrust2.
from picrust2.
Hi Gavinn
I lowered the threads to 2 and still the same error:
(picrust2-dev) igor@ubuntu:/Desktop/Alfaguara/Picrust2$ hsp.py -i EC -t alfaguara.tre -o EC_predicted -p 2/Desktop/Alfaguara/Picrust2$
Error in serialize(what, NULL, xdr = FALSE) : cannot allocate buffer
Calls: mclapply -> lapply -> FUN -> sendMaster -> serialize
Warning messages:
1: In mclapply(trait_states_mapped, hsp_max_parsimony, tree = full_tree, :
scheduled core 1 encountered error in user code, all values of the job will be affected
2: In mclapply(names(hsp_out_models), function(x) { :
all scheduled cores encountered errors in user code
Error in $<-.data.frame
(*tmp*
, sequence, value = c("M01028_125_000000000-AN36D_1_1111_16866_17721", :
replacement has 65410 rows, data has 0
Calls:
Execution halted
Error running this command:
Rscript /home/igor/picrust2-read_tree/picrust2/Rscripts/castor_hsp.R alfaguara.tre /home/igor/picrust2-read_tree/precalculated/prokaryotic/ec_counts_mean_round_var.txt mp FALSE FALSE FALSE 2 /tmp/4ljnwko_ /tmp/vmu2wrax None None FALSE
(picrust2-dev) igor@ubuntu:
Any help?
Thanks
from picrust2.
Hi Igor,
I re-wrote how the HSP parallelization step works (see the PR here: #13) and you shouldn't have this problem anymore if you update to the latest version. Note that there is now a new option for hsp.py
script (--chunk_size
), which specified how many gene families should be read into a given R environment at a time (and that will be run by 1 processor). There are ~3000 gene families in the E.C. table so you could set this to be 300 if you wanted to run 10 processes for instance. Note that if you set this option to be 3000 and set 10 processes that it would still only run on 1 CPU since there would only be one chunk of input data.
Best,
Gavin
from picrust2.
from picrust2.
from picrust2.
Hi Igor,
I'm glad that the HSP step is runnable for you now. I'm going to close this issue now so that it will be easier to keep them straight in the future. I posted your new issue here: #16 where I'll try to address it!
Best,
Gavin
from picrust2.
Related Issues (20)
- Installing problem into MacBook Air m1 terminal HOT 9
- help please : Error running picrust2_pipeline.py HOT 2
- maximum recursion depth exceeded while calling a Python object
- Question about previous filtering of otu table HOT 3
- Customize database guidance HOT 4
- running picrust2 on ITS HOT 2
- metagenome contributions analysis HOT 2
- Running picrust on shotgun output from bracken HOT 2
- Error runnig picrust due to cannot import name '_convert_node_to_root_polytomy' HOT 2
- Absolute abundance data for picrust2 HOT 3
- pytest fails 1 test on test_place_seqs.py HOT 2
- Macbook M1 chip pytest error (sqlite3 import??) HOT 2
- Assign general function from KEGG pathway in pathways_out HOT 1
- PICRTUST 2 No such file or directory: 'hmmalign' HOT 2
- Gene family abundances per ASV rather than per sample HOT 1
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte HOT 3
- During pytest place_seqs.py fails and cannot run the full pipeline HOT 6
- complex definition effect on abundance calculation HOT 1
- Please help for picrust2 tutorial: cannot download the data set HOT 4
- Running picrust on full geomes HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from picrust2.