picrust / picrust2 Goto Github PK
View Code? Open in Web Editor NEWCode, unit tests, and tutorials for running PICRUSt2
License: GNU General Public License v3.0
Code, unit tests, and tutorials for running PICRUSt2
License: GNU General Public License v3.0
Hello, I am trying to follow your workflow, I have successfully "placed study sequences into a reference multiple-sequence alignment" and finish step 2 (hsp.py). But in step 3 (metagenome_pipeline.py), where does the study_seqs.biom come from? I tried creating what I think is study_seqs.biom by exporting seqtab.nochim as a BIOM file from DADA2. (because actually, I am trying to evade the OTU picking step by directly converting DADA2 output for use in PICRUSt, following this pipeline, but that is for PICRUSt1.) But when I run step 3 I get ValueError: No common OTUs between the otu table and the genome table, so can't predict metagenome.
Since metagenome_pipeline.py is a wrapper script for normalize_by_copy_number.py and predict_metagenomes.py, I read the PICRUSt1 page about them, and seems that study_seqs.biom is analogous to your_otu_table.biom. If this is the case, does this mean I have to pick OTUs first? So PICRUSt2 is still a closed-reference OTU picking approach? Thank you very much for your attention.
P.S. I posted a separate question in the picrust-users Google group but feel this issue is more appropriate here, feel free to ask me to re-post this issue there if you think it's better.
Best,
Jamie
Hello,
Error: wrap_hsp.py:31
In full pipeline picrust2_pipeline.py.
Error:
Error running this command:
hsp.py --tree picrust2_out_pipeline/out.tre --output picrust2_out_pipeline/EC_predicted.tsv --observed_trait_table /data/apps/python/3.5.1/lib/python3.5/site-packages/picrust2/default_files/prokaryotic/ec.txt.gz --hsp_method mp --processes 4 --seed 100
STDOUT of failed command:
STDERR of failed command:
/data/apps/python/3.5.1/lib/python3.5/site-packages/picrust2/wrap_hsp.py:31: FutureWarning: read_table is deprecated, use read_csv instead.
dtype={'assembly' : str})
I have no idea what might be causing this error.
Hi @gavinmdouglas - Really glad to see the release of Picrust2! Thank you so much! I have a quick question about the reference files to use to run Picrust2 with Q2. I have a rooted and an unrooted tree that I created from Qiime2 already but looks like I need to place my ASVs onto a Picrust2 reference tree instead. Can you please clarify where these Picrust2 reference tree files are located? Should I use the same reference tree file that comes with the tutorial you have posted on Github? Please let me know. Thank you so much! -Taruna
Hi
I wanted to run the minpath for KO results obtained from
hsp.py -i KO -t kiwis.tre -o KO_kiwis_predicted -p 6 --chunk_size 300
and I run the command bellow, but the out put gave a txt with nothing besides the headline, despite KO_kiwis_predicted.tsv had 14.8MB:
run_minpath.py -i /home/igor/Desktop/KIWIS/Metagenome_prediction_KO/pred_metagenome_strat.tsv \
-m /home/igor/picrust2/MinPath/ec2metacyc_picrust_prokaryotic.txt \ -o kiwis_KO \ --intermediate IntermediateFiles_wikis_KO \ -p 4
Hi again,
I got this error several times while trying to run metagenome_pipeline.py
(v 2.0.0-b.5) in our server (CentOS). It happens after few minutes and seems to be an issue with joblib
v 0.12.1.
I forced the downgrade to joblib=0.11
in the conda environment and still running after way longer than with 0.12.1 (~30 min)
$ metagenome_pipeline.py -i CHI4831_mothur_biom.reformated.full.json.biom -m CHI4831.picrust2.16S.tsv -f CHI4831.picrust2.KO.tsv -p 4 --strat_out -o CHI4831.picrust2.meta
Traceback (most recent call last):
File "/home/z3382651/anaconda2/envs/picrust2-dev/bin/metagenome_pipeline.py", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/home/z3382651/bin/picrust2/scripts/metagenome_pipeline.py", line 102, in <module>
main()
File "/home/z3382651/bin/picrust2/scripts/metagenome_pipeline.py", line 88, in main
output_normfile=True)
File "/home/z3382651/bin/picrust2/picrust2/metagenome_pipeline.py", line 103, in run_metagenome_pipeline
strat_out=strat_out))
File "/home/z3382651/bin/picrust2/picrust2/metagenome_pipeline.py", line 174, in funcs_by_sample
for sample in sample_ids)
File "/home/z3382651/anaconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py", line 962, in __call__
self.retrieve()
File "/home/z3382651/anaconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py", line 865, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/home/z3382651/anaconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 515, in wrap_future_result
return future.result(timeout=timeout)
File "/home/z3382651/anaconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/externals/loky/_base.py", line 431, in result
return self.__get_result()
File "/home/z3382651/anaconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/externals/loky/_base.py", line 382, in __get_result
raise self._exception
joblib.externals.loky.process_executor.BrokenProcessPool: A process in the executor was terminated abruptly while the future was running or pending.
Cheers,
Xabi
Hi
I installed picrust2 on a Ubuntu16.06 VBox a few days back and the EPA-NG step in the sequence placement is not working. The installation was carried out as instructed without a single error. Pytest ran in all greens. I ran place.seqs with print_cmds and this is the error i am getting:
(picrust2-dev) arghya250@arghya250-VirtualBox:~/Apps/picrust2$ place_seqs.py -s example_files/tutorial.fna -o OUTPUT.tre --threads 10 --intermediate placement_working
papara called as:
papara -t /home/arghya250/Apps/picrust2/default_files/prokaryotic/img_centroid_16S_aligned.tree -s ref_seqs.phylip -q /home/arghya250/Apps/picrust2/example_files/tutorial.fna -j 10 -n out
gap rate: 1503347 21163428
gap rate: 0.0710351
rate matrix: 2,2
p: 2,2
references container instantiated as: papara::references<pvec_pgap, sequence_model::tag_dna>
edges: 32885
scoring scheme: -3 -1 2 -3
papara_core version 2.5
start scoring, using 10 threads
thread 5: 1.22437 gncup/s
thread 6: 1.23993 gncup/s
thread 1: 1.28678 gncup/s
thread 4: 1.26158 gncup/s
thread 3: 1.34938 gncup/s
thread 9: 1.31181 gncup/s
thread 8: 1.28195 gncup/s
thread 0: 1.33646 gncup/s
thread 2: 1.3424 gncup/s
thread 7: 1.26382 gncup/s
scoring finished: 9.65413
sweep: 98658 -> 0
38.1369
generating best scoring alignments
SUCCESS 38.1369
INFO Converting given FASTA file to BFAST format...
INFO Resulting bfast file was written to: placement_working/epa_out/study_seqs_papara.fasta.bfast
INFO Selected: Output dir: placement_working/epa_out/
INFO Selected: Query file: placement_working/epa_out/study_seqs_papara.fasta.bfast
INFO Selected: Tree file: /home/arghya250/Apps/picrust2/default_files/prokaryotic/img_centroid_16S_aligned.tree
INFO Selected: Reference MSA: placement_working/ref_seqs_papara.fasta
INFO Selected: Specified model: GTR+G
INFO Rate heterogeneity: GAMMA (4 cats, mean), alpha: 1 (ML), weights&rates: (0.25,0.136954) (0.25,0.476752) (0.25,1) (0.25,2.38629)
Base frequencies (ML): 0.25 0.25 0.25 0.25
Substitution rates (ML): 0.5 0.5 0.5 0.5 0.5 1
INFO Selected: Reading queries in chunks of: 5000
INFO Selected: Using threads: 10
INFO ______ ____ ___ _ __ ______
/ // __ \ / | / | / // /
/ __/ / // // /| | ______ / |/ // / __
/ / / // ___ |/_____// /| // // /
/_____/// // || // |/ __/ (v0.2.1-beta)
INFO Number of ranks: 1
INFO Output file: placement_working/epa_out/epa_result.jplace
Error running this command:
epa-ng --tree /home/arghya250/Apps/picrust2/default_files/prokaryotic/img_centroid_16S_aligned.tree --ref-msa placement_working/ref_seqs_papara.fasta --query placement_working/epa_out/study_seqs_papara.fasta.bfast --chunk-size 5000 -T 10 -m GTR+G -w placement_working/epa_out
Whats going wrong here ? Please help.
I am running run_minpathway.py using the suggested parameters and the pred_metagenome_unstrat.tsv file generated by metagenome_pipeline.py. I'm getting an error raised by pandas: "KeyError: "['KMGS1'] not in index"
KMGS1 is my first sample name, and is the name of the first column after 'function' in the pred_metagenome_unstrat.tsv file. I have no idea what might be causing this error; any input would be appreciated.
Hi there,
Today I installed what I believe is the newest version of picrust2. Everything worked fine until I executed the pytest. It appears that it cannot find the two scripts below:
FileNotFoundError: [Errno 2] No such file or directory: 'place_seqs.py': 'place_seqs.py'
FileNotFoundError: [Errno 2] No such file or directory: 'picrust2_pipeline.py': 'picrust2_pipeline.py'
Here is the output of the pytest. Please let me know if you know why this may be happening. Thanks!
Yakshi
(picrust2) [yortiz@boqueron ~]$ pytest
============================================ test session starts =============================================
platform linux -- Python 3.6.7, pytest-4.1.1, py-1.7.0, pluggy-0.8.1
rootdir: /home/crestrepo/yortiz, inifile:
plugins: cov-2.6.1
collected 46 items
picrust2/tests/test_hsp.py ........ [ 17%]
picrust2/tests/test_metagenome_pipeline.py ........... [ 41%]
picrust2/tests/test_place_seqs.py .... [ 50%]
picrust2/tests/test_run_minpath.py ... [ 56%]
picrust2/tests/test_util.py .................. [ 95%]
picrust2/tests/test_workflow.py FF [100%]
================================================== FAILURES ==================================================
____________________________________ workflow_test.test_full_pipeline_tsv ____________________________________
self = <tests.test_workflow.workflow_test testMethod=test_full_pipeline_tsv>
def test_full_pipeline_tsv(self):
'''Test that full pipeline can be run without error with
TSV sequence abundance table.'''
with TemporaryDirectory() as temp_dir:
out_tree = path.join(temp_dir, "out.tre")
system_call_check("place_seqs.py -s " + test_study_seqs + " -r " +
test_msa + " -t " + test_tree + " --hmm " +
test_hmm + " -o " + out_tree)
picrust2/tests/test_workflow.py:55:
picrust2/picrust2/util.py:211: in system_call_check
return_value = call(cmd, stdout=stdout, stderr=stderr)
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:267: in call
with Popen(*popenargs, **kwargs) as p:
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:709: in init
restore_signals, start_new_session)
self = <subprocess.Popen object at 0x2aab180ee4a8>
args = ['place_seqs.py', '-s', '/home/crestrepo/yortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta', '-r', '/home/crestrepo/yortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.fna', '-t', ...]
executable = b'place_seqs.py', preexec_fn = None, close_fds = True, pass_fds = (), cwd = None, env = None
startupinfo = None, creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1
errread = -1, errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""
if isinstance(args, (str, bytes)):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
orig_executable = executable
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = os.pipe()
# errpipe_write must not be in the standard io 0, 1, or 2 fd range.
low_fds_to_close = []
while errpipe_write < 3:
low_fds_to_close.append(errpipe_write)
errpipe_write = os.dup(errpipe_write)
for low_fd in low_fds_to_close:
os.close(low_fd)
try:
try:
# We must avoid complex work that could involve
# malloc or free in the child process to avoid
# potential deadlocks, thus we do all this here.
# and pass it to fork_exec()
if env is not None:
env_list = []
for k, v in env.items():
k = os.fsencode(k)
if b'=' in k:
raise ValueError("illegal environment variable name")
env_list.append(k + b'=' + os.fsencode(v))
else:
env_list = None # Use execv instead of execve.
executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env))
fds_to_keep = set(pass_fds)
fds_to_keep.add(errpipe_write)
self.pid = _posixsubprocess.fork_exec(
args, executable_list,
close_fds, tuple(sorted(map(int, fds_to_keep))),
cwd, env_list,
p2cread, p2cwrite, c2pread, c2pwrite,
errread, errwrite,
errpipe_read, errpipe_write,
restore_signals, start_new_session, preexec_fn)
self._child_created = True
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# self._devnull is not always defined.
devnull_fd = getattr(self, '_devnull', None)
if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
os.close(p2cread)
if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
os.close(c2pwrite)
if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
os.close(errwrite)
if devnull_fd is not None:
os.close(devnull_fd)
# Prevent a double close of these fds from __init__ on error.
self._closed_child_pipe_fds = True
# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 50000)
errpipe_data += part
if not part or len(errpipe_data) > 50000:
break
finally:
# be sure the FD is closed no matter what
os.close(errpipe_read)
if errpipe_data:
try:
pid, sts = os.waitpid(self.pid, 0)
if pid == self.pid:
self._handle_exitstatus(sts)
else:
self.returncode = sys.maxsize
except ChildProcessError:
pass
try:
exception_name, hex_errno, err_msg = (
errpipe_data.split(b':', 2))
# The encoding here should match the encoding
# written in by the subprocess implementations
# like _posixsubprocess
err_msg = err_msg.decode()
except ValueError:
exception_name = b'SubprocessError'
hex_errno = b'0'
err_msg = 'Bad exception data from child: {!r}'.format(
bytes(errpipe_data))
child_exception_type = getattr(
builtins, exception_name.decode('ascii'),
SubprocessError)
if issubclass(child_exception_type, OSError) and hex_errno:
errno_num = int(hex_errno, 16)
child_exec_never_called = (err_msg == "noexec")
if child_exec_never_called:
err_msg = ""
# The error must be from chdir(cwd).
err_filename = cwd
else:
err_filename = orig_executable
if errno_num != 0:
err_msg = os.strerror(errno_num)
if errno_num == errno.ENOENT:
err_msg += ': ' + repr(err_filename)
raise child_exception_type(errno_num, err_msg, err_filename)
E FileNotFoundError: [Errno 2] No such file or directory: 'place_seqs.py': 'place_seqs.py'
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:1344: FileNotFoundError
________________________________ workflow_test.test_picrust2_pipeline_script _________________________________
self = <tests.test_workflow.workflow_test testMethod=test_picrust2_pipeline_script>
def test_picrust2_pipeline_script(self):
'''Test that full pipeline can be run successfully with
picrust2_pipeline.py'''
with TemporaryDirectory() as temp_dir:
out_dir = path.join(temp_dir, "pipeline_out")
system_call_check("picrust2_pipeline.py -s " + test_study_seqs +
" -i " + test_seq_abun_tsv + " -o " + out_dir +
" -r " + test_msa + " -t " + test_tree +
" --hmm " + test_hmm +
" --custom_trait_tables " + test_known_traits +
" --marker_gene_table " + test_known_marker +
" --verbose")
picrust2/tests/test_workflow.py:100:
picrust2/picrust2/util.py:211: in system_call_check
return_value = call(cmd, stdout=stdout, stderr=stderr)
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:267: in call
with Popen(*popenargs, **kwargs) as p:
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:709: in init
restore_signals, start_new_session)
self = <subprocess.Popen object at 0x2aab18235550>
args = ['picrust2_pipeline.py', '-s', '/home/crestrepo/yortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta', '-i', '/home/crestrepo/yortiz/picrust2/tests/test_data/workflow/workflow_seq_abun.tsv', '-o', ...]
executable = b'picrust2_pipeline.py', preexec_fn = None, close_fds = True, pass_fds = (), cwd = None
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1, p2cwrite = -1, c2pread = -1
c2pwrite = -1, errread = -1, errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""
if isinstance(args, (str, bytes)):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
orig_executable = executable
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = os.pipe()
# errpipe_write must not be in the standard io 0, 1, or 2 fd range.
low_fds_to_close = []
while errpipe_write < 3:
low_fds_to_close.append(errpipe_write)
errpipe_write = os.dup(errpipe_write)
for low_fd in low_fds_to_close:
os.close(low_fd)
try:
try:
# We must avoid complex work that could involve
# malloc or free in the child process to avoid
# potential deadlocks, thus we do all this here.
# and pass it to fork_exec()
if env is not None:
env_list = []
for k, v in env.items():
k = os.fsencode(k)
if b'=' in k:
raise ValueError("illegal environment variable name")
env_list.append(k + b'=' + os.fsencode(v))
else:
env_list = None # Use execv instead of execve.
executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env))
fds_to_keep = set(pass_fds)
fds_to_keep.add(errpipe_write)
self.pid = _posixsubprocess.fork_exec(
args, executable_list,
close_fds, tuple(sorted(map(int, fds_to_keep))),
cwd, env_list,
p2cread, p2cwrite, c2pread, c2pwrite,
errread, errwrite,
errpipe_read, errpipe_write,
restore_signals, start_new_session, preexec_fn)
self._child_created = True
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# self._devnull is not always defined.
devnull_fd = getattr(self, '_devnull', None)
if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
os.close(p2cread)
if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
os.close(c2pwrite)
if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
os.close(errwrite)
if devnull_fd is not None:
os.close(devnull_fd)
# Prevent a double close of these fds from __init__ on error.
self._closed_child_pipe_fds = True
# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 50000)
errpipe_data += part
if not part or len(errpipe_data) > 50000:
break
finally:
# be sure the FD is closed no matter what
os.close(errpipe_read)
if errpipe_data:
try:
pid, sts = os.waitpid(self.pid, 0)
if pid == self.pid:
self._handle_exitstatus(sts)
else:
self.returncode = sys.maxsize
except ChildProcessError:
pass
try:
exception_name, hex_errno, err_msg = (
errpipe_data.split(b':', 2))
# The encoding here should match the encoding
# written in by the subprocess implementations
# like _posixsubprocess
err_msg = err_msg.decode()
except ValueError:
exception_name = b'SubprocessError'
hex_errno = b'0'
err_msg = 'Bad exception data from child: {!r}'.format(
bytes(errpipe_data))
child_exception_type = getattr(
builtins, exception_name.decode('ascii'),
SubprocessError)
if issubclass(child_exception_type, OSError) and hex_errno:
errno_num = int(hex_errno, 16)
child_exec_never_called = (err_msg == "noexec")
if child_exec_never_called:
err_msg = ""
# The error must be from chdir(cwd).
err_filename = cwd
else:
err_filename = orig_executable
if errno_num != 0:
err_msg = os.strerror(errno_num)
if errno_num == errno.ENOENT:
err_msg += ': ' + repr(err_filename)
raise child_exception_type(errno_num, err_msg, err_filename)
E FileNotFoundError: [Errno 2] No such file or directory: 'picrust2_pipeline.py': 'picrust2_pipeline.py'
miniconda3/envs/picrust2/lib/python3.6/subprocess.py:1344: FileNotFoundError
Hi!
I'm running the tutorial "CBW 2018 PICRUSt2 Tutorial" with the data you provided and after running the hidden-state prediction step, when I plot the NSTI values in RStudio, I get a plot very different from yours, there are more than one outliers. Moreover, when I run the metagenome pipeline code:
metagenome_pipeline.py -i input_files/ASV_abun.tsv -f EC_predicted.tsv -m 16S_predicted.tsv -p 4
the tutorial says that I should get as output 4 files, but I'm only getting 3, there's no output called pred_metagenome_strat.tsv.
I also tried running the line code of the tutorial with my own data (an ASV table and a FASTA file) and I get the same problem, no pred_metagenome_strat.tsv file
Any ideas why?
Thanks in advance!!
Camila
metagenome_contributions.py
no longer in this repository - there could be better way to output the contributions that could leverage the likelihoods outputted in R by the discrete hidden-state prediction methods.
Hi
I am getting this error message:
metagenome_pipeline.py -i FM.biom -m 16S_predicted.tsv -f EC_predicted.tsv -p 10 -o metagenome
Traceback (most recent call last):
File "/home/hh750/anaconda2/envs/picrust2-dev/bin/metagenome_pipeline.py", line 6, in
exec(compile(open(file).read(), file, 'exec'))
File "/home/hh750/Apps/picrust2/scripts/metagenome_pipeline.py", line 82, in
main()
File "/home/hh750/Apps/picrust2/scripts/metagenome_pipeline.py", line 70, in main
output_normfile=True)
File "/home/hh750/Apps/picrust2/picrust2/metagenome_pipeline.py", line 59, in run_metagenome_pipeline
pred_marker)
File "/home/hh750/Apps/picrust2/picrust2/util.py", line 246, in three_df_index_overlap_sort
"input files.")
ValueError: No sequence ids overlap between all three of the input files.
My input files are from qiime 2. I have checked the rep seq file which was used in the placement step and the .biom file along with the 16S and EC predicted files. The sequence IDs are there. What is going wrong?Please help.
Hi Gavin, I updated my local clone today and ran pytest
and got 9 out of 19 failed tests. Here is the first part of stderr/stdout, the entire output is at pastebin . May you please take a look and see if it's a serious error? Thank you.
platform linux -- Python 3.6.5, pytest-3.6.0, py-1.5.3, pluggy-0.6.0
rootdir: /home/ubuntu/downloads/picrust2, inifile:
plugins: cov-2.5.1
collected 28 items
tests/test_hsp.py FFFFFFF [ 25%]
tests/test_metagenome_pipeline.py ....... [ 50%]
tests/test_place_seqs.py ...... [ 71%]
tests/test_run_minpath.py .. [ 78%]
tests/test_util.py .... [ 92%]
tests/test_workflow.py FF [100%]
============================================================== FAILURES ===============================================================
___________________________________________ castor_hsp_workflow_tests.test_emp_prob_simple ____________________________________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_emp_prob_simple>
def test_emp_prob_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="emp_prob",
> ran_seed=10)
tests/test_hsp.py:73:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:779: in __call__
while self.dispatch_one_batch(iterator):
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:625: in dispatch_one_batch
self._dispatch(tasks)
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:588: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:111: in apply_async
result = ImmediateResult(func)
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:332: in __init__
self.results = batch()
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:131: in __call__
return [func(*args, **kwargs) for func, args, kwargs in self.items]
../../miniconda2/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:131: in <listcomp>
return [func(*args, **kwargs) for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/home/ubuntu/downloads/picrust2/picrust2/Rscripts/castor_hsp.R', '/home/ubuntu/downloads/picrust2/tests/test_data/hsp/tree.tre', '/tmp/tmpb3627unh/subset_tab_0', 'emp_prob', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /home/ubuntu/downloads/picrust2/picrust2/Rscripts/castor_hsp.R /home/ubuntu/downloads/picrust2/tests/test_data/hsp/tree.tre /tmp/tmpb3627unh/subset_tab_0 emp_prob FALSE FALSE /tmp/tmpaf7krjxy/predicted_counts.txt /tmp/tmpaf7krjxy/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
Best,
Jamie
Hi
I installed the latest version of picrust2, but when running the pytest command I've got 13 warnings that I don't know if they are crucial or not for the correct usage of the picrust2 pipeline. Could you please take a look and see what is wrong? Thank you for any help.
(picrust2) igor@ubuntu:~/picrust2$ pytest
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.0.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/igor/picrust2, inifile:
plugins: cov-2.6.0
collected 46 items
tests/test_hsp.py ........ [ 17%]
tests/test_metagenome_pipeline.py ........... [ 41%]
tests/test_place_seqs.py .... [ 50%]
tests/test_run_minpath.py ... [ 56%]
tests/test_util.py .................. [ 95%]
tests/test_workflow.py .. [100%]
=============================== warnings summary ===============================
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_strat_biom
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_strat_tsv
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_strat_tsv_2proc
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_unstrat_biom
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_unstrat_biom_when_no_strat
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_full_pipeline_unstrat_tsv_when_no_strat
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::metagenome_pipeline_test::test_norm_by_marker_copies
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::rare_seqs_test::test_full_pipeline_strat_tsv_rare_category
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::rare_seqs_test::test_rare_2_samp
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_metagenome_pipeline.py::rare_seqs_test::test_rare_4_reads
/home/igor/miniconda2/envs/picrust2/lib/python3.6/site-packages/numpy/matrixlib/defmatrix.py:68: PendingDeprecationWarning: the matrix subclass is not the recommended way to represent matrices or deal with linear algebra (see https://docs.scipy.org/doc/numpy/user/numpy-for-matlab-users.html). Please adjust your code to use regular ndarray.
return matrix(data, dtype=dtype, copy=False)
tests/test_run_minpath.py::run_minpath_tests::test_strat_default_pipeline
/home/igor/picrust2/picrust2/run_minpath.py:424: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=False'.
To retain the current behavior and silence the warning, pass 'sort=True'.
in_tab_df = pd.concat(in_tab, axis=1)
tests/test_util.py::convert_table_tests::test_humann2_strat_split_to_picrust2
/home/igor/picrust2/picrust2/util.py:391: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=False'.
To retain the current behavior and silence the warning, pass 'sort=True'.
humann2_combined = pd.concat(humann2_samples, axis=1)
tests/test_util.py::convert_table_tests::test_picrust2_to_humann2_merged
/home/igor/picrust2/picrust2/util.py:569: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version
of pandas will change to not sort by default.
To accept the future behavior, pass 'sort=False'.
To retain the current behavior and silence the warning, pass 'sort=True'.
new_tab = pd.concat([new_tab, in_table], axis=1)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=================== 46 passed, 13 warnings in 63.86 seconds ====================
(picrust2) igor@ubuntu:~/picrust2$
Hi Gavin, Morgan, and Daniel,
This is actually a feature or guideline request. I try to figure out how difficult is to get PICRUSt2 support HTCondor, based on this CERN presentation.
I saw on PICRUSt make_cluster_jobs.py and you guys have support Galaxy, SGE, and Torque job before. The particular interesting topics for me are:
Jamie (@jjmmii) and I are actually colleagues. She ran some jobs on EC2 r4.large instances with -4
and then the memory got blow-up on the last stage (not enough memory on the last merge stage). We moved to r4.xlarge then it got passed. This makes me wonder if I would like to run PICRUSt2 with full automatically way, the checkpoint making and resuming should be unavoidable. Just wonder how you guys handle this or previously you guys get Galaxy to handle this.
Continue on # 1, we try to come out some formula to estimate how many memory / CPU ratio, and how to manually specify the argument of a task to split multiple small jobs. If we fail to do so, no matter we submit to VM based scheduler (SGE, Torque, or AWS EMR) or container based scheduler (Kubernetes, Mesos, or AWS ECS, Google GKE) the outcome won't be good if we don't guard with checkpoint.
Thanks and have a weekend.
I was trying out the q2-picrust2 Tutorial (from https://github.com/picrust/picrust2/wiki/q2-picrust2-Tutorial).
I got through the first part but got stuck at the qiime picrust2 custom-tree-pipeline command...
The feedback from the command is as followed:
Error running this command:
hsp.py -i 16S -t /tmp/tmp1fyg7g9t/placed_seqs.tre -p 1 -n -o /tmp/tmp1fyg7g9t/picrust2_out/16S_predicted -m mp
I am using the QIIME2 in a VirtualBox.
How should I fix it...?
Thanks in advance!!
Hi
I tried to run the command bellow but it gave this error... Any help? Thanks.
hsp.py -i COG -t kiwis.tre -o COG_kiwis_predicted -p 1 --chunk_size 100
Error in WMPR_ASR_CPP(Ntips = Ntips, Nnodes = tree$Nnode, Nedges = Nedges, :
std::bad_alloc
Calls: lapply ... hsp_max_parsimony -> asr_max_parsimony -> WMPR_ASR_CPP -> .Call
Execution halted
Error running this command:
Rscript /home/igor/picrust2/picrust2/Rscripts/castor_hsp.R kiwis.tre /tmp/tmph76xe_st/subset_tab_27 mp FALSE FALSE /tmp/tmpvxx5iyyv/predicted_counts.txt /tmp/tmpvxx5iyyv/predicted_ci.txt None
In the Full Pipeline page of the Wiki it is written:
--per_sequence_contrib
- Run MinPath on the gene families contributed by each sequence (i.e. a predicted genome) individually. This will only matter--per_sequence_contrib
is set.
I believe it should be:
--per_sequence_contrib
- Run MinPath on the gene families contributed by each sequence (i.e. a predicted genome) individually. This will only matter--stratified
is set.
Hi, First of all, I want to thank Gavin for this new tool.
I'm trying to run the following command:
hsp.py -i 16S -t /home/igor/Desktop/Alfaguara/Picrust2/alfaguara.tre -o 16S_Alfaguara_predicted -p 10 -n
Then I have this error message:
(picrust2-dev) igor@ubuntu:~/Desktop/Alfaguara/Picrust2$ hsp.py -i 16S -t alfaguara.tre -o 16S_Alfaguara_predicted -p 7 -n
*** caught segfault ***
address 0x556a3, cause 'memory not mapped'
Traceback:
1: .Call(treeBuild, x)
2: FUN(X[[i]], ...)
3: lapply(STRING, .treeBuild)
4: read.tree(Args[1])
An irrecoverable exception occurred. R is aborting now ...
Error running this command:
Rscript /home/igor/picrust2-master/picrust2/Rscripts/castor_hsp.R /home/igor/Desktop/Alfaguara/Picrust2/alfaguara.tre /home/igor/picrust2-master/precalculated/prokaryotic/16S_counts_mean_round_var.txt mp TRUE FALSE FALSE 7 /tmp/0g39u50d /tmp/1fyxa9fn None None FALSE
(picrust2-dev) igor@ubuntu:~/Desktop/Alfaguara/Picrust2$
Any thoughts?
Thanks
Igor
I have run the same biom file and representative sequences with both the qiime2 plugin of picrust2 and the picrust2_pipeline.py program, and have found around 600 KOs to be present/absent in one v/s the other. Also the KOs which are present in both the results show different abundance values. Can this difference be attributed to SEPP v/s EPA-NG?
I followed the pre-requisite installation instructions and received several errors while running the make commands :
cd epa-ng-0.2.1-beta-dev
make
Running cmake
CMake Warning (dev) at CMakeLists.txt:119:
Syntax Warning in cmake code at column 71
Argument not separated from preceding token by whitespace.
This warning is for project developers. Use -Wno-dev to suppress it.
-- The CXX compiler identification is GNU 4.8.5
-- The C compiler identification is GNU 4.8.5
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
CMake Error at CMakeLists.txt:39 (message):
GNU compiler too old! Minimum required: 4.9.2
-- Configuring incomplete, errors occurred!
See also "/users/spenumut/picrust2/placement_tools/epa-ng-0.2.1-beta-dev/build/CMakeFiles/CMakeOutput.log".
make: *** [build/CMakeCache.txt] Error 1
cd gappa-0.0.0-dev
make
Running CMake...
-- The CXX compiler identification is GNU 4.8.5
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Gappa build type: RELEASE
-- Static linking of system libraries: OFF
-- CLI11 not found
-- Downloading CLI11 from https://github.com/CLIUtils/CLI11/archive/bd890e24a1f081d6db1a59a0cc372eccc64e9e52.zip
-- Configuring done
-- Generating done
-- Build files have been written to: /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download
gmake[1]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download' gmake[2]: Entering directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download' Scanning dependencies of target CLI11 gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download' [ 11%] Creating directories for 'CLI11' [ 22%] Performing download step (download, verify and extract) for 'CLI11' -- Downloading... dst='/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download/CLI11-prefix/src/bd890e24a1f081d6db1a59a0cc372eccc64e9e52.zip' timeout='none' -- Using src='https://github.com/CLIUtils/CLI11/archive/bd890e24a1f081d6db1a59a0cc372eccc64e9e52.zip' -- Downloading... done -- extracting... src='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download/CLI11-prefix/src/bd890e24a1f081d6db1a59a0cc372eccc64e9e52.zip' dst='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/libs/CLI11' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 33%] No patch step for 'CLI11' [ 44%] No update step for 'CLI11' [ 55%] No configure step for 'CLI11' [ 66%] No build step for 'CLI11' [ 77%] No install step for 'CLI11' [ 88%] Performing test step for 'CLI11' /bin/sh: DOWNLOAD_NO_PROGRESS 1: command not found gmake[3]: *** [CLI11-prefix/src/CLI11-stamp/CLI11-test] Error 127 gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download'
gmake[2]: *** [CMakeFiles/CLI11.dir/all] Error 2
gmake[2]: Leaving directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/CLI11Download'
-- Cannot build CLI11: 2
-- genesis not found
-- Downloading genesis from https://github.com/lczech/genesis/archive/6e1aa0665614df91ec51400356d339ca88c7134b.zip
-- Configuring done
-- Generating done
-- Build files have been written to: /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload
gmake[1]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload' gmake[2]: Entering directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload' Scanning dependencies of target genesis gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload' [ 11%] Creating directories for 'genesis' [ 22%] Performing download step (download, verify and extract) for 'genesis' -- Downloading... dst='/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload/genesis-prefix/src/6e1aa0665614df91ec51400356d339ca88c7134b.zip' timeout='none' -- Using src='https://github.com/lczech/genesis/archive/6e1aa0665614df91ec51400356d339ca88c7134b.zip' -- Downloading... done -- extracting... src='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload/genesis-prefix/src/6e1aa0665614df91ec51400356d339ca88c7134b.zip' dst='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/libs/genesis' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 33%] No patch step for 'genesis' [ 44%] No update step for 'genesis' [ 55%] No configure step for 'genesis' [ 66%] No build step for 'genesis' [ 77%] No install step for 'genesis' [ 88%] Performing test step for 'genesis' /bin/sh: DOWNLOAD_NO_PROGRESS 1: command not found gmake[3]: *** [genesis-prefix/src/genesis-stamp/genesis-test] Error 127 gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload'
gmake[2]: Leaving directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload' gmake[2]: *** [CMakeFiles/genesis.dir/all] Error 2 gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/genesisDownload'
-- Cannot build genesis: 2
-- sparsepp not found
-- Downloading sparsepp from https://github.com/greg7mdp/sparsepp/archive/6bfe3b4bdb364993e612d6bb729d680cf4c77649.zip
-- Configuring done
-- Generating done
-- Build files have been written to: /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload
gmake[1]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload' gmake[2]: Entering directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload' Scanning dependencies of target sparsepp gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload'
gmake[3]: Entering directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload' [ 11%] Creating directories for 'sparsepp' [ 22%] Performing download step (download, verify and extract) for 'sparsepp' -- Downloading... dst='/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload/sparsepp-prefix/src/6bfe3b4bdb364993e612d6bb729d680cf4c77649.zip' timeout='none' -- Using src='https://github.com/greg7mdp/sparsepp/archive/6bfe3b4bdb364993e612d6bb729d680cf4c77649.zip' -- Downloading... done -- extracting... src='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload/sparsepp-prefix/src/6bfe3b4bdb364993e612d6bb729d680cf4c77649.zip' dst='/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/libs/sparsepp' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 33%] No patch step for 'sparsepp' [ 44%] No update step for 'sparsepp' [ 55%] No configure step for 'sparsepp' [ 66%] No build step for 'sparsepp' [ 77%] No install step for 'sparsepp' [ 88%] Performing test step for 'sparsepp' /bin/sh: DOWNLOAD_NO_PROGRESS 1: command not found gmake[3]: *** [sparsepp-prefix/src/sparsepp-stamp/sparsepp-test] Error 127 gmake[3]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload'
gmake[2]: *** [CMakeFiles/sparsepp.dir/all] Error 2
gmake[2]: Leaving directory /gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload' gmake[1]: *** [all] Error 2 gmake[1]: Leaving directory
/gpfs_home/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build/sparseppDownload'
-- Cannot build sparsepp: 2
-- Configuring Genesis
-- CMake version 3.12.0
-- The C compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Genesis version: v0.19.0
-- Building Genesis as a dependency
-- Build type: RELEASE
-- Unity build: FULL
-- C++ compiler: GNU 4.8.5 at /usr/bin/c++
-- C compiler : GNU 4.8.5 at /usr/bin/cc
-- Check if the system is big endian
-- Searching 16 bit integer
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of unsigned short
-- Check size of unsigned short - done
-- Using unsigned short
-- Check if the system is big endian - little endian
-- Looking for Threads
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found Threads: -pthread
-- Using Threads
-- Looking for OpenMP
-- Found OpenMP_C: -fopenmp (found version "3.1")
-- Found OpenMP_CXX: -fopenmp (found version "3.1")
-- Found OpenMP: TRUE (found version "3.1")
-- Found OpenMP: -fopenmp
-- Using OpenMP
-- Building static lib
-- Finished configuring Genesis
-- CMAKE_EXE_LINKER_FLAGS
-- GENESIS_LINK_LIBRARIES -pthread;genesis_lib_static
-- Configuring done
-- Generating done
-- Build files have been written to: /users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/build
Running make...
make -s -C build
Scanning dependencies of target genesis_lib_static
[ 3%] Building CXX object libs/genesis/lib/genesis/CMakeFiles/genesis_lib_static.dir/////genesis_unity_sources/lib/all.cpp.o
[ 6%] Linking CXX static library ../../../../../libs/genesis/bin/libgenesis.a
[ 6%] Built target genesis_lib_static
Scanning dependencies of target gappa
[ 9%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/assign.cpp.o
[ 12%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/correlation.cpp.o
[ 15%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/dispersion.cpp.o
[ 18%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/edgepca.cpp.o
[ 21%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/graft.cpp.o
[ 24%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/kmeans.cpp.o
[ 27%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/kmeans_imbalance.cpp.o
[ 30%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/kmeans_phylogenetic.cpp.o
[ 33%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/krd.cpp.o
[ 36%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/nhd.cpp.o
[ 39%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/squash.cpp.o
[ 42%] Building CXX object CMakeFiles/gappa.dir/src/commands/analyze/visualize_color.cpp.o
[ 45%] Building CXX object CMakeFiles/gappa.dir/src/commands/prepare/chunkify.cpp.o
[ 48%] Building CXX object CMakeFiles/gappa.dir/src/commands/prepare/extract.cpp.o
[ 51%] Building CXX object CMakeFiles/gappa.dir/src/commands/prepare/phat.cpp.o
[ 54%] Building CXX object CMakeFiles/gappa.dir/src/commands/prepare/unchunkify.cpp.o
[ 57%] Building CXX object CMakeFiles/gappa.dir/src/main.cpp.o
[ 60%] Building CXX object CMakeFiles/gappa.dir/src/options/color/color_map.cpp.o
[ 63%] Building CXX object CMakeFiles/gappa.dir/src/options/color/color_norm.cpp.o
[ 66%] Building CXX object CMakeFiles/gappa.dir/src/options/color/single_color.cpp.o
[ 69%] Building CXX object CMakeFiles/gappa.dir/src/options/file_input.cpp.o
[ 72%] Building CXX object CMakeFiles/gappa.dir/src/options/file_output.cpp.o
[ 75%] Building CXX object CMakeFiles/gappa.dir/src/options/global.cpp.o
[ 78%] Building CXX object CMakeFiles/gappa.dir/src/options/jplace_input.cpp.o
[ 81%] Building CXX object CMakeFiles/gappa.dir/src/options/matrix_output.cpp.o
In file included from /users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/src/options/matrix_output.hpp:32:0,
from /users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/src/options/matrix_output.cpp:24:
/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/libs/genesis/lib/genesis/utils/containers/matrix/writer.hpp: In constructor ‘genesis::utils::MatrixWriter::MatrixWriter() [with T = double]’:
/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/libs/genesis/lib/genesis/utils/containers/matrix/writer.hpp:72:5: error: conversion from ‘const char [2]’ to non-scalar type ‘std::string {aka std::basic_string}’ requested
MatrixWriter() = default;
^
/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/src/options/matrix_output.cpp: In member function ‘void MatrixOutputOptions::write_matrix(const genesis::utils::Matrix&, const std::vector<std::basic_string >&, const std::vector<std::basic_string >&, const string&) const’:
/users/spenumut/picrust2/placement_tools/gappa-0.0.0-dev/src/options/matrix_output.cpp:99:40: note: synthesized method ‘genesis::utils::MatrixWriter::MatrixWriter() [with T = double]’ first required here
auto writer = MatrixWriter();
^
make[3]: *** [CMakeFiles/gappa.dir/src/options/matrix_output.cpp.o] Error 1
make[2]: *** [CMakeFiles/gappa.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [build] Error 2
Hi Gavin
I've updated picrust2 using the command bellow
I even tried to logout and run pytest again but the result was the same.
Any thoughts?
Thanks
Igor
igor@ubuntu:$ source activate picrust2$ conda install -c bioconda -c conda-forge picrust2
igor@ubuntu:
(picrust2) igor@ubuntu:~$ pytest
============================= test session starts ==============================
platform linux -- Python 3.6.7, pytest-4.2.1, py-1.7.0, pluggy-0.8.1
rootdir: /home/igor, inifile:
plugins: cov-2.6.1
collected 8 items / 45 errors
==================================== ERRORS ====================================
__________ ERROR collecting lxml/src/lxml/html/tests/test_autolink.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_autolink.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/html/tests/test_basic.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_basic.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/html/tests/test_clean.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_clean.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
____________ ERROR collecting lxml/src/lxml/html/tests/test_diff.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_diff.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
________ ERROR collecting lxml/src/lxml/html/tests/test_elementsoup.py _________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_elementsoup.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
______ ERROR collecting lxml/src/lxml/html/tests/test_feedparser_data.py _______
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_feedparser_data.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
__________ ERROR collecting lxml/src/lxml/html/tests/test_formfill.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_formfill.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/html/tests/test_forms.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_forms.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/html/tests/test_forms.txt ___________
miniconda2/envs/picrust2/lib/python3.6/doctest.py:669: in get_doctest
return DocTest(self.get_examples(string, name), globs,
miniconda2/envs/picrust2/lib/python3.6/doctest.py:683: in get_examples
return [x for x in self.parse(string, name)
miniconda2/envs/picrust2/lib/python3.6/doctest.py:645: in parse
self._parse_example(m, name, lineno)
miniconda2/envs/picrust2/lib/python3.6/doctest.py:726: in _parse_example
options = self._find_options(source, name, lineno)
miniconda2/envs/picrust2/lib/python3.6/doctest.py:757: in _find_options
(lineno+1, name, option))
E ValueError: line 52 of the doctest for test_forms.txt has an invalid option: '+NOPARSE_MARKUP'
___________ ERROR collecting lxml/src/lxml/html/tests/test_frames.py ___________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_frames.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
________ ERROR collecting lxml/src/lxml/html/tests/test_html5parser.py _________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_html5parser.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
________ ERROR collecting lxml/src/lxml/html/tests/test_rewritelinks.py ________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_rewritelinks.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/html/tests/test_xhtml.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/html/tests/test_xhtml.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_____________ ERROR collecting lxml/src/lxml/tests/test_builder.py _____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_builder.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_builder.py:9: in
from lxml import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/tests/test_classlookup.py ___________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_classlookup.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_classlookup.py:14: in
from common_imports import etree, HelperTestCase, SillyFileLike, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_______________ ERROR collecting lxml/src/lxml/tests/test_css.py _______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_css.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_css.py:3: in
import lxml.html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_________ ERROR collecting lxml/src/lxml/tests/test_doctestcompare.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_doctestcompare.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_doctestcompare.py:4: in
from lxml import etree
E ImportError: cannot import name 'etree'
_______________ ERROR collecting lxml/src/lxml/tests/test_dtd.py _______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_dtd.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_dtd.py:13: in
from common_imports import etree, html, BytesIO, _bytes, _str
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/tests/test_elementpath.py ___________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_elementpath.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_elementpath.py:11: in
from .common_imports import etree, HelperTestCase
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/tests/test_elementtree.py ___________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_elementtree.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_elementtree.py:18: in
from common_imports import BytesIO, etree
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_____________ ERROR collecting lxml/src/lxml/tests/test_errors.py ______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_errors.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_errors.py:10: in
from lxml import etree
E ImportError: cannot import name 'etree'
______________ ERROR collecting lxml/src/lxml/tests/test_etree.py ______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_etree.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_etree.py:25: in
from .common_imports import etree, StringIO, BytesIO, HelperTestCase
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
________ ERROR collecting lxml/src/lxml/tests/test_external_document.py ________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_external_document.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_external_document.py:11: in
from .common_imports import HelperTestCase, etree, skipIf
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/tests/test_htmlparser.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_htmlparser.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_htmlparser.py:14: in
from common_imports import etree, html, StringIO, BytesIO, fileInTestDir, _bytes, _str
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_____________ ERROR collecting lxml/src/lxml/tests/test_http_io.py _____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_http_io.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_http_io.py:19: in
from .common_imports import (
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_______ ERROR collecting lxml/src/lxml/tests/test_incremental_xmlfile.py _______
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_incremental_xmlfile.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_incremental_xmlfile.py:16: in
from lxml.etree import LxmlSyntaxError
E ModuleNotFoundError: No module named 'lxml.etree'
_______________ ERROR collecting lxml/src/lxml/tests/test_io.py ________________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_io.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_io.py:14: in
from common_imports import etree, ElementTree, _str, _bytes
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
__________ ERROR collecting lxml/src/lxml/tests/test_isoschematron.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_isoschematron.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_isoschematron.py:8: in
from lxml import isoschematron
lxml/src/lxml/isoschematron/init.py:7: in
from lxml import etree as _etree # due to validator init signature
E ImportError: cannot import name 'etree'
____________ ERROR collecting lxml/src/lxml/tests/test_nsclasses.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_nsclasses.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_nsclasses.py:14: in
from common_imports import etree, HelperTestCase, _bytes
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
____________ ERROR collecting lxml/src/lxml/tests/test_objectify.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_objectify.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_objectify.py:14: in
from common_imports import etree, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
__________ ERROR collecting lxml/src/lxml/tests/test_pyclasslookup.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_pyclasslookup.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_pyclasslookup.py:14: in
from common_imports import etree, StringIO, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_____________ ERROR collecting lxml/src/lxml/tests/test_relaxng.py _____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_relaxng.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_relaxng.py:13: in
from common_imports import etree, BytesIO, _bytes, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_______________ ERROR collecting lxml/src/lxml/tests/test_sax.py _______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_sax.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_sax.py:13: in
from common_imports import HelperTestCase, make_doctest, BytesIO, _bytes
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
___________ ERROR collecting lxml/src/lxml/tests/test_schematron.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_schematron.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_schematron.py:13: in
from common_imports import etree, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
____________ ERROR collecting lxml/src/lxml/tests/test_threading.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_threading.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_threading.py:17: in
from common_imports import etree, HelperTestCase, BytesIO, _bytes
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_____________ ERROR collecting lxml/src/lxml/tests/test_unicode.py _____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_unicode.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_unicode.py:10: in
from common_imports import StringIO, etree, SillyFileLike, HelperTestCase
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
____________ ERROR collecting lxml/src/lxml/tests/test_xmlschema.py ____________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_xmlschema.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_xmlschema.py:13: in
from common_imports import etree, BytesIO, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_________ ERROR collecting lxml/src/lxml/tests/test_xpathevaluator.py __________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_xpathevaluator.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_xpathevaluator.py:13: in
from common_imports import etree, HelperTestCase, _bytes, BytesIO
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
______________ ERROR collecting lxml/src/lxml/tests/test_xslt.py _______________
ImportError while importing test module '/home/igor/lxml/src/lxml/tests/test_xslt.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
lxml/src/lxml/tests/test_xslt.py:33: in
from .common_imports import etree, BytesIO, HelperTestCase, fileInTestDir
lxml/src/lxml/tests/common_imports.py:18: in
from lxml import etree, html
lxml/src/lxml/html/init.py:54: in
from .. import etree
E ImportError: cannot import name 'etree'
_________________ ERROR collecting picrust2/tests/test_hsp.py __________________
ImportError while importing test module '/home/igor/picrust2/tests/test_hsp.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_hsp.py:9: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
_________ ERROR collecting picrust2/tests/test_metagenome_pipeline.py __________
ImportError while importing test module '/home/igor/picrust2/tests/test_metagenome_pipeline.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_metagenome_pipeline.py:9: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
______________ ERROR collecting picrust2/tests/test_place_seqs.py ______________
ImportError while importing test module '/home/igor/picrust2/tests/test_place_seqs.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_place_seqs.py:10: in
from picrust2.util import get_picrust_project_dir, read_phylip, read_fasta
picrust2/picrust2/util.py:11: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
_____________ ERROR collecting picrust2/tests/test_run_minpath.py ______________
ImportError while importing test module '/home/igor/picrust2/tests/test_run_minpath.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_run_minpath.py:8: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
_________________ ERROR collecting picrust2/tests/test_util.py _________________
ImportError while importing test module '/home/igor/picrust2/tests/test_util.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_util.py:9: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
_______________ ERROR collecting picrust2/tests/test_workflow.py _______________
ImportError while importing test module '/home/igor/picrust2/tests/test_workflow.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
picrust2/tests/test_workflow.py:10: in
from picrust2.util import get_picrust_project_dir, system_call_check
picrust2/picrust2/util.py:11: in
import pandas as pd
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/init.py:42: in
from pandas.core.api import *
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/api.py:10: in
from pandas.core.groupby import Grouper
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/init.py:2: in
from pandas.core.groupby.groupby import (
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/groupby/groupby.py:48: in
from pandas.core.arrays import ExtensionArray, Categorical
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/init.py:1: in
from .base import ExtensionArray # noqa
miniconda2/envs/picrust2/lib/python3.6/site-packages/pandas/core/arrays/base.py:10: in
from pandas.errors import AbstractMethodError
E ImportError: cannot import name 'AbstractMethodError'
!!!!!!!!!!!!!!!!!!! Interrupted: 45 errors during collection !!!!!!!!!!!!!!!!!!!
========================== 45 error in 22.13 seconds ===========================
(picrust2) igor@ubuntu:~$
Hello,
I am trying to set up the picrust2 beta to play around with it. I have installed all dependencies and they seem to work independently of picrust2. When I run pytest I have 4 errors which all seem to be the same. epa-ng says it must be run with a model explicitly specified. I have pasted a part of the error below. Does this mean anything to you or does it seem to be an issue on my end?
Thanks,
Mike
--------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------
papara called as:
papara -t /Users/mish0397/git_sw/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre -s ref_seqs.phylip -q /Users/mish0397/git_sw/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta -j 1 -n out
gap rate: 1139 19290
gap rate: 0.0590461
rate matrix: 2,2
p: 2,2
references container instantiated as: papara::references<pvec_pgap, sequence_model::tag_dna>
edges: 27
scoring scheme: -3 -1 2 -3
papara_core version 2.5
start scoring, using 1 threads
thread 0: 2.25299 gncup/s
scoring finished: 0.0308421
0.0701151
generating best scoring alignments
SUCCESS 0.0701232
INFO Selected: Output dir: /tmp/tmp8_76gnoc/epa_out/
INFO Selected: Query file: /tmp/tmp8_76gnoc/study_seqs_papara.fasta
INFO Selected: Tree file: /Users/mish0397/git_sw/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre
INFO Selected: Reference MSA: /tmp/tmp8_76gnoc/ref_seqs_papara.fasta
INFO Selected: Specified model: GTR+G
INFO Rate heterogeneity: GAMMA (4 cats, mean), alpha: 1 (ML), weights&rates: (0.25,0.136954) (0.25,0.476752) (0.25,1) (0.25,2.38629)
Base frequencies (ML): 0.25 0.25 0.25 0.25
Substitution rates (ML): 0.5 0.5 0.5 0.5 0.5 1
INFO Selected: Reading queries in chunks of: 5000
INFO Selected: Using threads: 1
INFO ______ ____ ___ _ __ ______
/ // __ \ / | / | / // /
/ __/ / // // /| | ______ / |/ // / __
/ / / // ___ |/_____// /| // // /
/_____/// // || // |/ __/ (v0.2.1-beta)
When using epa-ng like this, a model has to be explicitly specified!
You may specify it generically (GTR+G), however parameters will not be optimized.
Instead we reccommend to use RAxML to re-evaluate the parameters and then pass the resulting
RAxML_info file to the epa-ng --model argument. epa-ng will then auto-parse the parameters.
( raxmlHPC -f e -s /tmp/tmp8_76gnoc/ref_seqs_papara.fasta -t /Users/mish0397/git_sw/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre -n info -m GTRGAMMAX )
Aborting with a failure.
--------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------
sweep: 84 -> 0
Error running this command:
epa-ng --tree /Users/mish0397/git_sw/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre --ref-msa /tmp/tmp8_76gnoc/ref_seqs_papara.fasta --query /tmp/tmp8_76gnoc/study_seqs_papara.fasta --chunk-size 5000 -T 1 -w /tmp/tmp8_76gnoc/epa_out
Hi Gavin,
I'm just wondering if there is a categorize_by_function.py script within PiCrust2 (Specifically within the QIIME2 plugin) like in the original PiCrust in order to collapse the pathways into higher level of the pathway hierarchy?
I've successfully produced the KO_metagenome.qza, EC_metagenome.qza and the pathway_abundance.qza and have also been able to export and convert them biom > .spf using your Microbiome helper but I'm still only able to run and visualize on individual KO instead of functional groups.
Any thoughts?
####Halo, I am new to picrust2 and I encounter some problem running the tutorial data.
####I am using virtual machine to run picrust2 and my installation process is following below:
conda create -n picrust2
source activate picrust2
conda install -c bioconda -c conda-forge picrust2
#####After the installation I have run downloaded the tutorial data and run the command as follows:
picrust2_pipeline.py -s /media/sf_microbiome_USB/dry_lab/raw_data/picrust_testing/tutorial_seqs.fna -i /media/sf_microbiome_USB/dry_lab/raw_data/picrust_testing/tutorial.biom -o picrust2_out_pipeline
#####After that I have encounter the following error message:
Error: File existence/permissions problem in trying to open HMM file /home/qiime2/miniconda/envs/picrust2/lib/python3.6/site-packages/default_files/prokaryotic/reference.hmm.
HMM file /home/qiime2/miniconda/envs/picrust2/lib/python3.6/site-packages/default_files/prokaryotic/reference.hmm not found (no
Error running this command:
hmmalign --trim --dna --mapali /home/qiime2/miniconda/envs/picrust2/lib/python3.6/site-packages/default_files/prokaryotic/reference.fna --informat FASTA -o picrust2_out_pipeline_5/intermediate/place_seqs/query_align.stockholm /home/qiime2/miniconda/envs/picrust2/lib/python3.6/site-packages/default_files/prokaryotic/reference.hmm /media/sf_microbiome_USB/dry_lab/raw_data/picrust_testing/tutorial_seqs.fna
####After investigation, I found that /home/qiime2/miniconda/envs/picrust2/lib/python3.6/site-packages have no any sub directory under it. However, I dun know why this will be happened and how to solve it.
Thank you so much for helping me
Trying to install PICRUST2. All prerequisites are installed form the "placement_tools" folder. pytest failed 5 out of 41 tests. I found that there is an issue with EPA-NG. It produces an error:
/lib64/libstdc++.so.6: version
GLIBCXX_3.4.20' not found (required by epa-ng)`
I could fix it with copiyng libstdc++.so.6.0.25 to /lib64 folder but I don't understand why EPA-NG search here. $LD_LIBRARY_PATH don't contain '/lib64' and it contains path to conda folder with library that satisfies requirements. I'm a little bit worried to change library version in the system folder but I can't force EPA-NG to search it anywhere else. Probably you know how to specify library path to EPA-NG?
Thanks,
Semion
Hi,
I am trying to run the picrust2 (using picrsut2_pipeline.py command) and am running into an error. I have used this exact command before (with the same exact files before with no problem and with the same versions of any dependent software) and had no issues. Below is the error I am getting:
joblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/util.py", line 249, in make_output_dir
makedirs(dirpath)
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/os.py", line 241, in makedirs
mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/data/c.older/FIV_ging/picrust2_pipe_output_12042018_2/intermediate/pathways/minpath_running'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/externals/loky/process_executor.py", line 420, in _process_worker
r = call_item.fn(*call_item.args, **call_item.kwargs)
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/_parallel_backends.py", line 563, in call
return self.func(*args, **kwargs)
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/parallel.py", line 261, in call
for func, args, kwargs in self.items]
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/parallel.py", line 261, in <listcomp>
for func, args, kwargs in self.items]
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/run_minpath.py", line 744, in unstrat_minpath
print_opt, extra_str)
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/run_minpath.py", line 536, in minpath_wrapper
make_output_dir(path.join(out_dir, "minpath_running"))
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/util.py", line 253, in make_output_dir
raise IOError(err_str)
OSError: Could not create directory '%s'. Are permissions set correctly? Got error: '[Errno 17] File exists: '/data/c.older/FIV_ging/picrust2_pipe_output_12042018_2/intermediate/pathways/minpath_running''
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/data/apps/miniconda/envs/qiime2-2018.8/bin/picrust2_pipeline.py", line 7, in <module>
exec(compile(f.read(), file, 'exec'))
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/scripts/picrust2_pipeline.py", line 227, in <module>
main()
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/scripts/picrust2_pipeline.py", line 220, in main
verbose=args.verbose)
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/pipeline.py", line 227, in full_pipeline
print_cmds=verbose)
File "/data/apps/miniconda/envs/qiime2-2018.8/plugins/picrust2/picrust2/run_minpath.py", line 397, in run_minpath_pipeline
for sample_id in samples)
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/parallel.py", line 996, in call
self.retrieve()
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/parallel.py", line 899, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/joblib/_parallel_backends.py", line 517, in wrap_future_result
return future.result(timeout=timeout)
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/concurrent/futures/_base.py", line 398, in result
return self.__get_result()
File "/data/apps/miniconda/envs/qiime2-2018.8/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
raise self._exception
OSError: Could not create directory '%s'. Are permissions set correctly? Got error: '[Errno 17] File exists: '/data/c.older/FIV_ging/picrust2_pipe_output_12042018_2/intermediate/pathways/minpath_running''
Any idea what might be going on? Would appreciate any help!
Thanks,
Caitlin
Posted here from another issue from @itiago:
Hi Gavin
so, the hsp.py worked fine (thanks for that), now the problem is the next
step: Metagenome prediction.
I don't understand this step, once the biom file from mothur only provides
OTUs and relative abundance, and how it will relate to the later results
obtained from picrust2.
I did run the make.biom file from mothur and have a biom file (please
remember that I only have one sample, so I did this with a biomfile for
just one sample)
then I run the command and got this error result:
(picrust2-dev) igor@ubuntu:~/Desktop/Alfaguara/Picrust2$
metagenome_pipeline.py -i shared.0.03.biom -m
16S_predicted.tsv -f EC_predicted.tsv
-p 4 -o metagenome_prediction
Traceback (most recent call last):
File
"/home/igor/miniconda2/envs/picrust2-dev/bin/metagenome_pipeline.py", line
6, in
exec(compile(open(file).read(), file, 'exec'))
File "/home/igor/picrust2/scripts/metagenome_pipeline.py", line 76, in
main()
File "/home/igor/picrust2/scripts/metagenome_pipeline.py", line 64, in
main
output_normfile=True)
File "/home/igor/picrust2/picrust2/metagenome_pipeline.py", line 46, in
run_metagenome_pipeline
pred_marker)
File "/home/igor/picrust2/picrust2/util.py", line 246, in
three_df_index_overlap_sort
"input files.")
ValueError: No sequence ids overlap between all three of the input files.
Thanks for any help
Best
Igor
Hello,
I have picrust2.1.0-b, pandas 0.23.4, and python version 3.7.1 installed.
I'm trying to run place_seqs.py and the process fails with the following error when I use my otu.fasta file, but not the tutorial one. Though, the process has never actually finished on the tutorial fasta either. Below is the error I get. Help is much appreciated. Thanks :)
Error running this command:
hmmalign --trim --dna --mapali /home/rick/Desktop/picrust2-2.1.0-b/picrust2/default_files/prokaryotic/pro_ref/pro_ref.fna --informat FASTA -o placement_working/query_align.stockholm /home/rick/Desktop/picrust2-2.1.0-b/picrust2/default_files/prokaryotic/pro_ref/pro_ref.hmm /media/rick/DATADRIVE1/Rick/WSU Projects/Biosolid Studies_Nate/2017_7_3_062317NS515F_fasta-qual-mapping/fasta-qual-mapping-files/Individual_Samples_Files/openref/otus.fna
STDOUT of failed command:
ERROR: Incorrect number of command line arguments.
Usage: hmmalign [-options]
Code should be split into functions and placed under "picrust" folder to make more modular.
Hi Gavin!
So I'm on step 4 of your pipeline now. I typed this command and got this error:
$ run_minpath.py -i OUT_PREFIX.genefamilies.biom -m /path/to/MinPath/data/ec2path -o Pathway_abundances.tsv --keep_tmp
File "/home/ubuntu/MinPath/MinPath12hmp.py", line 74
print "now get SEED"
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("now get SEED")?
Error running this command:
MinPath12hmp.py -any minpath_tmp_jam0tj_8/W0-PIS10005_minpath_in.txt -map /path/to/MinPath/data/ec2path -report minpath_tmp_jam0tj_8/W0-PIS10005_minpath_report.txt -details minpath_tmp_jam0tj_8/W0-PIS10005_minpath_details.txt -mps minpath_tmp_jam0tj_8/W0-PIS10005_minpath.mps
I think MinPath uses python2 while the PICRUSt2 conda environment uses python3. How shall we fix this?
Also, you said to install MinPath12hmp.py in the install wiki page but mentioned MinPath1.4.py in step 4. I think you meant to use MinPath1.4.py because you need its support files, but I had to symlink it as MinPath12hmp.py because your script calls so.
Thanks!
Jamie
Would be more informative to know how accuracy decreases as all other tips in a same species, genus, family, etc, are left out (and average could be reported for all tips within that lineage).
Hi there,
Would it be a possible to add a verbose option, a progress indicator or something along these lines? Right now hsp.py
is completely silent and it does not write on the output location until it finishes, so it's hard to know if it's even running, esp. when I do it remotely on a server as I can't at least hear the computer.
Cheers
I have just cloned this repository and am trying to install the prerequisites, but the placement_tools/
directory that is supposed to contain the epa-ng and gappa source code is missing.
HI there,
We were following the instructions to install PICRUSt2 (installing it on a Mac with 16Gb of ram). First we got the Clon Repository. Then we started to install the pre-requisites and this was when problems started to happen.
While installing PaPaRa we had issues because we didn't know that we had to install boost first. After installing boost, the installation kept failing.
papara_nt-2.5 yakshiortiz$ sh build_papara2.sh
In file included from papara.cpp:30:
In file included from ./papara.h:47:
In file included from ./pvec.h:41:
ivy_mike/src/ivymike/tree_traversal_utils.h:30:10: fatal error:
'boost/tr1/unordered_set.hpp' file not found
#include <boost/tr1/unordered_set.hpp>
^
1 error generated.
We managed to understand that the problem was because boost latest version removed a folder named tr1 (see here: ros/rospack@6fcee1b). We changed the code as it was shown in the link provided. After that, the installation was successful.
While installing epa-ng, another error appeared. First we had to install cmake. Then an error with Tiny_Tree persisted.
[ 97%] Building CXX object src/CMakeFiles/epa_module.dir/tree/Tiny_Tree.cpp.o
In file included from /Users/yakshiortiz/picrust2/placement_tools/epa-ng/src/tree/Tiny_Tree.cpp:1:
In file included from /Users/yakshiortiz/picrust2/placement_tools/epa-ng/src/tree/Tiny_Tree.hpp:13:
/Users/yakshiortiz/picrust2/placement_tools/epa-ng/src/core/Lookup_Store.hpp:166:27: error:
implicit instantiation of undefined template 'std::_1::array<unsigned
long, 128>'
std::array<size_t, 128> char_to_posish;
^
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/__tuple:116:65: note:
template is declared here
template <class _Tp, size_t _Size> struct _LIBCPP_TYPE_VIS_ONLY array;
^
1 error generated.
make[3]: *** [src/CMakeFiles/epa_module.dir/tree/Tiny_Tree.cpp.o] Error 1
make[2]: *** [src/CMakeFiles/epa_module.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [run_make] Error 2
To solve this, we modified the Tiny_Tree files emacs src/tree/Tiny_Tree.cpp src/tree/Tiny_Tree.hpp
to add a line that said Include <Array>
.
Finally, while installing GAPPA, we noticed a similar problem.
[ 68%] Building CXX object CMakeFiles/gappa.dir/src/options/file_input.cpp.o
/Users/yakshiortiz/picrust2/placement_tools/gappa/src/options/file_input.cpp:38:10: fatal error:
'linux/limits.h' file not found
#include <linux/limits.h>
^
1 error generated.
make[3]: *** [CMakeFiles/gappa.dir/src/options/file_input.cpp.o] Error 1
make[2]: *** [CMakeFiles/gappa.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [build] Error 2
To solve this problem, we ran this code: vi +38 src/options/file_input.cpp
and there we converted
this line
#include <linux/limits.h>
into a comment. This appeared to solve the problems. But after "successfully" creating PICRUSt environment with conda,
Installing collected packages: PICRUSt2
Running setup.py develop for PICRUSt2
Successfully installed PICRUSt2
You are using pip version 9.0.3, however version 10.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
we ran the self test (pytest). This test says that 13 tests failed and 15 passed. I am leaving the error output below. Its very long, but it may help solve the problem. Please let me know if any of you have any insights into what might be happening.
Thanks,
Yakshi Ortiz
(picrust2-dev) ursula:picrust2 yakshiortiz$ pytest
============================= test session starts ==============================
platform darwin -- Python 3.6.5, pytest-3.6.2, py-1.5.4, pluggy-0.6.0
rootdir: /Users/yakshiortiz/picrust2, inifile:
plugins: cov-2.5.1
collected 28 items
tests/test_hsp.py FFFFFFF [ 25%]
tests/test_metagenome_pipeline.py ....... [ 50%]
tests/test_place_seqs.py .FFFF. [ 71%]
tests/test_run_minpath.py .. [ 78%]
tests/test_util.py .... [ 92%]
tests/test_workflow.py FF [100%]
=================================== FAILURES ===================================
________________ castor_hsp_workflow_tests.test_emp_prob_simple ________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_emp_prob_simple>
def test_emp_prob_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="emp_prob",
> ran_seed=10)
tests/test_hsp.py:73:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp34b217n5/subset_tab_0', 'emp_prob', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp34b217n5/subset_tab_0 emp_prob FALSE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp7jma9xbe/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp7jma9xbe/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
_____________________ castor_hsp_workflow_tests.test_mp_ci _____________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_mp_ci>
def test_mp_ci(self):
'''Test that MP confidence intervals calculated correctly.'''
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="mp",
ran_seed=10,
> calc_ci=True)
tests/test_hsp.py:126:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp1fk2yvpl/subset_tab_0', 'mp', 'TRUE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp1fk2yvpl/subset_tab_0 mp TRUE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpurw96obt/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpurw96obt/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
___________________ castor_hsp_workflow_tests.test_mp_simple ___________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_mp_simple>
def test_mp_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="mp",
> ran_seed=10)
tests/test_hsp.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpt2imkmr_/subset_tab_0', 'mp', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpt2imkmr_/subset_tab_0 mp FALSE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpvtbmcd5n/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpvtbmcd5n/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
_____________________ castor_hsp_workflow_tests.test_nsti ______________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_nsti>
def test_nsti(self):
'''Test that calculated NSTI values match expected.'''
in_traits1_df = pd.read_csv(in_traits1, sep="\t",
index_col="assembly",
dtype={'assembly' : str})
nsti_out = castor_nsti(tree_path=in_tree1,
> known_tips=in_traits1_df.index.values)
tests/test_hsp.py:142:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:169: in castor_nsti
nsti_tmp_out]))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_nsti.R', '/Users/yakshiortiz/picrust2/tests/test_dat..._xk300000gr/T/tmpdsgk6pm2/known_tips.txt', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpdsgk6pm2/nsti_out.txt']
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_nsti.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpdsgk6pm2/known_tips.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpdsgk6pm2/nsti_out.txt
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
__________________ castor_hsp_workflow_tests.test_pic_simple ___________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_pic_simple>
def test_pic_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="pic",
> ran_seed=10)
tests/test_hsp.py:86:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpp5w68hjg/subset_tab_0', 'pic', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpp5w68hjg/subset_tab_0 pic FALSE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpgy9_zgq2/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpgy9_zgq2/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
__________________ castor_hsp_workflow_tests.test_scp_simple ___________________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_scp_simple>
def test_scp_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="scp",
> ran_seed=10)
tests/test_hsp.py:99:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp1d6j9yi3/subset_tab_0', 'scp', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp1d6j9yi3/subset_tab_0 scp FALSE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpkup1ddgv/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpkup1ddgv/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
____________ castor_hsp_workflow_tests.test_subtree_average_simple _____________
self = <tests.test_hsp.castor_hsp_workflow_tests testMethod=test_subtree_average_simple>
def test_subtree_average_simple(self):
predict_out, ci_out = castor_hsp_workflow(tree_path=in_tree1,
trait_table_path=in_traits1,
hsp_method="subtree_average",
> ran_seed=10)
tests/test_hsp.py:112:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/wrap_hsp.py:65: in castor_hsp_workflow
for trait_in in file_subsets)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:949: in __call__
if self.dispatch_one_batch(iterator):
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:791: in dispatch_one_batch
self._dispatch(tasks)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:748: in _dispatch
job = self._backend.apply_async(batch, callback=cb)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:183: in apply_async
result = ImmediateResult(func)
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/_parallel_backends.py:543: in __init__
self.results = batch()
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in __call__
for func, args, kwargs in self.items]
../miniconda3/envs/picrust2-dev/lib/python3.6/site-packages/joblib/parallel.py:224: in <listcomp>
for func, args, kwargs in self.items]
picrust2/wrap_hsp.py:126: in castor_hsp_wrapper
system_call_check(hsp_cmd)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['Rscript', '/Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R', '/Users/yakshiortiz/picrust2/tests/test_data...ree.tre', '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpxirgx95y/subset_tab_0', 'subtree_average', 'FALSE', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E Rscript /Users/yakshiortiz/picrust2/picrust2/Rscripts/castor_hsp.R /Users/yakshiortiz/picrust2/tests/test_data/hsp/tree.tre /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpxirgx95y/subset_tab_0 subtree_average FALSE FALSE /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp_bxftajt/predicted_counts.txt /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmp_bxftajt/predicted_ci.txt 10
picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
*** caught segfault ***
address 0x18, cause 'memory not mapped'
Traceback:
1: dyn.load(file, DLLpath = DLLpath, ...)
2: library.dynam(lib, package, package.lib)
3: loadNamespace(package, lib.loc)
4: doTryCatch(return(expr), name, parentenv, handler)
5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
6: tryCatchList(expr, classes, parentenv, handlers)
7: tryCatch({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)}, error = function(e) { P <- if (!is.null(cc <- conditionCall(e))) paste(" in", deparse(cc)[1L]) else "" msg <- gettextf("package or namespace load failed for %s%s:\n %s", sQuote(package), P, conditionMessage(e)) if (logical.return) message(paste("Error:", msg), domain = NA) else stop(msg, call. = FALSE, domain = NA)})
8: library(pkg, character.only = TRUE, logical.return = TRUE, lib.loc = lib.loc, quietly = quietly)
9: .getRequiredPackages2(pkgInfo, quietly = quietly)
10: library(castor, quietly = TRUE)
An irrecoverable exception occurred. R is aborting now ...
_________________ place_seqs_tests.test_gappa_jplace_to_newick _________________
self = <tests.test_place_seqs.place_seqs_tests testMethod=test_gappa_jplace_to_newick>
def test_gappa_jplace_to_newick(self):
'''Basic test for gappa_jplace_to_newick function.'''
# Read in expected newick output.
exp_newick_in = open(exp_newick).read()
with TemporaryDirectory() as temp_dir:
newick_out = path.join(temp_dir, "out.tre")
> gappa_jplace_to_newick(jplace_file=exp_jplace, outfile=newick_out)
tests/test_place_seqs.py:120:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/place_seqs.py:137: in gappa_jplace_to_newick
print_out=print_cmds)
picrust2/util.py:157: in system_call_check
return_value = call(cmd, stdout=stdout, stderr=stderr)
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:267: in call
with Popen(*popenargs, **kwargs) as p:
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:709: in __init__
restore_signals, start_new_session)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <subprocess.Popen object at 0x1197df278>
args = ['gappa', 'analyze', 'graft', '--jplace-path', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/place_seqs_output/place_seqs_working/epa_out/epa_result.jplace', '--fully-resolve', ...]
executable = b'gappa', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False
p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1
errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""
if isinstance(args, (str, bytes)):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
orig_executable = executable
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = os.pipe()
# errpipe_write must not be in the standard io 0, 1, or 2 fd range.
low_fds_to_close = []
while errpipe_write < 3:
low_fds_to_close.append(errpipe_write)
errpipe_write = os.dup(errpipe_write)
for low_fd in low_fds_to_close:
os.close(low_fd)
try:
try:
# We must avoid complex work that could involve
# malloc or free in the child process to avoid
# potential deadlocks, thus we do all this here.
# and pass it to fork_exec()
if env is not None:
env_list = []
for k, v in env.items():
k = os.fsencode(k)
if b'=' in k:
raise ValueError("illegal environment variable name")
env_list.append(k + b'=' + os.fsencode(v))
else:
env_list = None # Use execv instead of execve.
executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env))
fds_to_keep = set(pass_fds)
fds_to_keep.add(errpipe_write)
self.pid = _posixsubprocess.fork_exec(
args, executable_list,
close_fds, tuple(sorted(map(int, fds_to_keep))),
cwd, env_list,
p2cread, p2cwrite, c2pread, c2pwrite,
errread, errwrite,
errpipe_read, errpipe_write,
restore_signals, start_new_session, preexec_fn)
self._child_created = True
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# self._devnull is not always defined.
devnull_fd = getattr(self, '_devnull', None)
if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
os.close(p2cread)
if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
os.close(c2pwrite)
if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
os.close(errwrite)
if devnull_fd is not None:
os.close(devnull_fd)
# Prevent a double close of these fds from __init__ on error.
self._closed_child_pipe_fds = True
# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 50000)
errpipe_data += part
if not part or len(errpipe_data) > 50000:
break
finally:
# be sure the FD is closed no matter what
os.close(errpipe_read)
if errpipe_data:
try:
pid, sts = os.waitpid(self.pid, 0)
if pid == self.pid:
self._handle_exitstatus(sts)
else:
self.returncode = sys.maxsize
except ChildProcessError:
pass
try:
exception_name, hex_errno, err_msg = (
errpipe_data.split(b':', 2))
# The encoding here should match the encoding
# written in by the subprocess implementations
# like _posixsubprocess
err_msg = err_msg.decode()
except ValueError:
exception_name = b'SubprocessError'
hex_errno = b'0'
err_msg = 'Bad exception data from child: {!r}'.format(
bytes(errpipe_data))
child_exception_type = getattr(
builtins, exception_name.decode('ascii'),
SubprocessError)
if issubclass(child_exception_type, OSError) and hex_errno:
errno_num = int(hex_errno, 16)
child_exec_never_called = (err_msg == "noexec")
if child_exec_never_called:
err_msg = ""
# The error must be from chdir(cwd).
err_filename = cwd
else:
err_filename = orig_executable
if errno_num != 0:
err_msg = os.strerror(errno_num)
if errno_num == errno.ENOENT:
err_msg += ': ' + repr(err_filename)
> raise child_exception_type(errno_num, err_msg, err_filename)
E FileNotFoundError: [Errno 2] No such file or directory: 'gappa': 'gappa'
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:1344: FileNotFoundError
_______________________ place_seqs_tests.test_run_epa_ng _______________________
self = <tests.test_place_seqs.place_seqs_tests testMethod=test_run_epa_ng>
def test_run_epa_ng(self):
'''Basic test to check whether EPA-NG wrapper can be run. Exact
matches to a treefile are not checked since slight differences
are expected depending on different versions.'''
with TemporaryDirectory() as temp_dir:
run_epa_ng(tree=test_tree,
ref_msa_fastafile=exp_ref_fasta,
study_msa_fastafile=exp_study_fasta,
> out_dir=temp_dir)
tests/test_place_seqs.py:135:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
picrust2/place_seqs.py:125: in run_epa_ng
str(threads) + " -w " + out_dir, print_out=print_cmds)
picrust2/util.py:157: in system_call_check
return_value = call(cmd, stdout=stdout, stderr=stderr)
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:267: in call
with Popen(*popenargs, **kwargs) as p:
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:709: in __init__
restore_signals, start_new_session)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <subprocess.Popen object at 0x1196ed7b8>
args = ['epa-ng', '--tree', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre', '--...iortiz/picrust2/tests/test_data/place_seqs/place_seqs_output/place_seqs_working/ref_seqs_papara.fasta', '--query', ...]
executable = b'epa-ng', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False
p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1
errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""
if isinstance(args, (str, bytes)):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
orig_executable = executable
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = os.pipe()
# errpipe_write must not be in the standard io 0, 1, or 2 fd range.
low_fds_to_close = []
while errpipe_write < 3:
low_fds_to_close.append(errpipe_write)
errpipe_write = os.dup(errpipe_write)
for low_fd in low_fds_to_close:
os.close(low_fd)
try:
try:
# We must avoid complex work that could involve
# malloc or free in the child process to avoid
# potential deadlocks, thus we do all this here.
# and pass it to fork_exec()
if env is not None:
env_list = []
for k, v in env.items():
k = os.fsencode(k)
if b'=' in k:
raise ValueError("illegal environment variable name")
env_list.append(k + b'=' + os.fsencode(v))
else:
env_list = None # Use execv instead of execve.
executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env))
fds_to_keep = set(pass_fds)
fds_to_keep.add(errpipe_write)
self.pid = _posixsubprocess.fork_exec(
args, executable_list,
close_fds, tuple(sorted(map(int, fds_to_keep))),
cwd, env_list,
p2cread, p2cwrite, c2pread, c2pwrite,
errread, errwrite,
errpipe_read, errpipe_write,
restore_signals, start_new_session, preexec_fn)
self._child_created = True
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# self._devnull is not always defined.
devnull_fd = getattr(self, '_devnull', None)
if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
os.close(p2cread)
if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
os.close(c2pwrite)
if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
os.close(errwrite)
if devnull_fd is not None:
os.close(devnull_fd)
# Prevent a double close of these fds from __init__ on error.
self._closed_child_pipe_fds = True
# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 50000)
errpipe_data += part
if not part or len(errpipe_data) > 50000:
break
finally:
# be sure the FD is closed no matter what
os.close(errpipe_read)
if errpipe_data:
try:
pid, sts = os.waitpid(self.pid, 0)
if pid == self.pid:
self._handle_exitstatus(sts)
else:
self.returncode = sys.maxsize
except ChildProcessError:
pass
try:
exception_name, hex_errno, err_msg = (
errpipe_data.split(b':', 2))
# The encoding here should match the encoding
# written in by the subprocess implementations
# like _posixsubprocess
err_msg = err_msg.decode()
except ValueError:
exception_name = b'SubprocessError'
hex_errno = b'0'
err_msg = 'Bad exception data from child: {!r}'.format(
bytes(errpipe_data))
child_exception_type = getattr(
builtins, exception_name.decode('ascii'),
SubprocessError)
if issubclass(child_exception_type, OSError) and hex_errno:
errno_num = int(hex_errno, 16)
child_exec_never_called = (err_msg == "noexec")
if child_exec_never_called:
err_msg = ""
# The error must be from chdir(cwd).
err_filename = cwd
else:
err_filename = orig_executable
if errno_num != 0:
err_msg = os.strerror(errno_num)
if errno_num == errno.ENOENT:
err_msg += ': ' + repr(err_filename)
> raise child_exception_type(errno_num, err_msg, err_filename)
E FileNotFoundError: [Errno 2] No such file or directory: 'epa-ng': 'epa-ng'
../miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:1344: FileNotFoundError
_______________________ place_seqs_tests.test_run_papara _______________________
self = <tests.test_place_seqs.place_seqs_tests testMethod=test_run_papara>
def test_run_papara(self):
'''Basic test for run_papara function.'''
exp_phylip = read_phylip(exp_papara_phylip)
in_msa = read_fasta(test_msa)
with TemporaryDirectory() as temp_dir:
obs_phylip = run_papara(tree=test_tree,
ref_msa=in_msa,
out_dir=temp_dir,
> study_fasta=test_study_seqs)
/Users/yakshiortiz/picrust2/tests/test_place_seqs.py:82:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/yakshiortiz/picrust2/picrust2/place_seqs.py:85: in run_papara
" -n out", print_out=print_cmds)
/Users/yakshiortiz/picrust2/picrust2/util.py:157: in system_call_check
return_value = call(cmd, stdout=stdout, stderr=stderr)
/Users/yakshiortiz/miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:267: in call
with Popen(*popenargs, **kwargs) as p:
/Users/yakshiortiz/miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:709: in __init__
restore_signals, start_new_session)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <subprocess.Popen object at 0x1196fa898>
args = ['papara', '-t', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre', '-s', 'ref_seqs.phylip', '-q', ...]
executable = b'papara', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False
p2cread = -1, p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1
errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""
if isinstance(args, (str, bytes)):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
orig_executable = executable
# For transferring possible exec failure from child to parent.
# Data format: "exception name:hex errno:description"
# Pickle is not used; it is complex and involves memory allocation.
errpipe_read, errpipe_write = os.pipe()
# errpipe_write must not be in the standard io 0, 1, or 2 fd range.
low_fds_to_close = []
while errpipe_write < 3:
low_fds_to_close.append(errpipe_write)
errpipe_write = os.dup(errpipe_write)
for low_fd in low_fds_to_close:
os.close(low_fd)
try:
try:
# We must avoid complex work that could involve
# malloc or free in the child process to avoid
# potential deadlocks, thus we do all this here.
# and pass it to fork_exec()
if env is not None:
env_list = []
for k, v in env.items():
k = os.fsencode(k)
if b'=' in k:
raise ValueError("illegal environment variable name")
env_list.append(k + b'=' + os.fsencode(v))
else:
env_list = None # Use execv instead of execve.
executable = os.fsencode(executable)
if os.path.dirname(executable):
executable_list = (executable,)
else:
# This matches the behavior of os._execvpe().
executable_list = tuple(
os.path.join(os.fsencode(dir), executable)
for dir in os.get_exec_path(env))
fds_to_keep = set(pass_fds)
fds_to_keep.add(errpipe_write)
self.pid = _posixsubprocess.fork_exec(
args, executable_list,
close_fds, tuple(sorted(map(int, fds_to_keep))),
cwd, env_list,
p2cread, p2cwrite, c2pread, c2pwrite,
errread, errwrite,
errpipe_read, errpipe_write,
restore_signals, start_new_session, preexec_fn)
self._child_created = True
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# self._devnull is not always defined.
devnull_fd = getattr(self, '_devnull', None)
if p2cread != -1 and p2cwrite != -1 and p2cread != devnull_fd:
os.close(p2cread)
if c2pwrite != -1 and c2pread != -1 and c2pwrite != devnull_fd:
os.close(c2pwrite)
if errwrite != -1 and errread != -1 and errwrite != devnull_fd:
os.close(errwrite)
if devnull_fd is not None:
os.close(devnull_fd)
# Prevent a double close of these fds from __init__ on error.
self._closed_child_pipe_fds = True
# Wait for exec to fail or succeed; possibly raising an
# exception (limited in size)
errpipe_data = bytearray()
while True:
part = os.read(errpipe_read, 50000)
errpipe_data += part
if not part or len(errpipe_data) > 50000:
break
finally:
# be sure the FD is closed no matter what
os.close(errpipe_read)
if errpipe_data:
try:
pid, sts = os.waitpid(self.pid, 0)
if pid == self.pid:
self._handle_exitstatus(sts)
else:
self.returncode = sys.maxsize
except ChildProcessError:
pass
try:
exception_name, hex_errno, err_msg = (
errpipe_data.split(b':', 2))
# The encoding here should match the encoding
# written in by the subprocess implementations
# like _posixsubprocess
err_msg = err_msg.decode()
except ValueError:
exception_name = b'SubprocessError'
hex_errno = b'0'
err_msg = 'Bad exception data from child: {!r}'.format(
bytes(errpipe_data))
child_exception_type = getattr(
builtins, exception_name.decode('ascii'),
SubprocessError)
if issubclass(child_exception_type, OSError) and hex_errno:
errno_num = int(hex_errno, 16)
child_exec_never_called = (err_msg == "noexec")
if child_exec_never_called:
err_msg = ""
# The error must be from chdir(cwd).
err_filename = cwd
else:
err_filename = orig_executable
if errno_num != 0:
err_msg = os.strerror(errno_num)
if errno_num == errno.ENOENT:
err_msg += ': ' + repr(err_filename)
> raise child_exception_type(errno_num, err_msg, err_filename)
E FileNotFoundError: [Errno 2] No such file or directory: 'papara': 'papara'
/Users/yakshiortiz/miniconda3/envs/picrust2-dev/lib/python3.6/subprocess.py:1344: FileNotFoundError
________________ place_seqs_tests.test_run_place_seqs_pipeline _________________
self = <tests.test_place_seqs.place_seqs_tests testMethod=test_run_place_seqs_pipeline>
def test_run_place_seqs_pipeline(self):
'''Basic test of full place seqs pipeline. As for EPA-NG, exact
matches to a treefile are not checked since slight differences
are expected depending on different versions.'''
with TemporaryDirectory() as temp_dir:
tmp_tree = path.join(temp_dir, "out.tre")
place_seqs_pipeline(study_fasta=test_study_seqs,
ref_msa=test_msa,
tree=test_tree,
out_tree=tmp_tree,
threads=1,
papara_output=None,
out_dir=temp_dir,
chunk_size=5000,
> print_cmds=False)
/Users/yakshiortiz/picrust2/tests/test_place_seqs.py:153:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/yakshiortiz/picrust2/picrust2/place_seqs.py:35: in place_seqs_pipeline
threads=threads, print_cmds=print_cmds)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tree = '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre'
ref_msa = {'2501846300_cluster': 'ATTGAACGCTGGCGGCAGGCCTAACACATGCAAGTCGAGCGGAGCGGCGGACGGGTGAGTAATGCCTAGGA-ATCTGCCTGGTAGTGGGGGATA...CGTTCCCGGGCC-TGTACACACCGCCCGTCACACCATGGGAGTTTGTTGCACCAGAAGTA--TAC-CACGGTGTGGCCGATGACTGGGGTGAAGTCGTAACAAGGTAGCCGT', ...}
study_fasta = '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta'
out_dir = '/var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpixi1dk16'
threads = 1, print_cmds = False
def run_papara(tree: str, ref_msa: dict, study_fasta: str, out_dir: str,
threads=1, print_cmds=False):
'''Run PaPaRa to place study sequences into reference multiple-sequence
alignment (MSA). Will return dictionary of the the output MSA (sequence ids
as keys). Expects path to tree and study FASTA as strings. Expects
reference MSA as a dictionary output by read_fasta. This MSA will be
converted to phylip format before running PaPaRa.'''
# Get absolute paths to input files.
tree = path.abspath(tree)
study_fasta = path.abspath(study_fasta)
# Change working directory to out directory (but keep track of original).
# This is necessary because PaPaRa outputs into the current working
# directory.
> orig_wd = getcwd()
E FileNotFoundError: [Errno 2] No such file or directory
/Users/yakshiortiz/picrust2/picrust2/place_seqs.py:76: FileNotFoundError
____________________ workflow_test.test_full_pipeline_biom _____________________
self = <tests.test_workflow.workflow_test testMethod=test_full_pipeline_biom>
def test_full_pipeline_biom(self):
'''Test that full pipeline can be run without error with
BIOM sequence abundance table.'''
with TemporaryDirectory() as temp_dir:
out_tree = path.join(temp_dir, "out.tre")
system_call_check("place_seqs.py -s " + test_study_seqs + " -r " +\
test_msa + " -t " + test_tree + " -o " +\
> out_tree)
/Users/yakshiortiz/picrust2/tests/test_workflow.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['place_seqs.py', '-s', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta', '-r', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.fna', '-t', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E place_seqs.py -s /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta -r /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.fna -t /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre -o /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpuy2ivted/out.tre
/Users/yakshiortiz/picrust2/picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Traceback (most recent call last):
File "/Users/yakshiortiz/miniconda3/envs/picrust2-dev/bin/place_seqs.py", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/Users/yakshiortiz/picrust2/scripts/place_seqs.py", line 97, in <module>
main()
File "/Users/yakshiortiz/picrust2/scripts/place_seqs.py", line 93, in main
print_cmds=args.print_cmds)
File "/Users/yakshiortiz/picrust2/picrust2/place_seqs.py", line 35, in place_seqs_pipeline
threads=threads, print_cmds=print_cmds)
File "/Users/yakshiortiz/picrust2/picrust2/place_seqs.py", line 76, in run_papara
orig_wd = getcwd()
FileNotFoundError: [Errno 2] No such file or directory
_____________________ workflow_test.test_full_pipeline_tsv _____________________
self = <tests.test_workflow.workflow_test testMethod=test_full_pipeline_tsv>
def test_full_pipeline_tsv(self):
'''Test that full pipeline can be run without error with
TSV sequence abundance table.'''
with TemporaryDirectory() as temp_dir:
out_tree = path.join(temp_dir, "out.tre")
system_call_check("place_seqs.py -s " + test_study_seqs + " -r " +\
test_msa + " -t " + test_tree + " -o " +\
> out_tree)
/Users/yakshiortiz/picrust2/tests/test_workflow.py:54:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cmd = ['place_seqs.py', '-s', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta', '-r', '/Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.fna', '-t', ...]
print_out = False, stdout = None, stderr = None
def system_call_check(cmd, print_out=False, stdout=None, stderr=None):
"""Run system command and throw and error if return is not 0. Input command
can be a list containing the command or a string."""
# Print command out if option set.
if print_out:
if type(cmd) is list:
print(" ".join(cmd))
else:
print(cmd)
# Convert command to list if input as string.
if type(cmd) is str:
cmd = cmd.split()
return_value = call(cmd, stdout=stdout, stderr=stderr)
# Exit with error if command did not finish successfully.
if return_value != 0:
> raise SystemExit("Error running this command:\n" + " ".join(cmd))
E SystemExit: Error running this command:
E place_seqs.py -s /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/study_seqs_test.fasta -r /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.fna -t /Users/yakshiortiz/picrust2/tests/test_data/place_seqs/img_centroid_16S_aligned_head30.tre -o /var/folders/_2/bs2w2q6n6n555b_bj3__xk300000gr/T/tmpzh3vt4vu/out.tre
/Users/yakshiortiz/picrust2/picrust2/util.py:161: SystemExit
----------------------------- Captured stderr call -----------------------------
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
Traceback (most recent call last):
File "/Users/yakshiortiz/miniconda3/envs/picrust2-dev/bin/place_seqs.py", line 6, in <module>
exec(compile(open(__file__).read(), __file__, 'exec'))
File "/Users/yakshiortiz/picrust2/scripts/place_seqs.py", line 97, in <module>
main()
File "/Users/yakshiortiz/picrust2/scripts/place_seqs.py", line 93, in main
print_cmds=args.print_cmds)
File "/Users/yakshiortiz/picrust2/picrust2/place_seqs.py", line 35, in place_seqs_pipeline
threads=threads, print_cmds=print_cmds)
File "/Users/yakshiortiz/picrust2/picrust2/place_seqs.py", line 76, in run_papara
orig_wd = getcwd()
FileNotFoundError: [Errno 2] No such file or directory
===================== 13 failed, 15 passed in 7.44 seconds =====================
I've previously used PICRUSt2 with a small set of sequences and never got any problems. Now I'm using it with ~5,000 sequences and got this error:
Error running this command:
epa-ng --tree /strg1/users/apcamargo/PICRUSt2/picrust2-2.0.3-b/default_files/prokaryotic/reference.tre --ref-msa picrust2_out_pipeline/intermediate/place_seqs/ref_seqs_hmmalign.fasta --query picrust2_out_pipeline/intermediate/place_seqs/epa_out/study_seqs_hmmalign.fasta.bfast --chunk-size 5000 -T 100 -m GTR+G -w picrust2_out_pipeline/intermediate/place_seqs/epa_out
I'm using EPA-ng version 0.3.4.
Hi Gavin,
I am trying to install PICRUSt2 on a cluster. The pre-requisites installed ok but when running pytest I get the following error:
____________________________________ ERROR collecting tests/test_metagenome_pipeline.py____________________________________
tests/test_metagenome_pipeline.py:10: in <module>
import biom
../.conda/envs/picrust2/lib/python3.6/site-packages/biom/__init__.py:51: in <module>
from .table import Table
../.conda/envs/picrust2/lib/python3.6/site-packages/biom/table.py:200: in <module>
from ._filter import _filter
__init__.pxd:861: in init biom._filter (biom/_filter.c:6729)
???
E ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I have tried changing the version of numpy but still having the same issue. Do you have any idea why this might be?
best,
Adam
I installed picrust2 using the conda method. All steps run fine except the pathway abundance step. I get the error that pathway_pipeline.py not found although pytests doesnt report any error
Hi,
I'm wondering if there is a way to get an output of pathway abundances that show each pathway's abundance per sample. Currently I have the pathway_abundance.qzv, but as far as I can tell, there is not functionally within viewing this file that allows me to see which specific pathways were predicted per sample (can only see a pathway is found in x samples or a sample has x pathways). I could not find anything on the wiki page or qiime2 site.
Thanks,
Caitlin
I am doing a q2 picrust2 Tutorial (https://github.com/picrust/picrust2/wiki/q2-picrust2-Tutorial)
A Plugin error occurs when I am doing fragment-insertion:
What I have done is following the tutorial line by line, how could I solve this?
I download those 3 file by:
mkdir q2-picrust2_test
cd q2-picrust2_test
wget http://kronos.pharmacology.dal.ca/public_files/tutorial_datasets/picrust2_tutorial_files/mammal_biom.qza
wget http://kronos.pharmacology.dal.ca/public_files/tutorial_datasets/picrust2_tutorial_files/mammal_seqs.qza
wget http://kronos.pharmacology.dal.ca/public_files/tutorial_datasets/picrust2_tutorial_files/mammal_metadata.tsv
Then I download another two file by:
wget http://kronos.pharmacology.dal.ca/public_files/tutorial_datasets/picrust2_tutorial_files/reference.fna.qza
wget http://kronos.pharmacology.dal.ca/public_files/tutorial_datasets/picrust2_tutorial_files/reference.tre.qza
All these five files were successfully downloaded into q2-picrust2_test folder, which is created at the beginning.
I run the q2-fragmentation insertion by:
qiime fragment-insertion sepp --i-representative-sequences mammal_seqs.qza
–p-threads 1 --i-reference-alignment reference.fna.qza
–i-reference-phylogeny reference.tre.qza
–output-dir tutorial_placed_out
Then the error occurs here:
Plugin error from fragment-insertion:
Command ‘[‘run-sepp.sh’, ‘/tmp/qiime2-archive-pmh381na/38a5d748-739e-44d1-b547-a35c0c8b8f29/data/dna-sequences.fasta’, ‘q2-fragment-insertion’, ‘-x’, ‘1’, ‘-A’, ‘1000’, ‘-P’, ‘5000’, ‘-a’, ‘/tmp/qiime2-archive-yyddx_7u/87909524-fa4f-4b28-abd6-c7c89a3c7da7/data/aligned-dna-sequences.fasta’, ‘-t’, ‘/tmp/qiime2-archive-3jm7iibt/f4e69728-a745-4069-b432-3f14951089fd/data/tree.nwk’]’ returned non-zero exit status 1
Debug info has been saved to /tmp/qiime2-q2cli-err-cby4nvz1.log
Good afternoon!
I received this error after doing a fresh install into a conda environment. I'm guessing it has to do with my current version of numpy, which might be too old for picrust2.
(picrust2) []$ conda list | grep numpy
numpy 1.15.4 py36h7e9f1db_0
numpy-base 1.15.4 py36hde5b4d6_0
No specific version of numpy is pinned in the picrust2-env.yaml
. Which version of numpy from which bioconda channel should I try to install?
Main error:
_______________________ ERROR collecting tests/test_metagenome_pipeline.py _______________________
tests/test_metagenome_pipeline.py:10: in <module>
import biom
../anaconda/envs/picrust2/lib/python3.6/site-packages/biom/__init__.py:51: in <module>
from .table import Table
../anaconda/envs/picrust2/lib/python3.6/site-packages/biom/table.py:200: in <module>
from ._filter import _filter
__init__.pxd:861: in init biom._filter (biom/_filter.c:6729)
???
E ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
======================================== warnings summary ========================================
EDIT: Looks like this is an issue with a mismatch between numpy and the biom-format package that imports numpy. Neither numpy or biom-format are pinned. Here are the versions I have installed using pip:
biom-format 2.1.7
numpy 1.15.4
Using conda, I also have these versions of numpy:
numpy 1.15.4 py36h7e9f1db_0 anaconda
numpy-base 1.15.4 py36hde5b4d6_0
What is the current tree into which sequences are placed when using the EPA-NG pipeline? I was trying to figure out which database, but could not find that info
I was wondering if the EPA-NG pipeline can be used to place sequences onto the SILVA tree
Hi,
Can I export feature-table from dada2 and analyzed in PICRUST plugin in QIIME and use in PICRUST to have more options or should I use close-reference reads? Currently, I can only see how many pathways and what pathways we have but there is no information about which samples or which species have how many of which pathways so the information is very limited in PICRUST plugin in QIIME.
Very excited to be testing this out!
Right now, I'm trying to perform something similar to genome_prediction.py, but I don't see a way to do this with the current qiime2 picrust workflow.
Will there be plans on including this file? This can be especially useful when comparing 16S hits on a phylogenetic tree for instance.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.