PySlice_Unpack error about oyster_river_protocol HOT 18 OPEN

Fixed it by editing py36_env.yml to specify a newer python version (>=3.6.1), adding git to dependencies, and getting busco directly from the gitlab repository (v3.1) instead of the out-of-date anaconda repository (v3.02). Don't know if this latter bit was necessary, but I saw in the commits that there were problems with busco when using python>3.6. Anyway, it didn't give me any problems on the included test data. Here is my new py36_env.yml:

name: orp_v2
channels:
  - bioconda
  - conda-forge
  - defaults
  - r
  - antoined
dependencies:
  - python>=3.6.1
  - git
  - zsh
  - ruby
  - make
  - glib
  - zlib==1.2.11=0
  - libgcc>=7.2.0
  - salmon>=0.10.2
  - seqtk
  - trinity
  - spades
  - blast
  - hmmer
  - samtools
  - blat
  - parallel
  - last
  - rcorrector==1.0.3
  - mcl
  - abyss>=2.0.1
  - python-igraph
  - diamond
  - cd-hit
  - bwa
  - pip:
    - biopython
    - cvxopt
    - cython
    - numpy
    - pandas
    - pip
    - scipy
    - bashplotlib
    - "git+https://gitlab.com/ezlab/busco"

from oyster_river_protocol.

So just to confirm, the test dataset now runs properly, and gives an output something like

 6| #
 5| ##
 4| ###
 3| ###
 2| ###       #
 1| ### ###   #
   -----------

------------------------------------
|             Summary              |
------------------------------------
|         observations: 20         |
|       min value: -1.000000       |
|         mean : -0.986450         |
|       max value: -0.935000       |
------------------------------------


*****  See the following link for interpretation *****
*****  https://oyster-river-protocol.readthedocs.io/en/latest/strandexamine.html *****



*****  QUALITY REPORT FOR: test4 using the ORP version 2.1.0 ****
*****  THE ASSEMBLY CAN BE FOUND HERE: /mnt/lustre/macmaneslab/macmanes/fastq_files/assemblies/test4.ORP.fasta ****

*****  BUSCO SCORE ~~~~~>           	C:0.0%[S:0.0%,D:0.0%],F:0.3%,M:99.7%,n:303
*****  TRANSRATE SCORE ~~~~~>           0.42269
*****  TRANSRATE OPTIMAL SCORE ~~~~~>   0.57644
*****  UNIQUE GENES ORP ~~~~~>          39
*****  UNIQUE GENES TRINITY ~~~~~>      31
*****  UNIQUE GENES SPADES55 ~~~~~>     22
*****  UNIQUE GENES SPADES75 ~~~~~>     25
*****  UNIQUE GENES TRANSABYSS ~~~~~>   36

from oyster_river_protocol.

Here is the output:

[bam_sort_core] merging from 0 files and 10 in-memory blocks...
-parsing file: test.sorted.bam
-done parsing file, examining orientations of reads.

 7|  #         
 6| ##         
 5| ##         
 4| ##         
 3| ###        
 2| ###       #
 1| ###  ##   #
   -----------

------------------------
|       Summary        |
------------------------
|   observations: 20   |
| min value: -1.000000 |
|   mean : -0.987400   |
| max value: -0.935000 |
------------------------


*****  See the following link for interpretation ***** 
*****  https://oyster-river-protocol.readthedocs.io/en/latest/strandexamine.html ***** 



*****  QUALITY REPORT FOR: test using the ORP version 2.1.0 ****
*****  THE ASSEMBLY CAN BE FOUND HERE: /home/tc/sampledata2/assemblies/test.ORP.fasta **** 

*****  BUSCO SCORE ~~~~~>           	C:0.0%[S:0.0%,D:0.0%],F:0.3%,M:99.7%,n:303
*****  TRANSRATE SCORE ~~~~~>           0.41937
*****  TRANSRATE OPTIMAL SCORE ~~~~~>   0.58487
*****  UNIQUE GENES ORP ~~~~~>          39
*****  UNIQUE GENES TRINITY ~~~~~>      31
*****  UNIQUE GENES SPADES55 ~~~~~>     22
*****  UNIQUE GENES SPADES75 ~~~~~>     23
*****  UNIQUE GENES TRANSABYSS ~~~~~>   35
 

DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.

from oyster_river_protocol.

ok good.. the other issues aside, have you been able to successfully run a real dataset through the pipeline?

from oyster_river_protocol.

Sample problem not solved following the instructions above.
This is the way py36_env.yml looks after editing with nano (note:

(orp_v2) [ant@hillary Oyster_River_Protocol]$ cat py36_env.yml
name: orp_v2
channels:

• bioconda
• conda-forge
• defaults
• r
• antoined
  dependencies:
• python==3.6.0 {note: I check the version in my download}
• git
• zsh
• ruby
• make
• glib
• zlib==1.2.11=0
• libgcc>=7.2.0
• salmon>=0.10.2
• seqtk
• trinity
• spades
• busco
• blast
• hmmer
• samtools
• blat
• parallel
• last
• rcorrector==1.0.3
• mcl
• abyss>=2.0.1
• python-igraph
• diamond
• cd-hit
• bwa
• pip:
  • biopython
  • cvxopt
  • cython
  • numpy
  • pandas
  • pip
  • scipy
  • bashplotlib
  • "git+https://gitlab.com/ezlab/busco"
    (orp_v2) [ant@hillary Oyster_River_Protocol]$

The error message I get is the same than before editing py36 but slightly different to the one reported above by cooketho:

Thank you for using SPAdes!
Traceback (most recent call last):
File "/home/ant/Oyster_River_Protocol/software/transabyss/transabyss", line 18, in
from utilities.adj_utils import has_edges
File "/home/ant/Oyster_River_Protocol/software/transabyss/utilities/adj_utils.py", line 6, in
import igraph
File "/home/ant/Oyster_River_Protocol/software/anaconda/install/envs/orp_v2/lib/python3.6/site-packages/igraph/init.py", line 34, in
from igraph._igraph import *
ImportError: /home/ant/Oyster_River_Protocol/software/anaconda/install/envs/orp_v2/lib/python3.6/site-packages/igraph/_igraph.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_Unpack
make: *** [/home/ant/Oyster_River_Protocol/sampledata/assemblies/test.transabyss.fasta] Error 1
(orp_v2) [ant@hillary sampledata]$

Any thoughts? Thanks beforehand

from oyster_river_protocol.

You have python==3.6.0. You probably need python>=3.6.1, which is what I used (see previous post).

from oyster_river_protocol.

Yes, I did try it too:
I edited again py36 so that it looks like:

(orp_v2) [ant@hillary Oyster_River_Protocol]$ cat py36_env.yml
name: orp_v2
channels:

• bioconda
• conda-forge
• defaults
• r
• antoined
  dependencies:
• python>=3.6.1
• git
• zsh
• ruby
• make
• glib
• zlib==1.2.11=0
• libgcc>=7.2.0
• salmon>=0.10.2
• seqtk
• trinity
• spades
• busco
• blast
• hmmer
• samtools
• blat
• parallel
• last
• rcorrector==1.0.3
• mcl
• abyss>=2.0.1
• python-igraph
• diamond
• cd-hit
• bwa
• pip:
  • biopython
  • cvxopt
  • cython
  • numpy
  • pandas
  • pip
  • scipy
  • bashplotlib
  • "git+https://gitlab.com/ezlab/busco"

But I get the same error:

Thank you for using SPAdes!
Traceback (most recent call last):
File "/home/ant/Oyster_River_Protocol/software/transabyss/transabyss", line 18, in
from utilities.adj_utils import has_edges
File "/home/ant/Oyster_River_Protocol/software/transabyss/utilities/adj_utils.py", line 6, in
import igraph
File "/home/ant/Oyster_River_Protocol/software/anaconda/install/envs/orp_v2/lib/python3.6/site-packages/igraph/init.py", line 34, in
from igraph._igraph import *
ImportError: /home/ant/Oyster_River_Protocol/software/anaconda/install/envs/orp_v2/lib/python3.6/site-packages/igraph/_igraph.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PySlice_Unpack
make: *** [/home/ant/Oyster_River_Protocol/sampledata/assemblies/test.transabyss.fasta] Error 1
(orp_v2) [ant@hillary sampledata]$

Thanks!

from oyster_river_protocol.

You still have - busco in the main dependencies list. That will install the outdated anaconda repository for busco. Try deleting it. That's why I used the gitlab busco repository installed via pip. You can just cut and paste the code from my previous post.

from oyster_river_protocol.

got it! I will try

from oyster_river_protocol.

Nope. Still the same error! Not sure what else to do. Thanks

from oyster_river_protocol.

Hmm. Sorry! I don't have any more advice, except you might want to try deleting the entire installation and starting from scratch by downloading from github again, and following the installation instructions. But before you enter make, copy-paste my post above into your py36_env.yml

from oyster_river_protocol.

ok, great, will do, thanks again

from oyster_river_protocol.

I've updated the repo with the Python changes. You should be able to git update -> make at this point.

Let me know if starting from scratch works - I'm committed to helping you get this up and running.

from oyster_river_protocol.

Thanks Matt. I will keep you posted. Regards!

from oyster_river_protocol.

I uninstalled everything, and all went smoothly. Now, before doing make, I modified the py36 file, but then I am getting this error message after make:

Collecting git+https://gitlab.com/ezlab/busco (from -r /home/ant/Oyster_River_Protocol/condaenv._g9hpm_0.requirements.txt (line 9))
Cloning https://gitlab.com/ezlab/busco to /tmp/pip-req-build-kzyqy7sh
error: RPC failed; result=22, HTTP code = 404

This problem was solved by adding '.git' at the end of the web address in the py36 file

from oyster_river_protocol.

The installation went smoothly: I modified the py36 file according to the post above before 'make' and all run well. The previous error was resolved. However, I have a new issue. When I test the protocol, this is the message:

(orp_v2) [ant@hillary Oyster_River_Protocol]$ cd sampledata/

(orp_v2) [ant@hillary sampledata]$ /home/ant/Oyster_River_Protocol/oyster.mk main \

STRAND=RF
MEM=15
CPU=8
READ1=test.1.fq.gz
READ2=test.2.fq.gz
RUNOUT=test
which: no run_BUSCO.py in which: no run_BUSCO.py in (/home/ant/Oyster_River_Protocol/software/anaconda/install/envs/orp_v2/bin:/home/ant/Oyster_River_Protocol/software/anaconda/install/condabin:/usr/local/rvm/gems/ruby-1.9.3-p551/bin:/usr/local/rvm/gems/ruby-1.9.3-p551@global/bin:/usr/local/rvm/rubies/ruby-1.9.3-p551/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/rvm/bin:/home/ant/bin:/home/ant/Oyster_River_Protocol/software/anaconda/install/bin:/home/ant/Oyster_River_Protocol/software/OrthoFinder/orthofinder:/home/ant/Oyster_River_Protocol/software/orp-transrate:/home/ant/Oyster_River_Protocol/software/transabyss)
/home/ant/Oyster_River_Protocol/oyster.mk:96: *** "\n\n*** BUSCO is not installed, must fix ***". Stop.
(orp_v2) [ant@hillary sampledata]$

So, I went to github and downloaded the script 'run_BUSCO.py' to anaconda/bin
When I run the test sample, the protocol is getting stuck in orthofuse with the following message:

parallel: Error: --tollef has been retired.
parallel: Error: Remove --tollef or use --gnu to override --tollef.
make: *** [/home/ant/Oyster_River_Protocol/sampledata/orthofuse/test/orthotransrate.done] Error 255
(orp_v2) [ant@hillary sampledata]$

trying to solve this issue now: problem solved by adding flag --gnu to each parallel command found in each *.mk file.

from oyster_river_protocol.

I wonder about what can be done if you are working on a linux centos machine with no administrator privileges. I got stuck because the version of coreutils in the machine I am using is old enough so that the command sort does not have --parallel. I installed a new version of coreutils with conda in the ORP_v2 environment [together with other mandatory updates requested by conda when installing coreutils]. Of course, this last action [mandatory script updates] broke the pipeline. I assume that is too much work to rework the pipeline so to have coreutils be integral part of it? Any comment will be appreciated. Thanks

from oyster_river_protocol.

I will address ASAP! #

from oyster_river_protocol.