Hi abcbot123 :)

Thanks for reaching out. At a first glance, the problem originates within i-ADHoRe, while the other warnings in the later commands arise from the fact that some orthologs Ks data are still not generated (probably interrupted by the error?).

The i-ADHoRe error message says:

ERROR i-ADHoRe execution failed with return code: -6
ERROR i-ADHoRe standard error output:
i-adhore: /i-adhore/src/Profile.cpp:95: void Profile::createNodes(const std::set&, 
const std::vector<GeneList*>&, std::vector<std::vector<Node*> >&, bool, bool) 
const: Assertion `geneX.isPairWith(geneY)' failed.

It's thus related to line 95 in file Profile.cpp.
I never encountered such error and right now I don't know what can be causing it.

It seems to me that i-ADHoRe is not finding the data it expects to find. Could you double check if your GFF file follows the standard format and could you make sure you're parsing it with the correct gff_feature and gff_attribute?
If you have successfully run other species in the past, can you think about something different in the input files?

Cheers,
Cecilia

from ksrates.

Here's an update :)
I noticed that your error message is the same reported in a recent issue on i-ADHoRe itself.
When building the ksrates docker container, we are actually basing it on the vib/i-adhore GitHub repo (see first line in our dockerfile). This makes me think that your issue is inherited from i-ADHoRe. I'll get in touch with its maintainers to look into that!

If the above is correct, then you should still be able to run ksrates using the whole-paranome pipeline, deactivating the colinearity pipeline (see configuration).

Best,
Cecilia

from ksrates.

Hi :) It would also probably still be helpful if you could provide us with your input FASTA and GFF files for Oxalis, as well as with the parsing gff_features and gff_attribute you have chosen. Thanks!

from ksrates.

Hi Cecilia,
Thanks so much for your response. I have recheck my gff3 file and there was an issue with the format. I have fixed it and got the output files. But I'm struggling with how to explain my results. Could you please help me to figure out how to explain my output files? If so, I can send those .pdf files I got.
Thanks so much!

from ksrates.

Hi, I'm happy it was just a format issue! Feel free to share what it was if you think other people might encounter the same problem.
Sure, we can have a look together at your output files. What about using gitter to communicate? You can sign in with your GitHub account, then we can share files and text.
Cheers,
Cecilia

from ksrates.

Hi Cecilia,
Yes, I can do that. I have login to gitter using my GitHub account but I couldn't find you to sent a message.

from ksrates.

Hi, I'm having a similar problem, here's the code to run and the errors reported, can you help me see how to fix it?

ksrates plot-paralogs config_Bchinensis.txt

INFO Generating mixed paralog and ortholog distributions
INFO Sun Jul 21 15:14:52 2024
INFO Loading parameters and input files
ERROR Anchor pair Ks TSV file not found at default position [paralog_distributions/wgd_Bchinensis/Bchinensis.ks_anchors.tsv].
ERROR Exiting

There is also an error in the above step

ksrates orthologs-analysis config_Bchinensis.txt

INFO Computing ortholog distribution peaks with related error
INFO Sun Jul 21 15:31:43 2024

INFO Loading parameters and input files
INFO
INFO Belamcanda chinensis and Crocus sativus: data already present in ortholog database
INFO
INFO Asparagus officinal and Belamcanda chinensis: data already present in ortholog database
INFO
INFO Asparagus officinal and Crocus sativus: data already present in ortholog database
INFO
INFO All done

from ksrates.

Hi wjq1981,

Thanks for reaching out!

The error originating from command plot-paralogs says that, while trying to plot the anchor pair Ks distribution, it doesn't find the file listing all the anchor pair Ks values; it also says that this file is normally located at paralog_distributions/wgd_Bchinensis/Bchinensis.ks_anchors.tsv.
Have you already generated such file? To do so, you have to run command paralogs-ks also for the anchor pairs by setting collinearity = yes in configuration file config_Bchinensis.txt.
If you did already generate it, have you perhaps moved it or renamed it? In that case, it doesn't find it.
Or, perhaps you're lunching the command line from a wrong location, from which it can't find the file according to the default path.

The output from command orthologs-analysis doesn't have any error message :) The INFO lines say that the ortholog Ks distributions for all species pairs have been already generated and are already available in the ortholog database, so they are not calculated again.

Let me know how it goes, and if this doesn't help we can investigate further!
Cheers,
Cecilia

from ksrates.

Thank you for your prompt reply.
'collinearity = yes' is so in the “config_Bchinensis.txt” file.
At the same time I didn't find the file “Bchinensis.ks_anchors.tsv” in any of the generated directories.
And to rule out the possibility that it's just me, I ran “config_elaeis.txt” from test and got the same error!

ksrates plot-paralogs config_elaeis.txt
INFO - - - - - - - - - - - - - - - - - - - - - - - - - -
INFO Generating mixed paralog and ortholog distributions
INFO Mon Jul 22 16:21:45 2024
INFO - - - - - - - - - - - - - - - - - - - - - - - - - -
INFO Loading parameters and input files
ERROR Anchor pair Ks TSV file not found at default position [paralog_distributions/wgd_elaeis/elaeis.ks_anchors.tsv].
ERROR Exiting

from ksrates.

This is a list of all the files in the “paralog_distributions” directory.

paralog_distributions/wgd_elaeis
├── elaeis.blast.tsv
├── elaeis.ks_anchors_tmp
│   ├── families.tsv
│   ├── i-adhore.conf
│   ├── i-adhore_gene_lists
│   │   ├── EG5.1_Chr10.lst
│   │   ├── EG5.1_Chr11.lst
│   │   ├── EG5.1_Chr12.lst
│   │   ├── EG5.1_Chr13.lst
│   │   ├── EG5.1_Chr14.lst
│   │   ├── EG5.1_Chr15.lst
│   │   ├── EG5.1_Chr16.lst
│   │   ├── EG5.1_Chr1.lst
│   │   ├── EG5.1_Chr2.lst
│   │   ├── EG5.1_Chr3.lst
│   │   ├── EG5.1_Chr4.lst
│   │   ├── EG5.1_Chr5.lst
│   │   ├── EG5.1_Chr6.lst
│   │   ├── EG5.1_Chr7.lst
│   │   ├── EG5.1_Chr8.lst
│   │   ├── EG5.1_Chr9.lst
│   │   ├── p5_sc00010.lst
│   │   ├── ..............
│   │   └── p5_sc40286.lst
│   └── i-adhore_out
│   ├── AlignmentMultiplicon1.svg
│   ├── AlignmentMultiplicon2.svg
│   ├── AlignmentMultiplicon3.svg
│   ├── AlignmentMultiplicon4.svg
│   ├── AlignmentMultiplicon5.svg
│   ├── alignment.txt
│   ├── anchorpoints.txt
│   ├── baseclusters.txt
│   ├── genes.txt
│   ├── list_elements.txt
│   ├── multiplicon_pairs.txt
│   ├── multiplicons.txt
│   └── segments.txt
├── elaeis.ks.tsv
└── elaeis.mcl.tsv

Wouldn't 'elaeis.ks.tsv' be 'elaeis.ks_anchors.tsv' here either? Because it's the first time I've run it and I don't know what their contents should be!

from ksrates.

Alright! In the docs you can find the description of the output files: for example here it explains that elaeis.ks.tsv and elaeis.ks_anchors.tsv are similar in content, but the first is the list of Ks values for all paralogs (paranome), while the second is restricted to only anchor pairs.

The directory structure can tell us something: normally elaeis.ks_anchors_tmp is a temporary directory that is deleted once the final elaeis.ks_anchors.tsv is obtained. Therefore, I think that command paralogs-ks encountered an error during its execution while computing anchor pairs; could you check its output log?

from ksrates.

This is the output log.

ksrates paralogs-ks config_elaeis.txt --n-threads=64

INFO - - - - - - - - - - - - - - - - - - - - -
INFO Paralog wgd analysis for species [elaeis]
INFO Mon Jul 22 16:57:41 2024
INFO - - - - - - - - - - - - - - - - - - - - -
INFO Checking if sequence data files exist and if sequence IDs are compatible with wgd pipeline...
INFO Completed
INFO Creating directory [paralog_distributions/]
INFO Running wgd paralog Ks pipeline...
INFO ---
INFO Checking external software...
INFO makeblastdb: 2.5.0+
INFO blastp: 2.5.0+
INFO mcl 14-137
INFO MUSCLE v3.8.1551 by Robert C. Edgar
INFO AAML in paml version 4.10.7, June 2023
INFO Usage for FastTree version 2.1.8 Double precision (No SSE3):
INFO Creating output directory /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis
INFO Translating CDS file elaeis.fasta...
INFO ---
INFO Running all versus all Blastp
INFO Writing protein Blastdb sequences to /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/...
INFO Writing protein query sequences to /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/...
INFO Performing all versus all Blastp (this might take a while)...
INFO Making Blastdb
INFO makeblastdb -in /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/elaeis.db.fasta -dbtype prot
INFO makeblastdb output:
Building a new DB, current time: 07/22/2024 16:57:42
New DB name: /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/elaeis.db.fasta
New DB title: /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/elaeis.db.fasta
Sequence type: Protein
Keep MBits: T
Maximum file size: 1000000000B
Adding sequences from FASTA; added 500 sequences in 0.108267 seconds.
INFO Running Blastp
INFO blastp -db /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/elaeis.db.fasta -query /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast_tmp/elaeis.query.fasta -evalue 1e-10 -outfmt 6 -num_threads 64 -out /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.blast.tsv
INFO All versus all Blastp done
INFO Removing tmp directory
INFO ---
INFO Running gene family construction (MCL clustering with inflation factor = 2.0)
INFO Started MCL clustering (mcl)
INFO ---
INFO Running whole paranome Ks analysis...
INFO Started analysis of 66 gene families in parallel using 64 threads
INFO Performing analysis on gene family GF_000001 (size 13)
INFO Performing analysis on gene family GF_XXXXXX (size 2)
INFO Performing analysis on gene family GF_000066 (size 2)
INFO Finished 2/66 gene family analyses...
INFO Finished 47/66 gene family analyses...
INFO Finished 63/66 gene family analyses...
INFO Finished 64/66 gene family analyses...
INFO Finished 65/66 gene family analyses...
INFO Finished 66/66 gene family analyses...
INFO Finished all gene family analyses
INFO Analysis done
INFO Making results data frame
INFO Computing weights, outlier cut-off at Ks > 20
INFO Removing tmp directory
INFO ---
INFO Done
INFO ---
INFO Running wgd colinearity Ks pipeline...
INFO No colinearity anchor pair Ks data, will run wgd colinearity Ks analysis
INFO Checking external software...
INFO This is i-ADHoRe v3.0.
INFO Creating i-ADHoRe tmp directory /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.ks_anchors_tmp
INFO Parsing GFF file
INFO Writing gene lists for i-ADHoRe
INFO Writing families file for i-ADHoRe
INFO Writing i-ADHoRe configuration file
INFO Running i-ADHoRe 3.0...
INFO i-adhore /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.ks_anchors_tmp/i-adhore.conf
INFO i-ADHoRe output:
This is i-ADHoRe v3.0.
Copyright (c) 2002-2010, Flanders Interuniversity Institute for Biotechnology, VIB.
Algorithm designed by Klaas Vandepoele, Cedric Simillion, Jan Fostier, Dieter De Witte,
Koen Janssens, Sebastian Proost, Yvan Saeys and Yves Van de Peer.

Process 1/1 is alive on suma-Nd.

************* i-ADHoRe parameters *************
Number of genelists = 2643
Blast table = /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.ks_anchors_tmp/families.tsv
Output path = /dev/hdd/soft/ksrates/test/paralog_distributions/wgd_elaeis/elaeis.ks_anchors_tmp/i-adhore_out/
Gap size = 30
Cluster gap size = 35
Cloud gap size = 0
Cloud cluster gap size = 0
Max gaps in alignment = 35
Tandem gap = 15
Flush output = 1000
Q-value = 0.75
Anchorpoints = 3
Probability cutoff = 0.01
Cloud filtering method = Binomial
Level 2 only = false
Use family = true
Write statistics = false
Alignment method = GreedyGraphbased4
Multiple hypothesis correction = FDR
Number of threads = 64
Compare aligners = false
Collinear searches only
Visualize GHM.png = false
Visualize Alignment = true
Verbose output = true
************ END i-AdDHoRe parameters *********

Creating dataset... done. (time: 0.132908s)
Mapping gene families... done. (time: 0.0924711s)
Remapping tandem duplicates... done. (time: 0.043623s)
Writing genelists file... done. (time: 0.480689s)
Collinear Search
Level 2 multiplicon detection... done. (time: 1.85592s)
Profile detection...
5 multiplicons to evaluate - evaluating level 2 multiplicon... 0 new multiplicons found.
4 multiplicons to evaluate - evaluating level 2 multiplicon... 0 new multiplicons found.
3 multiplicons to evaluate - evaluating level 2 multiplicon... level-2 multiplicon is redundant
2 multiplicons to evaluate - evaluating level 2 multiplicon... level-2 multiplicon is redundant
1 multiplicons to evaluate - evaluating level 2 multiplicon... level-2 multiplicon is redundant
Flushing output files...Visualize AlignedProfiles
done.
Time for Higher Level Detection: 0.075561s.

All Done! Bye...
ERROR i-ADHoRe execution failed with standard error output:
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
ERROR Exiting.

from ksrates.

Based on the output log, i-ADHoRe was doing alright until the very end: it printed "All Done! Bye..." and then something generated an error with message "Authorization required, but no authorization protocol specified".
I've never encountered this error and I don't know what it refers to. After a quick search online and a question to ChatGPT, I see that it is related to "Unix-like system using the X Window System (X11)", so to your environment where you are running ksrates/i-ADHoRe. I suggest you search this error and see whether some other users already had similar problems, or that you ask ChatGPT and/or your IT group for help. I'll have a look myself as well in the meantime, because I don't fully grasp the topic...

For example, here the problem was that the user was running commands as "root" and not as "user": https://www.reddit.com/r/linux4noobs/comments/lu1plx/hi_i_get_this_authorization_required_but_no/ and https://stackoverflow.com/questions/48833451/no-protocol-specified-when-running-a-sudo-su-app-on-ubuntu-linux

from ksrates.

Thank you very much for your answer.
Another problem I have is that I'm getting errors running with docker, I uninstalled and reinstalled it and still get the same error, can you see if it's a software problem or a docker problem? I'm using other docker software and it's working.

docker run --rm -v $PWD:/temp -w /temp vibpsb/ksrates -h
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "-h": executable file not found in $PATH: unknown.

from ksrates.

Let's see, here in the docs I showed how to use the docker commands, and it looks like you chose the one to "launch a single command through the container":

In order to run such command, for example -h, you need to typeksrates -h (and not only -h) at the end of the docker command line (compare with the command line in your previous message). Can you try again with the following edit?
docker run --rm -v $PWD:/temp -w /temp vibpsb/ksrates ksrates -h
Hope this helps!

from ksrates.

Yeah, you're right. Now that it's working with “ksrates”, I'm crying at my own stupidity. Good day！

from ksrates.

No worries, great to hear this worked! Should you find an answer for the authorization-related error, let me know :)

from ksrates.