The interactive profile module currently fails to detect the absence of a dependent binary mmseqs on the environmental path.

The program won't run as expected when the module runs interactively on the system where mmseqs is not/improperly installed.

In such cases, please refrain using the interactive mode before this issue being handled.

Hello,

I created a phylogenetic tree from 68 genome assemblies using ufcg profile & ufcg tree and the pipeline worked perfectly! Thank you! However, I wanted to add few markers that I know would help with resolving the phylogeny of the group I am working on.

To be exact, I wanted to use ufcg to create hmm profiles of 6 genes. For each marker I have around 250 fasta sequences. I tried to run different combinations of the following command
ufcg train -i LOCI4TRAINING -g ../assemblies_renamed/ -o LOCI_HMM -t 8 -v
and every genome from "assemblies renamed" is missing every marker and the .hmm files are empty.

The markers are extracted from the fungal genomes from the same family I want to resolve. I also tried to use one of the "core genes" that was extracted by ufcg profile from the assemblies and it also says that it's missing from all of the assemblies when I use it as the argument for ufcg train -i.

Could you point me to where I am making a mistake? Even if the markers are not present in any of the genome (which I doubt, especially with the core gene test), why are the .hmm profiles empty?

All best,
Alis

Hello @endixk ,

Thank you for developing this amazing pipeline! Our team are currently working on the clinical outbreak investigation on one of the most challenging multidrug-resistant fungi - Candida auris.

We would like to build a specific marker gene set for Candida auris by using the train module base on all the Candida auris genomes available on NCBI with complete/chromosome assembly level.

We have already downloaded a total of 45 genomes and created a directory containing all these reference genomes. However, when we tried to use the train module, we found that there was one more required option: -i STR Directory containing marker sequences in FASTA format (should be able to build an MSA).

May we know what data should we provide for this option?

Thank you very much!

Best regards,
Eddie

Hi, this is less of a bug and more of an unexpected behaviour! Posting here so others who might have had this issue know a fix.

I'm using ufcg with the v1.0.5 container in Nextflow. Nextflow executes each software process in a separate work directory and passes input files into this directory via symbolic links, rather than copying them over. For my purposes, I was supplying the metadata.tsv file via an input channel, which meant it was a symbolic link, and ufcg wasn't liking it:

      __  __ _____ _____ _____
     / / / // ___// ___// ___/
    / / / // /_  / /   / / __
   / /_/ // __/ / /___/ /_/ /
   \____//_/    \____/\____/ v1.0.5
  
  
  [JUN 24 00:54:18] UFCG  |:  Verbose option check.
  [JUN 24 00:54:18] UFCG  |:  Timestamp printing option check.
  [JUN 24 00:54:18] UFCG  |:  Input file check : GCA_013839505.1_ASM1383950v1_genomic.fna
  [JUN 24 00:54:18] UFCG  |:  Input argument : symbolic link to /group/pathogens/IAWS/Personal/JackS/dev/fungal-phylo/work/a4/656055c898c37762af308a88ba3c0e/GCA_013839505.1_ASM1383950v1_genomic.fna
  [JUN 24 00:54:18] UFCG  |:  Output directory check : .
  [JUN 24 00:54:18] UFCG  |:  Temporary directory check : /tmp/GCA_013839505.1
  [JUN 24 00:54:18] UFCG  |:  Custom CPU thread count check : 1
  [JUN 24 00:54:18] UFCG  |:  Metadata file check : repository_metadata.tsv
  [JUN 24 00:54:18] UFCG  |:  ERROR! Invalid file given : repository_metadata.tsv
  [JUN 24 00:54:18] UFCG  |:  Run with "profile -h" option to see the user manual.

The fix is to use the readlink command in bash to convert the symbolic path (here repository_metadata.tsv) to an absolute path:

META_PATH=$(readlink repository_metadata.tsv -fn)

...and then use this as the metadata file path in ufcg:

ufcg profile -i ./input -o ./output -m $META_PATH

Because the fix is so easy in bash, not sure if it's worth trying to fix this within ufcg, but might be worth adding a note to the documentation? Although it is strange to me that ufcg recognises and allows input files to be symbolic links (see output above).

• Environment: UFCG 1.0.3, conda installed version, ubuntu 20.04
• Steps that I've done:

0. Prepare input file
   /tmp
   GCA_024703775.1.fna
   .... (many other genome fna files)

1. Call ucg file with "profile"
   ufcg profile -i ./tmp 0o ./101_ufcg_profile -t 32 -v --developer --force 1

2. Align with ufcg
   ufcg align -i ./101_ufcg_profile -o ./201_ufcg_align_nuc -l acc -a nucleotide -n align -l strain -t 24

• Problems
  The resulting alignment includes nucleotide "X", which makes many other programs (such as MAFFT without sequence designation, or geneious) to recognize alignment file as protein alignment. It would be better alignment to change "X" to "N" if "-a nucleotide" flag included

Hello, thanks for your tool, this is quite handy!

I am analysing metagenomic datasets and while I do not have bins which can be considered to be MAGs, I would still like to assess broad fungi presence from detection of ufcg directly from the assembly.
I would like to apply your tool directly to an assembly and assess broad Fungi coverage.

First question, can ufcg be applied to other things than isolated genomes? If I do gene calling with augustus, it requires a unique organism.
Second question, which of your core/canonical genes can be considered single copy? From your publication, the core seems to be single copy?

Greetings,

I getting said error when running the tree command, it also seems the process does not complete because of this. I'm not able to determine what is causing the error. I checked the COX2_pro.zZ.fasta files:

zZ7200270472082568702zZ
MYFQDSATPNQEEDGQLRLLDTDTSIVAPVDTHIRFIVSAADVIHDFAIPSLGIKIDACPGRLNQVSALIEREGVFYGQCSELCGVAHSAMPIKLEVVSLPEFLE

zZ5232740519337231783zZ
MGRESLVSPRRSRAASARRLLPGLSRRVLTSLLLFSRRRSYGDLSGVSPEICNNGLGCGSPLDTSVAPEGMLGVSRPPALVDSPTSSDDPPSVLPAQNISATHFYVGSNVVRNYGIFLQARNIPGQHFAVHTWSHQYTTSLTNEQVVAEIGWTMQILADNNFGLIPAHWRPRELVSSRFFTSHDSDSSPLPSPRPAYGDVDNRVRAIAREVFGLKTITWNPDEYEARVRGSKSPGLIPLEHVTSVEAVDGELIYVLTPPKYLLLTLYPFVNQSSSRPTLSSSPKAGTSPTSFVASPLAFLPFVLLFLADAKFFLSVFSFSLISGQKPGTRTPTPATSRRVSLQESPSSRLASPMAPLLPWLQQEHPLLDTQNGGAFCCGPEFSWELWERGEGGVGGVGCSWEHSRFGHWFRLGRNELDQIVLIFDTLPRHFDSKTPSELSRPLFLLQPPTLSFSSEDAALSIKQPLQADETDLLDNLVSRLPPLPSPPPSMSISLLPQEVLEPILHLAIQPSTTEGASILLVCTLWHNLGREKLYEHVTLSSQAAYDSYFLLGGSKASWRPLAQAQRTLDYQNLRSLHLRFGPLTKLPFSLSSSNPSPPYLPRFRNLKLIHLDLAKGSSYLSCRPKPMARRVAKLMGGFSPETMILARSSSAEISLSAVMPHHLRRTKYLLLASHHVSHLPSLTPQSCPAMIKNVGFQLLTGVLLPPSALAPLSETTKPSSTRSSPSSSAVGGLQPAVLRDSTFAVCHFTASSVCRMAAVVVSSRQDTSLASRFRSARPNVLFSPPRGLKPSSSQLHPHFPPACSPPVSHLSPRPSPPGYVSPPPSFKDTGSKSCRESRKAKPISSHLSHPFPFLFPSSLPQAFSSTPRSAPSAAIVGNSMLAASGAEGRADLSRWIETQPGSLPTTSTTTSTRRTLPSRSSLPPPRRMYLFPSISGRKGRGADLWSELTSSSSFFFLLLPITLRSSTSSNHPSSTPHSPLHPWILHHRIYHQRHHHQLHSLPLFKTPSTNAFSNLGSLTTSLLAHADEVGVSFDSYMVPDNEIADGQPRLLDVDARVVLPIETHTRFILSSTDVIHDWAVPSLGIKMDAMPGRLNQTSTLIERKGLFFGQCSELCGVYHGFMPIVVEAVELPEYLAWLLAQE

zZ7320208565470240394zZ
MYFQDSATPNQEEDGQLRLLDTDTSIVAPVDTHIRFIVSAADVIHDFAIPSLGIKIDACPGRLNQVSALIEREGVFYGQCSELCGVAHSAMPIKLEVVSLPEFLE

The only weird thing that I am able to discern is that the sequence is significantly larger than the others, also with less identity. What could be causing this error?

I'd tried to install ufcg in conda environment following the install guideline.
but I cannot able to install ufcg by conda install -y -c bioconda ufcg.

I've got the messages Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
and Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.,
and then Collecting package metadata (repodata.json): goes to infinite.

so I've tried to build it from source, although the installation succeeds, but when run it, I've repeatedly got an error UFCG |: ERROR! Following dependency binary remains unsolved : fastBlockSearch.

How can I bypass all these errors?
Any advice would be many appreciated.

Hi, I was just wondering if there is a way to add an extra gene(s)on top of 61 marker genes, in the core gene list before we run the analysis? Thanks, Sandeep

Thank you for this interesting pipeline, but now I have a problem. When I run the "ufcg download -t minimal", it shows the error message "ERROR! No internet connection", could you help me figure out the reason? I can downoload things from other website, so the internet should be connected. Thank you.

If existing output directory is given, current version (v1.0.3) of the tree and align modules check the list of files that is already present in the directory to reduce running time by skipping some stages (which can be referred to as a checkpoint feature).

However it does not work properly if there happens to be an empty file with a proper name, or the file list is incomplete e.g. due to the interrupted process.

To produce a liable result, please only provide an empty/nonexistent output directory to these modules for now, until this issue being handled in the newer version.

Hi,

I finally managed to get ufcg running with --set NUC, only to realise that it then only uses ITS.

Why isn't it using all the core genes in NUC mode as it does in PRO mode?
Can that be added?

I am working with lots of isolates of the same species and was hoping for better resolution using the nucleotide sequences instead of the protein sequences.

Cheers.

Hello,
Please, I have done the "ufcg download -t full" and when I run the profile analysis with --set NUC , the tool is not finding any ITS gene for any of the genomes.
The other --set options are working.

command:

ufcg profile --input ./genomes/ --output results_phylo_2_nuc --set NUC --thread 28 -k -w tmp_results_phylo_2_nuc

How can I solve this?
Thank you in advance

Based on the tutorial, I tried to command with my sample and got the above error, This is my command:

ufcg profile --input WI0722303A01_sample/seq --output WI0722303A01_sample/out --set PRO --thread 10 --metadata QI0722303A01_sample/metadata_full_QI0722303A01_added.tsv

I have 7 samples, the first 6 samples' runnings are done successfully, but cause an error on the 7th, I suspect the error was caused by missing the input metadata field of the 7th sample. I only filled the filename, taxon_name, strain_name, and taxonomy filed of the 7th sample and missing the accession, and ncbi_name fields (because they are unknown information). I really need some help. Thanks.

Hello,

I am using the latest version of the pipeline (v.1.0.3) and I am testing a large group of genomes from ncbi (very variable in quality) using the profile module both with PROT and BUSCO.
For some genomes I got a java exception as follow:

In this case, the genome GCA_017580835.1 had no errors with PRO profiling but the BUSCO profiling did not work.
A few genomes failed (with the same error) with the PRO profiling:
GCA_900068945.1
GCA_900068915.1
GCA_900069095.1
GCA_900068965.1
GCA_900068985.1
GCA_018221805.1
GCA_900068975.1
GCA_900068955.1

What does the error mean and do you have any idea about what is causing it?

Thanks a lot for your help and the amazing pipeline!

Best,
Brigida

• version 1.0.4 on Linux 20.04, conda build

• What I did

source ${PATH_ACTIVATE}/activate ufcg
for file in ${INPUT}/*.fna; 
do
    if [ ! -d "./tmp" ] 
    then
        mkdir ./tmp
    else
        rm -r ./tmp
        mkdir ./tmp
    fi
    cp ${INPUT}/${file##*/} ./tmp/${file##*/}
    ufcg profile -i ./tmp -o ./101_ufcg_profile -t ${THREAD} -v --developer --force 1
    rm -r ./tmp
done

• What happened

[JUN 08 15:57:13] UFCG  |:  AUGUSTUS is predicting genes... (contig chr14, position 444431-450895)
[JUN 08 15:57:13] DEBUG |:  exec: augustus --optCfgFile=/home/genome/anaconda3/envs/ufcg/share/ufcg-1.0.4-0/config/ppx.cfg --predictionStart=437264 --predictionEnd=457264 --proteinprofile=/home/genome/anaconda3/envs/ufcg/share/ufcg-1.0.4-0/config/model/pro/POL2.hmm /tmp/KMRB18080202/KMRB18080202_chr14.fna > /tmp/KMRB18080202/KMRB18080202_chr14_p447264_POL2.gff
[JUN 08 15:57:18] DEBUG |:  exec: head -1 /tmp/KMRB18080202/KMRB18080202_chr14_p447264_POL2.gff
[JUN 08 15:57:18] UFCG  |:  Parsing gene prediction result written on : /tmp/KMRB18080202/KMRB18080202_chr14_p447264_POL2.gff
[JUN 08 15:57:18] UFCG  |:  [Phase 3 : Validation]
[JUN 08 15:57:18] UFCG  |:  Exporting predicted genes to FASTA file : /tmp/KMRB18080202/KMRB18080202_POL2.fasta
[JUN 08 15:57:18] DEBUG |:  exec: mmseqs easy-search /tmp/KMRB18080202/KMRB18080202_POL2.fasta /home/genome/anaconda3/envs/ufcg/share/ufcg-1.0.4-0/config/seq/pro/POL2.fa /tmp/KMRB18080202/UFCG_b72a895b2b963217_POL2.fa.m8 /tmp/KMRB18080202/ --search-type 3 -e 0.001 --cov-mode 1 -c 0.8 --threads 1
[JUN 08 15:57:23] UFCG  |:  Extracting cDNA sequence of gene POL2...
[JUN 08 15:57:23] UFCG  |:  Query genome contains a single copy of gene POL2
[JUN 08 15:57:23] UFCG  |:  RESULT : [Single: 56 ; Duplicated: 5 ; Missing: 0]
[JUN 08 15:57:23] UFCG  |:  Writing results on : ./101_ufcg_profile/KMRB18080202.ucg
[JUN 08 15:57:23] UFCG  |:  Cleaning temporary files up...
[JUN 08 15:57:23] UFCG  |:  ERROR! java.util.ConcurrentModificationException: null
[JUN 08 15:57:23] UFCG  |:      at java.util.ArrayList$Itr.next(ArrayList.java:867)
[JUN 08 15:57:23] UFCG  |:      at envs.toolkit.FileStream.wipeOut(FileStream.java:101)
[JUN 08 15:57:23] UFCG  |:      at module.ProfileModule.run(ProfileModule.java:921)
[JUN 08 15:57:23] UFCG  |:      at pipeline.ModuleHandler.handle_profile(ModuleHandler.java:28)
[JUN 08 15:57:23] UFCG  |:      at pipeline.ModuleHandler.handle(ModuleHandler.java:66)
[JUN 08 15:57:23] UFCG  |:      at pipeline.UFCGMainPipeline.main(UFCGMainPipeline.java:281)

I got same error issue with #21.
But in my case, I checked CMD1_pro.zZ.fasta file, but there was no any different sequences (every sequences were same).

Though I got 1 warning message as...

WARN |: Less than 4 species have 'COB'. This gene will be excluded.
COB had no problem and its nwk file is successfully generated.

I also tried --fasttree STR and --raxml STR option, but I got same result. What other option can I adjust?

Hi -

I installed ufcg using conda, and used the setup commands in the instructions.
I am hit with this error when using the ufcg profile:

ufcg profile -i directory_with_sequences -o outdir

Any insight would be helpful.

When running ufcg (v1.0.2) profile, I get this error message:
ERROR! Failed subcommand :augustus --optCfgFile=/mnt/gpfs/seb/Applications/UFCG/config/ppx.cfg --predictionStart=24228 --predictionEnd=44228 --proteinprofile=/mnt/gpfs/seb/Applications/UFCG/config/model/pro/RPB2.hmm /home/sebr/seb/Database/Fungus_DB/UFCG/tmp/Lobtra1/tmp/Lobtra1/Lobtra1_scaffold_92.fna > /home/sebr/seb/Database/Fungus_DB/UFCG/tmp/Lobtra1/tmp/Lobtra1/Lobtra1_scaffold_92_p34228_RPB2.gff

I checked and this same genome dealt with normaly in ufcg v1.0.1. I have now to have ufcg (v1.0.1) for profiling and ufcg (v1.0.2) for building tree (it does not work in v1.0.1).

I have intalled ufcg on WSL, however, when I used the same way (mamba install -y ufcg) to install it on ubuntu 22.04.2 LTS, it got wroing.
ufcg -u Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class envs.config.GenericConfig at envs.toolkit.Prompt.print(Prompt.java:31) at pipeline.UFCGMainPipeline.main(UFCGMainPipeline.java:289)
I checked my java is openjdk-17-jdk, how can I process it?

Hello I have a new issue for you :)
Some of the genomes I'm looking at are taking forever to be profiled. When looking at it in detail augustus is being slow (50h+) as well as hogging ram (worst so far is 420G ram) on some genes. As I don't monitor all the time I don't know if some of theses managed to complete before.
From what Is running in my server I have 6 ELP3 and 3 RPO21 running for more than 40h.

I can share with you an example where it does not end so far.
Here is a command running forever:
augustus --optCfgFile=/mnt/gpfs/seb/Applications/UFCG_v1.0.1/UFCG/config/ppx.cfg --predictionStart=165215 --predictionEnd=185215 --proteinprofile=/mnt/gpfs/seb/Applications/UFCG_v1.0.1/UFCG/config/model/pro/ELP3.hmm /home/sebr/seb/Database/Fungus_DB/UFCG/tmp/Conco1/tmp/Conco1/Conco1_scaffold_37.fna

with genome and temp dir:
Conco1_issue.tar.gz

Hi, I have tried to resolve this issue but failed to do so while using UFCG. Please help asap.

ERROR! AUGUSTUS-PPX config file is absent or improperly formatted.

I get this in the latest version as well as the old one.

For example, I want to use the agaricales_odb10 as reference database to pick single copy orthologs, how can I change the Fungi_odb10 to Agaricales_odb10?

I am encountering small issue with makes me unable to use ufcg on more than a file at a time.

After profiling a genome and writing results there is an attempt to delete temporary files, this fail in a non deterministic way and may be partially related to my server file system speed in closing handle on files.

I suppose ufcg attempt to remove these files too quickly or without having closed them beforehand? In any way, it interrupt execution and I need to work around it by launching ufcg on a unique genome at a time.

In conda installation, UFCG requires
ufcg -t download to finish installation.

However, current error message

"Failed to detect environment path. --ppxcfg, --seqpath, and --modelpath options should be specified."
"ERROR! Block profile model directory is incomplete."

is difficult for users to figure out needs of ufcg -t download

I suggest to add messages such as these

If this is your first time using UFCG, use "ufcg -t download config
If you haven't downloaded model, use "ufcg -t download minimum" or "ufcg -t download full"

Dear Daniel,

After the extraction of the core genes I manipulated the ucg files to get counts for each gene in order to understand which genes are still single-copy and can be carried along for phylogenetic analyses.
This is surprisingly variable, partially biologically and partially technical (assembly-related) but the majority of the genes are indeed single-copy.
It would be super useful to be able to extract all the unaligned sequences as fasta files so the user can make decisions on what to bring to the next stage. Currently, using the module "align" all the genes for all the species are included in one single alignment (including multi copy genes), is this correct?

E.g. the MIP2 gene. I searched in 54 species. In 48 species, it was single copy, in 2 species it was missing and in 4 species it was duplicated. My MIP2 alignment includes 52 species, so I assumed the 48 + 4 but I cannot see any label for the duplicated species which appear in there once. Can you tell me how the genes are carried along and labeled?

Gene Category (missing, 1 copy, 2 copies) Nr. species Freq
MIP1 0 2 0.03703704
MIP1 1 48 0.88888889
MIP1 2 4 0.07407407

Thank you in advance your help!

Next update will include a new feature for the profile module that allows for the customization of profiling sensitivity.

Current algorithm validates the AUGUSTUS search result using 3-step fallback search, which uses 80% coverage first, 50% coverage upon failure, and 0% coverage (e-value only) upon failure.

A sensitivity option will be included that can tweak with this algorithm. In detail:

• --sensitivity 3 (default): Use original algorithm
• --sensitivity 2 : 2-step search (80%, 50%)
• --sensitivity 1 : 1-step search (80%)

Lowering sensitivity will significantly reduce the amount of false positive gene calls from the result, however, will become prone to miss some true positive genes as a trade-off. (e.g. failing to call genes with subtle evidences)