Giter VIP home page Giter VIP logo

nextits's Introduction

NextITS

GitHub (latest release) Nextflow Runs with singularity GitHub license

NextITS is an automated pipeline for metabarcoding fungi and other eukaryotes with full-length ITS sequenced with PacBio. Amplicons obtained with Illumina are also supported.

Introduction

The most widely used genetic markers for metabarcoding fungal communities are highly variable rRNA ITS1 and ITS2 sub-regions of the internal transcribed spacer. High-throughput metabarcoding has greatly improved our understanding of fungal community ecology. Here, we present NextITS, an automated pipeline for analyzing full-length ITS sequences (ITS1-5.8S-ITS2) from the Pacific Biosciences (PacBio) third-generation sequencing platform. Although the PacBio HiFi reads are highly accurate, the primary type of sequencing error is insertions or deletions in homopolymeric sites, which are also naturally common in fungal ITS. In the pipeline, we implemented correction of homopolymer errors, detection of tag-switching artefacts, and recovery of sequences false-positively annotated as chimeric. The pipeline is built using Nextflow workflow manager, with all the software dependencies packaged into Docker and Singularity containers.

User Documentation

User documentation: https://Next-ITS.github.io/

Quick Start

nextflow run vmikk/NextITS -r main \
  -profile singularity \
  -resume \
  --input          "pacbio_ccs.fastq.gz" \
  --barcodes       "sample_barcodes.fasta" \
  --primer_forward "GTACACACCGCCCGTCG" \
  --primer_reverse "CCTSCSCTTANTDATATGC" \
  --its_region     "full" \
  --blast_taxdb    "false" \
  --outdir         "Results"

Citation

Mikryukov V., Anslan S., Tedersoo L. NextITS: a pipeline for metabarcoding fungi and other eukaryotes with full-length ITS sequenced with PacBio. https://github.com/vmikk/NextITS

nextits's People

Contributors

vmikk avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

nextits's Issues

test run not working

Hi Vladimir,

On the doc page (https://next-its.github.io/installation/) there is a test command that I do not succeed to run.

nextflow run vmikk/nextits -r main -profile test
N E X T F L O W  ~  version 22.10.1
Unknown configuration profile: 'test'

I tried replacing main by 0.5.0 without more success

$ nextflow run vmikk/nextits -r 0.5.0 -profile test
N E X T F L O W  ~  version 22.10.1
Unknown configuration profile: 'test'
log
Aug-29 12:14:28.633 [main] DEBUG nextflow.cli.Launcher - $> nextflow run vmikk/nextits -r 0.5.0 -profile test
Aug-29 12:14:28.693 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 22.10.1
Aug-29 12:14:28.717 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/luna.kuleuven.be/u0002316/.nextflow/plugins; core-plugins: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]
Aug-29 12:14:28.727 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Aug-29 12:14:28.728 [main] INFO  org.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Aug-29 12:14:28.731 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.4.1 in 'deployment' mode
Aug-29 12:14:28.741 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Aug-29 12:14:28.756 [main] DEBUG nextflow.scm.ProviderConfig - Using SCM config path: /home/luna.kuleuven.be/u0002316/.nextflow/scm
Aug-29 12:14:29.692 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/luna.kuleuven.be/u0002316/.nextflow/assets/vmikk/nextits/.git/config; branch: master; remote: origin; url: https://github.com/vmikk/NextITS.git
Aug-29 12:14:29.714 [main] DEBUG nextflow.scm.RepositoryFactory - Found Git repository result: [RepositoryFactory]
Aug-29 12:14:29.722 [main] DEBUG nextflow.scm.AssetManager - Git config: /home/luna.kuleuven.be/u0002316/.nextflow/assets/vmikk/nextits/.git/config; branch: master; remote: origin; url: https://github.com/vmikk/NextITS.git
Aug-29 12:14:30.533 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /home/luna.kuleuven.be/u0002316/.nextflow/assets/vmikk/nextits/nextflow.config
Aug-29 12:14:30.534 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /home/luna.kuleuven.be/u0002316/.nextflow/assets/vmikk/nextits/nextflow.config
Aug-29 12:14:30.542 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `test`
Aug-29 12:14:30.737 [main] DEBUG nextflow.config.ConfigBuilder - Available config profiles: [singularity, hpc, docker]
Aug-29 12:14:30.754 [main] DEBUG nextflow.cli.Launcher - Operation aborted
nextflow.exception.AbortOperationException: Unknown configuration profile: 'test'
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:72)
	at org.codehaus.groovy.reflection.CachedConstructor.doConstructorInvoke(CachedConstructor.java:59)
	at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrap.callConstructor(ConstructorSite.java:84)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:59)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
	at nextflow.config.ConfigBuilder.checkValidProfile(ConfigBuilder.groovy:511)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:61)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
	at nextflow.config.ConfigBuilder.buildConfig0(ConfigBuilder.groovy:385)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:61)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:194)
	at nextflow.config.ConfigBuilder.buildGivenFiles(ConfigBuilder.groovy:317)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:193)
	at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:61)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:51)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:171)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
	at nextflow.config.ConfigBuilder.buildConfigObject(ConfigBuilder.groovy:780)
	at nextflow.config.ConfigBuilder.build(ConfigBuilder.groovy:793)
	at nextflow.cli.CmdRun.run(CmdRun.groovy:307)
	at nextflow.cli.Launcher.run(Launcher.groovy:487)
	at nextflow.cli.Launcher.main(Launcher.groovy:646)

I cloned the git and have the test_data folder local, should I do something extra to run the test?

but the image seems to be ready to do something

$ nextflow run vmikk/nextits -r 0.5.0 --help
N E X T F L O W  ~  version 22.10.1
Launching `https://github.com/vmikk/nextits` [suspicious_banach] DSL2 - revision: 6f9e110b9d [0.5.0]
Please provide the input file with sequences in FASTQ.gz format with `--input` parameter.

S

failed run with Zymo control data

Hi Vladimir,

This is for when you return from vacation, no hurry!

We got today the first run completed and I tried to analyse the data as you told me, providing the fastq folder as input and adding --demultiplexed true

The sequences blast nicely and returns many times Saccharomyces which is what we expect from the Zymo D6305 sample, however the nextflow run failed, apparently due to reference data not found (FATAL: container creation failed: mount /mnt/Dat2/DB/UNITE/Leho_Subset->/mnt/Dat2/DB/UNITE/Leho_Subset error: while mounting /mnt/Dat2/DB/UNITE/Leho_Subset: mount source /mnt/Dat2/DB/UNITE/Leho_Subset doesn't exist)

I attach an archive with the .nextflow log and my input data as well as my full command, I hope you will spot the issue and let me know (if you do not get the zip, just let me know).

Also, the amplicons from feces samples run in parallel (for which we have nice 16S) return much larger ITS-amplicon sequences 2-3kb which so far all blast to bacterial entities with various levels of identity.

Can you share what the expected on-target frequency for this PCR is with mixed populations with bacterial content? It seems very low in our case, I did not find yet a single non-bacterial hit (blast is running...)

Thanks for your tips and info

failed_run.zip

loading RData objects fails

HI Vladimir,

I copied locally the two RData objects found at

04.PooledResults/OTU_table_long.RData
04.PooledResults/OTU_table_wide.RData

but RStudio does not like them

> load("~/work/NGS-Analyses/NextITS_analysis/NextITS_Results_HiFi_reads/OTU_table_long.RData")
Error in load("~/work/NGS-Analyses/NextITS_analysis/NextITS_Results_HiFi_reads/OTU_table_long.RData") : 
  bad restore file magic number (file may be corrupted) -- no data loaded
In addition: Warning messages:
1: In readChar(con, 5L, useBytes = TRUE) :
  truncating string with embedded nuls
2: file ‘OTU_table_long.RData’ has magic number 'X'
  Use of save versions prior to 2 is deprecated 
> load("~/work/NGS-Analyses/NextITS_analysis/NextITS_Results_HiFi_reads/OTU_table_wide.RData")
Error in load("~/work/NGS-Analyses/NextITS_analysis/NextITS_Results_HiFi_reads/OTU_table_wide.RData") : 
  bad restore file magic number (file may be corrupted) -- no data loaded
In addition: Warning messages:
1: In readChar(con, 5L, useBytes = TRUE) :
  truncating string with embedded nuls
2: file ‘OTU_table_wide.RData’ has magic number 'X'
  Use of save versions prior to 2 is deprecated 

Q about asymetrical barcodes

Dear Vladimir,

We work with assymetrical barcodes in order to multiplex a large number of samples on our Sequel-IIe (inspired from the 16S pipeline and using the same layouts).

I read in your usage page that assymetrical barcodes are not currently supported. Very sad! This marks the end of my exploration as I cannot change that in the wet lab and multiple samples always share the same barcode on either end.

Do you have an estimate on when this could be implemented?

Thanks in advance

secondary analysis

Dear Vladimir,

I am finally starting to use your pipeline and fixed a few issues at my side that were crashing step2.
I now have al expected folders but do not find how to proceed and get classification (I now ran a manual blast) as well nice bar plots and also phylogeny plots.

I expect this can be done using Qiime2 but I am still not too familiar with it and which files need be imported to create the right objects for plotting.

Would you by any chance have a tutorial explaining the next steps for visualisation to obtain results similar to the pb-16S-nf pipeline does for 16S. Ideally, an extension to your current https://next-its.github.io/ doc would be perfect with results from some test data.

Thanks in advance

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.