systemsgenetics / pynome Goto Github PK
View Code? Open in Web Editor NEWGenome collection script for the SciDAS project.
License: GNU General Public License v3.0
Genome collection script for the SciDAS project.
License: GNU General Public License v3.0
Hi.
In transferring iRODS genome data to Colorado State for the NDN project, I noticed that these things (see 'ils' example below)
[ffeltus@scidas-scratch1 MG2]$ ils
/scidasZone/sysbio/genomes/Zymoseptoria_tritici/MG2:
Zymoseptoria_tritici-MG2.1.ht2
Zymoseptoria_tritici-MG2.2.ht2
Zymoseptoria_tritici-MG2.3.ht2
Zymoseptoria_tritici-MG2.4.ht2
Zymoseptoria_tritici-MG2.5.ht2
Zymoseptoria_tritici-MG2.6.ht2
Zymoseptoria_tritici-MG2.7.ht2
Zymoseptoria_tritici-MG2.8.ht2
Zymoseptoria_tritici-MG2.fa
Zymoseptoria_tritici-MG2.gff3
Zymoseptoria_tritici-MG2.gtf
Zymoseptoria_tritici-MG2.Splice_sites.txt
GEMmaker now supports Kallisto and Salmon. Pynome should create those indexes too:
Hi. I was working with Colorado State to move the iRODs genome data into an NDN network when I realized there are some substantial naming/parsing issues:::
It says that genome files will be in this directory format:
[genus][species]{[infraspecific name]}/[assembly_name]/
that contains these files:
[genus]_[species]{_[infraspecific name]}-[assembly_name].{hisat2 extension}
[genus]_[species]{_[infraspecific name]}-[assembly_name].gff3
[genus]_[species]{_[infraspecific name]}-[assembly_name].fasta
[genus]_[species]{_[infraspecific name]}-[assembly_name].gtf
[genus]_[species]{_[infraspecific name]}-[assembly_name].Splice_Sites.txt
[genus]_[species]{_[infraspecific name]}-[assembly_name].meta.json
However, the files are not in an "infraspecific name" subdirectory and the file names are not in the '[genus][species]{[infraspecific name]}-[assembly_name].' format:
C- /scidasZone/sysbio/genomes/Absidia_glauca
C- /scidasZone/sysbio/genomes/Acanthamoeba_castellanii_str_neff
C- /scidasZone/sysbio/genomes/Acidomyces_richmondensis
C- /scidasZone/sysbio/genomes/Acidomyces_richmondensis_bfw
C- /scidasZone/sysbio/genomes/Acremonium_chrysogenum_atcc_11550
BIG ISSUES:
Shall we meet to discuss these issues?
Alex
This worked:
'ichmod -rM read susmit /scidasZone/sysbio'
Over 150 genomes failed on the last run of pynome. On inspecting one of them (./2792677/ASM1369444v1-ncbi), there were problems creating the cDNA FASTA file.
.fa.fai
did not have the sequence names in the first column. Not sure what caused that.gffread
utility on Kamiak I got the following message Error: no ID found for GFF record start
. It turns out there is a transcript_id = "";
entry in the gene
feature of the GTF file. When I manually removed that entry for each gene the cDNA was built just fine.We need to make the following changes to the NCBI downloader of Pynome:
assembly_summary_genbank.txt
file.infraspecific_name
column of the assembly_summary_genbank.txt
file. Just use the organism_name
column as it does have the infraspecific name in it. The only thing, it appears, that the infraspecific_name
column is holding is a key/value pair specifying the cultivar or the strain. We can just ignore this in the file naming.https://api.ncbi.nlm.nih.gov/datasets/v1alpha/genome/accession/$id
where $id
is the NCBI ID (first column of the assembly_summary_genbank.txt
file.We should explore the possibility of adding NCBI genome assemblies:
ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/
It seems Pynome is using the entire genome assembly for Salmon and Kallisto indexing rather than the cDNA sequences. The following should be used instead.
For both Salmon and Kallisto we need to use the cDNA file that Ensembl provides. You can find the cDNA files in an FTP directory similar to the following:
ftp://ftp.ensemblgenomes.org/pub/plants/release-48/fasta/oryza_sativa/cdna/
We need to retrieve the file with the suffix .cdna.all.fa.gz
. The file just needs to be uncompressed and used instead of the whole genome FASTA file.
If a GTF file exists we need to download it and run the following command to create a cDNA FASTA file:
gffread -w transcripts.fa -g genome.fa transcripts.gtf
Where:
transcript.fa
is the name of the output FASTA file that will have the cDNA entries.genome.fa
is the name of the whole genome FASTA filetranscript.gtf
is the name of the GTF file.We want to name the output file transcript.fa
to follow our naming convention for all assembly files.
If a GTF file is not available but a GFF is then I believe Pynome already has code to convert it to GTF. Although, I think currently happens after genome indexing. Instead the converion of a GFF to GTF should happen prior to indexing and then that GTF file can be used to create a cDNA FASTA file just as described above.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.