carrascomj / drastic Goto Github PK

The preprocessing task involves sampling from the genomes on varying length chunks.

• Find the minimum_size of the chunks (maximum length of a gene)
• For every gene, expand or decrease (to 90%) and label as gene.
• Sample varying size from no genes.
• Define partial genes as 50% of a genes in a chunk

• notes while we work and develop models

• first draft in latex

• review 1

The README should have the models and features we are going to accomplish.

We should account sometime to prepare the poster and coordinate the presentation.

Which license should we use for the repository? I guess it's fine with a MIT or GNU license.

• Choose some license.
• Add it to the repository.
• Add it to the header of all files if necessary.

Given that we have defined the functions for extracting labeled ranges from the data (#3), all that is left is to build a dataframe (I suppose in pandas) using these ranges and the genome sequence.

My first guess is that the dataset would contain two columns, one for the sequence and other for the label.

Having defined all of the pre-processing functions (embeddings not included), we need to define some API for the user to transform the raw data into a suitable format:

• Functions in python that joins all of the pre-processing functions into a pipeline.
• Filter big genes too in the pipeline.
• Command Line Interface in /bin to go from paths to files (FASTA and feature table) to the pre-processed dataframe (maybe a CSV).

Christmas is coming and the deadline is looming! Please tell others if you are working on one of the tasks and edit this issue as you please.

Urgent and doable right now

• Introduction (in Overleaf).
• Methods.
• Draft of Abstract.
• Pre-processing exploitation.
• Pre-processing explain: add information in the input pd.DataFrame about the
  functions of each gene so we can find explanations later.
• Post-processing gathering (put all the functions together).
• Table of results/performance/confusion matrix (try stuff and annotate
  results).

Urgent but blocked

• Results.
• Discussion.
• Conclusion (I am keen on separating all the sections, but these three
  may be combined if needed).
• Finish Abstract.
• Analysis of difficult cases.

Not urgent but doable

• Convnet: Try one hot encoding instead of index input (4 input channels)
• Convnet: analysis of activations.
• CBOW: tSNE/PCA.
• Proper handling of UNK characters (N, S, etc.), just remove them and break
  the window at this point.

Proposed in feedback

• Add more conv layers with different kernel size.
• Explore CBOW k.
• There were more things but I don't remember them, just edit this section if you feel like doing so.

1. Decide on amount of nucleotides for embeddings. Start with 7?
2. Create/import function to go from nucleotide string to embedding.
3. Clarify how the 7-ker sequence is encoded in word embeddings

First model to consider for the first milestone.

• First model implementation.
• Accepting varying size input.

Reference: Stephen Merity, Nitish Shirish Keskar, Richard Socher (2017). Regularizing and Optimizing LSTM Language Models.

Now that we have something working, we should prepare the notebooks and organize the different models:

• Pass the notebook of CBOW to a .py ready to be imported.
• Define the classes for the different NN Models that we have at src/data/, ready to be imported.
• Clean the notebooks and standardize the paths (Colab/drive should be changed with relative paths starting from the repo tree).