Mohammed Sabry's Projects
dada2 Nextflow pipeline
An implementation of Mamba to develop an understanding of its functioning.
Comparing between residual stream and highway stream in transformers(BERT) .
Educational project
Articles
Emergent Communication Pretraining for Few-Shot Machine Translation
Playing with SVD and Word Vectors
learning project
Comparing the full-precision Query, Keys, and Values matrices with their 1-bit counterparts in a two-layer, attention-only transformer trained on a synthetic copying task.
Function Vectors in Large Language Models
Supplementary Machine Learning and Deep Learning tutorial sessions with code to AMMI program 2019-2020 cohort.
NeuSpell: A Neural Spelling Correction Toolkit
Python implementation of TextRank algorithms ("textgraphs") for phrase extraction
Querying a Language model in a residual path for drug-target interaction task
NLP
Visualizing the change of Sudan temperature from 1901 to 2016