Background: We originally started tracking the FastAI course.

But because of a lack of depth, we decided instead to dissect blogs and tutorials.

Introductions!

What does your current work look like? How do you plan to use NLP?

1. What is NLP?

• A changing field
• Resources
• Tools
• Python libraries
• Example applications
• Ethics issues

Notebook: What is NLP?

Video(s):

• FastAI: What is NLP?
• SOTA 2019

Video(s):

• FastAI: Topic Modeling with SVD & NMF

Notebook:

• Topic Modeling with SVD & NMF

Resources:

• A Tutorial on PCA (closely related to SVD)
• SVD article, explaining intuition behind factorizing to 3 different matrices

Video(s):

• FastAI: Logistic Regression & Naive Bayes

Notebook:

• Logistic Regression & Naive Bayes

Resources:

• Baysean Inference

Questions:

• Why is it when you plot the sorted document-term matrix it shows those waves

Skipped

Video(s):

• FastAI: Transfer Learning

Notebook:

• Transfer Learning

Resources:

• NLP's ImageNet Movement
• Cyclical Learning Rates
• Stop using Word2Vec

Questions:

• When we pick a learning rate off the LR vs. Loss graph, we pick it halway down the descent to the lowest point. Why is that?

Blogs:

• Word Embeddings - Part 1: Intro
• Word Embeddings - Part 2: Approximating SoftMax

• Word Embeddings - Part 3: Secret Ingredients of Word2Vec
• Word Embeddings - Part 5: Future Trends

Optional:

• The illustrated Word2Vec

• The Unreasonable Effectiveness of RNNs
• Illustrated Guide to LSTMs and GRUs
• Understanding LSTM Networks

• Tutorial on LSTMs: A Computational Perspective
• Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention)
• Seq2Seq with Attention and Beam Search

• Attention and Augmented Recurrent Neural Networks
• Attention? Attention!
• Attention is all you need

Optional:

• Intuitive Understanding of Attention Mechanism in Deep Learning
• NLP FROM SCRATCH: TRANSLATION WITH A SEQUENCE TO SEQUENCE NETWORK AND ATTENTION
• NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE

• Deep Learning Book
• FastAI Book

• Linear Algebra with code examples
• Linear Algebra Class(MIT Opencourseware)
• FastAI: Computational Linear Algebra

• A Primer in BERTology: What we know about how BERT works
• Awesome NLP
• The Illustrated Transformer
• The Illustrated BERT, ELMo, and co.

• Distill
• OpenAI
• StichFix
• The Gradient