# Roadmap: Natural Language Processing Fast Ai to practical transformer

Suitable for anyone who have trained deeplearning networks and want to try out NLP

Suggested Route: 5 hrs a week
1 hr of lesson
3 hrs of code in notebook

---
### Week 1
- [ ] 1. What is NLP?
- [ ] A changing field
- [ ] Resources
- [ ] Tools
- [ ] Python libraries
- [ ] Example applications
- [ ] Ethics issues

### Week 2
- [ ] 2. Topic Modeling with NMF and SVD
- [ ] Stop words, stemming, & lemmatization
- [ ] Term-document matrix
- [ ] Topic Frequency-Inverse Document Frequency (TF-IDF)
- [ ] Singular Value Decomposition (SVD)
- [ ] Non-negative Matrix Factorization (NMF)
- [ ] Truncated SVD, Randomized SVD

### Week 3
- [ ] 3. Sentiment classification with Naive Bayes, Logistic regression, and ngrams
- [ ] Sparse matrix storage
- [ ] Counters
- [ ] the fastai library
- [ ] Naive Bayes
- [ ] Logistic regression
- [ ] Ngrams
- [ ] Logistic regression with Naive Bayes features, with trigrams

### Week 4
- [ ] 4. Regex (and re-visiting tokenization)

### Week 5
- [ ] Deeplearning.ai Recurrent Neural Networks
- [ ] Deeplearning.ai Natural Language Processing & Word Embeddings

### Week 6
- [ ] 5. Language modeling & sentiment classification with deep learning
- [ ] Language model
- [ ] Transfer learning
- [ ] Sentiment classification

### Week 7
- [ ] 6. Translation with RNNs
- [ ] Review Embeddings
- [ ] Bleu metric
- [ ] Teacher Forcing
- [ ] Bidirectional
- [ ] Attention

### Week 8 and 9
- [ ] Deeplearning.ai Sequence models & Attention mechanism
- [ ] 7. Translation with the Transformer architecture
- [ ] Transformer Model
- [ ] Multi-head attention
- [ ] Masking
- [ ] Label smoothing

### Week 10 11
- [ ] Implement a tranformer in tensorflow from scratch

### Week 12
- [ ] 8. Bias & ethics in NLP
- [ ] bias in word embeddings
- [ ] types of bias
- [ ] attention economy
- [ ] drowning in fraudulent/fake info