# Group 2: Natural Language Processing Fast Ai to practical transformer Suitable for anyone who have trained deeplearning networks and want to try out NLP Suggested Route: 5 hrs a week 1 hr of lesson 3 hrs of code in notebook ## Balaji Week 1 - [x] 1. What is NLP? - [x] A changing field - [x] Resources - [x] Tools - [x] Python libraries - [x] Example applications - [x] Ethics issues Week 2 - [x] 2. Topic Modeling with NMF and SVD - [x] Stop words, stemming, & lemmatization - [x] Term-document matrix - [x] Topic Frequency-Inverse Document Frequency (TF-IDF) - [x] Singular Value Decomposition (SVD) - [x] Non-negative Matrix Factorization (NMF) - [x] Truncated SVD, Randomized SVD Week 3 - [ ] 3. Sentiment classification with Naive Bayes, Logistic regression, and ngrams - [x] Sparse matrix storage - [x] Counters - [x] the fastai library - [ ] Naive Bayes - [ ] Logistic regression - [ ] Ngrams - [ ] Logistic regression with Naive Bayes features, with trigrams Week 4 - [ ] 4. Regex (and re-visiting tokenization) Week 5 - [ ] Deeplearning.ai Recurrent Neural Networks - [ ] Deeplearning.ai Natural Language Processing & Word Embeddings Week 6 - [ ] 5. Language modeling & sentiment classification with deep learning - [ ] Language model - [ ] Transfer learning - [ ] Sentiment classification Week 7 - [ ] 6. Translation with RNNs - [ ] Review Embeddings - [ ] Bleu metric - [ ] Teacher Forcing - [ ] Bidirectional - [ ] Attention Week 8 and 9 - [ ] Deeplearning.ai Sequence models & Attention mechanism - [ ] 7. Translation with the Transformer architecture - [ ] Transformer Model - [ ] Multi-head attention - [ ] Masking - [ ] Label smoothing Week 10 11 - [ ] Implement a tranformer in tensorflow from scratch Week 12 - [ ] 8. Bias & ethics in NLP - [ ] bias in word embeddings - [ ] types of bias - [ ] attention economy - [ ] drowning in fraudulent/fake info ## Aditya Week 1 - [x] 1. What is NLP? - [x] A changing field - [x] Resources - [x] Tools - [x] Python libraries - [x] Example applications - [x] Ethics issues Week 2 - [x] 2. Topic Modeling with NMF and SVD - [x] Stop words, stemming, & lemmatization - [x] Term-document matrix - [x] Topic Frequency-Inverse Document Frequency (TF-IDF) - [x] Singular Value Decomposition (SVD) - [x] Non-negative Matrix Factorization (NMF) - [x] Truncated SVD, Randomized SVD Week 3 - [x] 3. Sentiment classification with Naive Bayes, Logistic regression, and ngrams - [x] Sparse matrix storage - [x] Counters - [x] the fastai library - [ ] Naive Bayes - [ ] Logistic regression - [ ] Ngrams - [ ] Logistic regression with Naive Bayes features, with trigrams Week 4 - [ ] 4. Regex (and re-visiting tokenization) Week 5 - [ ] Deeplearning.ai Recurrent Neural Networks - [ ] Deeplearning.ai Natural Language Processing & Word Embeddings Week 6 - [ ] 5. Language modeling & sentiment classification with deep learning - [ ] Language model - [ ] Transfer learning - [ ] Sentiment classification Week 7 - [ ] 6. Translation with RNNs - [ ] Review Embeddings - [ ] Bleu metric - [ ] Teacher Forcing - [ ] Bidirectional - [ ] Attention Week 8 and 9 - [ ] Deeplearning.ai Sequence models & Attention mechanism - [ ] 7. Translation with the Transformer architecture - [ ] Transformer Model - [ ] Multi-head attention - [ ] Masking - [ ] Label smoothing Week 10 11 - [ ] Implement a tranformer in tensorflow from scratch Week 12 - [ ] 8. Bias & ethics in NLP - [ ] bias in word embeddings - [ ] types of bias - [ ] attention economy - [ ] drowning in fraudulent/fake info ## Kaiwei Week 1 - [x] 1. What is NLP? - [x] A changing field - [x] Resources - [x] Tools - [x] Python libraries - [x] Example applications - [x] Ethics issues Week 2 - [x] 2. Topic Modeling with NMF and SVD - [x] Stop words, stemming, & lemmatization - [x] Term-document matrix - [x] Topic Frequency-Inverse Document Frequency (TF-IDF) - [x] Singular Value Decomposition (SVD) - [x] Non-negative Matrix Factorization (NMF) - [x] Truncated SVD, Randomized SVD Week 3 - [x] 3. Sentiment classification with Naive Bayes, Logistic regression, and ngrams - [x] Sparse matrix storage - [x] Counters - [x] the fastai library - [ ] Naive Bayes - [ ] Logistic regression - [ ] Ngrams - [ ] Logistic regression with Naive Bayes features, with trigrams Week 4 - [ ] 4. Regex (and re-visiting tokenization) Week 5 - [ ] Deeplearning.ai Recurrent Neural Networks - [ ] Deeplearning.ai Natural Language Processing & Word Embeddings Week 6 - [ ] 5. Language modeling & sentiment classification with deep learning - [ ] Language model - [ ] Transfer learning - [ ] Sentiment classification Week 7 - [ ] 6. Translation with RNNs - [ ] Review Embeddings - [ ] Bleu metric - [ ] Teacher Forcing - [ ] Bidirectional - [ ] Attention Week 8 and 9 - [ ] Deeplearning.ai Sequence models & Attention mechanism - [ ] 7. Translation with the Transformer architecture - [ ] Transformer Model - [ ] Multi-head attention - [ ] Masking - [ ] Label smoothing Week 10 11 - [ ] Implement a tranformer in tensorflow from scratch Week 12 - [ ] 8. Bias & ethics in NLP - [ ] bias in word embeddings - [ ] types of bias - [ ] attention economy - [ ] drowning in fraudulent/fake info ### Suggestions: -dont make it in weeks. Make it in traunches ## Veri Week 1 - [ ] 1. What is NLP? - [ ] A changing field - [ ] Resources - [ ] Tools - [ ] Python libraries - [ ] Example applications - [ ] Ethics issues Week 2 - [ ] 2. Topic Modeling with NMF and SVD - [ ] Stop words, stemming, & lemmatization - [ ] Term-document matrix - [ ] Topic Frequency-Inverse Document Frequency (TF-IDF) - [ ] Singular Value Decomposition (SVD) - [ ] Non-negative Matrix Factorization (NMF) - [ ] Truncated SVD, Randomized SVD Week 3 - [ ] 3. Sentiment classification with Naive Bayes, Logistic regression, and ngrams - [ ] Sparse matrix storage - [ ] Counters - [ ] the fastai library - [ ] Naive Bayes - [ ] Logistic regression - [ ] Ngrams - [ ] Logistic regression with Naive Bayes features, with trigrams Week 4 - [ ] 4. Regex (and re-visiting tokenization) Week 5 - [ ] Deeplearning.ai Recurrent Neural Networks - [ ] Deeplearning.ai Natural Language Processing & Word Embeddings Week 6 - [ ] 5. Language modeling & sentiment classification with deep learning - [ ] Language model - [ ] Transfer learning - [ ] Sentiment classification Week 7 - [ ] 6. Translation with RNNs - [ ] Review Embeddings - [ ] Bleu metric - [ ] Teacher Forcing - [ ] Bidirectional - [ ] Attention Week 8 and 9 - [ ] Deeplearning.ai Sequence models & Attention mechanism - [ ] 7. Translation with the Transformer architecture - [ ] Transformer Model - [ ] Multi-head attention - [ ] Masking - [ ] Label smoothing Week 10 11 - [ ] Implement a tranformer in tensorflow from scratch Week 12 - [ ] 8. Bias & ethics in NLP - [ ] bias in word embeddings - [ ] types of bias - [ ] attention economy - [ ] drowning in fraudulent/fake info