# Deep Learning Congratulations! You have sucessfully (hopefully) completed Maths and Basic Machine Learning. Now you get to dive into Deep Learning. The stuff thats actually all the rage ! ## Basic #### Introduction If you know nothing about **Neural Networks** start from this small and intutive playlist: * [3B1B Neural Networks Playlist](https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi) - Entire playlist is gold but you can start with just the first 4 videos initially and cover rest when you reach L12-13 in UMich DL playlist. #### PyTorch Use of frameworks are ubiquitous and essential in modern implementation of Deep Learning which makes it essential to familiarize yourselves with frameworks like PyTorch, TensorFlow and Jax. **We focus on PyTorch here and require all assignments to be in PyTorch only.** Learning other frameworks later is on your own discretion. * [PyTorch Documentation](https://pytorch.org/docs/stable/index.html)- The best resource on PyTorch ! Obviously we dont expect you to read the whole documentation initially. You will learn most stuff while referencing it while you code. * [Youtube Playlist in Documentation](https://pytorch.org/tutorials/beginner/introyt.html)- To get started watch this playlist to give initial feel of PyTorch. Also have a look at [PyTorch Recipies](https://pytorch.org/tutorials/recipes/recipes_index.html) * [Another Good Playlist](https://www.youtube.com/playlist?list=PLqnslRFeH2UrcDBWF5mfPGpqQDSta6VK4)- Some people like this playlist for PyTorch better. Incase you are that person we got you covered ! However we don't recommend watching all of this one go interleave this with theory otherwise it might get boring. **Note:** Do either of the two playlists. Don't waste your time doing both. Actual practice would be better use of your time #### Theory Now comes actual Deep Learning theory! **Recommended Pace:** 2 lectures everday ! * [Deep Learning - UMich](https://www.youtube.com/playlist?list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r)- You need to cover 1st 13 lectures to cover Deep Learning Basics * L1-6: NN Basics like optimization, backpropagation, etc * L7-8: CNNs and vision architectures * L9: Pretty nice lecture on GPUs, you can skip it initially in case of time shortage but make sure you cover it later ! * L10-11: Training NNs: Very Important for practical knowledge! Seems trivial but probably the most useful stuff. * L12-13: RNNs, LSTMs, GRUs and Transformers! * [The Illustrated Transformer:](https://jalammar.github.io/illustrated-transformer/) Read this excellent blog to understrand how transformers work in a visual way. ## Advanced Once you get done with the basics you need to choose a field of Deep learning to dive even DEEPER ! These resources are there to just get you started in the respective fields. We recommend starting with either **Vision or NLP**. * **How to Read Papers**: [This awesome lecture by Andrew NG](https://www.youtube.com/watch?v=733m6qBH-jI&t=1057s) is the perfect guide on paper reading and how to shape your future in Machine Learning. * **Computer Vision**: L14-20 of [Deep Learning - UMich](https://www.youtube.com/playlist?list=PL5-TkQAfAZFbzxjBHtzdVCWE0Zbhomg7r) course are excellent to get started with CV. * **Natural Language Processing:** [CS224N](https://www.youtube.com/playlist?list=PLoROMvodv4rMFqRtEuo6SGjY4XbRIVRd4) of Stanford is one of the best courses there to get started with NLP * **Reinforcement Learning**: [C285 - UC Berkeley](https://www.youtube.com/playlist?list=PL_iWQOsE6TfVYGEGiAOMaOzzv41Jfm_Ps) Comprehensive coruse on Deep Learning * **AI Security:** [Clever Hans Blogs](https://www.cleverhans.io/) are a good starting point. ## Other Important Resources **Note**: These are not mutually exculusive among itself or with the above resources. Refer them as per need. * [Goodfellow](https://www.deeplearningbook.org/): The OG Deep learning Book * [Nielson](http://neuralnetworksanddeeplearning.com/): Short and Intuitive book for beginners. * [D2L-Book](https://d2l.ai/): Excellent resource that balances theory and implementation. * [CS182](https://www.youtube.com/playlist?list=PL_iWQOsE6TfVmKkQHucjPAoRtIJYt8a5A): Deep learning by UC Berkeley * [Deepmind](https://www.youtube.com/playlist?list=PLqYmG7hTraZCDxZ44o4p3N5Anz3lLRVZF): Deepmind's lecture series on Deep learning * [CS 230](https://www.youtube.com/playlist?list=PLoROMvodv4rOABXSygHTsbvUz4G_YQhOb): Andrew NG's 2018 Stanford course on Deep Leanring * [CS 25 Transformers United](https://www.youtube.com/playlist?list=PLoROMvodv4rNiJRchCzutFw5ItR_Z27CM)- Great videos of transformers from industry experts !