# Probs-Stats II

After having finished Probs-Stats I, we move onto the final half of the Probs-Stats lecture, viz. Probs-Stats II. Focusing sometimes on theoretical concepts, we urge you to refer to these resources for clarity. Some of the resources are not disjoint and cover essentially the same material, such redundant resources have been purposefully added so that you can figure out what works best for you. Please refer to individual sections (in order) to read (or watch) the relevant resources. For any doubts, do not hesitate in contacting a DSG member (these resources were specifically prepared by Shorya Singhal and Sukrit Jindal). While the optional resources are, as the name suggests, optional, we still encourage you to go through them at least once.

### Bayesian Probability
* [Understanding Bayes' Theorem](https://www.yudkowsky.net/rational/bayes)
* [Bayesian vs. Frequentist Statistics](https://medium.com/@roshmitadey/frequentist-v-s-bayesian-statistics-24b959c96880)
* [Bayesian Demonstration](https://seeing-theory.brown.edu/bayesian-inference/index.html) - check out the other sections for frequentist demonstration and other demos
* [Intuition of Bayes' Theorem](https://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/) - optional
* [Video Lecture Series for Bayesian Statistics](https://m.youtube.com/playlist?list=PLwJRxp3blEvZ8AKMXOy0fc0cqT61GsKCG) - optional

### Estimators and Estimation
* [Deep Learning by Ian Goodfellow Chapter 5](https://www.deeplearningbook.org/contents/ml.html) - Sections 5.4-5.6
* [Maximum Likelihood Estimation Video (StatQuest)](https://www.youtube.com/watch?v=XepXtl9YKwc)
* [MLE vs. MAP](https://blog.christianperone.com/2019/01/mle/#)
* [MLE vs. MAP with an example](https://automaticaddison.com/difference-between-maximum-likelihood-and-maximum-a-posteriori-estimation/)
* [Maximum Likelihood Estimation Theory](https://minerva.it.manchester.ac.uk/~saralees/statbook2.pdf) - optional, Chapter 6
* [Probabilistic Modeling and Bayesian Analysis](https://ocw.mit.edu/courses/15-097-prediction-machine-learning-and-statistics-spring-2012/553a0822984b08bc611306c93533a0a3_MIT15_097S12_lec15.pdf) - optional

### Error Generalisation
* [Deep Learning by Ian Goodfellow Chapter 5](https://www.deeplearningbook.org/contents/ml.html) - Section 5.2
* [CS229 Lecture 9](https://www.youtube.com/watch?v=iVOxMcumR4A&list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU&index=9)
* [Introduction to Statistical Learning Theory](https://ocw.mit.edu/courses/15-097-prediction-machine-learning-and-statistics-spring-2012/3f3332b76e8248226fb2285b91cfc6db_MIT15_097S12_lec14.pdf)

### Kullback-Leibler Divergence
* [Kullback-Leibler Divergence](https://www.countbayesie.com/blog/2017/5/9/kullback-leibler-divergence-explained)
* [Intuition for KL Divergence with an example](https://towardsdatascience.com/light-on-math-machine-learning-intuitive-guide-to-understanding-kl-divergence-2b382ca2b2a8)
* [MLE and KL-Divergence](https://agustinus.kristia.de/techblog/2017/01/26/kl-mle/)
* [Forward and Reverse KL Divergence](https://blog.evjang.com/2016/08/variational-bayes.html) - optional, covers mean-field approximation as well
* [Deep Learning by Ian Goodfellow Chapter 3](https://www.deeplearningbook.org/contents/prob.html) - Section 3.13 - optional