# Bayesian Machine Learning
###### tags: `shared` `technical`

Introduction to BML
---
+ Bayesian generative models
+ Probablistic graphical models (PGM)
+ Integration with deep learning: Bayesian deep learning (BDL)

Advantages
---
+ Integration of perception task and inference task
+ Automatic model comparison/selection
+ Incorporating priors
+ Avoid overfitting if less data available
+ Principled way to solve complex tasks
+ Often good performance

Disadvantages
---
+ Computationally demanding
+ Subjective
+ Need priors

Architecture
---
+ Perception component and task-specific component
+ Three uncertainties to handle
- Neural network paramters
- Task-specific parameters
- Exchange of information between two components
+ A simple example: Multilayer perceptron
- $$ \min_{\{W_l\},\{b_l\}} \Vert X_L - Y \Vert_F + \lambda \sum_l \Vert W_l \Vert_F^2 $$ $$s.t. X_l = \sigma (X_{t-1}W_l + b_l), l=1, \dots, L $$
- Certain assumptions on distribution of variables
- Various way to train the network

Inference
---
+ An algorithm to figure out the "optimal" parameters
- Expectation Maximization (EM)
- Variational Bayesian (VB)
- Sampling methods (Gibb's, CD-k, SML)
+ Often used: EM & VB
- Some notes from sklearn document
- EM is commonly used to train GMM/HMM
- Variational Bayesian concentrates on maximizing EBLO and thus minimizing the KL divergence

Reference
---
+ Toward Bayesian Deep Learning: A Survey
- Formulation of BDL
- Interesting discussion on RS and topic models
{%pdf https://arxiv.org/pdf/1604.01662.pdf %}

+ Bayesian Machine Learning
- Application on EEG/ECG signal analysis
- Rich mathematical contents on inference methods
- http://ieeexplore.ieee.org/document/7366707/