---
# System prepended metadata

title: Bayesian Machine Learning
tags: [technical, shared]

---

# Bayesian Machine Learning
###### tags: `shared` `technical`

Introduction to BML
---
+ Bayesian generative models
+ Probablistic graphical models (PGM)
+ Integration with deep learning: Bayesian deep learning (BDL)

Advantages
---
+ Integration of perception task and inference task
+ Automatic model comparison/selection
+ Incorporating priors
+ Avoid overfitting if less data available
+ Principled way to solve complex tasks
+ Often good performance

Disadvantages
---
+ Computationally demanding
+ Subjective
+ Need priors

Architecture
---
+ Perception component and task-specific component
+ Three uncertainties to handle
  - Neural network paramters
  - Task-specific parameters
  - Exchange of information between two components
+ A simple example: Multilayer perceptron
  - $$ \min_{\{W_l\},\{b_l\}} \Vert X_L - Y \Vert_F + \lambda \sum_l \Vert W_l \Vert_F^2 $$ $$s.t. X_l = \sigma (X_{t-1}W_l + b_l), l=1, \dots, L $$
  - Certain assumptions on distribution of variables
  - Various way to train the network

Inference
---
+ An algorithm to figure out the "optimal" parameters
  - Expectation Maximization (EM)
  - Variational Bayesian (VB)
  - Sampling methods (Gibb's, CD-k, SML)
+ Often used: EM & VB
  - Some notes from sklearn document
    ![](https://i.imgur.com/ZoUYqf6.png)
  - EM is commonly used to train GMM/HMM
  - Variational Bayesian concentrates on maximizing EBLO and thus minimizing the KL divergence

Reference
---
+ Toward Bayesian Deep Learning: A Survey
  - Formulation of BDL
  - Interesting discussion on RS and topic models
{%pdf https://arxiv.org/pdf/1604.01662.pdf %}

+ Bayesian Machine Learning
  - Application on EEG/ECG signal analysis
  - Rich mathematical contents on inference methods
  - http://ieeexplore.ieee.org/document/7366707/
