---
# System prepended metadata

title: 'VL Machine Learning: Unsupervised Techniques Exam Part 1/2'
tags: [Exam]

---

# VL Machine Learning: Unsupervised Techniques Exam Part 1/2
###### tags: `Exam`

#### Which statements about Unsupervised Machine Learning are true?
* Model selection is called "training" or "learning".
* Generative models should be able to produce the distribution of the observed data.
* In parametric models, every parameter (vector) represents a model. Factor analysis is a parametric model. 
* Examples of unsupervised methods are: PCA, ICA, Factor Analysis, k-means clustering, and mixture models.
* ...

---

#### Which statements about Parameter Estimation and Maximum Likelihood are true?
* The mean squared error can be written as the sum of the variance of the estimator and the squared bias of the estimator.
* If the data are distributed i.i.d., the likelihood can be written as a product of individual probabilities over the data samples.
* An estimator w^ is unbiased if its expected value (over the training set distribution) equals the true parameter: E(w^)=w.
* ...

---

#### Which statements about Expectation Maximization (EM) and Maximum Entropy are correct
* The E-step increases the lower bound on the log-likelihood.
* Maximum Entropy requires minimal prior assumptions to assign a-priori-probabilities.
* The EM algorithm can be applied to hidden Markov models and mixture models.
* The E-step increases the lower bound on the log-likelihood.
* ...

---

![](https://i.imgur.com/lhT2fKh.png)
![](https://i.imgur.com/KzGoUHI.png)
![](https://i.imgur.com/CZOgrmx.png)


---

#### Which statements about Principal Component Analysis (PCA) are correct?
* The sample covariance matrix is symmetric and positive-semidefinite.
* The first principal component corresponds to the largest eigenvalue in the eigendecomposition of the sample covariance matrix. 
* Eigenvectors can be computed iteratively with Oja's rule.
* The matrix of eigenvectors U is an orthogonal matrix.
* The sample variance of the k-th projection is equal to the k-th eigenvalue of the sample covariance matrix.
* ...

---

![](https://i.imgur.com/6RLFjZn.png)



---
#### We stick to the notation from the lecture, i.e. the data points are denoted by $\mathbf{x}_1,...\mathbf{x}_n \in \mathcal{X}=\mathbb{R}^m$ the kernel by $k: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$ and the feature map by  $\Phi : \mathcal{X} \to \mathcal{H}$ The feauture covariance matrix is defined by $\mathbf{C}= \frac1n \sum_{i=1}^n \Phi(\mathbf{x}_i) \Phi(\mathbf{x}_j)^T$ and the Gram matrix $K$ by $K_{ij}=\Phi(\mathbf{x}_i)^T \Phi(\mathbf{x}_j)$ for $i,j=1,...,n$

* To compute the projection of a data point x onto the eigenvectors of K, you need to center x and K, as well as to normalize the eigenvectors of  K
* In general, $\Phi$ is a nonlinear map that maps into a higher dimensional linear feature space $\mathcal{H}$. 
* Linear PCA can be seen as a special case of kernel PCA by using k(x,y)=xTy as the associated kernel function.
* The main task of kernel PCA is to solve the eigenvalue problem of C in feature space. This can be achieved by solving the eigenvalue problem of K in a Euclidean space.
* ...


---

#### Which statements about Independent Component Analysis (ICA) are correct?

* Fast ICA relies on whitening and rotating the data.
* Criteria for independence are mutual information between components and non-Gaussianity of components.
* In general, ICA components are neither ranked nor orthogonal.
* ...

---

![](https://i.imgur.com/txr0Rq5.png)
![](https://i.imgur.com/coYU9l7.png)

---

![](https://i.imgur.com/kWJEtp5.png)
![](https://i.imgur.com/nCoCppn.png)
![](https://i.imgur.com/UccGWMJ.png)

