# generative model

$$
Z \sim P_s
$$

$$
X = A Z
$$

$$
P_s(Z) = \prod_i P_s(Z_i)
$$

$$
W = A^{-1}
$$

## Maximum likelihood

$$
\log p(x) = \log p_s(Wx) - \log \vert W \vert
$$

$$
\log p(x) = -\|Wx\|_1 + \log \vert W \vert
$$
## PCA

$$
P_s = \mathcal{N}(0, I)
$$

### Rotation invariance

$R$ is a rotation, then

$$
\log \vert RW \vert = \log \vert W \vert
$$

and

$$
\log p_s(RWx) = \log p_s(Wx)
$$

$$
\log p_W(x) = \log p_{RW}(x)
$$

PCA: additional constraint

$$
W = U^T S V
$$

PCA "canonical" rotation:

$$
W^* = UU^T S V
$$

### ICA

$P_s$ non-Gaussian