---
# System prepended metadata

title: Random Constrained Reformulations
tags: [Deep Lagrangian Optimizer]

---

# Random Constrained Reformulations

# Robust optimization
Find the parameters $\theta$ that minimize the loss on a fraction $\alpha$ of the dataset.

$$\min_{\theta, w} \sum w_i \ell_\theta(x_i) \\
s.t. \sum w_i \geq \alpha \\
s.t. w_i\ge0\ \forall i\\
$$

We can enforce positivity of $w_i$ by
$w_i = \text{relu}(\hat w_i)$

# Gaussian Mixture Models
Find the parameters $\theta$ representing a mixture of $N$ gaussians $\mathcal{N}(\mu, \Sigma)$, $\theta_i=(\pi_i, \mu_i, \sigma_i)$

$\max_{\theta} p(X|\theta)=\Pi p(x_i|\theta)=\Pi_i^N \pi_1 \mathcal{N}(x_i|\mu_1,\Sigma_1)+\dots$
$s.t. \sum \pi_i=1$
$s.t. \pi_i\ge0\ \forall i \in {1..N}$

Maximize likelyhood subject to guassian weights $\pi$ are probabilities.

The last set of constraints can become
$\pi_i\ge0 = relu(-\pi_i)=0$
since we don't want to deal with inequalities.

$\Sigma$ should also be positive semi-definite, but we can deal with that later maybe, e.g. with diagonal covariances assumptions.

###### tags: `Deep Lagrangian Optimizer`