# Controllers

### Overal Objective:
$$
\sum_{t=1}^T r_t^\top \Sigma_t u - \phi^\top \left(\tau - \sum_{t=1}^T W\Sigma_t e \right)_+
$$

<hr />

### Oracle:
$$
\sum_{t=1}^T r_t^\top \Sigma_t u - \phi^\top \left(\tau - \sum_{t=1}^T W\Sigma_t e \right)_+
$$

<hr />

### Baseline Controller - BPC:

\begin{align*}
\Sigma_t = \arg\max_\Sigma & & r_t^\top \Sigma u \\
\text{s.t.} & & W \Sigma e + s_{t-1} \geq \frac{t}{T} \tau \\
\end{align*}

<span style="color:red">

\begin{align*}
\Sigma_t = \arg\max_\Sigma & & r_t^\top \Sigma u - \mathbb{1}^\top \left(\frac{t}{T}\tau - [W\Sigma e + s_{t-1}] \right)_+ \\
\end{align*}

</span>

<span style="color:blue">

\begin{align*}
\Sigma_t = \arg\max_\Sigma & & [r_t^\top \Sigma u - \mathbb{1} \kappa] \\
\text{s.t.} & & \kappa \geq \frac{t}{T} \tau - [ W \Sigma e + s_{t-1}] \\
& & \kappa \geq 0 ~\text{and}~\kappa \in \mathbb{R}^{k \times 1}~\text{and}~ \mathbb{1} \in \mathbb{R}^{k \times 1}
\end{align*}

</span>

<hr />

### Baseline Controller - BPC-$\phi$:

<span style="color:red">

\begin{align*}
\Sigma_t = \arg\max_\Sigma & & r_t^\top \Sigma u - \phi^\top \left(\frac{t}{T}\tau - [ W\Sigma e +s_{t-1}] \right)_+ \\
\end{align*}

</span>

<span style="color:blue">

\begin{aligned}
\Sigma_t = \arg\max_\Sigma & & \left[ r_t^\top \Sigma e - \phi \kappa \right]\\
\text{s.t.} & & \kappa \geq \frac{t}{T} \tau - [ W\Sigma e + s_{t-1}]\\
& & \kappa \geq 0 ~\text{and}~\kappa \in \mathbb{R}^{k \times 1}\\
\end{aligned}

</span>

<hr />

### Baseline - BPC-$\frac{t}{T}\phi$:

<span style="color:red">

\begin{align*}
\Sigma_t = \arg\max_\Sigma & & r_t^\top \Sigma u - \left(\frac{t}{T} \phi\right)^\top \left(\frac{t}{T}\tau - [ W\Sigma e +s_{t-1}] \right)_+ \\
\end{align*}

</span>

<span style="color:blue">

\begin{aligned}
\Sigma_t = \arg\max_\Sigma & & \left[ r_t^\top \Sigma e - \left(\frac{t}{T} \phi\right)^\top \kappa \right]\\
\text{s.t.} & & \kappa \geq \frac{t}{T} \tau - [ W\Sigma e + s_{t-1}]\\
& & \kappa \geq 0 ~\text{and}~\kappa \in \mathbb{R}^{k \times 1}\\
\end{aligned}

</span>

<hr />

### Cost-Aware Reactive-Controller:

#### Objective:

$$ \min_{0\leq \lambda\leq \phi} \frac{1}{T}\sum_{t=1}^T u(a_t|x_t) - \lambda^T \left(\frac{1}{T}\tau - \frac{1}{T}\sum_{t=1}^T c(a_t|x_t)\right),
$$

#### Iterative Objective:

$$ \Sigma_t = \arg\max_\Sigma r_t^\top \Sigma u + (\lambda_{t-1})^\top W\Sigma e
$$

$$\lambda_t =\color{red}{\Gamma_{[0,\phi]}} \left[ \lambda_{t-1} + \gamma \left(\frac{1}{T}\tau - W\Sigma_t e \right) \right],
$$

</span>

<hr \>

<!--
### Cost-Aware Predictive Controller-$\phi$
#### Objective:

$$
\Sigma_t = \arg\max_\Sigma r_t^\top \Sigma_t u - \phi^T \left(\tau - \left[s_{t-1} + W\Sigma_t e + C(A_t|C_t) \right]\right)_+
$$

$$ \frac{1}{T}\sum_{t=1}^T u(a_t|x_t) - \phi^\top \Big(\tau - \left[s_{t-1} + c(a_t|x_t) + C(A_t|C_t)\right]\Big)_+,
$$

#### Multi-forecast Iterative Solution:
$$
\Sigma_t = \arg\max_\Sigma~ r_t^\top \Sigma u - \frac{1}{B}\sum_{b=1}^B
\phi^\top \left(\tau - [s_{t-1} +W\Sigma e + \hat{C}_t^b] \right)_+
$$

<hr \>
-->

### Cost-Aware Predictive Controller
#### Objective:
<!--
$$
\Sigma_t = \arg\max_\Sigma r_t^\top \Sigma_t u - \phi^T \left(\tau - \left[s_{t-1} + W\Sigma_t e + C(A_t|C_t) \right]\right)_+
$$
-->
$$ \min_{0\leq \lambda\leq \phi} \frac{1}{T}\sum_{t=1}^T u(a_t|x_t) - \lambda^\top \Big(\tau - \left[s_{t-1} + c(a_t|x_t) + C(A_t|C_t)\right]\Big)_+,
$$

#### [A] Offline-Objective:

<!-- <span style="color:red">

\begin{aligned}
\widehat{\boldsymbol{\Sigma}} = \arg\max_{(\Sigma^1, \dots, \Sigma^b) \in \Pi} \Big[ \frac{T}{|N|} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right)r_n^\top \Sigma^n u - \phi^\top \frac{1}{|B|}\sum_{i=1}^{|B|} \Big(\tau - \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right) \Sigma^n e \Big)_+ \Big]\\
\end{aligned}

</span> -->

\begin{aligned}
\widehat{\boldsymbol{\Sigma}} = \arg\max_{(\Sigma^1, \dots, \Sigma^b) \in \Pi} \Big[ \frac{1}{|B|}\sum_{i=1}^{|B|} \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right)r_n^\top \Sigma^n u - \phi^\top \frac{1}{|B|}\sum_{i=1}^{|B|} \Big(\tau - \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right) \Sigma^n e \Big)_+ \Big]\\
\end{aligned}

<!--
\begin{aligned}
\widehat{\boldsymbol{\Sigma}} = \arg\max_{(\Sigma^1, \dots, \Sigma^b) \in \Pi} \Big[ \cdots - \phi^\top \frac{1}{|B|}\sum_{i=1}^{|B|} \Big(\tau - \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right) \Sigma^n e \Big)_+ \Big]\\
\end{aligned}
-->

#### [B] Offline-Objective:

\begin{aligned}
\widehat{\boldsymbol{\Sigma}} = \arg\max_{(\Sigma^1, \dots, \Sigma^b) \in \Pi} \Big[ \frac{1}{|B|}\sum_{i=1}^{|B|} \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right)r_n^\top \Sigma^j u - \phi^\top \frac{1}{|B|}\sum_{i=1}^{|B|} \Big(\tau - \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right) \Sigma^j e \Big)_+ \Big]\\
\end{aligned}

<!--
\begin{aligned}
\widehat{\boldsymbol{\Sigma}} = \arg\max_{(\Sigma^1, \dots, \Sigma^b) \in \Pi} \Big[ \cdots - \phi^\top \frac{1}{|B|}\sum_{i=1}^{|B|} \Big(\tau - \sum_{j=1}^{T} \sum_{n=1}^{|N|} \left( z^i_{n,j} \right) \Sigma^j e \Big)_+ \Big]\\
\end{aligned}
-->

#### Multi-forecast Iterative Solution:
<!--
$$
\Sigma_t = \arg\max_\Sigma~ r_t^\top \Sigma u - \frac{1}{B}\sum_{b=1}^B
\left(\lambda^b_{t-1}
\right)^\top \left(\tau - [s_{t-1} + W\Sigma e + \hat{C}_t^b ]\right)_+
$$
-->

$$
\Sigma_t = \arg\max_\Sigma~ r_t^\top \Sigma u - \frac{1}{B}\sum_{b=1}^B
\left(\lambda^b_{t-1}
\right)^\top W\Sigma e
$$

$$
\lambda_t^b = \Gamma_{[0,\phi]} \left[ \lambda_{t-1}^b + \gamma \left(\tau - [s_{t-1} + c(a_t|x_t) + \hat{C}_t^b] \right) \right],~~b=1,\dots,B.
$$