---
# System prepended metadata

title: 'Brief updates (April 30, 2021)'

---

# Brief updates (April 30, 2021)

Let $w^*, w \in \mathbb{R}^p$, and $||w^*||=||w|| = 1$. Also let $X \in \mathbb{R}^{n \times p}$ where each column $X_{.i}$ is a feature with a variace of $\sigma_i ^2$.
The teacher is defined as:
$$
y^* = X \gamma w^* + \epsilon \quad \text{where}, \quad \epsilon \sim \mathcal{N}(0, \sigma_{\epsilon}^2),\ \text{and} \ \gamma \in \mathbb{R}^p \ \text{with} \ \gamma_i = \frac{1}{\sigma_i}.
$$
The student is defined as:
$$
y = Xw.
$$
Let's use the following change of variables:
$$
h^* := X \gamma w^* \quad \text{and} \quad h := Xw.
$$
We have: (change of notation)
$$
Q^* := <h^*, h^*> = 1, \\
H := <y^*, h^*> = Q^* = 1,\\
G := <y^*, y^*> = 1 + \sigma_{\epsilon} ^ 2,\\
Q := <h, h> = \frac{1}{p}\sum w_i^2 \gamma_i^2, \\
R := <h^*, h> = \frac{1}{p}\sum w^*_i w_i \gamma_i.
$$

The following equations still holds and I have validated it experimentally:
$$
E_G = \frac{1}{2} (G - 2R + Q).
$$
![](https://i.imgur.com/zomC8Nc.png)

On the plot on the right, the values of $R$ and $Q$ and  are computed experimentally. However, the ultimate goal is to derive analytical expresstions for $R(\alpha, t)$ and $Q(\alpha, t)$ as functions of $\alpha$ and time or equivalently a function of $\lambda$, the regularization coefficient.

Right now, using the following equations and varying $a$ from $\infty$ to $a^{exh}$ leads to the follwing $r-q$ phase diagram:
![](https://i.imgur.com/atQPzji.png)

(x-axis is $r$ and y-axis is $q$):
![](https://i.imgur.com/YXmcEPc.png)

However, the actual emperical $r-q$ plane is the following:
![](https://i.imgur.com/I5m0pfa.png)


