---
title: Ch6-5
tags: Linear algebra
GA: G-77TT93X4N1
---

# Chapter 6 extra note 5

> Schmidt decomposition
> singular value decomposition
> image of unit ball in a linear transformation

## Selected lecture notes

Let $T\in\mathcal{L}(V, W)$, where $V$ and $W$ are inner product spaces we can define the following operators:
* $T^*\in\mathcal{L}(W, V)$;
* $T^*T\in\mathcal{L}(V)$;
* $|T|=\sqrt{T^*T}\in\mathcal{L}(V)$;
* $TT^*\in\mathcal{L}(W)$;
* $|T^*|=\sqrt{TT^*}\in\mathcal{L}(W)$;
* $T^\dagger\in\mathcal{L}(W, V)$.

### Reduced SVD

Since $T^*T\ge 0$, we can find $\{{\bf v}_1, \cdots, {\bf v}_n\}\subset V$ and $\lambda_1\ge \cdots \ge\lambda_n\ge 0$ such that
$$
\tag{1}
T^*T{\bf v}_i = \lambda_i{\bf v}_i.
$$
Define $r$ be such that $\lambda_r>0$ and $\lambda_{r+1}=0$, then the singular values of $T$ are $\{\sigma_i\}^r_{i=1}$ where $\sigma_i=\sqrt{\lambda_i}$. We then have
$$
\tag{2}
T^*T{\bf v}_i = \sigma^2_i{\bf v}_i.
$$
Let us also define
$$
\tag{3}
{\bf w}_i = \frac{1}{\sigma_i}T({\bf v}_i).
$$
The reduced singular value decomposition (reduced SVD) of $T$ is then given by
$$
\tag{4}
T{\bf v} = \sum^r_{i=1}\sigma_i\langle{\bf v}, {\bf v}_i\rangle{\bf w}_i.
$$

**Remark:**
$$
\tag{5}
\begin{align}
TT^*{\bf w}_i &= TT^*\left(\frac{1}{\sigma_i}T({\bf v}_i)\right) \\
&= \frac{1}{\sigma_i}T\left(T^*T{\bf v}_i\right) \\
&= \frac{1}{\sigma_i}T\left(\sigma^2_i{\bf v}_i\right) \\
&=\sigma_i T{\bf v}_i = \sigma^2_i{\bf w}_i.
\end{align}
$$

### Reduced SVD 2

Since $TT^*\ge 0$, we can find $\{{\bf w}_1, \cdots, {\bf w}_m\}\subset W$ and $\lambda_1\ge \cdots \ge\lambda_m\ge 0$ such that
$$
\tag{6}
TT^*{\bf w}_i = \lambda_i{\bf w}_i.
$$
Define $r$ be such that $\lambda_r>0$ and $\lambda_{r+1}=0$, then the singular values of $T$ are $\{\sigma_i\}^r_{i=1}$ where $\sigma_i=\sqrt{\lambda_i}$. We then have
$$
\tag{7}
TT^*{\bf w}_i = \sigma^2_i{\bf w}_i.
$$
Let us also define
$$
\tag{8}
{\bf v}_i = \frac{1}{\sigma_i}T^*({\bf w}_i).
$$
The reduced singular value decomposition (reduced SVD) of $T$ is then given by
$$
T{\bf v} = \sum^r_{i=1}\sigma_i\langle{\bf v}, {\bf v}_i\rangle{\bf w}_i.
$$

**Remark:**
One can either solve the eigenvalue problem for $T^*T$ or $TT^*$ to obtain the singular values and SVD.

---

### The adjoint transformation

$$
\tag{9}
\begin{align}
\langle {\bf v}, T^*{\bf w}\rangle &= \langle T{\bf v}, {\bf w}\rangle\\
&= \langle \sum^r_{i=1}\sigma_i\langle{\bf v}, {\bf v}_i\rangle{\bf w}_i, {\bf w}\rangle\\
&= \sum^r_{i=1}\sigma_i\langle{\bf v}, {\bf v}_i\rangle\langle{\bf w}_i, {\bf w}\rangle\\
&= \sum^r_{i=1}\sigma_i\langle{\bf w}_i, {\bf w}\rangle\langle{\bf v}, {\bf v}_i\rangle\\
&= \langle {\bf v}, \sum^r_{i=1}\sigma_i\overline{\langle{\bf w}_i, {\bf w}\rangle}{\bf v}_i\rangle\\
&= \langle {\bf v}, \sum^r_{i=1}\sigma_i\langle{\bf w}, {\bf w}_i\rangle{\bf v}_i\rangle.
\end{align}
$$
Therefore, the reduced SVD of $T^*$ is given by
$$
\tag{10}
T^*{\bf w} = \sum^r_{i=1}\sigma_i\langle{\bf w}, {\bf w}_i\rangle{\bf v}_i.
$$

---

### Pseudoinverse

The pseudoinverse of $T$ is given by
$$
\tag{11}
T^\dagger{\bf w} = \sum^r_{i=1}\frac{1}{\sigma_i}\langle{\bf w}, {\bf w}_i\rangle{\bf v}_i.
$$

---

### Image of the unit ball

Define an unit ball in $V$ as
$$
\tag{12}
B_1 = \{{\bf v}\in V, \quad \|{\bf v}\|\le 1.\}
$$
Given ${\bf v}\in B_1$, we have
$$
\tag{13}
{\bf v} = \sum^n_{i=1}\langle{\bf v}, {\bf v}_i\rangle{\bf v}_i, \quad
\|{\bf v}\| = \sum^n_{i=1}\langle{\bf v}, {\bf v}_i\rangle^2\le 1.
$$
Based on SVD, we know that
$$
T{\bf v} = \sum^r_{i=1}\sigma_i\langle{\bf v}, {\bf v}_i\rangle{\bf w}_i\in W.
$$
Let us denote the coordinate of $T{\bf v}$ in the basis $\{{\bf w}_1, \cdots, {\bf w}_m\}$ as $(y_1, \cdots, y_m)$, then we have
$$
\tag{14}
y_i = \begin{cases}
\sigma_i\langle{\bf v}, {\bf v}_i\rangle, & 1\le i\le r \\
{\bf 0}, & r<i\le m.
\end{cases}
$$
Therefore
$$
\tag{15}
\frac{y^2_1}{\sigma^2_1} + \cdots + \frac{y^2_r}{\sigma^2_r} = \sum^r_{i=1}\langle{\bf v}, {\bf v}_i\rangle^2\le 1,
$$
that is, $T{\bf v}$ lies inside the ellipse
$$
\tag{16}
E_1 = \left\{{\bf w}=(y_1, \cdots, y_m)\in W, \quad \frac{y_1}{\sigma^2_1} + \cdots + \frac{y_r}{\sigma^2_r}\le 1.\right\}
$$