{%hackmd 5xqeIJ7VRCGBfLtfMi0_IQ %}

# Find the singular value decomposition

## Problem

Let

$$
A = \begin{bmatrix}
1 & 1 & 1 & 1 \\
1 & 1 & -1 & -1 \\
0 & 0 & 0 & 0
\end{bmatrix}.
$$

Find the singular value decomposition of $A$.

## Thought

The singular value decomposition of $A$ is $A = U\Sigma V\trans$, where $U$ and $V$ are orthogonal matrices, and $\Sigma$ is a matrix of the same order as $A$ with its only nonzero entries on its $1,1$-, $\ldots$, $r,r$-entries as the singular values $\sigma_1 \geq \cdots \geq \sigma_r$.

Recall that the singular values are the square roots of the nonzero eigenvalues of $A\trans A$ (or of $AA\trans$). Also, the columns of $V$ are the eigenvectors of $A\trans A$, and the columns of $U$ are the eigenvectors of $AA\trans$.

## Sample answer

Compute

$$
A\trans A = \begin{bmatrix}
2 & 2 & 0 & 0 \\
2 & 2 & 0 & 0 \\
0 & 0 & 2 & 2 \\
0 & 0 & 2 & 2
\end{bmatrix},
$$

whose eigenvalues are $4,4,0,0$ with eigenvectors

$$
\bv_1 = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 1 \\ 0 \\ 0 \end{bmatrix},\
\bv_2 = \frac{1}{\sqrt{2}}\begin{bmatrix} 0 \\ 0 \\ 1 \\ 1 \end{bmatrix},\
\bv_3 = \frac{1}{\sqrt{2}}\begin{bmatrix} -1 \\ 1 \\ 0 \\ 0 \end{bmatrix},\
\bv_4 = \frac{1}{\sqrt{2}}\begin{bmatrix} 0 \\ 0 \\ -1 \\ 1 \end{bmatrix}.
$$

Thus, we know the singular values are $2,2$, and

$$
\Sigma = \begin{bmatrix}
2 & 0 & 0 & 0 \\
0 & 2 & 0 & 0 \\
0 & 0 & 0 & 0
\end{bmatrix}.
$$

According to the singular decomposition, $\alpha = \{\bv_1, \bv_2, \bv_3, \bv_4\}$ as the columns of $V$ and $\beta = \{\bu_1, \bu_2, \bu_3\}$ as the columns of $U$ are orthonormal bases of $\mathbb{R}^2$ and $\mathbb{R}^3$ such that $[f_A]_\alpha^\beta = \Sigma$. This means we have $A\bv_1 = 2\bu_1$ and $A\bv_2 = 2\bu_2$.

By direct computation, we have

$$
\begin{aligned}
A\bv_1 &= \sqrt{2}\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} = 2 \cdot \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \text{ and } \\
A\bv_2 &= \sqrt{2}\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix} = 2 \cdot \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}.
\end{aligned}
$$

By choosing

$$
\bu_1 = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix},\
\bu_2 = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix},\
\bu_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix},
$$

where $\bu_3$ is chosen as a unit vector orthogonal to $\bu_1$ and $\bu_2$, we have $[f_A]_\alpha^\beta = \Sigma$. Equivalently, $U$ is the matrix whose columns are $\bu_1, \ldots, \bu_3$, and $V$ is the matrix whose columns are $\bv_1, \ldots, \bv_4$. Then $A = U\Sigma V\trans$.

*This note can be found at Course website > Learning resources.*