$$

Find the singular value decomposition

Problem

Let

$$A=\left[\begin{array}{cccc}1& 1& 1& 1\\ 1& 1& -1& -1\\ 0& 0& 0& 0\end{array}\right].$$

Find the singular value decomposition of

$A$.

Thought

The singular value decomposition of

$A$ is

$A=U\mathrm{\Sigma }{V}^{\mathrm{\top }}$, where

$U$ and

$V$ are orthogonal matrices, and

$\mathrm{\Sigma }$ is a matrix of the same order as

$A$ with its only nonzero entries on its

$1,1$-,

$\dots$,

$r,r$-entries as the singular values

${\sigma }_1\ge \cdots \ge {\sigma }_r$.

Recall that the singular values are the square roots of the nonzero eigenvalues of

$A^{\mathrm{\top }}A$ (or of

$A{A}^{\mathrm{\top }}$). Also, the columns of

$V$ are the eigenvectors of

$A^{\mathrm{\top }}A$, and the columns of

$U$ are the eigenvectors of

$A{A}^{\mathrm{\top }}$.

Sample answer

Compute

$$A^{\mathrm{\top }}A=\left[\begin{array}{cccc}2& 2& 0& 0\\ 2& 2& 0& 0\\ 0& 0& 2& 2\\ 0& 0& 2& 2\end{array}\right],$$

whose eigenvalues are

$4,4,0,0$ with eigenvectors

$${\mathbf{v}}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\\ 0\end{array}\right],{\mathbf{v}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}0\\ 0\\ 1\\ 1\end{array}\right],{\mathbf{v}}_3=\frac{1}{\sqrt{2}}\left[\begin{array}{c}-1\\ 1\\ 0\\ 0\end{array}\right],{\mathbf{v}}_4=\frac{1}{\sqrt{2}}\left[\begin{array}{c}0\\ 0\\ -1\\ 1\end{array}\right].$$

Thus, we know the singular values are

$2,2$, and

$$\mathrm{\Sigma }=\left[\begin{array}{cccc}2& 0& 0& 0\\ 0& 2& 0& 0\\ 0& 0& 0& 0\end{array}\right].$$

According to the singular decomposition,

$\alpha =\{{\mathbf{v}}_1,{\mathbf{v}}_2,{\mathbf{v}}_3,{\mathbf{v}}_4\}$ as the columns of

$V$ and

$\beta =\{{\mathbf{u}}_1,{\mathbf{u}}_2,{\mathbf{u}}_3\}$ as the columns of

$U$ are orthonormal bases of

${\mathbb{R}}^2$ and

${\mathbb{R}}^3$ such that

$[f_A{]}_{\alpha }^{\beta }=\mathrm{\Sigma }$. This means we have

$A{\mathbf{v}}_1=2{\mathbf{u}}_1$ and

$A{\mathbf{v}}_2=2{\mathbf{u}}_2$.

By direct computation, we have

$$\begin{array}{rl}A{\mathbf{v}}_1& =\sqrt{2}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]=2\cdot \frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]\text{ and }\\ A{\mathbf{v}}_2& =\sqrt{2}\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right]=2\cdot \frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right].\end{array}$$

By choosing

$${\mathbf{u}}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],{\mathbf{u}}_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}1\\ -1\\ 0\end{array}\right],{\mathbf{u}}_3=\left[\begin{array}{c}0\\ 0\\ 1\end{array}\right],$$

where

${\mathbf{u}}_3$ is chosen as a unit vector orthogonal to

${\mathbf{u}}_1$ and

${\mathbf{u}}_2$, we have

$[f_A{]}_{\alpha }^{\beta }=\mathrm{\Sigma }$. Equivalently,

$U$ is the matrix whose columns are

${\mathbf{u}}_1,\dots ,{\mathbf{u}}_3$, and

$V$ is the matrix whose columns are

${\mathbf{v}}_1,\dots ,{\mathbf{v}}_4$. Then

$A=U\mathrm{\Sigma }{V}^{\mathrm{\top }}$.

This note can be found at Course website > Learning resources.