{%hackmd 5xqeIJ7VRCGBfLtfMi0_IQ %}

# Find an orthonormal basis of a subspace

## Problem

Let

$$
A = \begin{bmatrix}
1 & 1 & 0 & 1 \\
0 & 0 & 1 & 1
\end{bmatrix}.
$$

Find an orthonormal basis of $\ker(A)$ and the projection matrix onto $\ker(A)$.

## Thought

We may find a basis by the process in [How to solve a system of linear equations?](https://hackmd.io/@jephianlin/HkM6o-cAh), but the given basis is likely to be not orthogonal. To find an orthonormal basis, we may find a nonzero vector $\bu_1$ in the subspace first, and then add the condition $\bu_1\trans\bx = 0$ to our equations. For example, it is not hard to find one vector in $\ker(A)$, say, for example, $(-1,1,0,0)\trans$. Now, we are looking for a vector $\bu_2$ such that $A\bu_2 = \bzero$ and $\bu_1\trans\bu_2 = 0$. Equivalently, $\bu_2$ is a vector in the kernel of

$$
\begin{bmatrix}
1 & 1 & 0 & 1 \\
0 & 0 & 1 & 1 \\
-1 & 1 & 0 & 0
\end{bmatrix},
$$

which can be easily found.

Once we have an orthogonal basis, one may normalize the lengths to make it orthonormal. Moreover, let $U$ be the matrix whose columns are composed of the vectors in the orthonormal basis, then $U(U\trans U)^{-1} U$ is the projection matrix. Since the columns of $U$ are orthonormal, $U\trans U = I$, and the projection matrix is simply $UU\trans$.

## Sample answer

Pick a nonzero vector $\bv\in\ker(A)$, for example, $\bu_1 = (-1, 1, 0, 0)\trans$. Then we are looking for a vector $\bu_2$ such that $A\bu_2 = \bzero$ and $\bu_1\trans\bu_2 = 0$. Equivalently, $\bu_2$ is in the kernel of

$$
\begin{bmatrix}
1 & 1 & 0 & 1 \\
0 & 0 & 1 & 1 \\
-1 & 1 & 0 & 0
\end{bmatrix},
$$

whose reduced echelon form is

$$
\begin{bmatrix}
1 & 0 & 0 & 0.5 \\
0 & 1 & 0 & 0.5 \\
0 & 0 & 1 & 1 \\
\end{bmatrix}.
$$

Thus, $\bu_2$ can be chosen as $(-1,-1,-2,2)\trans$. By replacing $\bu_i$ with $\frac{1}{\|\bu_i\|}\bu_i$ for each $i$, we have

$$
\bu_1 = \frac{1}{\sqrt{2}}\begin{bmatrix} -1 \\ 1 \\ 0 \\ 0 \end{bmatrix} \text{ and }
\bu_2 = \frac{1}{\sqrt{10}}\begin{bmatrix} -1 \\ -1 \\ -2 \\ 2 \end{bmatrix}.
$$

Thus, $\{\bu_1, \bu_2\}$ is an orthornormal basis of $\ker(A)$.

For the projection matrix, we may let

$$
U = \begin{bmatrix}
| & | \\
\bu_1 & \bu_2 \\
| & |
\end{bmatrix} =
\begin{bmatrix}
-\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{10}} \\
\frac{1}{\sqrt{2}} & -\frac{1}{\sqrt{10}} \\
0 & -\frac{2}{\sqrt{10}} \\
0 & \frac{2}{\sqrt{10}} \\
\end{bmatrix}.
$$

Then

$$
P = U(U\trans U)^{-1}U\trans = UU\trans =
\begin{bmatrix}
\frac{3}{5} & -\frac{2}{5} & \frac{1}{5} & -\frac{1}{5} \\
-\frac{2}{5} & \frac{3}{5} & \frac{1}{5} & -\frac{1}{5} \\
\frac{1}{5} & \frac{1}{5} & \frac{2}{5} & -\frac{2}{5} \\
-\frac{1}{5} & -\frac{1}{5} & -\frac{2}{5} & \frac{2}{5}
\end{bmatrix}.
$$

## Note

By choosing a different $\bu_1$, you might get a different $\bu_2$. However, the projection matrix remains the same. Give a try and think about why.

In general, every subspace has an orthonormal basis, which can be found by the [Gram–Schmidth process](https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process).

*This note can be found at Course website > Learning resources.*