$$

Find an orthonormal basis of a subspace

Problem

Let

$$A=\left[\begin{array}{cccc}1& 1& 0& 1\\ 0& 0& 1& 1\end{array}\right].$$

Find an orthonormal basis of

$\ker (A)$ and the projection matrix onto

$\ker (A)$.

Thought

We may find a basis by the process in How to solve a system of linear equations?, but the given basis is likely to be not orthogonal. To find an orthonormal basis, we may find a nonzero vector

${\mathbf{u}}_1$ in the subspace first, and then add the condition

${\mathbf{u}}_1^{\mathrm{\top }}\mathbf{x}=0$ to our equations. For example, it is not hard to find one vector in

$\ker (A)$, say, for example,

$(-1,1,0,0{)}^{\mathrm{\top }}$. Now, we are looking for a vector

${\mathbf{u}}_2$ such that

$A{\mathbf{u}}_2=\mathbf{0}$ and

${\mathbf{u}}_1^{\mathrm{\top }}{\mathbf{u}}_2=0$. Equivalently,

${\mathbf{u}}_2$ is a vector in the kernel of

$$\left[\begin{array}{cccc}1& 1& 0& 1\\ 0& 0& 1& 1\\ -1& 1& 0& 0\end{array}\right],$$

which can be easily found.

Once we have an orthogonal basis, one may normalize the lengths to make it orthonormal. Moreover, let

$U$ be the matrix whose columns are composed of the vectors in the orthonormal basis, then

$U(U^{\mathrm{\top }}U{)}^{-1}U$ is the projection matrix. Since the columns of

$U$ are orthonormal,

$U^{\mathrm{\top }}U=I$, and the projection matrix is simply

$U{U}^{\mathrm{\top }}$.

Sample answer

Pick a nonzero vector

$\mathbf{v}\in \ker (A)$, for example,

${\mathbf{u}}_1=(-1,1,0,0{)}^{\mathrm{\top }}$. Then we are looking for a vector

${\mathbf{u}}_2$ such that

$A{\mathbf{u}}_2=\mathbf{0}$ and

${\mathbf{u}}_1^{\mathrm{\top }}{\mathbf{u}}_2=0$. Equivalently,

${\mathbf{u}}_2$ is in the kernel of

$$\left[\begin{array}{cccc}1& 1& 0& 1\\ 0& 0& 1& 1\\ -1& 1& 0& 0\end{array}\right],$$

whose reduced echelon form is

$$\left[\begin{array}{cccc}1& 0& 0& 0.5\\ 0& 1& 0& 0.5\\ 0& 0& 1& 1\end{array}\right].$$

Thus,

${\mathbf{u}}_2$ can be chosen as

$(-1,-1,-2,2{)}^{\mathrm{\top }}$. By replacing

${\mathbf{u}}_i$ with

$\frac{1}{\Vert {\mathbf{u}}_i\Vert }{\mathbf{u}}_i$ for each

$i$, we have

$${\mathbf{u}}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}-1\\ 1\\ 0\\ 0\end{array}\right]\text{ and }{\mathbf{u}}_2=\frac{1}{\sqrt{10}}\left[\begin{array}{c}-1\\ -1\\ -2\\ 2\end{array}\right].$$

Thus,

$\{{\mathbf{u}}_1,{\mathbf{u}}_2\}$ is an orthornormal basis of

$\ker (A)$.

For the projection matrix, we may let

$$U=\left[\begin{array}{cc}|& |\\ {\mathbf{u}}_1& {\mathbf{u}}_2\\ |& |\end{array}\right]=\left[\begin{array}{cc}-\frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{10}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{10}}\\ 0& -\frac{2}{\sqrt{10}}\\ 0& \frac{2}{\sqrt{10}}\end{array}\right].$$

Then

$$P=U(U^{\mathrm{\top }}U{)}^{-1}{U}^{\mathrm{\top }}=U{U}^{\mathrm{\top }}=\left[\begin{array}{cccc}\frac{3}{5}& -\frac{2}{5}& \frac{1}{5}& -\frac{1}{5}\\ -\frac{2}{5}& \frac{3}{5}& \frac{1}{5}& -\frac{1}{5}\\ \frac{1}{5}& \frac{1}{5}& \frac{2}{5}& -\frac{2}{5}\\ -\frac{1}{5}& -\frac{1}{5}& -\frac{2}{5}& \frac{2}{5}\end{array}\right].$$

Note

By choosing a different

${\mathbf{u}}_1$, you might get a different

${\mathbf{u}}_2$. However, the projection matrix remains the same. Give a try and think about why.

In general, every subspace has an orthonormal basis, which can be found by the Gram–Schmidth process.

This note can be found at Course website > Learning resources.