$$

$$

Projection

Problem

Let

$V$ be the subspace of

${\mathbb{R}}^3$ defined by the equation

$x+y+z=0$ and

$$\mathbf{b}=\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right].$$

Find

${\mathrm{proj}}_V(\mathbf{b})$, the projection of

$\mathbf{b}$ onto

$V$.

Thought

Let

$V$ be a subspace of

${\mathbb{R}}^n$. We first recall that

$$V^{\perp }=\{\mathbf{x}:⟨\mathbf{x},\mathbf{v}⟩=0\text{ for all }\mathbf{v}\in V\}.$$

That is

$V^{\perp }$ collects all vectors

$\mathbf{x}$ that is perpendicular to any vector

$\mathbf{v}\in V$. For example, if

$V$ is the subspace of

${\mathbb{R}}^3$ defined by

$x+y+z=0$, which is a plane, then

$V^{\perp }$ is

$\mathrm{span}\{\mathbf{1}\}$, which is the straight line defined by the normal vector

$\mathbf{1}$ of

$V$. It is intuitive and true that

$\dim (V)+\dim (V^{\perp })=\dim ({\mathbb{R}}^n)=n$.

Given a vector

$\mathbf{b}\in {\mathbb{R}}^n$ and a subspace

$V\subseteq {\mathbb{R}}^n$, it can be written as

$\mathbf{b}=\mathbf{w}+\mathbf{h}$ such that

$\mathbf{w}\in V$ and

$\mathbf{h}\in V^{\perp }$. It is like in physics we often write a vector as the sum of its horizontal and vertical components. Our goal is to find

$\mathbf{w}={\mathrm{proj}}_V(\mathbf{b})$.

As every subspace has a basis, let

$\{{\mathbf{u}}_1,\dots ,{\mathbf{u}}_d\}$ be a basis of

$V$. And let

$A$ be the

$n\times d$ matrix whose columns are

${\mathbf{u}}_1,\dots ,{\mathbf{u}}_d$. Let us observe a few properties.

1. Since
   $\mathbf{w}\in V$,
   $\mathbf{w}=c_1{\mathbf{u}}_1+\cdots +c_d{\mathbf{u}}_d$ for some scalars
   $c_1,\dots ,c_d$. Equivalently,
   $\mathbf{w}=A\mathbf{c}$ for some
   $\mathbf{c}=(c_1,\dots ,c_d{)}^{\mathrm{\top }}$.
2. Since
   $\mathbf{h}\in V^{\perp }$, the fact
   $⟨\mathbf{h},{\mathbf{u}}_1⟩=\cdots =⟨\mathbf{h},{\mathbf{u}}_d⟩=0$ implies
   $A^{\mathrm{\top }}\mathbf{h}=\mathbf{0}$.

Now, by pre-multiplying

$A^{\mathrm{\top }}$ to

$\mathbf{b}=\mathbf{w}+\mathbf{h}$, we have

$A^{\mathrm{\top }}\mathbf{b}=A^{\mathrm{\top }}\mathbf{w}$. Since

$\mathbf{w}=A\mathbf{c}$, we have

$A^{\mathrm{\top }}\mathbf{b}=A^{\mathrm{\top }}A\mathbf{c}$. Consequently,

$\mathbf{c}=(A^{\mathrm{\top }}A{)}^{-1}{A}^{\mathrm{\top }}\mathbf{b}$ and

$$\mathbf{w}=A\mathbf{c}=A(A^{\mathrm{\top }}A{)}^{-1}{A}^{\mathrm{\top }}\mathbf{b},$$

which is the projection of

$\mathbf{b}$ onto

$V$.

Sample answer

We may pick

$${\mathbf{u}}_1=\left[\begin{array}{c}-1\\ 1\\ 0\end{array}\right]\text{ and }{\mathbf{u}}_1=\left[\begin{array}{c}-1\\ 0\\ 1\end{array}\right]$$

such that

$\{{\mathbf{u}}_1,{\mathbf{u}}_2\}$ is a basis of

$V$. (Review How to solve a system of linear equations? to see how to find

${\mathbf{u}}_1$ and

${\mathbf{u}}_2$.) Then let

$$A=\left[\begin{array}{cc}-1& -1\\ 1& 0\\ 0& 1\end{array}\right]$$

be the matrix whose columns are

${\mathbf{u}}_1$ and

${\mathbf{u}}_2$.

The rest is the computation. By the projection formula, we have

$$\begin{array}{rl}{\mathrm{proj}}_V(\mathbf{b})& =A(A^{\mathrm{\top }}A{)}^{-1}{A}^{\mathrm{\top }}\mathbf{b}\\ & =\left[\begin{array}{cc}-1& -1\\ 1& 0\\ 0& 1\end{array}\right]{\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]}^{-1}\left[\begin{array}{ccc}-1& 1& 0\\ -1& 0& 1\end{array}\right]\left[\begin{array}{c}1\\ 0\\ 0\end{array}\right]\\ & =\left[\begin{array}{c}\frac{2}{3}\\ -\frac{1}{3}\\ -\frac{1}{3}\end{array}\right],\end{array}$$

which is the desired answer. We may double-check that this vector is in

$V$ since the sum of its entries is

$0$.

This note can be found at Course website > Learning resources.