{%hackmd 5xqeIJ7VRCGBfLtfMi0_IQ %}

$\newcommand{\proj}{\operatorname{proj}}$

# Projection

## Problem

Let $V$ be the subspace of $\mathbb{R}^3$ defined by the equation $x + y + z = 0$ and

$$
\bb = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}.
$$

Find $\proj_V(\bb)$, the projection of $\bb$ onto $V$.

## Thought

Let $V$ be a subspace of $\mathbb{R}^n$. We first recall that

$$
V^\perp = \{\bx: \inp{\bx}{\bv} = 0 \text{ for all } \bv\in V\}.
$$

That is $V^\perp$ collects all vectors $\bx$ that is perpendicular to any vector $\bv\in V$. For example, if $V$ is the subspace of $\mathbb{R}^3$ defined by $x + y + z = 0$, which is a plane, then $V^\perp$ is $\vspan\{\bone\}$, which is the straight line defined by the normal vector $\bone$ of $V$. It is intuitive and true that $\dim(V) + \dim(V^\perp) = \dim(\mathbb{R}^n) = n$.

Given a vector $\bb\in\mathbb{R}^n$ and a subspace $V\subseteq\mathbb{R}^n$, it can be written as $\bb = \bw + \bh$ such that $\bw\in V$ and $\bh\in V^\perp$. It is like in physics we often write a vector as the sum of its horizontal and vertical components. Our goal is to find $\bw = \proj_V(\bb)$.

As every subspace has a basis, let $\{\bu_1, \ldots, \bu_d\}$ be a basis of $V$. And let $A$ be the $n\times d$ matrix whose columns are $\bu_1, \ldots, \bu_d$. Let us observe a few properties.

1. Since $\bw\in V$, $\bw = c_1\bu_1 + \cdots + c_d\bu_d$ for some scalars $c_1, \ldots, c_d$. Equivalently, $\bw = A\bc$ for some $\bc = (c_1, \ldots, c_d)\trans$.
2. Since $\bh\in V^\perp$, the fact $\inp{\bh}{\bu_1} = \cdots = \inp{\bh}{\bu_d} = 0$ implies $A\trans\bh = \bzero$.

Now, by pre-multiplying $A\trans$ to $\bb = \bw + \bh$, we have $A\trans\bb = A\trans\bw$. Since $\bw = A\bc$, we have $A\trans\bb = A\trans A\bc$. Consequently, $\bc = (A\trans A)^{-1}A\trans \bb$ and

$$
\bw = A\bc = A(A\trans A)^{-1}A\trans \bb,
$$

which is the projection of $\bb$ onto $V$.

## Sample answer

We may pick

$$
\bu_1 = \begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix} \text{ and }
\bu_1 = \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}
$$

such that $\{\bu_1, \bu_2\}$ is a basis of $V$. (Review [How to solve a system of linear equations?](https://hackmd.io/@jephianlin/HkM6o-cAh) to see how to find $\bu_1$ and $\bu_2$.) Then let

$$
A = \begin{bmatrix}
-1 & -1 \\
1 & 0 \\
0 & 1
\end{bmatrix}
$$

be the matrix whose columns are $\bu_1$ and $\bu_2$.

The rest is the computation. By the projection formula, we have

$$
\begin{aligned}
\proj_V(\bb) &= A(A\trans A)^{-1}A\trans \bb \\
&= \begin{bmatrix}
-1 & -1 \\
1 & 0 \\
0 & 1
\end{bmatrix}
\begin{bmatrix}
2 & 1 \\
1 & 2
\end{bmatrix}^{-1}
\begin{bmatrix}
-1 & 1 & 0 \\
-1 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} \\
&= \begin{bmatrix} \frac{2}{3} \\ -\frac{1}{3} \\ -\frac{1}{3} \end{bmatrix},
\end{aligned}
$$

which is the desired answer. We may double-check that this vector is in $V$ since the sum of its entries is $0$.

*This note can be found at Course website > Learning resources.*