Chapter 5 extra note 4

Selected lecture notes

orthogonal complement
direct sum
linear functional
Riesz Representation Theorem
adjoint linear transformation

Orthogonal complement

Definition:
For a vector subspace

$E$, its orthogonal complement

$E^{\perp }=\{\mathbf{v}\in V|\mathbf{v}\perp E\}$.

Proposition:

$E^{\perp }$ is a vector subspace.

• Proof:

  Let

  ${\mathbf{v}}_1,{\mathbf{v}}_2\in E^{\perp }$, fix

  $\alpha ,\beta \in F$.

  claim:

  ${\alpha }_1{\mathbf{v}}_1+{\alpha }_2{\mathbf{v}}_2\in E^{\perp }$:

  Given

  $\mathbf{u}\in E$, since

  ${\mathbf{v}}_1,{\mathbf{v}}_2\in E^{\perp }$, we have
  

  $$⟨{\mathbf{v}}_1,\mathbf{u}⟩=⟨{\mathbf{v}}_2,\mathbf{u}⟩=0.$$
  So
  

  $$0={\alpha }_1⟨{\mathbf{v}}_1,\mathbf{u}⟩+{\alpha }_2⟨{\mathbf{v}}_2,\mathbf{u}⟩=⟨{\alpha }_1{\mathbf{v}}_1+{\alpha }_2{\mathbf{v}}_2,\mathbf{u}⟩,$$
  that is,

  $({\alpha }_1{\mathbf{v}}_1+{\alpha }_2{\mathbf{v}}_2)\perp \mathbf{u}$.
  Therefore,

  $({\alpha }_1{\mathbf{v}}_1+{\alpha }_2{\mathbf{v}}_2)\perp \mathbf{u}$,

  $\mathrm{\forall }\mathbf{u}\in E$.

Proposition:

$E\cap E^{\perp }=\{\mathbf{0}\}$.

• Proof:

  At first, we show that

  $E\cap E^{\perp }$ is not an empty set, and it contains at least one element which is the zero vector,

  $\mathbf{0}$.

  Since

  $E$ and

  $E^{\perp }$ are both vector subspaces, so

  $\mathbf{0}\in E$ and

  $\mathbf{0}\in E^{\perp }$, and we have

  $\mathbf{0}\in (E\cap E^{\perp })$.

  Secondly, we show that

  $\mathbf{0}$ is the only element.

  Let

  $\mathbf{v}\in E\cap E^{\perp }$. We have

  $\mathbf{v}\in E$ and

  $\mathbf{v}\in E^{\perp }$, so
  

  $$0=⟨\mathbf{v},\mathbf{v}⟩=\Vert \mathbf{v}{\Vert }^2.$$
  Hence

  $\mathbf{v}=\mathbf{0}$.

Proposition:
Let

$E\subset V$, for any

$\mathbf{v}\in V$, there exists an uniqe decomposition

$$\mathbf{v}={\mathbf{v}}_1+{\mathbf{v}}_2,{\mathbf{v}}_1\in E,{\mathbf{v}}_2\in E^{\perp }.$$

• Proof:

  (Existence)
  Given

  $\mathbf{v}\in V$, we define

  ${\mathbf{v}}_1=P_E\mathbf{v}\in E$, and

  ${\mathbf{v}}_2=\mathbf{v}-{\mathbf{v}}_1$.
  By the definition of orthogonal projection we have

  ${\mathbf{v}}_2\in E^{\perp }$.

  (Uniqueness)
  Assume
  

  $$\mathbf{v}={\mathbf{v}}_1+{\mathbf{v}}_2={\mathbf{w}}_1+{\mathbf{w}}_2,$$
  where

  ${\mathbf{v}}_1,{\mathbf{w}}_1\in E$ and

  ${\mathbf{v}}_2,{\mathbf{w}}_2\in E^{\perp }$.
  We define

  $\mathbf{u}={\mathbf{v}}_1-{\mathbf{w}}_1$. Since

  ${\mathbf{v}}_1,{\mathbf{w}}_1\in E$ we must have

  $\mathbf{u}\in E$. In addition, we find
  

  $$\mathbf{u}={\mathbf{v}}_1-{\mathbf{w}}_1=(\mathbf{v}-{\mathbf{v}}_2)-(\mathbf{v}-{\mathbf{w}}_2)={\mathbf{w}}_2-{\mathbf{v}}_2.$$
  Since

  ${\mathbf{v}}_2,{\mathbf{w}}_2\in E^{\perp }$ we must have

  $\mathbf{u}\in E^{\perp }$. As a result,

  $\mathbf{u}\in (E\cap E^{\perp })=\mathbf{0}$. We must have

  $\mathbf{u}=\mathbf{0}$, and

  ${\mathbf{v}}_1={\mathbf{w}}_1$,

  ${\mathbf{v}}_2={\mathbf{w}}_2$.

Proposition:
Let

$E\subset V$, for any

$\mathbf{v}\in V$,

$$P_{E^{\perp }}\mathbf{v}=\mathbf{v}-P_E\mathbf{v}.$$

• Proof:

  Let

  ${\mathbf{v}}_2=\mathbf{v}-P_E\mathbf{v}$, then

  ${\mathbf{v}}_2\perp E$, so we have
  (1)

  ${\mathbf{v}}_2\in E^{\perp }$.
  Besides,

  $\mathbf{v}-{\mathbf{v}}_2=P_E\mathbf{v}\perp E^{\perp }$. So we obtain
  (2)

  $(\mathbf{v}-{\mathbf{v}}_2)\perp E^{\perp }$.
  By (1), (2) and the definition of orthogonal projection, we have
  

  $$P_{E^{\perp }}\mathbf{v}=\mathbf{v}-P_E\mathbf{v}.$$

Definition:
Let

$W_1$ and

$W_2$ be vector subspaces, we define

$W_1+W_2$ be a set satisfies

$$W_1+W_2=\{\mathbf{v}|\mathbf{v}={\mathbf{w}}_1+{\mathbf{w}}_2,{\mathbf{w}}_1\in W_1,{\mathbf{w}}_2\in W_2\}.$$

Proposition:

$W_1+W_2$ is a vector subspace.

Proposition:
A vector space

$V$ is called a direct sum of subspaces

$W_1$ and

$W_2$ if

$W_1\cap W_2=\{\mathbf{0}\}$ and

$V=W_1+W_2$. We denote it by

$V=W_1\oplus W_2$.

Proposition:

$V=E\oplus E^{\perp }$.

Proposition:

${\left(E^{\perp }\right)}^{\perp }=E$.

• Proof:

  Let

  $\mathbf{u}\in E$, then

  $⟨\mathbf{u},\mathbf{v}⟩=0$, for all

  $\mathbf{v}\in E^{\perp }$.
  Since

  $⟨\mathbf{u},\mathbf{v}⟩=0$, for all

  $\mathbf{v}\in E^{\perp }$, so

  $\mathbf{u}\in {\left(E^{\perp }\right)}^{\perp }$.
  Therefore,

  $E\subseteq {\left(E^{\perp }\right)}^{\perp }$.

  Let

  $\mathbf{u}\in {\left(E^{\perp }\right)}^{\perp }\subseteq V$, then

  $\mathbf{u}=\mathbf{v}+\mathbf{w}$ where

  $\mathbf{v}\in E$ and

  $\mathbf{w}\in E^{\perp }$.
  Since

  $\mathbf{u}\in {\left(E^{\perp }\right)}^{\perp }$ and

  $\mathbf{v}\in E\subseteq {\left(E^{\perp }\right)}^{\perp }$, so

  $\mathbf{w}=\mathbf{u}-\mathbf{v}\in {\left(E^{\perp }\right)}^{\perp }$.
  But we also have

  $\mathbf{w}\in E^{\perp }$.
  As a result we must have

  $\mathbf{w}=\mathbf{0}$, and

  $\mathbf{u}=\mathbf{v}\in E$.
  Therefore,

  ${\left(E^{\perp }\right)}^{\perp }\subseteq E$.

Linear functional

Definition:
A linear functional on a vector space

$V$ is a linear map from

$V$ to

$\mathbb{R}$. That is, a linear functional belongs to

$\mathcal{L}(V,\mathbb{R})$.

Examples of linear functionals

• Example 1:
  
  $${\varphi }_1:{\mathbb{P}}_2\to \mathbb{R},\varphi (p)=p(1).$$
• Example 2:
  
  $${\varphi }_1:{\mathbb{P}}_2\to \mathbb{R},\varphi (p)={\int }_0^{1}p(x)dx.$$
• Example 3:
  
  $${\varphi }_1:{\mathbb{P}}_2\to \mathbb{R},\varphi (p)={\int }_0^{1}p(x)\sin (x)dx.$$

Riesz Representation Theorem:
Suppose

$V$ is a finite dimentional inner product space and

$\varphi :V\to \mathbb{R}$ is a linear functional, then there exists an unique

$\mathbf{u}\in V$ such that

$$\varphi (\mathbf{v})=⟨\mathbf{v},\mathbf{u}⟩,\mathrm{\forall }\mathbf{v}\in V.$$

• Proof:

  (existence)
  Let

  $\{e_1,\cdots ,e_n\}\subset V$ be an orthonormal basis of

  $V$.

  Given

  $\mathbf{v}\in V$, we then have
  

  $$\begin{array}{rl}\mathbf{v}& ={\alpha }_1{e}_1+\cdots +{\alpha }_n{e}_n\\ & =⟨\mathbf{v},e_1⟩e_1+\cdots +⟨\mathbf{v},e_n⟩e_n.\end{array}$$
  Since

  $\varphi$ is linear,
  

  $$\begin{array}{rl}\varphi (\mathbf{v})& =\varphi (⟨\mathbf{v},e_1⟩e_1+\cdots +⟨\mathbf{v},e_n⟩e_n)\\ & =⟨\mathbf{v},e_1⟩\varphi (e_1)+\cdots +⟨\mathbf{v},e_n⟩\varphi (e_n)\\ & =⟨\mathbf{v},\varphi (e_1)e_1⟩+\cdots +⟨\mathbf{v},\varphi (e_n)e_n⟩\\ & =⟨\mathbf{v},\varphi (e_1)e_1+\cdots +\varphi (e_n)e_n⟩.\end{array}$$
  Let us define
  

  $$\mathbf{u}=\varphi (e_1)e_1+\cdots +\varphi (e_n)e_n,$$
  it is then clear that

  $\varphi (\mathbf{v}\mathbf{)}\mathbf{=}⟨\mathbf{v}\mathbf{,}\mathbf{u}⟩$.

  (uniqueness)
  Suppose

  $\mathrm{\exists }{\mathbf{u}}_1,{\mathbf{u}}_2\in V$ such that
  

  $$\varphi (\mathbf{v})=⟨\mathbf{v},{\mathbf{u}}_1⟩=⟨\mathbf{v},{\mathbf{u}}_2⟩,\mathrm{\forall }\mathbf{v}\in V.$$
  Then we have
  

  $$0=⟨\mathbf{v},{\mathbf{u}}_1-{\mathbf{u}}_2⟩,\mathrm{\forall }\mathbf{v}\in V.$$
  Therefore, we must have

  ${\mathbf{u}}_1-{\mathbf{u}}_2=\mathbf{0}$ and

  ${\mathbf{u}}_1={\mathbf{u}}_2$.

Adjoint linear transformation

Definition:
Let

$V$ and

$W$ be inner product space and

$T:V\to W$ be a linear transformation. The map

$T^{\ast }:W\to V$ is called an adjoint of

$T$ if

$$⟨T\mathbf{v},\mathbf{w}⟩=⟨\mathbf{v},T^{\ast }\mathbf{w}⟩,\mathrm{\forall }\mathbf{v}\in V,\mathbf{w}\in W.$$

Remark:
Here we show the existence of the map

$T^{\ast }$.

Given

$\mathbf{w}\in W$, we define

$\varphi (\mathbf{v})=⟨T\mathbf{v},\mathbf{w}⟩$. It is clear that

$\varphi$ is a linear functional. Hence, based on the Riese representation Theorem, there exists

$\mathbf{u}\in V$ such that

$$\varphi (\mathbf{v})=⟨\mathbf{v},\mathbf{u}⟩.$$
We then define

$T^{\ast }\mathbf{w}=\mathbf{u}$ and we have

$$⟨T\mathbf{v},\mathbf{w}⟩=\varphi (\mathbf{v})=⟨\mathbf{v},\mathbf{u}⟩=⟨\mathbf{v},T^{\ast }\mathbf{w}⟩.$$
So the existence of

$T^{\ast }$ is assured.

Proposition

$T^{\ast }$ is an adjoint of

$T$ if

$$⟨\mathbf{w},T\mathbf{v}⟩=⟨T^{\ast }\mathbf{w},\mathbf{v}⟩,\mathrm{\forall }\mathbf{v}\in V,\mathbf{w}\in W.$$

Proposition

$T^{\ast }$ is unique.

• Proof:

  Let

  $S:W\to V$ be a map that also satisfies
  

  $$⟨T\mathbf{v},\mathbf{w}⟩=⟨\mathbf{v},S\mathbf{w}⟩,\mathrm{\forall }\mathbf{v}\in V,\mathbf{w}\in W.$$
  Then
  

  $$⟨\mathbf{v},S\mathbf{w}⟩=⟨T\mathbf{v},\mathbf{w}⟩=⟨\mathbf{v},T^{\ast }\mathbf{w}⟩,\mathrm{\forall }\mathbf{v}\in V,\mathbf{w}\in W.$$
  So
  

  $$⟨\mathbf{v},(S-T^{\ast })\mathbf{w}⟩=0,\mathrm{\forall }\mathbf{v}\in V,\mathbf{w}\in W.$$
  Hence,

  $S=T^{\ast }$.

Proposition

$T^{\ast }$ is linear.

• Proof:

  Fix

  ${\mathbf{w}}_1,{\mathbf{w}}_2\in W$,

  ${\alpha }_1,{\alpha }_2\in F$, given

  $\mathbf{v}\in V$,
  

  $$\begin{array}{rl}⟨\mathbf{v},T^{\ast }({\alpha }_1{\mathbf{w}}_1+{\alpha }_2{\mathbf{w}}_2)⟩& =⟨T(\mathbf{v}),{\alpha }_1{\mathbf{w}}_1+{\alpha }_2{\mathbf{w}}_2⟩\\ & =\overline{{\alpha }_1}⟨T(\mathbf{v}),{\mathbf{w}}_1⟩+\overline{{\alpha }_2}⟨T(\mathbf{v}),{\mathbf{w}}_2⟩\\ & =\overline{{\alpha }_1}⟨\mathbf{v},T^{\ast }({\mathbf{w}}_1)⟩+\overline{{\alpha }_2}⟨\mathbf{v},T^{\ast }({\mathbf{w}}_2)⟩\\ & =⟨\mathbf{v},{\alpha }_1{T}^{\ast }({\mathbf{w}}_1)+{\alpha }_2{T}^{\ast }({\mathbf{w}}_2)⟩.\end{array}$$
  Therefore
  

  $$⟨\mathbf{v},T^{\ast }({\alpha }_1{\mathbf{w}}_1+{\alpha }_2{\mathbf{w}}_2)⟩=⟨\mathbf{v},{\alpha }_1{T}^{\ast }({\mathbf{w}}_1)+{\alpha }_2{T}^{\ast }({\mathbf{w}}_2)⟩,\mathrm{\forall }\mathbf{v}\in V.$$
  That leads to
  

  $$T^{\ast }({\alpha }_1{\mathbf{w}}_1+{\alpha }_2{\mathbf{w}}_2)={\alpha }_1{T}^{\ast }({\mathbf{w}}_1)+{\alpha }_2{T}^{\ast }({\mathbf{w}}_2).$$