Chapter 6 extra note 1

self-adjoint operator
matrix representation of an adjoint linear transformation

Youtube

• 3Blue1Brown - Eigenvectors and eigenvalues

Selected lecture notes

self-adjoint operator

Definition:
Let V be an inner product space.

$T\in \mathcal{L}(V)$ is called self-adjoint if

$T=T^{\ast }$, in other word,

$$\begin{array}{}\text{(1)}& ⟨T(\mathbf{v}),\mathbf{w}⟩=⟨\mathbf{v},T(\mathbf{w})⟩,\mathrm{\forall }\mathbf{v},\mathbf{w}\in V.\end{array}$$

Theorem:
Let V be an inner product space over

$\mathbb{C}$,

$T\in \mathcal{L}(V)$ and

$T=T^{\ast }$, then

1. All the eigenvalues are real.
2. Eigenvectors correspond to different eigenvalues are orthogonal.

• Proof:

  Let

  $\mathbf{v}$ be an eigenvector of

  $T$ correspond to

  $\lambda$, i.e.,
  

  $$T(\mathbf{v})=\lambda \mathbf{v},\mathbf{v}\ne \mathbf{0}.$$
  Then
  

  $$\begin{array}{}\text{(2)}& ⟨T(\mathbf{v}),\mathbf{v}⟩=⟨\lambda \mathbf{v},\mathbf{v}⟩=\lambda ⟨\mathbf{v},\mathbf{v}⟩=\lambda \Vert \mathbf{v}{\Vert }^2,\end{array}$$
  and
  

  $$\begin{array}{}\text{(3)}& ⟨T(\mathbf{v}),\mathbf{v}⟩=⟨\mathbf{v},T(\mathbf{v})⟩=⟨\mathbf{v},\lambda \mathbf{v}⟩=\overline{\lambda }\Vert \mathbf{v}{\Vert }^2.\end{array}$$
  Therefore
  

  $$\begin{array}{}\text{(4)}& (\lambda -\overline{\lambda })\Vert \mathbf{v}{\Vert }^2=0.\end{array}$$
  But since

  $\mathbf{v}$ is an eigenvector,

  $\mathbf{v}\ne \mathbf{0}$ and

  $\Vert \mathbf{v}\Vert \ne 0$, we have
  

  $$\begin{array}{}\text{(5)}& \lambda -\overline{\lambda }=0,\end{array}$$
  that gives

  $\lambda \in \mathbb{R}$.

  Let

  $\mathbf{u}$ and

  $\mathbf{v}$ be non-zero vectors such that
  

  $$\begin{array}{}\text{(6)}& T(\mathbf{u})=\mu \mathbf{u},T(\mathbf{v})=\lambda \mathbf{v},\end{array}$$
  where

  $\mu \ne \lambda$.
  Then
  

  $$\begin{array}{}\text{(7)}& ⟨T(\mathbf{u}),\mathbf{v}⟩=⟨\mu \mathbf{u},\mathbf{v}⟩=\mu ⟨\mathbf{u},\mathbf{v}⟩,\end{array}$$
  and
  

  $$\begin{array}{}\text{(8)}& ⟨T(\mathbf{u}),\mathbf{v}⟩=⟨\mathbf{u},T(\mathbf{v})⟩=⟨\mathbf{u},\lambda \mathbf{v}⟩=\lambda ⟨\mathbf{u},\mathbf{v}⟩.\end{array}$$
  Therefore
  

  $$\begin{array}{}\text{(9)}& (\lambda -\mu )⟨\mathbf{u},\mathbf{v}⟩=0.\end{array}$$
  Since

  $\lambda \ne \mu$, we must have

  $⟨\mathbf{u},\mathbf{v}⟩=0$.

---

Lemma:
Let V be an inner product space,

$T\in \mathcal{L}(V)$ and

$T=T^{\ast }$, and

$b,c\in \mathbb{R}$ such that

$b^2<4c$, then the operator

$T^2+bT+cI$ is invertible.

• Proof:

  Choose

  $\mathbf{v}\ne \mathbf{0}$, we consider the following inner product
  

  $$\begin{array}{}\text{(10)}& \begin{array}{rl}⟨(T^2+bT+cI)\mathbf{v},\mathbf{v}⟩& =⟨T^2\mathbf{v},\mathbf{v}⟩+⟨bT\mathbf{v},\mathbf{v}⟩+⟨c\mathbf{v},\mathbf{v}⟩\\ & =⟨T\mathbf{v},T\mathbf{v}⟩+⟨bT\mathbf{v},\mathbf{v}⟩+⟨c\mathbf{v},\mathbf{v}⟩\\ & =\Vert T\mathbf{v}{\Vert }^2+⟨bT\mathbf{v},\mathbf{v}⟩+c\Vert \mathbf{v}{\Vert }^2\\ & \ge \Vert T\mathbf{v}{\Vert }^2-|b|\Vert T\mathbf{v}\Vert \Vert \mathbf{v}\Vert +c\Vert \mathbf{v}{\Vert }^2\\ & ={(\Vert T\mathbf{v}\Vert -\frac{|b|}{2}\Vert \mathbf{v}\Vert )}^2-\frac{|b{|}^2}{4}\Vert \mathbf{v}{\Vert }^2+c\Vert \mathbf{v}{\Vert }^2\\ & ={(\Vert T\mathbf{v}\Vert -\frac{|b|}{2}\Vert \mathbf{v}\Vert )}^2+\frac{-|b{|}^2+4c}{4}\Vert \mathbf{v}{\Vert }^2\\ & >0,\end{array}\end{array}$$
  where the first inequality comes from Cauchy-Schwartz inequality, and the final inequality is the restul of

  $\mathbf{v}\ne \mathbf{0}$ and

  $4c-b^2>0$.

  We therefore conclude that

  $(T^2+bT+cI)\mathbf{v}\ne \mathbf{0}$ for

  $\mathbf{v}\ne \mathbf{0}$, that gives

  $\text{Ker}(T^2+bT+cI)=\{\mathbf{0}\}$.

  Since

  $T^2+bT+cI\in \mathcal{L}(V)$, it must be invertible.

Theorem:
Let V be an non-zero, finite dimensional, real inner product space,

$T\in \mathcal{L}(V)$ and

$T=T^{\ast }$, then

$T$ has an eigenvalue.

• Proof:

  Assume

  $\text{dim}(V)=n$, given

  $\mathbf{v}\in V\setminus \{\mathbf{0}\}$, then the set
  

  $$\{\mathbf{v},T\mathbf{v},\cdots ,T^n\mathbf{v}\}$$
  contains

  $n+1$ vectors and is a linearly dependent set. There exists

  $a_0,\cdots ,a_n\in \mathbb{R}$ not all zero such that
  

  $$\begin{array}{}\text{(11)}& a_0\mathbf{v}+a_{1}T\mathbf{v}+\cdots +a_n{T}^n\mathbf{v}=\mathbf{0}.\end{array}$$

  Let

  $m$ be such that

  $a_m\ne 0$ and

  $a_{m+1}=\cdots =a_n=0$.

  It is easy to see that

  $m\ge 1$.

  We define a polynomial

  $p(x)=a_0+a_{1}x+\cdots a_m{x}^m$. According to the Fundamental theorem of Algebra, and since this is a polynomial of real coefficients, we have
  

  $$p(x)=a_m(x-{\mu }_1)\cdots (x-{\mu }_M)(x^2+b_{1}x+c_1)\cdots (x^2+b_{N}x+c_N),$$
  where

  $b_i^2<4c_i$ for all

  $i$ and

  $M+2N=m$.

  We can then rewrite (11) similarly as
  

  $$\begin{array}{}\text{(12)}& \begin{array}{rl}\mathbf{0}& =a_0\mathbf{v}+a_{1}T\mathbf{v}+\cdots +a_n{T}^n\mathbf{v}\\ & =p(T)\mathbf{v}\\ & =a_m(T-{\mu }_{1}I)\cdots (T-{\mu }_{M}I)(T^2+b_{1}T+c_{1}I)\cdots (T^2+b_{N}T+c_{N}I)\mathbf{v}.\end{array}\end{array}$$
  According to the previous lemma, we have

  $T^2+b_{i}T+c_{i}I$ are invertible for all

  $i$, we can then apply the inverse of these operator to both side of (11) to have
  

  $$\begin{array}{}\text{(13)}& \mathbf{0}=a_m(T-{\mu }_{1}I)\cdots (T-{\mu }_{M}I)\mathbf{v}.\end{array}$$
  Since

  $\mathbf{v}\ne \mathbf{0}$ and

  $a_m\ne 0$, there must be a

  $i\in \{1,\cdots ,M\}$ such that

  $\text{Ker}(T-{\mu }_{i}I)\ne \{\mathbf{0}\}$, that is,

  ${\mu }_i$ is an eigenvalue of

  $T$.

---

Theorem:
Let V be a finite dimensional, real inner product space,

$T\in \mathcal{L}(V)$ and

$T=T^{\ast }$, then there exists a basis of orthonormal eigenvectors.

• Proof:

  Assume

  $\text{dim}(V)=n$.

  Since

  $T\in \mathcal{L}(V)$ and

  $T=T^{\ast }$, based on the previous Theorem, there exists

  ${\lambda }_1\in \mathbb{R}$ and

  ${\mathbf{v}}_1\in V$ such that
  

  $$T({\mathbf{v}}_1)={\lambda }_1{\mathbf{v}}_1.$$
  Let

  $U_1=\text{span}\{{\mathbf{v}}_1\}$, then

  $U_1^{\perp }\subseteq V$ is a vector subspace and is also an inner product space. We also have

  $\text{dim}(U_1^{\perp })=n-1$.

  Now we want to define a new map by restricting

  $T$ on the domain

  $U_1^{\perp }$. Let's find out its range (or codomain) first.

  • claim:
    $T(\mathbf{v})\in U_1^{\perp }$ for
    $\mathbf{v}\in U_1^{\perp }$.

  $$\begin{array}{}\text{(14)}& ⟨T\mathbf{v},{\mathbf{v}}_1⟩=⟨\mathbf{v},T{\mathbf{v}}_1⟩={\lambda }_1⟨\mathbf{v},{\mathbf{v}}_1⟩.\end{array}$$
  If

  $\mathbf{v}\in U_1^{\perp }$, we have

  $⟨\mathbf{v},{\mathbf{v}}_1⟩=0$ that gives

  $⟨T\mathbf{v},{\mathbf{v}}_1⟩=0$. That is,

  $T(\mathbf{v})\in U_1^{\perp }$.

  Therefore, we can then define a map

  $T_1:U_1^{\perp }\to U_1^{\perp }$. It should be clear that

  $T_1$ is linear,

  $T_1\in \mathcal{L}(U_1^{\perp })$, and

  $T_1$ is self-adjoint.

  We use the previous Theorem again on

  $T_1$. There exists

  ${\lambda }_2\in \mathbb{R}$ and

  ${\mathbf{v}}_2\in U_1^{\perp }$ such that
  

  $$T({\mathbf{v}}_2)={\lambda }_2{\mathbf{v}}_2.$$
  Since

  ${\mathbf{v}}_2\in U_1^{\perp }$,

  $\{{\mathbf{v}}_1,{\mathbf{v}}_2\}$ is orthogonal.

  We can repeat the process to find

  $n$ eigenvalue and orthogonal eigenvectors. The eigenvectors can then be normalized to be orthonormal.

---

Matrix representation of an adjoint linear transformation

Inner product with orthonormal basis

Let

$V$ be an innver product space and

$\beta =\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}\subset V$ be an orthonormal basis. Given

$\mathbf{u}$ and

$\mathbf{v}\in V$, there exists

$\mathbf{x}$ and

$\mathbf{y}\in {\mathbb{C}}^n$ such that

$$\mathbf{u}=\sum _i{x}_i{\mathbf{v}}_i=\left[\begin{array}{c}{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\end{array}\right]\mathbf{x},\mathbf{v}=\sum _i{y}_i{\mathbf{v}}_i=\left[\begin{array}{c}{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\end{array}\right]\mathbf{y}.$$
Furthermore, we have

$$\begin{array}{}\text{(15)}& ⟨\mathbf{u},\mathbf{v}{⟩}_V=\sum _i{x}_y\overline{y_i}=⟨\mathbf{x},\mathbf{y}{⟩}_{{\mathbb{C}}^n}.\end{array}$$

Linear transformation between two inner product spaces

Let

$V$ and

$W$ be inner product spaces with orthonormal basis

$\beta =\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}\subset V$ and

$\mu =\{{\mathbf{w}}_1,\cdots ,{\mathbf{w}}_m\}\subset W$, respectively. Let

$T\in \mathcal{L}(V,W)$, we have

$$\begin{array}{}\text{(16)}& [T{]}_{\beta }^{\mu }=A\in M_{m\times n}.\end{array}$$

Given

$\mathbf{v}\in V$ and

$\mathbf{w}\in W$, there exists

$\mathbf{x}\in {\mathbb{C}}^n$ and

$\mathbf{y}\in {\mathbb{C}}^m$ such that

$$\mathbf{v}=\sum _{i=1}^n{x}_i{\mathbf{v}}_i,\mathbf{w}=\sum _{i=1}^m{y}_i{\mathbf{w}}_i.$$
Also we know that the "coordinate" of

$T\mathbf{v}$ in the basis

$\mu$ is given by

$A\mathbf{x}$. Therefore,

$$\begin{array}{}\text{(17)}& \begin{array}{rl}⟨T\mathbf{v},\mathbf{w}{⟩}_W& =⟨A\mathbf{x},\mathbf{y}{⟩}_{{\mathbb{C}}^m}\\ & ={\mathbf{y}}^{\ast }A\mathbf{x}\\ & =(A^{\ast }\mathbf{y}{)}^{\ast }\mathbf{x}\\ ⟨\mathbf{v},T^{\ast }\mathbf{w}{⟩}_V& =⟨\mathbf{x},A^{\ast }\mathbf{y}{⟩}_{{\mathbb{C}}^n},\end{array}\end{array}$$
where the superscript

${}^{\ast }$ denotes conjugate transpose.

As a result, we know that the "coordinate" of

$T^{\ast }\mathbf{w}$ in the basis

$\beta$ is given by

$A^{\ast }\mathbf{y}$, and the matrix representation for

$T^{\ast }$ must be

$$\begin{array}{}\text{(18)}& [T^{\ast }{]}_{\mu }^{\beta }=A^{\ast }.\end{array}$$