---
title: Ch6-1
tags: Linear algebra
GA: G-77TT93X4N1
---

# Chapter 6 extra note 1

> self-adjoint operator
> matrix representation of an adjoint linear transformation

## Youtube

* [3Blue1Brown - Eigenvectors and eigenvalues](https://youtu.be/PFDu9oVAE-g?si=RT4LB1LccIssxBmp)

## Selected lecture notes

### self-adjoint operator

:::info
**Definition:**
Let V be an inner product space. $T\in\mathcal{L}(V)$ is called **self-adjoint** if $T=T^*$, in other word,
$$
\tag{1}
\langle T({\bf v}), {\bf w}\rangle = \langle {\bf v}, T({\bf w})\rangle, \quad \forall {\bf v}, {\bf w}\in V.
$$
:::

**Theorem:**
Let V be an inner product space over $\mathbb{C}$, $T\in\mathcal{L}(V)$ and $T=T^*$, then
1. All the eigenvalues are real.
2. Eigenvectors correspond to different eigenvalues are orthogonal.
* Proof:
> 1)
> Let ${\bf v}$ be an eigenvector of $T$ correspond to $\lambda$, i.e.,
> $$
> T({\bf v}) = \lambda {\bf v}, \quad {\bf v}\ne {\bf 0}.
> $$
> Then
> $$
> \tag{2}
> \langle T({\bf v}), {\bf v}\rangle = \langle \lambda{\bf v}, {\bf v}\rangle = \lambda\langle {\bf v}, {\bf v}\rangle = \lambda\|{\bf v}\|^2,
> $$
> and
> $$
> \tag{3}
> \langle T({\bf v}), {\bf v}\rangle = \langle {\bf v}, T({\bf v})\rangle = \langle {\bf v}, \lambda{\bf v}\rangle = \bar{\lambda} \|{\bf v}\|^2.
> $$
> Therefore
> $$
> \tag{4}
> (\lambda-\bar{\lambda}) \|{\bf v}\|^2 =0.
> $$
> But since ${\bf v}$ is an eigenvector, ${\bf v}\ne {\bf 0}$ and $\|{\bf v}\|\ne 0$, we have
> $$
> \tag{5}
> \lambda-\bar{\lambda} =0,
> $$
> that gives $\lambda\in\mathbb{R}$.
>
> 2)
> Let ${\bf u}$ and ${\bf v}$ be non-zero vectors such that
> $$
> \tag{6}
> T({\bf u}) = \mu {\bf u}, \quad T({\bf v}) = \lambda {\bf v},
> $$
> where $\mu\ne\lambda$.
> Then
> $$
> \tag{7}
> \langle T({\bf u}), {\bf v}\rangle = \langle \mu{\bf u}, {\bf v}\rangle = \mu\langle {\bf u}, {\bf v}\rangle,
> $$
> and
> $$
> \tag{8}
> \langle T({\bf u}), {\bf v}\rangle = \langle {\bf u}, T({\bf v})\rangle = \langle {\bf u}, \lambda{\bf v}\rangle = \lambda\langle {\bf u}, {\bf v}\rangle.
> $$
> Therefore
> $$
> \tag{9}
> (\lambda-\mu) \langle {\bf u}, {\bf v}\rangle =0.
> $$
> Since $\lambda\ne\mu$, we must have $\langle {\bf u}, {\bf v}\rangle =0$.

---

**Lemma:**
Let V be an inner product space, $T\in\mathcal{L}(V)$ and $T=T^*$, and $b,c\in\mathbb{R}$ such that $b^2<4c$, then the operator $T^2+bT+cI$ is invertible.
* Proof:
> Choose ${\bf v}\ne {\bf 0}$, we consider the following inner product
> $$
> \tag{10}
> \begin{align}
> \langle (T^2+bT+cI){\bf v}, {\bf v}\rangle &= \langle T^2{\bf v}, {\bf v}\rangle+\langle bT{\bf v}, {\bf v}\rangle+\langle c{\bf v}, {\bf v}\rangle\\
> &=\langle T{\bf v}, T{\bf v}\rangle+\langle bT{\bf v}, {\bf v}\rangle+\langle c{\bf v}, {\bf v}\rangle\\
> &=\|T{\bf v}\|^2 +\langle bT{\bf v}, {\bf v}\rangle+c\|{\bf v}\|^2\\
> &\ge \|T{\bf v}\|^2 - |b|\|T{\bf v}\| \|{\bf v}\|+c\|{\bf v}\|^2\\
> &=\left(\|T{\bf v}\| - \frac{|b|}{2}\|{\bf v}\|\right)^2-\frac{|b|^2}{4}\|{\bf v}\|^2+c\|{\bf v}\|^2\\
> &= \left(\|T{\bf v}\| - \frac{|b|}{2}\|{\bf v}\|\right)^2+\frac{-|b|^2+4c}{4}\|{\bf v}\|^2\\
> &> 0,
> \end{align}
> $$
> where the first inequality comes from Cauchy-Schwartz inequality, and the final inequality is the restul of ${\bf v}\ne {\bf 0}$ and $4c-b^2>0$.
>
> We therefore conclude that $(T^2+bT+cI){\bf v}\ne {\bf 0}$ for ${\bf v}\ne {\bf 0}$, that gives $\text{Ker}(T^2+bT+cI)=\{{\bf 0}\}$.
>
> Since $T^2+bT+cI\in\mathcal{L}(V)$, it must be invertible.

**Theorem:**
Let V be an non-zero, finite dimensional, *real* inner product space, $T\in\mathcal{L}(V)$ and $T=T^*$, then $T$ has an eigenvalue.
* Proof:
> Assume $\text{dim}(V)=n$, given ${\bf v}\in V\setminus\{\bf 0\}$, then the set
> $$
> \{{\bf v}, T{\bf v}, \cdots, T^n{\bf v}\}
> $$
> contains $n+1$ vectors and is a linearly dependent set. There exists $a_0, \cdots, a_n\in\mathbb{R}$ not all zero such that
> $$
> \tag{11}
> a_0{\bf v}+a_1T{\bf v}+ \cdots +a_nT^n{\bf v} = {\bf 0}.
> $$
>
> Let $m$ be such that $a_m\ne 0$ and $a_{m+1}=\cdots =a_n=0$.
> > It is easy to see that $m\ge 1$.
>
> We define a polynomial $p(x)=a_0+a_1 x+\cdots a_m x^m$. According to the [Fundamental theorem of Algebra](https://en.wikipedia.org/wiki/Fundamental_theorem_of_algebra), and since this is a polynomial of real coefficients, we have
> $$
> p(x) = a_m(x-\mu_1)\cdots(x-\mu_M)(x^2+b_1x+c_1)\cdots (x^2+b_Nx+c_N),
> $$
> where $b_i^2<4c_i$ for all $i$ and $M+2N=m$.
>
> We can then rewrite (11) similarly as
> $$
> \tag{12}
> \begin{align}
> {\bf 0} &= a_0{\bf v}+a_1T{\bf v}+ \cdots +a_nT^n{\bf v} \\
> &= p(T){\bf v}\\
> &= a_m(T-\mu_1 I)\cdots(T-\mu_M I)(T^2+b_1T+c_1 I)\cdots (T^2+b_NT+c_NI){\bf v}.
> \end{align}
> $$
> According to the previous lemma, we have $T^2+b_i T + c_iI$ are invertible for all $i$, we can then apply the inverse of these operator to both side of (11) to have
> $$
> \tag{13}
> {\bf 0} = a_m(T-\mu_1 I)\cdots(T-\mu_M I){\bf v}.
> $$
> Since ${\bf v}\ne {\bf 0}$ and $a_m\ne 0$, there must be a $i\in \{1, \cdots, M\}$ such that $\text{Ker}(T-\mu_i I)\ne \{{\bf 0}\}$, that is, $\mu_i$ is an eigenvalue of $T$.

---

**Theorem:**
Let V be a *finite dimensional*, *real* inner product space, $T\in\mathcal{L}(V)$ and $T=T^*$, then there exists a basis of orthonormal eigenvectors.
* Proof:
> Assume $\text{dim}(V)=n$.
>
> Since $T\in\mathcal{L}(V)$ and $T=T^*$, based on the previous Theorem, there exists $\lambda_1\in\mathbb{R}$ and ${\bf v}_1\in V$ such that
> $$
> T({\bf v}_1) = \lambda_1{\bf v}_1.
> $$
> Let $U_1=\text{span}\{{\bf v}_1\}$, then $U_1^\perp\subseteq V$ is a vector subspace and is also an inner product space. We also have $\text{dim}(U_1^\perp)=n-1$.
>
> Now we want to define a new map by restricting $T$ on the domain $U_1^\perp$. Let's find out its range (or codomain) first.
>
> * claim: $T({\bf v})\in U_1^\perp$ for ${\bf v}\in U_1^\perp$.
> > $$
> > \tag{14}
> > \langle T{\bf v}, {\bf v}_1\rangle = \langle{\bf v}, T{\bf v}_1\rangle=\lambda_1\langle{\bf v}, {\bf v}_1\rangle.
> > $$
> > If ${\bf v}\in U_1^\perp$, we have $\langle{\bf v}, {\bf v}_1\rangle=0$ that gives $\langle T{\bf v}, {\bf v}_1\rangle=0$. That is, $T({\bf v})\in U_1^\perp$.
>
> Therefore, we can then define a map $T_1:U_1^\perp\to U_1^\perp$. It should be clear that $T_1$ is linear, $T_1\in\mathcal{L}(U_1^\perp)$, and $T_1$ is self-adjoint.
>
> We use the previous Theorem again on $T_1$. There exists $\lambda_2\in\mathbb{R}$ and ${\bf v}_2\in U_1^\perp$ such that
> $$
> T({\bf v}_2) = \lambda_2{\bf v}_2.
> $$
> Since ${\bf v}_2\in U_1^\perp$, $\{{\bf v}_1, {\bf v}_2\}$ is orthogonal.
>
> We can repeat the process to find $n$ eigenvalue and orthogonal eigenvectors. The eigenvectors can then be normalized to be orthonormal.

---

### Matrix representation of an adjoint linear transformation

#### Inner product with orthonormal basis

Let $V$ be an innver product space and $\beta=\{{\bf v}_1, \cdots, {\bf v}_n\}\subset V$ be an orthonormal basis. Given ${\bf u}$ and ${\bf v}\in V$, there exists ${\bf x}$ and ${\bf y}\in\mathbb{C}^n$ such that
$$
{\bf u} = \sum_i x_i{\bf v}_i=
\begin{bmatrix}{\bf v}_1, \cdots, {\bf v}_n\end{bmatrix}{\bf x}, \quad
{\bf v} = \sum_i y_i{\bf v}_i=
\begin{bmatrix}{\bf v}_1, \cdots, {\bf v}_n\end{bmatrix}{\bf y}.
$$
Furthermore, we have
$$
\tag{15}
\langle{\bf u}, {\bf v}\rangle_V = \sum_i x_y \bar{y_i}=\langle{\bf x}, {\bf y}\rangle_{\mathbb{C}^n}.
$$

#### Linear transformation between two inner product spaces

Let $V$ and $W$ be inner product spaces with orthonormal basis $\beta=\{{\bf v}_1, \cdots, {\bf v}_n\}\subset V$ and $\mu=\{{\bf w}_1, \cdots, {\bf w}_m\}\subset W$, respectively. Let $T\in\mathcal{L}(V, W)$, we have
$$
\tag{16}
[T]^{\mu}_{\beta} = A\in M_{m\times n}.
$$

Given ${\bf v}\in V$ and ${\bf w}\in W$, there exists ${\bf x}\in\mathbb{C}^n$ and ${\bf y}\in\mathbb{C}^m$ such that
$$
{\bf v} = \sum^n_{i=1} x_i{\bf v}_i, \quad
{\bf w} = \sum^m_{i=1} y_i{\bf w}_i.
$$
Also we know that the "coordinate" of $T{\bf v}$ in the basis $\mu$ is given by $A{\bf x}$. Therefore,
$$
\tag{17}
\begin{align}
\langle T{\bf v}, {\bf w}\rangle_W &= \langle A{\bf x}, {\bf y}\rangle_{\mathbb{C}^m}\\
&={\bf y}^*A{\bf x}\\
&=(A^*{\bf y})^*{\bf x}\\
\langle {\bf v}, T^*{\bf w}\rangle_V &=\langle {\bf x}, A^*{\bf y}\rangle_{\mathbb{C}^n},
\end{align}
$$
where the superscript $^*$ denotes conjugate transpose.

As a result, we know that the "coordinate" of $T^*{\bf w}$ in the basis $\beta$ is given by $A^*{\bf y}$, and the matrix representation for $T^*$ must be
$$
\tag{18}
[T^*]^{\beta}_{\mu} = A^*.
$$