Chapter 6 extra note 2

positive definite/semidefinite operators
square root of an operator
positive square root of an operator
modulus of a linear transformation

Youtube

• 3Blue1Brown - Linear transformation and matrices
• 3Blue1Brown - Three-dimensional linear transformation

Selected lecture notes

Definition:
A self-adjoint operator

$T\in \mathcal{L}(V)$ is called positive definite, or

$T>0$, if

$$\begin{array}{}\text{(1)}& ⟨T(\mathbf{v}),\mathbf{v}⟩>0,\mathrm{\forall }\mathbf{v}\in V\setminus \{\mathbf{0}\},\end{array}$$
and is called positive semidefinite, or

$T\ge 0$, if

$$\begin{array}{}\text{(2)}& ⟨T(\mathbf{v}),\mathbf{v}⟩\ge 0,\mathrm{\forall }\mathbf{v}\in V.\end{array}$$

Theorem:
Let

$V$ be an inner product space and

$T\in \mathcal{L}(V)$ be a self-adjoint operator.

$T>0$ if and only if all the engenvalues are positive, and

$T\ge 0$ if and only if all the engenvalues are non-negative.

• Proof:

  We prove for the case

  $T>0$. The case of

  $T\ge 0$ can be shown similarly.

  (

  $\Rightarrow$)
  Suppose

  $T=T^{\ast }$ and

  $T>0$. Let

  $\mathbf{v}\ne 0$ be such that

  $T(\mathbf{v})=\lambda \mathbf{v}$. Then
  

  $$\begin{array}{}\text{(3)}& 0<⟨T(\mathbf{v}),\mathbf{v}⟩=⟨\lambda \mathbf{v},\mathbf{v}⟩=\lambda \Vert \mathbf{v}{\Vert }^2.\end{array}$$
  Since

  $\mathbf{v}\ne 0$, we have

  $\Vert \mathbf{v}{\Vert }^2>0$ and hence

  $\lambda >0$.
  Therefore, all the eigenvalues are positive.

  (

  $\Leftarrow$)
  Since

  $T$ is self-adjoint, there exists an orthonormal basis of eigenvectors, i.e., we can find a basis

  $\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}\subset V$ such that, for

  $1\le i\le n$,

  1. $T({\mathbf{v}}_i)={\lambda }_i{\mathbf{v}}_i$;
  2. $⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩={\delta }_{ij}$;
  3. ${\lambda }_i\in \mathbb{R}$.

  Given

  $\mathbf{v}\in V\setminus \{\mathbf{0}\}$, it can then be represented as

  $\mathbf{v}=\sum _k{c}_k{\mathbf{v}}_k$ and

  $T(\mathbf{v})=\sum _k{c}_k{\lambda }_k{\mathbf{v}}_k$.

  So we have
  

  $$\begin{array}{}\text{(4)}& ⟨T(\mathbf{v}),\mathbf{v}⟩=⟨\sum _k{c}_k{\lambda }_k{\mathbf{v}}_k,\sum _k{c}_k{\mathbf{v}}_k⟩=\sum _k{c}_k^2{\lambda }_k.\end{array}$$
  Since

  ${\lambda }_k>0$ for all

  $k$, and

  $\mathbf{v}\ne \mathbf{0}$, we have

  $⟨T(\mathbf{v}),\mathbf{v}⟩>0$.

---

Definition:
An operator

$B$ is called a square root of an operator

$T$ if

$B^2=T$.

Definition:
Let

$T\ge 0$, a self-adjoint operator

$B=\sqrt{T}$ if

$B^2=T$ and

$B\ge 0$.

Theorem:
Let

$V$ be an inner product space and

$T\in \mathcal{L}(V)$ be a positive semidefinite operator. There exists an unique

$B\in \mathcal{L}(V)$ that is positive semidefinite and

$B^2=T$, that is,

$B$ is the positive square root of

$T$, and is denoted by

$B=\sqrt{T}$ or

$T^{1/2}$.

• Proof:

  (existence)
  Since

  $T$ is positive semidefinite, there exists

  $\{{\mathbf{v}}_1,\cdots ,{\mathbf{v}}_n\}\subset V$ such that, for

  $1\le i,j\le n$,

  1. $T({\mathbf{v}}_i)={\lambda }_i{\mathbf{v}}_i$;
  2. $⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩={\delta }_{ij}$;
  3. ${\lambda }_i\ge 0$.

  Let us define

  $B:V\to V$ be such that

  $B({\mathbf{v}}_i)=\sqrt{{\lambda }_i}{\mathbf{v}}_i$. Also, given

  $\mathbf{v}\in V$,

  $\mathbf{v}=\sum _k{c}_k{\mathbf{v}}_k$, we define
  

  $$\begin{array}{}\text{(5)}& B(\mathbf{v})=\sum _k{c}_k\sqrt{{\lambda }_k}{\mathbf{v}}_k.\end{array}$$
  It is easy to check that

  $B\in \mathcal{L}(V)$,

  $B=B^{\ast }$, and

  $B\ge 0$.
  Finally,
  

  $$\begin{array}{}\text{(6)}& B^2(\mathbf{v})=B(B(\mathbf{v}))=\sum _k{c}_k{\lambda }_k{\mathbf{v}}_k=T(\mathbf{v}).\end{array}$$

  (uniqueness)
  Let

  $\tilde{B}$ be such that

  $\tilde{B}\in \mathcal{L}(V)$,

  $\tilde{B}={\tilde{B}}^{\ast }$,

  $\tilde{B}\ge 0$ and

  ${\tilde{B}}^2=T$.
  Based on the following lemma, we must have
  

  $$\begin{array}{}\text{(7)}& \tilde{B}\mathbf{v}=\sqrt{\lambda }\mathbf{v}=B\mathbf{v},\end{array}$$
  for all eigenvector

  $\mathbf{v}$ and eigenvalue

  $\lambda$ of

  $T$.
  Since there exists a basis of orthonormal eigenvectors of

  $T$, we must have

  $\tilde{B}=B$.

Lemma:
Let

$V$ be an inner product space and

$T\in \mathcal{L}(V)$ be a positive semidefinite operator. Suppose

$\mathbf{v}\in V$ satisfies

$T(\mathbf{v})=\lambda \mathbf{v}$,

$\lambda \ge 0$, then

$B\mathbf{v}=\sqrt{\lambda }\mathbf{v}$, where

$B=\sqrt{T}$.

• Proof:

  Suppose

  $B=\sqrt{T}$, so

  $B=B^{\ast }\ge 0$. There exists

  $\{{\mathbf{u}}_1,\cdots ,{\mathbf{u}}_n\}\subset V$ such that, for

  $1\le i,j\le n$,

  1. $B({\mathbf{u}}_i)={\mu }_i{\mathbf{u}}_i$;
  2. $⟨{\mathbf{u}}_i,{\mathbf{u}}_j⟩={\delta }_{ij}$;
  3. ${\mu }_i\ge 0$.

  Let

  $\mathbf{v}\in V\setminus \{\mathbf{0}\}$ satisfies

  $T(\mathbf{v})=\lambda \mathbf{v}$,

  $\lambda \ge 0$, then

  $\mathbf{v}=\sum _k{c}_k{\mathbf{u}}_k$ and

  $B\mathbf{v}=\sum _k{c}_k{\mu }_k{\mathbf{u}}_k$.
  Since

  $B^2=T$,
  

  $$\begin{array}{}\text{(8)}& T(\mathbf{v})=B^2(\mathbf{v})=\sum _k{c}_k{\mu }_k^2{\mathbf{u}}_k,\end{array}$$
  also,
  

  $$\begin{array}{}\text{(9)}& T(\mathbf{v})=\lambda \mathbf{v}=\sum _k{c}_k\lambda {\mathbf{u}}_k.\end{array}$$
  From (8) and (9) we obtain
  

  $$\begin{array}{}\text{(10)}& \sum _k{c}_k({\mu }_k^2-\lambda ){\mathbf{u}}_k=\mathbf{0}.\end{array}$$
  Since

  $\{{\mathbf{u}}_{\mathbf{k}}\}$ is a basis, all the coefficients have to be zero. Therefore

  $c_k({\mu }_k^2-\lambda )=0$ for all

  $k$.
  Since

  $\mathbf{v}\ne \mathbf{0}$, there exists an index

  $j$ such that

  $c_j\ne 0$, that gives

  ${\mu }_j^2=\lambda$.
  Besides, for

  $k\ne j$,

  ${\mu }_k^2-\lambda \ne 0$, so

  $c_k=0$ for all

  $k\ne j$, that gives

  $\mathbf{v}={\mathbf{u}}_j$.

  Therefore,
  

  $$\begin{array}{}\text{(11)}& B(\mathbf{v})=B({\mathbf{u}}_j)={\mu }_j({\mathbf{u}}_j)=\sqrt{\lambda }\mathbf{v}.\end{array}$$

---

Given

$T\in \mathcal{L}(V,W)$, where

$V$ and

$W$ are inner product spaces, its adjoint linear transformation

$T^{\ast }:W\to V$ is uniquely determined. We also have

1. $T^{\ast }\in \mathcal{L}(W,V)$;
2. $(T^{\ast }{)}^{\ast }=T$.

We can then define a linear transformation that is a composition of

$T$ and

$T^{\ast }$ as

$T^{\ast }\circ T:V\to V$. We have

1. $T^{\ast }T\in \mathcal{L}(V,V)$;
2. $(T^{\ast }T{)}^{\ast }=T^{\ast }T$, i.e.,
   $T^{\ast }T$ is self-adjoint;
3. $⟨T^{\ast }T\mathbf{v},\mathbf{v}⟩=⟨T\mathbf{v},T\mathbf{v}⟩=\Vert T\mathbf{v}{\Vert }^2\ge 0$, so
   $T^{\ast }T\ge 0$;
4. $\sqrt{T^{\ast }T}$ exists and is unique.

Definition:
Let

$T\in \mathcal{L}(V,W)$ where

$V$ and

$W$ are inner product spaces. The modulus of

$T$, denoted by

$|T|$, is defined as

$|T|=\sqrt{T^{\ast }T}$.

Note that

$|T|=\sqrt{T^{\ast }T}$ is self-adjoint and

$|T|\ge 0$.

Proposition:
Let

$T\in \mathcal{L}(V,W)$ where

$V$ and

$W$ are inner product spaces, we have

$$\Vert T\mathbf{v}\Vert =\Vert |T|\mathbf{v}\Vert .$$

• Proof:

  $$\begin{array}{}\text{(12)}& \begin{array}{rl}\Vert |T|\mathbf{v}{\Vert }^2& =⟨|T|\mathbf{v},|T|\mathbf{v}⟩\\ & =⟨|T{|}^2\mathbf{v},\mathbf{v}⟩\\ & =⟨T^{\ast }T\mathbf{v},\mathbf{v}⟩\\ & =⟨T\mathbf{v},T\mathbf{v}⟩\\ & =\Vert T\mathbf{v}{\Vert }^2.\end{array}\end{array}$$