---
title: Ch6-2
tags: Linear algebra
GA: G-77TT93X4N1
---

# Chapter 6 extra note 2

> positive definite/semidefinite operators
> square root of an operator
> positive square root of an operator
> modulus of a linear transformation

## Youtube

* [3Blue1Brown - Linear transformation and matrices](https://youtu.be/kYB8IZa5AuE)
* [3Blue1Brown - Three-dimensional linear transformation](https://youtu.be/rHLEWRxRGiM)

## Selected lecture notes

:::info
**Definition:**
A self-adjoint operator $T\in\mathcal{L}(V)$ is called *positive definite*, or $T>0$, if
$$
\tag{1}
\langle T({\bf v}), {\bf v}\rangle >0, \quad \forall {\bf v}\in V\setminus\{{\bf 0}\},
$$
and is called *positive semidefinite*, or $T\ge 0$, if
$$
\tag{2}
\langle T({\bf v}), {\bf v}\rangle \ge 0, \quad \forall {\bf v}\in V.
$$
:::

**Theorem:**
Let $V$ be an inner product space and $T\in\mathcal{L}(V)$ be a self-adjoint operator. $T>0$ if and only if all the engenvalues are positive, and $T\ge 0$ if and only if all the engenvalues are non-negative.
* Proof:
> We prove for the case $T>0$. The case of $T\ge 0$ can be shown similarly.
>
> ($\Rightarrow$)
> Suppose $T=T^*$ and $T>0$. Let ${\bf v}\ne 0$ be such that $T({\bf v})=\lambda {\bf v}$. Then
> $$
> \tag{3}
> 0 < \langle T({\bf v}), {\bf v}\rangle = \langle \lambda{\bf v}, {\bf v}\rangle = \lambda \|{\bf v}\|^2.
> $$
> Since ${\bf v}\ne 0$, we have $\|{\bf v}\|^2>0$ and hence $\lambda>0$.
> Therefore, all the eigenvalues are positive.
>
> ($\Leftarrow$)
> Since $T$ is self-adjoint, there exists an orthonormal basis of eigenvectors, i.e., we can find a basis $\{{\bf v}_1, \cdots, {\bf v}_n\}\subset V$ such that, for $1\le i\le n$,
> 1. $T({\bf v}_i)=\lambda_i{\bf v}_i$;
> 2. $\langle {\bf v}_i, {\bf v}_j\rangle = \delta_{ij}$;
> 3. $\lambda_i\in\mathbb{R}$.
>
> Given ${\bf v}\in V\setminus\{{\bf 0}\}$, it can then be represented as ${\bf v} = \sum_k c_k{\bf v}_k$ and $T({\bf v}) = \sum_k c_k\lambda_k{\bf v}_k$.
>
> So we have
> $$
> \tag{4}
> \langle T({\bf v}), {\bf v}\rangle = \langle \sum_k c_k\lambda_k{\bf v}_k, \sum_k c_k{\bf v}_k\rangle = \sum_k c^2_k\lambda_k.
> $$
> Since $\lambda_k>0$ for all $k$, and ${\bf v}\ne {\bf 0}$, we have $\langle T({\bf v}), {\bf v}\rangle>0$.
>

---

:::info
**Definition:**
An operator $B$ is called a *square root* of an operator $T$ if $B^2=T$.
:::

:::info
**Definition:**
Let $T\ge 0$, a self-adjoint operator $B=\sqrt{T}$ if $B^2=T$ and $B\ge 0$.
:::

**Theorem:**
Let $V$ be an inner product space and $T\in\mathcal{L}(V)$ be a positive semidefinite operator. There exists an unique $B\in\mathcal{L}(V)$ that is positive semidefinite and $B^2 = T$, that is, $B$ is the *positive square root* of $T$, and is denoted by $B = \sqrt{T}$ or $T^{1/2}$.
* Proof:
> (existence)
> Since $T$ is positive semidefinite, there exists $\{{\bf v}_1, \cdots, {\bf v}_n\}\subset V$ such that, for $1\le i,j\le n$,
> 1. $T({\bf v}_i)=\lambda_i{\bf v}_i$;
> 2. $\langle {\bf v}_i, {\bf v}_j\rangle = \delta_{ij}$;
> 3. $\lambda_i\ge 0$.
>
> Let us define $B:V\to V$ be such that $B({\bf v}_i)=\sqrt{\lambda_i}{\bf v}_i$. Also, given ${\bf v}\in V$, ${\bf v} = \sum_k c_k{\bf v}_k$, we define
> $$
> \tag{5}
> B({\bf v}) = \sum_k c_k\sqrt{\lambda_k}{\bf v}_k.
> $$
> It is easy to check that $B\in\mathcal{L}(V)$, $B=B^*$, and $B\ge 0$.
> Finally,
> $$
> \tag{6}
> B^2({\bf v})=B(B({\bf v})) = \sum_k c_k\lambda_k{\bf v}_k = T({\bf v}).
> $$
>
> (uniqueness)
> Let $\tilde{B}$ be such that $\tilde{B}\in \mathcal{L}(V)$, $\tilde{B}=\tilde{B}^*$, $\tilde{B}\ge 0$ and $\tilde{B}^2=T$.
> Based on the following lemma, we must have
> $$
> \tag{7}
> \tilde{B}{\bf v} = \sqrt{\lambda}{\bf v} = B{\bf v},
> $$
> for all eigenvector ${\bf v}$ and eigenvalue $\lambda$ of $T$.
> Since there exists a basis of orthonormal eigenvectors of $T$, we must have $\tilde{B}=B$.

**Lemma:**
Let $V$ be an inner product space and $T\in\mathcal{L}(V)$ be a positive semidefinite operator. Suppose ${\bf v}\in V$ satisfies $T({\bf v})=\lambda{\bf v}$, $\lambda\ge 0$, then $B{\bf v}=\sqrt{\lambda}{\bf v}$, where $B=\sqrt{T}$.
* Proof:
> Suppose $B=\sqrt{T}$, so $B=B^*\ge 0$. There exists $\{{\bf u}_1, \cdots, {\bf u}_n\}\subset V$ such that, for $1\le i,j\le n$,
> 1. $B({\bf u}_i)=\mu_i{\bf u}_i$;
> 2. $\langle {\bf u}_i, {\bf u}_j\rangle = \delta_{ij}$;
> 3. $\mu_i\ge 0$.
>
> Let ${\bf v}\in V\setminus\{{\bf 0}\}$ satisfies $T({\bf v})=\lambda{\bf v}$, $\lambda\ge 0$, then ${\bf v}=\sum_kc_k {\bf u}_k$ and $B{\bf v}=\sum_kc_k \mu_k{\bf u}_k$.
> Since $B^2=T$,
> $$
> \tag{8}
> T({\bf v}) = B^2({\bf v}) = \sum_kc_k \mu^2_k{\bf u}_k,
> $$
> also,
> $$
> \tag{9}
> T({\bf v}) = \lambda{\bf v} = \sum_kc_k \lambda {\bf u}_k.
> $$
> From (8) and (9) we obtain
> $$
> \tag{10}
> \sum_k c_k (\mu^2_k - \lambda){\bf u}_k = {\bf 0}.
> $$
> Since $\{\bf u_k\}$ is a basis, all the coefficients have to be zero. Therefore $c_k (\mu^2_k - \lambda)=0$ for all $k$.
> Since ${\bf v}\ne {\bf 0}$, there exists an index $j$ such that $c_j\ne 0$, that gives $\mu^2_j=\lambda$.
> Besides, for $k\ne j$, $\mu^2_k - \lambda\ne 0$, so $c_k=0$ for all $k\ne j$, that gives ${\bf v}={\bf u}_j$.
>
> Therefore,
> $$
> \tag{11}
> B({\bf v}) = B({\bf u}_j) = \mu_j ({\bf u}_j) = \sqrt{\lambda}{\bf v}.
> $$

---

Given $T\in\mathcal{L}(V, W)$, where $V$ and $W$ are inner product spaces, its adjoint linear transformation $T^*:W\to V$ is uniquely determined. We also have
1. $T^*\in\mathcal{L}(W, V)$;
2. $(T^*)^*=T$.

We can then define a linear transformation that is a composition of $T$ and $T^*$ as $T^*\circ T:V\to V$. We have
1. $T^*T\in \mathcal{L}(V, V)$;
2. $(T^*T)^*=T^*T$, i.e., $T^*T$ is self-adjoint;
3. $\langle T^*T{\bf v}, {\bf v}\rangle=\langle T{\bf v}, T{\bf v}\rangle = \|T{\bf v}\|^2\ge 0$, so $T^*T\ge 0$;
4. $\sqrt{T^*T}$ exists and is unique.

:::info
**Definition:**
Let $T\in\mathcal{L}(V, W)$ where $V$ and $W$ are inner product spaces. The *modulus* of $T$, denoted by $|T|$, is defined as $|T|=\sqrt{T^*T}$.
:::

> Note that $|T|=\sqrt{T^*T}$ is self-adjoint and $|T|\ge 0$.

*Proposition:*
Let $T\in\mathcal{L}(V, W)$ where $V$ and $W$ are inner product spaces, we have
$$
\|T{\bf v}\| = \| |T|{\bf v}\|.
$$
* Proof:
> $$
> \tag{12}
> \begin{align}
> \| |T|{\bf v}\|^2 &= \langle|T|{\bf v}, |T|{\bf v}\rangle\\
> &=\langle|T|^2{\bf v}, {\bf v}\rangle\\
> &=\langle T^*T{\bf v}, {\bf v}\rangle\\
> &=\langle T{\bf v}, T{\bf v}\rangle\\
> &=\|T{\bf v}\|^2.
> \end{align}
> $$