---
# System prepended metadata

title: Linear Algebra Note 5
tags: [Linear Algebra, ' 國立陽明交通大學', ' 魏群樹', ' 大學', ' 線性代數', ' 筆記']

---

---
title: Linear Algebra Note 5
tags: Linear Algebra, 線性代數, 魏群樹, 大學, 國立陽明交通大學, 筆記
---

# Linear Algebra Note 5

## Determinants (2021.11.24)

- Properties
    9. $\begin{vmatrix}
        AB
        \end{vmatrix} = \begin{vmatrix}
        A
        \end{vmatrix}\begin{vmatrix}
        B
        \end{vmatrix}$
    - Recall: $rank(AB) \le min(rank(A),\ rank(B))$
    - Proof
        Let $A \in \mathbb{R}^{k \times l},\ B \in \mathbb{R}^{l \times m}$
        $\forall\ \vec{v} \in C(AB),\ \vec{v} = (AB)\vec{x}$ where $\vec{x}$ is the $m \times 1$ vector
        Also, $\vec{v} = A(B\vec{x})$ where $B\vec{x}$ is the $l \times 1$ vector
        $\therefore\ \vec{v} \in C(A),\ i.e.\ C(AB) \subseteq C(A) \\
        \implies dim(C(AB)) \le dim(C(A)),\ rank(AB) \le rank(A)$
        Similary, $rank(AB) \le rank(B)$
        1. when $\begin{vmatrix}
            B
            \end{vmatrix} = 0$ , $B$ is singular
            $rank(AB) \le min(rank(A),\ rank(B))$
            $\implies AB$ is also singular
            $\implies \begin{vmatrix}
            AB
            \end{vmatrix} = 0 \implies \begin{vmatrix}
            AB
            \end{vmatrix} = \begin{vmatrix}
            A
            \end{vmatrix}\begin{vmatrix}
            B
            \end{vmatrix} = 0$
        2. When $\begin{vmatrix}
            B
            \end{vmatrix} \not= 0$ , Let $D(A) \equiv {\begin{vmatrix}
            AB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}}$
            To prove $D(A) = \begin{vmatrix}
            A
            \end{vmatrix}\ ({\begin{vmatrix}
            AB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = \begin{vmatrix}
            A
            \end{vmatrix} \implies \begin{vmatrix}
            AB
            \end{vmatrix} = \begin{vmatrix}
            A
            \end{vmatrix}\begin{vmatrix}
            B
            \end{vmatrix})$
            show $D(A)$ satisfies (i)-(iii)
            ==(i)== $\begin{vmatrix}
            I_n
            \end{vmatrix} = 1$
            $D(A=I) = {\begin{vmatrix}
            IB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = {\begin{vmatrix}
            B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = 1 = \begin{vmatrix}
            A
            \end{vmatrix}$
            ==(ii)== Exchange of two rows leads to change of sign
            $\text{Let } A = \left[\begin{array}{c}
            \vec{a_1}^T \\
            \vec{a_2}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right],\ A' = \left[\begin{array}{c}
            \vec{a_2}^T \\
            \vec{a_1}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right],\ B = \left[\begin{array}{c c c c}
            \vec{b_1} & \vec{b_2} & \cdots & \vec{b_m} \\
            \end{array}\right] \\
            AB = \left[\begin{array}{c}
            \vec{a_1}^T \\
            \vec{a_2}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right]\left[\begin{array}{c c c c}
            \vec{b_1} & \vec{b_2} & \cdots & \vec{b_m} \\
            \end{array}\right] = \left[\begin{array}{c c c c}
            \vec{a_1}^T\vec{b_1} & \vec{a_1}^T\vec{b_2} & \cdots & \vec{a_1}^T\vec{b_m} \\
            \vec{a_2}^T\vec{b_1} & \vec{a_2}^T\vec{b_2} & \cdots & \vec{a_2}^T\vec{b_m} \\
            \vdots & \vdots & \ddots & \vdots \\
            \vec{a_n}^T\vec{b_1} & \vec{a_n}^T\vec{b_2} & \cdots & \vec{a_n}^T\vec{b_m} \\
            \end{array}\right] \\
            \begin{split}A'B = \left[\begin{array}{c}
            \vec{a_2}^T \\
            \vec{a_1}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right]\left[\begin{array}{c c c c}
            \vec{b_1} & \vec{b_2} & \cdots & \vec{b_m} \\
            \end{array}\right] &= \left[\begin{array}{c c c c}
            \vec{a_2}^T\vec{b_1} & \vec{a_2}^T\vec{b_2} & \cdots & \vec{a_2}^T\vec{b_m} \\
            \vec{a_1}^T\vec{b_1} & \vec{a_1}^T\vec{b_2} & \cdots & \vec{a_1}^T\vec{b_m} \\
            \vdots & \vdots & \ddots & \vdots \\
            \vec{a_n}^T\vec{b_1} & \vec{a_n}^T\vec{b_2} & \cdots & \vec{a_n}^T\vec{b_m} \\
            \end{array}\right] \\
            &= \text{row exchange of } AB\end{split} \\
            \therefore \begin{vmatrix}
            A'B
            \end{vmatrix} = -\begin{vmatrix}
            AB
            \end{vmatrix},\ D(A') = {\begin{vmatrix}
            A'B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = {-\begin{vmatrix}
            AB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = -D(A)$
            ==(iii)== The determinant is a linear function of each row separately
            **(scalar multiplication)**
             $A' = \left[\begin{array}{c}
            \vec{a_1}^T \\
            t\vec{a_2}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right],\ A'B = \left[\begin{array}{c c c c}
            \vec{a_1}^T\vec{b_1} & \vec{a_1}^T\vec{b_2} & \cdots & \vec{a_1}^T\vec{b_m} \\
            t\vec{a_2}^T\vec{b_1} & t\vec{a_2}^T\vec{b_2} & \cdots & t\vec{a_2}^T\vec{b_m} \\
            \vdots & \vdots & \ddots & \vdots \\
            \vec{a_n}^T\vec{b_1} & \vec{a_n}^T\vec{b_2} & \cdots & \vec{a_n}^T\vec{b_m} \\
            \end{array}\right] \\
            \begin{vmatrix}
            A'B
            \end{vmatrix} = t\begin{vmatrix}
            AB
            \end{vmatrix} \\
            \implies D(A') = {\begin{vmatrix}
            A'B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = {t\begin{vmatrix}
            AB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = tD(A)$
            **(vector addition)**
            $A'' = \left[\begin{array}{c}
            \vec{a_1}^T+\vec{a_1}'^T \\
            \vec{a_2}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right],\ A''B = \left[\begin{array}{c c c c}
            \vec{a_1}^T\vec{b_1}+\vec{a_1}'^T\vec{b_1} & \vec{a_1}^T\vec{b_2}+\vec{a_1}'^T\vec{b_2} & \cdots & \vec{a_1}^T\vec{b_m}+\vec{a_1}'^T\vec{b_m} \\
            \vec{a_2}^T\vec{b_1} & \vec{a_2}^T\vec{b_2} & \cdots & \vec{a_2}^T\vec{b_m} \\
            \vdots & \vdots & \ddots & \vdots \\
            \vec{a_n}^T\vec{b_1} & \vec{a_n}^T\vec{b_2} & \cdots & \vec{a_n}^T\vec{b_m} \\
            \end{array}\right] \\
            A''' = \left[\begin{array}{c}
            \vec{a_1}'^T \\
            \vec{a_2}^T \\
            \vdots \\
            \vec{a_n}^T \\
            \end{array}\right],\ A'''B = \left[\begin{array}{c c c c}
            \vec{a_1}'^T\vec{b_1} & \vec{a_1}'^T\vec{b_2} & \cdots & \vec{a_1}'^T\vec{b_m} \\
            \vec{a_2}^T\vec{b_1} & \vec{a_2}^T\vec{b_2} & \cdots & \vec{a_2}^T\vec{b_m} \\
            \vdots & \vdots & \ddots & \vdots \\
            \vec{a_n}^T\vec{b_1} & \vec{a_n}^T\vec{b_2} & \cdots & \vec{a_n}^T\vec{b_m} \\
            \end{array}\right] \\
            \begin{vmatrix}
            A''B
            \end{vmatrix} = \begin{vmatrix}
            AB
            \end{vmatrix} + \begin{vmatrix}
            A'''B
            \end{vmatrix} \\
            \implies D(A'') = {\begin{vmatrix}
            A''B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = {\begin{vmatrix}
            AB
            \end{vmatrix}+\begin{vmatrix}
            A'''B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = {\begin{vmatrix}
            AB
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} + {\begin{vmatrix}
            A'''B
            \end{vmatrix} \over \begin{vmatrix}
            B
            \end{vmatrix}} = D(A) + D(A''')$
            $D(A)$ satisfies rule (i)-(iii) $\implies D(A) = \begin{vmatrix}
            A
            \end{vmatrix}$
     - $det(A^{-1}) = {1 \over det(A)}$
         - Proof
            $AA^{-1} = I,\ \begin{vmatrix}
            AA^{-1}
            \end{vmatrix} = \begin{vmatrix}
            A
            \end{vmatrix}\begin{vmatrix}
            A^{-1}
            \end{vmatrix} = \begin{vmatrix}
            I
            \end{vmatrix} = 1$

    10. $\begin{vmatrix}
		A^T
		\end{vmatrix} = \begin{vmatrix}
		A
		\end{vmatrix}$
		- Proof
			1. when $\begin{vmatrix}
				A
				\end{vmatrix} = 0,\ A$ is singular $\implies A^T$ is singular $\implies \begin{vmatrix}
				A^T
				\end{vmatrix} = 0$
			2. when $\begin{vmatrix}
				A
				\end{vmatrix} \not= 0$ , apply $PA=LU$ ($P$ = permutation matrix)
				$(PA)^T = (LU)^T \implies A^TP^T = U^TL^T$
				from 9, $\begin{vmatrix}
				P
				\end{vmatrix}\begin{vmatrix}
				A
				\end{vmatrix} = \begin{vmatrix}
				PA
				\end{vmatrix} = \begin{vmatrix}
				LU
				\end{vmatrix} = \begin{vmatrix}
				L
				\end{vmatrix}\begin{vmatrix}
				U
				\end{vmatrix},\ \begin{vmatrix}
				A^T
				\end{vmatrix}\begin{vmatrix}
				P^T
				\end{vmatrix} = \begin{vmatrix}
				U^T
				\end{vmatrix}\begin{vmatrix}
				L^T
				\end{vmatrix}$
			$\because PP^T = I \implies \begin{vmatrix}
				P
				\end{vmatrix}\begin{vmatrix}
				P^T
				\end{vmatrix} = 1$
				Moreover, a permutation matrix $P$ can be regarded as s row-exchanged identity matrix
				$\begin{vmatrix}
				P
				\end{vmatrix} = (-1)^k\begin{vmatrix}
				I
				\end{vmatrix} = 1\ or\ -1 \\
				\because \begin{vmatrix}
				P
				\end{vmatrix}\begin{vmatrix}
				P^T
				\end{vmatrix} = 1 \implies \begin{vmatrix}
				P^T
				\end{vmatrix} = \begin{vmatrix}
				P
				\end{vmatrix}$
				Also, $\begin{vmatrix}
				L
				\end{vmatrix} = 1 = \begin{vmatrix}
				L^T
				\end{vmatrix}$ due to all-ones diagonal
				$\begin{vmatrix}
				U
				\end{vmatrix} = \begin{vmatrix}
				U^T
				\end{vmatrix} =$ product of the pivots
				$\begin{vmatrix}
				P
				\end{vmatrix}\begin{vmatrix}
				A
				\end{vmatrix} = \begin{vmatrix}
				L
				\end{vmatrix}\begin{vmatrix}
				U
				\end{vmatrix} = \begin{vmatrix}
				U^T
				\end{vmatrix}\begin{vmatrix}
				L^T
				\end{vmatrix} = \begin{vmatrix}
				A^T
				\end{vmatrix}\begin{vmatrix}
				P^T
				\end{vmatrix},\ \begin{vmatrix}
				P^T
				\end{vmatrix} = \begin{vmatrix}
				P
				\end{vmatrix} \implies \begin{vmatrix}
				A
				\end{vmatrix} = \begin{vmatrix}
				A^T
				\end{vmatrix}$

- Summary: How to find determinant (algorithm)
Use Gaussian elimination $PA=LU,\ \begin{vmatrix}
P
\end{vmatrix}\begin{vmatrix}
A
\end{vmatrix} = \begin{vmatrix}
L
\end{vmatrix}\begin{vmatrix}
U
\end{vmatrix}$
$\begin{vmatrix}
P
\end{vmatrix} = \pm 1,\ \begin{vmatrix}
L
\end{vmatrix} = 1,\ \begin{vmatrix}
U
\end{vmatrix} =$ product of the pivots
$\therefore \begin{vmatrix}
A
\end{vmatrix} = (-1)^k\begin{vmatrix}
U
\end{vmatrix} = (-1)^k \times (product\ of\ the\ pivots)$ , $k =$ the number of row exchange

## Eigenvectors & Eigenvalues (2021.11.26 ~ 2021.12.03)

- Example
$A = \left[\begin{array}{c c}
0.8 & 0.3 \\
0.2 & 0.7 \\
\end{array}\right],\ \vec{x_1} = \left[\begin{array}{c}
0.6 \\
0.4 \\
\end{array}\right],\ \vec{x_2} = \left[\begin{array}{c}
1 \\
-1 \\
\end{array}\right] \\
A\vec{x_1} = \left[\begin{array}{c c}
0.8 & 0.3 \\
0.2 & 0.7 \\
\end{array}\right]\left[\begin{array}{c}
0.6 \\
0.4 \\
\end{array}\right] = \left[\begin{array}{c}
0.6 \\
0.4 \\
\end{array}\right] = \vec{x_1} \\
A\vec{x_2} = \left[\begin{array}{c c}
0.8 & 0.3 \\
0.2 & 0.7 \\
\end{array}\right]\left[\begin{array}{c}
1 \\
-1 \\
\end{array}\right] = \left[\begin{array}{c}
0.5 \\
-0.5 \\
\end{array}\right] = 0.5\vec{x_2}$
$A\vec{x_1}$ and $A\vec{x_2}$ lie in the same direction with $\vec{x_1}$ and $\vec{x_2}$
$\text{Therefore, } \begin{split} &A\vec{x_1} = \lambda_1 \vec{x_1},\ \text{where } \lambda_1 = 1 \\
&A\vec{x_2} = \lambda_2 \vec{x_2},\ \text{where }\lambda_2 = 0.5\end{split}$
$\vec{x_1},\ \vec{x_2}$ are called ==eigenvectors== and $\lambda_1,\ \lambda_2$ are the corresponding ==eigenvalues==

- Define(eigenvector and eigenvalue): Let $A$ be a $n \times n$ matrix. A non-zero vector $\vec{x} \in \mathbb{V}$ is called an eigenvector of $A$ if there exists a scalar $\lambda$ such that $A\vec{x} = \lambda\vec{x}$ . This $\lambda$ is called the eigenvalue corresponding to the eigenvector $\vec{x}$

- Theorem
A scalar $\lambda$ is an eigenvalue if and only if $\begin{vmatrix}
A - \lambda I
\end{vmatrix} = 0$
	- Proof
	If $\vec{x}$ is an eigenvector of $A$ and $\lambda$ is the corresponding eigenvalue
	$\begin{split}A\vec{x} = \lambda \vec{x} &\iff A\vec{x} - \lambda \vec{x} = 0 \\
	&\iff (A-\lambda I) \vec{x} = 0 \\
	&\iff \vec{x} \in N(A-\lambda I)\end{split} \\
	\begin{split}\because \vec{x} \not= \vec{0} &\implies N(A-\lambda I) \not= \{ \vec{0} \} \\
	&\iff A-\lambda I \text{ is singular } \implies \begin{vmatrix}
A-\lambda I
\end{vmatrix} = 0\end{split}$

- Example
**(Finding $\lambda$)**
$A =\left[\begin{array}{c c}
0.8 & 0.3 \\
0.2 & 0.7 \\
\end{array}\right]$ , Let $\begin{vmatrix}
A-\lambda I
\end{vmatrix} = 0$
$\begin{vmatrix}
\left[\begin{array}{c c}
0.8-\lambda & 0.3 \\
0.2 & 0.7-\lambda \\
\end{array}\right]
\end{vmatrix} = 0.56-1.5\lambda+\lambda^2=0.06 = 0 \\
\lambda^2-1.5\lambda+0.5 = (\lambda - 1)(\lambda - 0.5) = 0 \implies \lambda = 1\ or\ 0.5$
**(Finding eigenvectors)**
Solve $N(A-\lambda I)$ to obtain eigenvectors
$\begin{split}\lambda = 1,\ &A - \lambda I = \left[\begin{array}{c c}
-0.2 & 0.3 \\
0.2 & -0.3 \\
\end{array}\right] \to \left[\begin{array}{c c}
1 & {-3 \over 2} \\
0 & 0 \\
\end{array}\right] \\
&\vec{x_1} = N(A - \lambda I) = \{ c\left[\begin{array}{c c}
{3 \over 2} \\
1 \\
\end{array}\right] \mid c \in \mathbb{R} \}\end{split}$
$\begin{split}\lambda = 0.5,\ &A - \lambda I = \left[\begin{array}{c c}
0.3 & 0.3 \\
0.2 & 0.2 \\
\end{array}\right] \to \left[\begin{array}{c c}
1 & 1 \\
0 & 0 \\
\end{array}\right] \\
&\vec{x_2} = N(A - \lambda I) = \{ c\left[\begin{array}{c c}
-1 \\
1 \\
\end{array}\right] \mid c \in \mathbb{R} \}\end{split}$

- Define(trace): Let $A = \left[\begin{array}{c c c c}
a_{11} & a_{12} & \cdots & a_{1n} \\
a_{21} & a_{22} & \cdots & a_{2n} \\
\vdots & \vdots & \ddots & \vdots \\
a_{n1} & a_{n2} & \cdots & a_{nn} \\
\end{array}\right]$ be a $n \times n$ square matrix, $trace(A)$ or $tr(A) \triangleq \sum\limits_{i = 1}^n{a_{ii}}$

- Theorem
Let $A$ be a square matrix and $\lambda_1,\ \lambda_2,\ \cdots,\ \lambda_n$ be its eigenvalues, then
    1. $det(A) = \lambda_1 \times \lambda_2 \times \cdots \times \lambda_n$
    2. $tr(A) = \sum\limits_{i = 1}^n{\lambda_i}$
    - Proof
    $A = \left[\begin{array}{c c c c}
    a_{11} & a_{12} & \cdots & a_{1n} \\
    a_{21} & a_{22} & \cdots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{n1} & a_{n2} & \cdots & a_{nn} \\
    \end{array}\right],\ A-\lambda I_n = \left[\begin{array}{c c c c}
    a_{11}-\lambda & a_{12} & \cdots & a_{1n} \\
    a_{21} & a_{22}-\lambda & \cdots & a_{2n} \\
    \vdots & \vdots & \ddots & \vdots \\
    a_{n1} & a_{n2} & \cdots & a_{nn}-\lambda \\
    \end{array}\right] \\
    det(A-\lambda I_n) = det(A) + \cdots + \sum\limits_{i=1}^n{a_{ii}(-\lambda)^{n-1}} + (-\lambda)^n$
    Let $\lambda_1,\ \lambda_2,\ \cdots,\ \lambda_n$ be the roots of the n-order polynomial
    $\begin{split}p(\lambda) &= d(\lambda_1-\lambda)(\lambda_2-\lambda)\cdots(\lambda_n-\lambda) \\
    &= d[(\lambda_1 \times \lambda_2 \times \cdots \times \lambda_n) + \cdots + \sum\limits_{i=1}^n{\lambda_i(-\lambda)^{n-1}} + (-\lambda)^n]\end{split}$
    match $det(A-\lambda I)$ and $p(\lambda)$
    $\implies det(A) = \lambda_1 \times \lambda_2 \times \cdots \times \lambda_n,\ tr(A) = \sum\limits_{i=1}^n{a_{ii}} = \sum\limits_{i=1}^n{\lambda_i}$

- Example (Projection matrix)
$P = \left[\begin{array}{c c}
{1 \over 2} & {1 \over 2} \\
{1 \over 2} & {1 \over 2} \\
\end{array}\right],\ \begin{vmatrix}
P - \lambda I
\end{vmatrix} = \begin{vmatrix}
\left[\begin{array}{c c}
{1 \over 2}-\lambda & {1 \over 2} \\
{1 \over 2} & {1 \over 2}-\lambda \\
\end{array}\right]
\end{vmatrix} = \lambda^2-\lambda = 0 \\
\lambda = 1\ or\ 0 \implies \vec{x_1} = \left[\begin{array}{c}
1 \\
1 \\
\end{array}\right],\ \vec{x_2} = \left[\begin{array}{c}
-1 \\
1 \\
\end{array}\right]$
Note that $P$ is a projection matrix onto a subspace spanned by $\left[\begin{array}{c}
1 \\
1 \\
\end{array}\right]$
$\begin{split}\therefore\  &\forall\ \vec{x_1} \text{ already in this subspace } &P\vec{x_1} = \lambda_1\vec{x_1} = \vec{x_1} \text{ (itself)} \\
&\forall\ \vec{x_2} \text{ orthogonal to this subspace } &P\vec{x_2} = \lambda_2\vec{x_2} = 0\end{split}$

- Example (Rotation matrix)
$Q = \left[\begin{array}{c c}
0 & -1 \\
1 & 0 \\
\end{array}\right]$ rotates a vector $\in \mathbb{R}^2$ by $90^\circ$
check $Q\vec{v} = \left[\begin{array}{c c}
0 & -1 \\
1 & 0 \\
\end{array}\right]\left[\begin{array}{c}
a \\
b \\
\end{array}\right] = \left[\begin{array}{c}
-b \\
a \\
\end{array}\right]$
$(Q\vec{v})^T\vec{v} = \left[\begin{array}{c}
-b \\
a \\
\end{array}\right]^T\left[\begin{array}{c}
a \\
b \\
\end{array}\right] = -ba+ab = 0$ (orthogonal)
**(Find $\lambda$)**
$\begin{vmatrix}
Q-\lambda I
\end{vmatrix} = \begin{vmatrix}
\left[\begin{array}{c c}
-\lambda & -1 \\
1 & -\lambda \\
\end{array}\right]
\end{vmatrix} = \lambda^2+1 = 0,\ \lambda = \pm i$
There doesn't exist any real number $\lambda$ and vector $\vec{x}$ s.t. $Q\vec{x} = \lambda \vec{x}$
since **NO** vector stay in the same direction after rotation by degrees other than $n\pi,\ n \in \mathbb{Z}$
