# 垂直子空間

Direct sum of orthogonal subspaces

This work by Jephian Lin is licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/).

$\newcommand{\trans}{^\top}
\newcommand{\adj}{^{\rm adj}}
\newcommand{\cof}{^{\rm cof}}
\newcommand{\inp}[2]{\left\langle#1,#2\right\rangle}
\newcommand{\dunion}{\mathbin{\dot\cup}}
\newcommand{\bzero}{\mathbf{0}}
\newcommand{\bone}{\mathbf{1}}
\newcommand{\ba}{\mathbf{a}}
\newcommand{\bb}{\mathbf{b}}
\newcommand{\bc}{\mathbf{c}}
\newcommand{\bd}{\mathbf{d}}
\newcommand{\be}{\mathbf{e}}
\newcommand{\bh}{\mathbf{h}}
\newcommand{\bp}{\mathbf{p}}
\newcommand{\bq}{\mathbf{q}}
\newcommand{\br}{\mathbf{r}}
\newcommand{\bx}{\mathbf{x}}
\newcommand{\by}{\mathbf{y}}
\newcommand{\bz}{\mathbf{z}}
\newcommand{\bu}{\mathbf{u}}
\newcommand{\bv}{\mathbf{v}}
\newcommand{\bw}{\mathbf{w}}
\newcommand{\tr}{\operatorname{tr}}
\newcommand{\nul}{\operatorname{null}}
\newcommand{\rank}{\operatorname{rank}}
%\newcommand{\ker}{\operatorname{ker}}
\newcommand{\range}{\operatorname{range}}
\newcommand{\Col}{\operatorname{Col}}
\newcommand{\Row}{\operatorname{Row}}
\newcommand{\spec}{\operatorname{spec}}
\newcommand{\vspan}{\operatorname{span}}
\newcommand{\Vol}{\operatorname{Vol}}
\newcommand{\sgn}{\operatorname{sgn}}
\newcommand{\idmap}{\operatorname{id}}
\newcommand{\am}{\operatorname{am}}
\newcommand{\gm}{\operatorname{gm}}
\newcommand{\mult}{\operatorname{mult}}
\newcommand{\iner}{\operatorname{iner}}$

from lingeo import random_int_list, random_good_matrix, kernel_matrix

## Main idea

Let $U$ and $V$ be two subspaces under the same inner product space.
We say $U$ and $V$ are orthogonal if $\inp{\bu}{\bv} = 0$ for any $\bu\in U$ and $\bv\in V$.
Similarly, we say a collection of subspaces $\{V_1,\ldots,V_k\}$ is orthogonal if they are pairwisely orthogonal.

If $\{V_1,\ldots,V_k\}$ is orthogonal and none of them are $\{\bzero\}$, then they are necssarily independent.
Therefore, we have the direct sum $V_1 \oplus \cdots \oplus V_k$.
Suppose $V = V_1 \oplus \cdots \oplus V_k$.
Then every vector $\bv\in V$ can be uniquely written as $\bv = \bv_1 + \cdots + \bv_k$ with $\bv_i\in V_i$ for each $i = 1,\ldots,k$.

Let $A$ be an $m\times n$ matrix.
We have seen several cases of mutually orthogonal subspaces related to $A$.
With the new terminology, we may safely say:
1. The subspaces $\Row(A)$ and $\ker(A)$ are orthogonal, and $\mathbb{R}^n = \Row(A) \oplus \ker(A)$.
1. The subspaces $\Col(A)$ and $\ker(A\trans)$ are orthogonal, and $\mathbb{R}^m = \Col(A) \oplus \ker(A\trans)$.

Suppose $V$ is a subspace in $\mathbb{R}^n$.
We also have $\mathbb{R}^n = V \oplus V^\perp$.

## Side stories
- projection matrix

## Experiments

##### Exercise 1

執行以下程式碼。
已知 $R$ 為 $A$ 的最簡階梯形式矩陣。

<!-- eng start -->
Run the code below. Let $R$ be the reduced echelon form of $A$.
<!-- eng end -->

### code
set_random_seed(0)
print_ans = False
m,n,r = 2,4,2
A = random_good_matrix(m,n,r)
R = A.rref()
H = kernel_matrix(R)
c = random_int_list(2, r=3)
b = c[0]*R[0] + c[1]*H.transpose()[0]

print("A =")
show(A)
print("R =")
show(R)
print("b = ", b)

if print_ans:
    r = c[0]*R[0]
    h = c[1]*H.transpose()[0]
    print("r =", r)
    print("h =", h)
    print("|b|^2 =", b.norm()**2)
    print("|r|^2 + |h|^2 = %s + %s = %s"%(r.norm()**2, h.norm()**2, r.norm()**2 + h.norm()**2))

##### Exercise 1(a)

將 $\bb$ 寫成 $\br + \bh$
其中 $\br\in\Row(A)$ 而 $\bh\in\ker(A)$。

<!-- eng start -->
Write $\bb$ as $\br + \bh$ such that $\br\in\Row(A)$ and $\bh\in\ker(A)$.
<!-- eng end -->

##### Exercise 1(b)

證驗 $\br$ 和 $\bh$ 垂直﹐
而且 $\|\bb\|^2 = \|\br\|^2 + \|\bh\|^2$。

<!-- eng start -->
Verify that $\br$ and $\bh$ are orthogonal and $\|\bb\|^2 = \|\br\|^2 + \|\bh\|^2$.
<!-- eng end -->

##### Exercise 1(c)

因為每一個 $\mathbb{R}^n$ 中的向量都可以分解成 $\br\in\Row(A)$ 和 $\bh\in\ker(A)$ 中的向量相加。
說明對任何 $m\times n$ 矩陣都有

$$
\{A\bx: \bx\in\mathbb{R}^n \} = \{ A\br : \br\in\Row(A)\}.
$$

<!-- eng start -->
Since any vector in $\mathbb{R}^n$ can be written as the sum of vectors $\br\in\Row(A)$ and $\bh\in\ker(A)$. Show that

$$
\{A\bx: \bx\in\mathbb{R}^n \} = \{ A\br : \br\in\Row(A)\}.
$$

for any $m\times n$ matrix $A$.
<!-- eng end -->

## Exercises

##### Exercise 2

令 $S = \{V_1,\ldots,V_k\}$ 為一群子空間。
證明若 $S$ 是垂直的集合且 $S$ 不包含 $\{\bzero\}$﹐則 $S$ 線性獨立。

<!-- eng start -->
Let $S = \{V_1,\ldots,V_k\}$ be a family of subspaces. Show that if $S$ is orthogonal and none of its elements is $\{\bzero\}$, then $S$ is linearly independent.
<!-- eng end -->

##### Exercise 3

若 $S = \{V_1, V_2, V_3\}$ 垂直。
令 $V = V_1 \oplus V_2 \oplus V_3$、
$P$ 為 $V$ 的投影矩陣、
$P_1,P_2,P_3$ 分別為 $V_1,V_2,V_3$ 的投影矩陣。

<!-- eng start -->
Suppose $S = \{V_1,\ldots,V_k\}$ is an orthogonal set of subspaces. Let $V = V_1 \oplus V_2 \oplus V_3$, $P$ the projection matrix onto $V$, and $P_1, P_2, P_3$ the projection matrices onto $V_1, V_2, V_3$, respectively.
<!-- eng end -->

##### Exercise 3(a)

說明 $P_1P_2 = P_2P_1$。

<!-- eng start -->
Give some intuition of why $P_1P_2 = P_2P_1$.
<!-- eng end -->

##### Exercise 3(b)

說明 $P = P_1 + P_2 + P_3$。

<!-- eng start -->
Give some intuition of why $P = P_1 + P_2 + P_3$.
<!-- eng end -->

##### Exercise 3(c)

若 $V = \mathbb{R}^n$ 為全空間。
說明 $I_n = P_1 + P_2 + P_3$。

<!-- eng start -->
Suppose $V = \mathbb{R}^n$ is the whole space. Show that $I_n = P_1 + P_2 + P_3$.
<!-- eng end -->

##### Exercise 4

依照步驟證明以下敘述等價：

1. $P$ 是某個空間的投影矩陣。
2. $P$ 對稱、而且 $P = P^2$。

<!-- eng start -->
Use the given instructions to show that the following are equivalent:

1. $P$ is the projection matrix onto some subspace.
2. $P$ is symmetric and $P = P^2$.
<!-- eng end -->

##### Exercise 4(a)

證明若 $P$ 是一個投影矩陣﹐
則 $P$ 是 $\Col(P)$ 的投影矩陣。
因此如果 $\bu\in\Col(P)$ 則 $P\bu = \bu$、
如果 $\bu\in\ker(P\trans)$ 則 $P\bu = \bzero$、
同時每個向量都可以分成 $\bu = P\bu + (I - P)\bu$
使得 $P\bu\in\Col(P)$ 且 $(I - P)\bu\in\ker(P\trans)$。

藉由這些性質說明如果條件一成立則條件二成立。

<!-- eng start -->
If $P$ is a projection matrix, then $P$ is the projection matrix onto $\Col(P)$. Therefore, $P\bu = \bu$ if $\bu\in\Col(P)$ and $P\bu = \bzero$ if $\bu\in\ker(P\trans)$. Also, every vector can be written as $\bu = P\bu + (I - P)\bu$ such that $P\bu\in\Col(P)$ and $(I - P)\bu\in\ker(P\trans)$.

Justify these statements and use them to show that Condition 1 implies Condition 2.
<!-- eng end -->

##### Exercise 4(b)

若 $P$ 對稱且 $P = P^2$。
說明 $\mathbb{R}^n = \Col(P) \oplus \ker(P)$ 且
如果 $\bu\in\Col(P)$ 則 $P\bu = \bu$、
如果 $\bu\in\ker(P)$ 則 $P\bu = \bzero$。

藉由這些性質說明如果條件二成立則條件一成立。

<!-- eng start -->
Suppose $P$ is a symmetric matrix and $P = P^2$. Then $\mathbb{R}^n = \Col(P) \oplus \ker(P)$. Also, $P\bu = \bu$ if $\bu\in\Col(P)$ and $P\bu = \bzero$ if $\bu\in\ker(P)$.

Justify these statements and use them to show that Condition 2 implies Condition 1.
<!-- eng end -->

##### Exercise 5

證明若 $V = V_1 \oplus \cdots \oplus V_k$﹐
則每一個向量 $\bv\in V$ 都可以被寫成 $\bv = \bv_1 + \cdots + \bv_k$﹐
使得對每一個 $i = 1,\ldots,k$ 都有 $\bv_i\in V_i$﹐
而且這種寫法唯一。

<!-- eng start -->
Show that if $V = V_1 \oplus \cdots \oplus V_k$, then every vector $\bv\in V$ can be written as $\bv = \bv_1 + \cdots + \bv_k$ such that $\bv_i\in V_i$ for each $i = 1, \ldots, k$. Moreover, such a representation is unique.
<!-- eng end -->

##### Exercise 6

利用垂直空間分解母空間的現象在其它向量空間也很常見。

<!-- eng start -->
It is also common to see other vector spaces being decomposed into orthogonal subspaces.
<!-- eng end -->

##### Exercise 6(a)

令 $U = \mathcal{M}_{n\times n}$ 為一向量空間，搭配內積 $\inp{A}{B} = \tr(B\trans A)$。
令 $V$ 為 $U$ 中所有對稱矩陣的集合、
令 $W$ 為 $U$ 中所有反對稱矩陣的集合。
說明 $\{V, W\}$ 垂直且 $U = V \oplus W$。

<!-- eng start -->
Let $U = \mathcal{M}_{n\times n}$ be the vector space equipped with the inner product $\inp{A}{B} = \tr(B\trans A)$. Let $V$ be the subspace of $U$ consisting of all symmetric matrices. Let $W$ be the subspace of $U$ consisting of all skew-symmetric matrices. Show that $\{V, W\}$ is orthogonal and $U = V \oplus W$.
<!-- eng end -->

##### Exercise 6(a)

令 $U = \mathcal{P}_{d}$ 為一向量空間，搭配內積

$$
\inp{a_0 + a_1x + \cdots + a_dx^d}{b_0 + b_1x + \cdots + b_dx^d} = a_0b_0 + a_1b_1 + \cdots + a_db_d.
$$

令 $V$ 為 $U$ 中所有偶函數的集合、
令 $W$ 為 $U$ 中所有奇函數的集合。
說明 $V$ 和 $W$ 垂直且 $U = V \oplus W$。

<!-- eng start -->
Let $U = \mathcal{P}_{d}$ be the vector space equipped with the inner product

$$
\inp{a_0 + a_1x + \cdots + a_dx^d}{b_0 + b_1x + \cdots + b_dx^d} = a_0b_0 + a_1b_1 + \cdots + a_db_d.
$$

Let $V$ be the subspace of $U$ consisting of all even functions. Let $W$ be the subspace of $U$ consisting of all odd functions. Show that $\{V, W\}$ is orthogonal and $U = V \oplus W$.
<!-- eng end -->