Try   HackMD

Linear Algebra Note 2

3. Vector Spaces and Subspaces (2021.10.13 ~ 2021.10.22)

  • Define:

    Rn={(a1, a2, ⋯, an)∣ai∈R}
    (The space
    Rn     consists of all vectors
    V→     with
    n     components)

  • Example

    [12]∈R2, [123]∈R3, [2π]∈R2

  • vector operations

    • vector addition:
      v→+w→       (element wise addition)
      • Example
        [123]+[2π0]=[1+22+π3+0]=[32+π3]
    • scalar multiplication:
      cv→       where
      c∈R
      • Example
        c=3, v→=[12]cv→=3[12]=[36]

Vector Space

  • Define: A vector space
    V     is a collection of vectors with two operations: (1) vector addition (2) scalar multiplication, so that
    (1)
    ∀v→, w→∈V, v→+w→∈V
    (2)
    ∀c∈R, cv→∈V

Any linear combination of vectors in

Rn results in a vector in
Rn

Subspace

  • Define: A subset
    W     of a vector space
    V     is called a subspace if
    W     itself form a vector space with the two operations defined on
    V     and contains
    O→

Since

V has been known as a vector space, so
W should satisty the 8 axiom of vector as well
⟹ verify the closedness of
W

  • Theorem
    Let
    V     a vector space and
    W     is a subspace of
    V     if and only if
    1. O→∈W       (the same
      O→       in
      V      )
    2. x→+y→∈W for x→, y→∈W       (closed under addition)
    3. cx→∈W for any c∈R, any x→∈W       (closed under multiplication)
    • Example1
      Let
      V       be a vector space,
      V       itself is a subspace of
      V
      1. O→∈V
      2. x→+y→∈V for x→, y→∈V
      3. cx→∈V for any c∈R, any x→∈V
    • Example2
      Z={O→}       is a subspace of
      V
      1. O→∈Z
      2. O→+O→=O→∈Z
      3. cO→=O→∈Z
    • Example3
      M      : The matrix space of all real
      2×2       matrices
      M={[abcd]∣a, b, c, d∈R}Let D={[e00f]∣e, f∈R}
      The collection of all
      2×2       diagonal matrices
      D⊆M       is a subspace
      1. [0000]∈D
      2. [e100f1]+[e200f2]∈D
      3. [ce100cf1]∈D
    • Example4
      R3       is a vector space, and
      W1={[a1, a2, a3]∈R3∣a1=3a2, a3=a2}∀x→, y→∈W1, x→=[x1, x2, x3], x1=3x2, x3=x2y→=[y1, y2, y3], y1=3y2, y3=y21. [0, 0, 0]∈W12. x→+y→=[x1+y1, x2+y2, x3+y3]=[3x2+3y2, x2+y2, x2+y2]∈W13. cx→=[cx1, cx2, cx3]=[c(3x2), cx2, cx2]∈W1⟹W1 is a subspace of R3W2={[a1, a2, a3]∈R3∣a1=a3+2}[0, 0, 0]∉W2⟹W2 is not a subspace of R3

Four Fundamental Subspaces

  1. Column space of
    Am×n    :
    C(A)≜{Ax→∣x→∈Rn}
  2. Row space of
    Am×n    :
    C(AT)≜{ATy→∣y→∈Rm}
  3. Null space of
    A    :
    N(A)≜{x→∈Rn∣Ax→=O→}
  4. Left Null space of
    A    :
    N(AT)≜{y→∈Rm∣ATy→=O→} (ATy→=O→⟹y→TA=O→)

Column Space

C(A) (The column space of
A) is defined by all linear combinations of the columns of
A
Let A=[104323], C(A)={Ax→∣x→∈R2}={x1[142]+x2[033]∣x1, x2∈R}
Hence
C(A) is a collection of
b→ for which
Ax→=b→ is solvable, i.e.
Ax→=b→ if and only if
b→∈C(A)

  • Example1

    A=I2×2=[1001]C(A)={Ax→∣x→∈R2}={x1[10]+x2[01]∣x1, x2∈R}

  • Example2

    A=[1224]C(A)={Ax→∣x→∈R2}={x1[12]+x2[24]∣x1, x2∈R}={(x1+2x2)[12]∣x1, x2∈R}Let x1+2x2=y⟹{y[12]∣y∈R}⟹The solution of Ax→=b→ form a line

  • Example3

    A=[123004]2×3C(A)={x1[10]+x2[20]+x3[34]∣x1, x2, x3∈R}={x1[10]+2x2[10]+x3[34]∣x1, x2, x3∈R}={y[10]+x3[34]∣y=x1+2x2, x3∈R}

  • Theorem

    C(A) is a subspace of
    Rm

    • Proof
      1. AO→n×1=O→m×1⟹O→m×1∈C(A), O→m×1∈Rm
      2. ∀x→, y→∈C(A), we have x→m×1=Ax′→n×1 and y→m×1=Ay′→n×1 where x′→, y′→∈Rnx→+y→=Ax′→+Ay′→=A(x′→+y′→)=Az′→ where z′→=x′→+y′→∈Rn⟹x→+y→∈C(A)
      3. ∀c∈Rcx→=cAx′→=A(cx′→)=Az″→ where z″→=cx′→∈Rn⟹cx→∈C(A)

Null Space

  • Define: The null space of

    Am×n consists of all solutions of
    Ax→=O→    , denoted by
    N(A)     , i.e.
    N(A)≜{x→∈Rn∣Ax→=O→}

  • Theorem

    N(A) is a subspace of
    Rn

    • Proof
      1. Let x→=O→AO→=O→⟹O→∈N(A)
      2. ∀x→, y→∈N(A), Ax→=O→, Ay→=O→A(x→+y→)=Ax→+Ay→=O→+O→=O→⟹x→+y→∈N(A)
      3. ∀c∈RA(cx→)=cAx→=O→⟹cx→∈N(A)

  • Example1

    A=[1236]
    Solve
    Ax→=O→
    [AO→]=[120360]→[120000]⟹ x1+2x2=0⟹ x1=−2x2⟹N(A)={c[−21]∣c∈R}
    (
    N(A)     is a line consisting all solutions to
    Ax→=O→     )

  • Example2

    A=[1238]2×2, B=[123824610]4×2, C=[122438616]2×4A=[1238]→[1001]⟹x1=0x2=0⟹The only solution to Ax→=O→ is x→=O→ i.e. N(A)={O→}B=[123824610]→[10010000]⟹x1=0x2=0⟹Again, the only solution to Bx→=O→ is x→=O→ i.e. N(B)={O→}C=[122438616]→[10200102]⟹x1+2x3=0x2+2x4=0⟹x1=−2x3x2=−2x4⟹N(C)={[x1x2x3x4]∣x1=−2x3, x2=−2x4, x1,x2,x3,x4∈R}={[−2x3−2x4x3x4]∣x3,x4∈R}={x3[−2010]+x4[0−201]∣x3,x4∈R}
    i.e.
    N(C)     has all linear combination of
    [−2010], [0−201]

    Since

    N(C) is a subspace, any linear combination of special solutions (
    [−2010], [0−201]    ) lies in
    N(C)

    Variables corresponding to pivot columns → pivot variables, ex.

    x1,
    x2
    Variables corresponding to free columns → free variables, ex.
    x3    ,
    x4

  • Example3

    A=[1302−10014−31316−4]→R=[1302−10014−300000]⟹x1+3x2+2x4−x5=0x3+4x4−3x5=0⟹N(A)=N(R)={[x1x2x3x4x5]∣x1+3x2+2x4−x5=0x3+4x4−3x5=0, x1,x2,x3,x4,x5∈R}={[−3x2−2x4+x5x2−4x4+3x5x4x5]∣x2,x4,x5∈R}={x2[−31000]+x4[−20−410]+x5[10301]∣x2,x4,x5∈R}
    Let
    N=[−3−211000−43010001]    ,
    span(N)=     linear combination of the column vectors in
    N=N(A)=N(R)
    Rearrange
    N     (null space matrix)
    →N′=[−3−210−43100010001]=[−F2×3I3×3]r pivot variablesn−r free variables
    Rearrange
    R→R′=[1032−10104−300000]=[I2×2F2×3O1×2O2×3]r pivot rowsm−r zero rows
    Check
    R′N′=[I2×2F2×3O1×2O2×3][−F2×3I3×3]=[−F2×3+F2×3O1×3+O1×3]=O3×3

  • A general approach to find

    N(A)

    1. Identify special solutions
    2. span these special solutions to get
      N(A)

  • Summary: To find

    N(A) of
    A

    1. Use Gauss-Jordan Elimination
      A→R       (reduced row echelon form)
    2. Identify pivot variables and free variables
    3. Identify special solutions (the number of special solutions is equal to the number of free variables)
    4. span({special solutions})=N(A)

    Note that in step 2, if there is no free variables

    ⟹N(A)={O→}

  • Define: The dimension of a null space is the number of gree variables,
    i.e.

    dim(N(A))=the number of special solutions=the number of free variables

  • Define: The rank of

    Am×n ,
    r=rank(A)=the number of pivots

Span

  • Define: We defined the span of
    U     to be all linear combination of vectors in
    U     , denoted by
    span(U)
  • span(U)     is always a subspace of
    V(Rm)
  • Example
    U={u1→, u2→, ⋯, un→}⊆V    , i.e.
    ui→∈V     as well
    span(U)={c1u1→+c2u2→+⋯+cnun→∣c1, c2, ⋯, cn∈R}
  • C(A)=span(U)     where
    U={column vectors of A}

The Complete Solution to
Ax→=b→

  • Define: For

    Ax→=b→, when
    {b→=0→         Homogeneous linear equationb→≠0→ Non-homogeneous linear equation

  • Example

    Ax→=b→, A=[130200141316], b→=[b1b2b3][Ab→]=[1302b10014b21316b3]→[1302b10014b20000b3−b2−b1]
    Ax→=b→     has solutions if and only if
    b3−b2−b1=0⟹b3=b1+b2
    Assume
    B3=b1+b2, {x1+3x2+2x4=b1x3+4x4=b2⟹{x1=−3x2−2x4+b1x3=−4x4+b2x→=[x1x2x3x4]=[−3x2−2x4+b1x2−4x4+b2x4]=x2[−3100]+x4[−20−41]⏟element of N(A)+[b10b20]⏟particular solution

    1. when b1=b2=0, b→=[b1b2b3]=[b1b2b1+b2]=0→x→n≜x2[−3100]+x4[−20−41]∈N(A) is a solution Ax→=0→In fact N(A)={x2[−3100]+x4[−20−41]∣x2,x4∈R}
    2. For any case of
      x2,x4      ,
      x2[−3100]+x4[−20−41]+[b10b20]       is a solution to
      Ax→=b→      , setting
      x2=x4=0,x→p≜[b10b20]       is a particular solution to
      Ax→=b→
    3. The solution space of
      Ax→=b→       is
      K={x→p}+KH       where
      KH       is the solution space of homogeneous equation
      Ax→=0→      , i.e.
      N(A)

  • Theorem
    Let

    K be the solution set of
    Ax→=b→    ,
    KH     be the solution set of the corresponding homogeneous system
    (Ax→=0→)    , then for any solution
    x→p     to
    Ax→=b→    ,
    K=x→p+KH={x→p+x→n∣x→n∈KH}

    • Proof
      Let
      x→p       be the solution s.t.
      Ax→p=b→
      Let
      x′→       be a solution to
      Ax→=b→
      Ax′→=b→       i.e.
      x′→∈K
      A(x′→−x→p)=Ax′→−Ax→p=b→−b→=0→
      ∴x′→−x→p       is a solution to
      Ax→=0→⟹x′→−x→p∈KH
      Let
      x→n=x′→−x→p∈KH⟹x′→=x→p+x→n
      ∴K⊆{x→p}+KH ...(i)
      Next, let
      x′→∈{x→p}+KH⟹x′→=x→p+x→n       for some
      x→n∈KH
      Ax′→=Ax→p+Ax→n=b→+0→
      ∴x′→       is a solution to
      Ax→=b→⟹x′→∈K⟹{x→p}+KH⊆K ...(ii)
      From
      (i)       and
      (ii)      ,
      K={x→p}+KH
Last changed by 
 
TC
0
109

Read more

Linear Algebra Note 7

Linear Algebra Note 7 Eigenvectors & Eigenvalues (2021.12.22 ~ 2021.12.29) Singular Value Decomposition (SVD) Theorem $A_{m \times n} = U_{m \times m} \Sigma_{m \times n} V_{n \times n}$ where $U$ and $V$ are orthogonal matrix (orthogonal columns) $\implies U^{-1} = U^T, V^{-1} = V^T$ $\Sigma = \left[\begin{array}{c c c c c c c} \sigma_1 & & & & & & \ & \sigma_2 & & & & & \

Dec 30, 2021
Linear Algebra Note 6

Linear Algebra Note 6 Eigenvectors & Eigenvalues (2021.12.08 ~ 2021.12.17) Eigenvalue Decomposition (EVD) or Matrix Diagonalization Theorem Let $A$ be a $n \times n$ matrix. $A$ can be decomposed into $A=S \Lambda S^{-1}$ or $\Lambda = S^{-1}AS$ (diagonalization) where $\Lambda = \left[\begin{array}{c c c c} \lambda_1 & 0 & \cdots & 0 \ 0 & \lambda_2 & \cdots & 0 \

Dec 22, 2021
Linear Algebra Note 5

Linear Algebra Note 5 Determinants (2021.11.24) Properties 9. $\begin{vmatrix} AB \end{vmatrix} = \begin{vmatrix} A \end{vmatrix}\begin{vmatrix} B

Dec 3, 2021
Linear Algebra Note 4

Linear Algebra Note 4 Orthogonality (2021.11.10 ~ 2021.11.17) Example $A = \left[\begin{array}{c c} 1 & 0 \ 1 & 1 \ 1 & 2 \ \end{array}\right],\ \vec{b} = \left[\begin{array}{c} 6 \

Nov 25, 2021
Read more from TC
Published on HackMD