---
tags: Giang's linear algebra notes
---

# Chapter 3: Linear transformation

[toc]

## Linear transformation

**Transformation**: A transformation from one set $X$ to another set $Y$ is a function $f:X\to Y$. We call $X$ the domain of $f$, and $Y$ the range of $f$

**Linear transformation**: A linear transformation is a transformation which preserves additive and scalar multiplicative structure. Formally, let $X$ and $Y$ be vector spaces, and $T:X\to Y$ be a transformation. Then $T$ is a linear transformation if

1. $T$ preserves vector addition, i.e. $T(u+v)=T(u)+T(v),\forall u,v\in X$

2. $T$ preserves scalar multiplication, i.e. $T(av)=aT(v),\forall v\in X,a\in\textbf{R}$

*Examples of linear transformations*

1. Linear transformations defined by matrices

2. Differentiation as linear transformation

3. Sampling as linear transformation, i.e. $S(f)=[f(1),f(2),\dots,f(n)]$

4. Lagrange interpolation as linear transformations, i.e. $S:\textbf{R}^{n}\to P_{n-1}(\textbf{R})$ where $S(y)=f(x)$

## Null spaces and nullity

**Null space**: Let $T:V\to W$ be a linear transformation. Then the null
space of $T$ is defined as $$\text{null}(T)=\{v\in V:T(v)=0\}$$ The null
space of $T$ can be called the *kernel of $T$* and be denoted as
$\text{ker}(T)$

**Null space as a subspace**. Given a linear transformation $T:V\to W$ then $\text{null}(T)$ is always a subspace of $V$

*Proof*

Since $\text{null}(T)\subseteq V$ and it is closed under addition and
scalar multiplication
- [ ]

>**NOTE**: If $T:V\to W$ is a linear transformation and $T$ is one-to-one if and only if $\text{null}(T)=\{\textbf{0}\}$

**Nullity of $T$**: Let $T$ be a linear transform, then the nullity of $T$ is defined as $$\text{nullity}(T)=\dim[\text{null}(T)]$$

## Range and rank

**Range of linear transformation**. Let $T:V\to W$ be a linear transformation, then the range of $T$ is defined as
$$\text{range}(T)=\{T(v):v\in V\}$$

>**NOTE**: Sometimes, $W$ is called the range of $T$ instead

**Range as a subspace**. Given a linear transformation $T:V\to W$ then $\text{range}(T)$ is a subspace of $W$

*Proof*

Since $\text{range}(T)\subseteq W$ and it is closed under addition and
scalar multiplication
- [ ]

**Domain and target of linear transformation**. Let $T:V\to W$ be a linear transformation, then *the domain of $T$* is $V$ and *the target space* *of $T$* is $W$

**Rank of linear transformation**: The rank of a linear transformation $T$
is defined as $$\text{rank}(T)=\dim[\text{range}(T)]$$

## The dimension theorem

**Intuition of null space and range**:

1. Null space $\text{null}(T)$ is the set of all stuff which are sent to $\textbf{0}$ by $T$

2. $\text{rank}(T)$ measures how much information, i.e. degrees of freedom, is retained by $T$, i.e. when applying $T$ on a nonsingular matrix

**The dimension theorem**: Let $V$ be a finite-dimensional space and $T:V\to W$ be a linear transformation. Then
$$\text{nullity}(T)+\text{rank}(T)=\dim(V)$$
In other words, by applying $T$ on $V$, we lost $\text{nullity}(T)$ degrees of freedom

*Example*: Every polynomial of degree at most $4$ is the derivative of some
polynomial of degree at most $5$

**Theorem**: Let $V$ and $W$ be finite-dimensional vector spaces and $\dim(V)=\dim(W)$. Let $T:V\to W$ be a linear transformation, then $T$ is one-to-one if and only if $T$ is onto

*Proof*:

First, we prove that onto mapping implies one-to-one mapping. Suppose that $T$ is an onto mapping, i.e. $W\subseteq\text{range}(T)$, we have that $W=\text{range}(T)$ since $T$ is a mapping form $V$ to $W$. In this case
$$\text{rank}(T)=\dim(W)=\dim(V)$$
Thus $\text{nullity}(T)=0$. In other words, we have that
$$\forall v_{1},v_{2}\in V,T(v_{1})=T(v_{2})$$
if and only if $v_{1}=v_{2}+v_{n}$ for some $v_{n}\in\text{null}(T)$. But since $\text{nullity}(T)=0$, i.e. the basis of $\text{null}(T)$ is $\emptyset$, thus $\text{null}(T)=\text{span}(\emptyset)=\boldsymbol{0}$. Thus, $T(v_{1})=T(v_{2})$ if and only if $v_{1}=v_{2}$. This implies that $T$ is a one-to-one mapping. Now we prove that one-to-one mapping implies onto mapping. Suppose now that $T$ is an one-to-one mapping, we have that
$$|\{v:T(v)=\boldsymbol{0}\}|=1$$
Due to the linearity of $T$, we have that $v$ can only be $\boldsymbol{0}$, otherwise $T(av)=aT(v)=0$ for every scalar $a$, and consequentially $|\{v:T(v)=\boldsymbol{0}\}|>1$. Thus $\text{null}(T)=\{\boldsymbol{0}\}$ and $\text{nullity}(T)=0$, hence $\text{rank}(T)=\dim(V)$ due to *dimension theorem*. Let $B$ be a basis of $\text{range}(T)$, we have that $|B|=\dim(V)=\dim(W)$ and $B\subseteq W$, thus $B$ is a basis of $W$, hence $\text{span}(B)=W$. In other words
$$\text{range}(T)=W=\text{span}(B)$$
Thus $T$ is an onto mapping.
- [ ]

## Linear transformation and bases

**Linear transformation and spanning**. Let $T:V\to W$ be a linear transformation, and $\{v_{1},\dots,v_{n}\}$ be a set of vectors that
$$\text{\text{span}}(\{v_{1},\dots,v_{n}\})=V$$
Then
$$\text{span}(\{T(v_{1}),\dots,T(v_{n})\})=\text{range}(T)$$

**Linear transformation and linear independence**. Let $T:V\to W$ be a linear transformation, which is one-to-one, and $\{v_{1},\dots,v_{n}\}$ be a set of linearly independent vectors. Then $\{T(v_{1}),\dots,T(v_{n})\}$ is linearly independent

*Proof*

Since $T$ is an one-to-one mapping, it follows that $\text{null}(T)=\{\boldsymbol{0}\}$ and $\text{nullity}(T)=0$. Now we have that $\{T(v_{1})\}$ is linearly independent. Suppose that $\{T(v_{i})\}_{i=1}^{n-1}$ is linearly independent, and $\{T(v_{i})\}_{i=1}^{n}$ is linearly dependent, i.e.
$$\exists\{a_{i}\}_{i=1}^{n},\exists i\in\{1,\dots,n\},a_{i}\neq0\land\sum_{i=1}^{n}a_{i}T(v_{i})=0$$
We have that $a_{n}\neq0$ must hold, otherwise $\sum_{i=1}^{n-1}a_{i}T(v_{i})=0$ thus $a_{1}=\dots=a_{n-1}=a_{n}=0$, which implies that $\{T(v_{i})\}_{i=1}^{n}$ is linearly independent. Thus we have that
$$T(v_{n})=\sum_{i=1}^{n-1}\frac{a_{i}}{a_{n}}T(v_{i})$$
Since $T$ is one-to-one, this implies that
$$v_{n}=\sum_{i=1}^{n-1}\frac{a_{i}}{a_{n}}v_{i}$$
and hence $\{v_{i}\}_{i=1}^{n}$ is linearly dependent. This contradicts the assumption that they are linearly independent. Thus $\{T(v_{i})\}_{i=1}^{n}$ must be linearly independent
- [ ]

**Theorem**: Let $T:V\to W$ be an one-to-one and onto linear transformation, and $\{v_{1},\dots,v_{n}\}$ be a basis of $V$. Then $\{T(v_{1}),\dots,T(v_{n})\}$ is a basis of $W$. Conversely, if $\{T(v_{1}),\dots,T(v_{n})\}$ is a basis of $W$, then $\{v_{1},\dots,v_{n}\}$ is a basis of $V$

*Proof*

Since $T$ is an one-to-one mapping, it follows that $\{T(v_{i})\}_{i=1}^{n}$ is linearly independent. Since $T$ is an onto mapping and $\text{range}(T)\subseteq W$, it follows that $\text{range}(T)=W$. From theorem about *linear transformation and spanning*, it follows also that
$$\text{span}(\{T(v_{i})\}_{i=1}^{n})=\text{range}(T)=W$$
Thus $\{T(v_{i})\}_{i=1}^{n}$ is a basis of $W$
- [ ]

## Use basis to specify a linear transformation

**Theorem**: Let $V$ be a finite-dimensional vector space and $\{v_{1},\dots,v_{n}\}$ be a basis of $V$. Let $W$ be another vector space and $w_{1},\dots,w_{n}$ be some vectors in $W$. Then there exists exactly one linear transformation $T:V\to W$ that
$$\forall j\in\{1,\dots,n\},T(v_{j})=w_{j}$$

*Proof*:

We have that, since $\{v_{i}\}_{i=1}^{n}$ is a basis of $V$, it follows
that $\text{range}(T)=\text{span}(\{T(v_{i})\}_{i=1}^{n})$. Let
$v=\sum_{i=1}^{n}a_{i}v_{i}$ be an arbitrary vector in $V$, we have that
$$\begin{aligned}T(v) & =T(\sum_{i=1}^{n}a_{i}v_{i})\\ & =\sum_{i=1}^{n}a_{i}T(v_{i})\\ & =\sum_{i=1}^{n}a_{i}w_{i}\end{aligned}$$
Suppose that there exists another $\hat{T}:V\to W$ so that
$$\forall i\in\{1,\dots,n\},T(v_{i})=w_{i}$$
Consider the previous $v$, we have that
$$\hat{T}(v)=\sum_{i=1}^{n}a_{i}w_{i}$$
Thus, we have that
$$\forall v\in V,T(v)=\hat{T}(v)$$
Hence, $T=\hat{T}$. Thus, there exists exactly one transformation mapping $\{v_{i}\}_{i=1}^{n}$ to $\{w_{i}\}_{i=1}^{n}$
- [ ]

>**NOTE**: Given a basis of $V$, we can describe any linear transformation $T:V\to W$ in a unique way

## Coordinate bases

**Ordered basis**. Let $V$ be a finite dimensional vector space, then *an ordered basis of $V$* is an ordered sequence $\{v_{i}\}_{i=1}^{n}$ of vectors in $V$ so that $\{v_{i}\}_{i=1}^{n}$ is a basis of $V$

>**NOTE**: In some materials, ordered bases are also called *coordinate bases*

>**NOTE**: Ordered bases are important since they enable us to refer to "first basis vector" or "second basis vector"

**Standard ordered basis**

1. For $\textbf{R}^{n}$, it is $\{e_{1},\dots,e_{n}\}$ where $e_{j}$ is
the $j$th unit vector

2. For $P_{n}(\textbf{R})$, it is $\{1,x,\dots,x^{n}\}$

**Coordinate of vectors $v$ w.r.t a basis $\beta=\{v_{1},\dots,v_{n}\}$** is $(a_{1},\dots,a_{n})$ where
$$v=\sum_{i=1}^{n}a_{i}v_{i}$$
*The coordinate vector of* $v$ *relative to $\beta$* is then
$$[v]^{\beta}=\begin{bmatrix}a_{1}\\ \vdots\\ a_{n} \end{bmatrix}$$

>**NOTE**: By convention, $[v]_{\beta}$ refers to coordinate vector of $v$ relative to $\beta$ if $v$ is a row vector, and $[v]^{\beta}$ refers to coordinate of $v$ relative to $\beta$ if $v$ is a column vector

>**NOTE**: With coordinate vectors, we can represent any vector $v$ as a familiar column vector, provided that a basis $\beta$ is supplied

>**NOTE**: The choice of $\beta$ is important since different bases give different coordinate vectors

**Intuition of linear transformations**: Any operation $T:V\to\textbf{R}^{n}$ of sending a vector $v$ to its coordinate vector $[v]^{\beta}$

**$\textbf{R}^{2}$ plane and Cartesian coordinate system**. The flexibility in choosing bases underlies a basic fact about Cartersian grid structure, with its $x$ and $y$ axes: it is artifact. Concretely, $\textbf{R}^{2}$ plane is a very natural object, but our Caterian grid is not, e.g. $\textbf{R}^{2}$ has $(1,0)$ and $(0,1)$ as its axes but Cartesian grid does not necessarily have such coordinates. In mathematical's description, the $\textbf{R}^{2}$ is canonical but the Cartesian coordinate system is not. Here, "canonical" means that there is a natural way to define the object uniquely, without recourse to any artificial convention. As a consequence, we can shift one's coordinate system to suit our situation

>**NOTE**: In the majority of cases, the standard basis suffices, if for no reason other than tradition

## Matrix representation of linear transformation

By using an ordered basis of $V$, and another ordered basis of $W$, we can represent linear transformations from $V$ to $W$ as matrices. Consequentially, we can study linear transformations by focusing on matrices

**Theorem**: Let $V$ and $W$ be finite-dimensional vector spaces, $\beta=\{v_{1},\dots,v_{n}\}$ be an ordered basis of $V$, and $\gamma=\{w_{1},\dots,w_{n}\}$ be an ordered basis of $W$. Let $T:V\to W$ be a transformation. Let $v\in V$ where
$$[v]^{\beta}=\begin{bmatrix}x_{1}\\ \vdots\\ x_{n} \end{bmatrix}$$
and $T(v)\in W$ where
$$[T(v)]^{\gamma}=\begin{bmatrix}y_{1}\\ \vdots\\ y_{m} \end{bmatrix}=\sum_{i=1}^{n}x_{i}\cdot[T(v_{i})]^{\gamma}$$
Assume also that
$$T(v_{j})=\sum_{i=1}^{m}a_{ij}w_{i}$$
Then the relation between $[v]^{\beta}$ and $[T(v)]^{\gamma}$ is
$$\begin{bmatrix}y_{1}\\ \vdots\\ y_{n} \end{bmatrix}=\begin{bmatrix}a_{11} & \cdots & a_{1n}\\ \vdots & \ddots & \vdots\\ a_{m1} & \cdots & a_{mn} \end{bmatrix}\cdot\begin{bmatrix}x_{1}\\ \vdots\\ x_{n} \end{bmatrix}$$

*Proof*

The idea is, coordinate vector of linear combination of vectors is linear combination of coordinate vectors. We have that
$$\begin{aligned}
[T(v)]^{\gamma}=\begin{bmatrix}y_{1}\\
\vdots\\
y_{n}
\end{bmatrix} & =\begin{bmatrix}[T(v_{1})]^{\gamma} & \cdots & [T(v_{n})]^{\gamma}\end{bmatrix}\cdot\begin{bmatrix}x_{1}\\
\vdots\\
x_{n}
\end{bmatrix}\\
& =\begin{bmatrix}a_{11} & \cdots & a_{1n}\\
\vdots & \ddots & \vdots\\
a_{m1} & \cdots & a_{mn}
\end{bmatrix}\cdot\begin{bmatrix}x_{1}\\
\vdots\\
x_{n}
\end{bmatrix}\end{aligned}$$
- [ ]

**Matrix representation of $T$ w.r.t the bases $\beta$ and $\gamma$** is
$$[T]_{\beta}^{\gamma}=\begin{bmatrix}a_{11} & \cdots & a_{1n}\\` \vdots & \ddots & \vdots\\ a_{m1} & \cdots & a_{mn} \end{bmatrix}$$
Especially, when $\beta=\gamma$ and $V=W$, $[T]_{\beta}^{\gamma}$ is written as $[T]_{\beta}$

>**NOTE**: $[T(v)]^{\gamma}=[T]_{\beta}^{\gamma}\cdot[v]^{\beta}$

**From matrix representation of $T$ to $T$**

1. Write $v$ in terms of $\beta$, obtaining $[v]^{\beta}$

2. Compuse $[T]_{\beta}^{\gamma}\cdot[v]^{\beta}$ to obtain $[T(v)]^{\gamma}$

3. Use $\gamma$ to convert $[T(v)]^{\gamma}$ back to $T(v)$

>**NOTE**: The procedure above can be represented as
$$v\to[v]^{\beta}\to[T]_{\beta}^{\gamma}\cdot[v]^{\beta}\to[T(v)]^{\gamma}\to T(v)$$

## Things to do with linear transformation

**Operations to do with linear transformation**

1. Addition

2. Sclar multiplication

3. Multiplication, under certain conditions

**Linear transformation addition** Let $V$ and $W$ be vector spaces, $S:V\to W$ and $T:V\to W$ be linear transformations. Then $S+T:V\to W$ is defined as
$$(S+T)(v)=S(v)+T(v)$$
With conditions for addition are

1. $S$ and $T$ have the same domain

2. $S$ and $T$ have the same target space

>**NOTE**: $S+T$ is a linear transformation

**Linear transformation scalar multiplication** Let $V$ and $W$ be vector spaces, $T:V\to W$ be a linear transformation, and $c$ be a scalar. Then $cT:V\to W$ is defined as $$(cT)(v)=c[T(v)]$$

>**NOTE**: $cT$ is a linear transformation

**Space of linear transformations from $V$ to $W$** is denoted as
${\mathcal L}(V,W)$

>**NOTE**: ${\mathcal L}(V,W)$ is a subspace of ${\mathcal F}(V,W)$, i.e. the space of all functions from $V$ to $W$

**Linear transformation multiplication**. Let $U,V,W$ be vector spaces, $S:V\to W$ be a linear transformation from $V$ to $W$, and $T:U\to V$ be a linear transformation from $U$ to $V$. Then $ST:U\to V$ is defined as
$$(ST)(u)=S[T(u)]$$

>**NOTE**: $ST$ is a linear transformation

## Addition and multiplication of matrices

Let $V$ and $W$ be vector spaces with ordered bases $\beta,\gamma$ respectively. Let $S:V\to W$ and $T:V\to W$ be linear transformations. Then

1. $[S+T]_{\beta}^{\gamma}=[S]_{\beta}^{\gamma}+[T]_{\beta}^{\gamma}$

2. $[cT]_{\beta}^{\gamma}=c[T]_{\beta}^{\gamma}$

## Expansions

**Onto mapping**: A transformation $T:V\to W$ is onto if
$$\forall w\in W,\exists v\in V,T(v)=w$$