# Homography Transformation
<style>
figure {
padding: 4px;
margin: auto;
text-align: center;
}

figcaption {
background-color: black;
color: white;
font-style: italic;
padding: 1px;
text-align: center;
}
</style>

:::success
Homography matrix describes the mapping relationship of points on the same ==plane== between different images.
:::

## Application
Before delving in, let's look at some cool applications about homography transformation first!

**application 1: align pictures**
<figure>
<img src="https://hackmd.io/_uploads/ryYXswX4C.png" width="400px">
</figure>

**application 2: camera calibration**
<figure>
<img src="https://hackmd.io/_uploads/ryN9oPQ4C.png" width="400px">
</figure>

**application 3: image stitching**
<figure>
<img src="https://hackmd.io/_uploads/SJCenPX4A.png" width="400px">
</figure>

## Homography Matrix Introduction

<figure>
<img src="https://hackmd.io/_uploads/Sybbwm9QR.png" width="400px">
</figure>

From [camera model post](https://hackmd.io/@jackyyeh/BJxMZtUUT/%2FI2EeEnKSSW6606aA1Gkdcg), we know how to transform 3D world coordinates into image coordinates as follow:
$$
\begin{bmatrix}
u \\
v \\
1
\end{bmatrix}
= \frac{1}{Z}
\begin{bmatrix}
f_{x} & 0 & c_{x} \\
0 & f_{y} & c_{y} \\
0 & 0 & 1 \\
\end{bmatrix}
\begin{bmatrix} R & t \\ 0^{T} & 1 \end{bmatrix}
\begin{bmatrix}
x_{w} \\
y_{w} \\
z_{w} \\
1
\end{bmatrix}
$$

Then let us simplify expression:
$$
\begin{bmatrix}
u \\
v \\
1
\end{bmatrix}
= \begin{bmatrix}
m_{11} & m_{12} & m_{13} & m_{14} \\
m_{21} & m_{22} & m_{23} & m_{24} \\
m_{31} & m_{32} & m_{33} & m_{34} \\
\end{bmatrix}
\begin{bmatrix}
x_{w} \\
y_{w} \\
z_{w} \\
1
\end{bmatrix}
$$

The matrix $M$ is a 4 x 4 matrix. If the 3D world points lie on the same plane, we can consider $z_{w} = 0$, which simplifies the matrix M to a 3 x 3 matrix. For two different cameras, the image coordinates and 3D world point coordinates can be expressed as follows, where $M$ is a 3 x 3 matrix.

$$
\begin{bmatrix}
u_{1} \\
v_{1} \\
1
\end{bmatrix}
= M_{1}
\begin{bmatrix}
x_{w} \\
y_{w} \\
1
\end{bmatrix},
\begin{bmatrix}
u_{2} \\
v_{2} \\
1
\end{bmatrix}
= M_{2}
\begin{bmatrix}
x_{w} \\
y_{w} \\
1
\end{bmatrix}
$$

By combining the two equations above, we obtain the following equation, where $H$ is the homography matrix! The $H$ matrix relates pairs of matching points between the two images. In other words, ==the homography matrix H maps points on the same plane in 3D space to the image coordinates in the two cameras.==

$$
\begin{bmatrix}
u_{2} \\
v_{2} \\
1
\end{bmatrix}
= M_{2}M_{1}^{-1}
\begin{bmatrix}
u_{1} \\
v_{1} \\
1
\end{bmatrix}
= H
\begin{bmatrix}
u_{1} \\
v_{1} \\
1
\end{bmatrix}
$$

### Solve Homography Matrix
Next question is how to solve homography matrix?

We know that one correspondence contributes for 2 constraints. Homography matrix $H$ is a 3x3 matrix and has 8 degrees of freedom. So we need 4 corresponding pairs to solve it.

## Ref
- [三维重建基础与极几何](https://youtu.be/e417umfxFRk?list=PLFI1Cd4723_SQ-h8W0kT3igOoUCYjWRHG&t=4860)
- [从零开始一起学习SLAM | 神奇的单应矩阵](https://mp.weixin.qq.com/s?__biz=MzIxOTczOTM4NA==&mid=2247486191&idx=1&sn=3b33d748dd4cd035e429665ef1e40299&chksm=97d7ef78a0a0666ebdd86886241b19c0a77a0ec8fa5e1a1ff5de2ae3e89740ffc5516f942bab&scene=21#wechat_redirect)