{%hackmd SybccZ6XD %}
# Parallax-Tolerant Image Stitching Based on Robust Elastic Warping
http://web.cecs.pdx.edu/~fliu/project/stitch/dataset.html
###### tags: `paper`

## Abstract
- based on elastic warping

- step
- analytical warping function: eliminate the parallax errors
- warp the input image according to computed deformations
- reprojecting the warped images
- Bayesian model: remove the incorrect local matches
- highly compatible with different transformation types

## INTRODUCTION

Image Stitching application
- surveillance
- immersive communication
- virtual reality

concentrate: accurate and efficient

step
- find feature points and projection bias
- TPS model: make the bias more smooth
- Bayesian model: remove the local outliers of the matching data.
- change local warps to global transformation in order to preserved global projectivity
- combine with global similarity transformation: suitable for another similarity transformation

## Related work
the comparison between global method and local method
- global method: camera translation is negligible
- local method: more suitable for feature point

global approaches: assume images are captured rotationally only
- AutoStitch, which was proposed by Brown and Lowe [4]

parallax issues
- seam-driven stitching
- 
- Evaluating the Cut
- content-preserving warps
- 
- local adaptive transformations
- this paper use this method

## III. ROBUST ELASTIC LOCAL ALIGNMENT

### Elastic Local Alignment

- Given two overlapped images $I_p$ and $I_q$
- Matched point $p_i = (x_i, y_i)^T$, $q_i = (u_i, v_i)^T$

==find global homography==
- use AutoStitch [4](http://matthewalunbrown.com/papers/ijcv2007.pdf) to estimate global transformation
- SIFT features are located at scale-space maxima/minima of a difference of Gaussian function [4](http://matthewalunbrown.com/papers/ijcv2007.pdf)
- RANSAC 隨機找點，並找出轉換公式，選最多inliers的
$\hat x$ and $\hat x'$ are a pair of matching points
H: global homography

==parallax error==
- The projection of $p_i$ in $I_q$: $p'_i = (x'_i, y'_i)^T$
- parallax error: projection bias $\text g = p'_i-q_i = (g_i, h_i)^T$
- g: the deformation in the x direction
- h: the deformation in the y direction

==energy function for optimal warp==
alignment and smoothness
Previous section get: $\text g = (g_i, h_i)^T$
- Overall energy function: $J_\lambda = J_D + \lambda J_S$
- $\lambda$: balance the two terms, 0.1%
- correspond optimal solution using TPS
- 

==accelerate the computation==
grid mesh
- find feature point: from AutoStitch[4]
- establish grid: accelerate the computation
- destortion

### Bayesian Refinement of Feature Matches
position-error and mismatch problems: remove outliers

previous: RANSAC remove
problem: mismatch
- inliers might be removed if their projection biases are larger than the threshold
- outliers might be preserved if their projection biases are less than the threshold.

previous: DLT remove
problem: fail for areas with no sufficient features.
- local homography in R = 50 small than r = 5

proposed method
in the TPS properties, the w is normal distribution
Remove outliers when $|w_i|>3\sigma_w$

remove outliers algorithm

### Smooth Transition to Global Transformation

if use the transformation from the above we get, it will suffer from the over-fitting problems in the non-overlapping region.

Suppressed deformation
- $\text g_s(x,y)= \eta \text g(x,y)$
- η(scale parameter) is gradually changed from 1 to 0 when moving away from the overlapping region
- 
- 
- 
- K is a scale parameter, 5

### Combination With Global Similarity Transformation
if use the global homography, it can lead to projective distortions
- Global Similarity Transformation: good performance in mitigating projective distortions
- simeilarity transformation: choose the lowest one don't have many rotation

using the techniques proposed in ANAP [12].
- $u_h$ and $u_s$: weighting coefficients
- $u_h$ linearly varies from 1 to 0 across the source image $I_q$

## V. EXPERIMENTS

feature point: guassian
$\lambda$: 0.1%
$K$: 5

### Comparison of Stitching Quality

==from image==
Row 1: global homography didn't use outlier removal
Rows 2 to 4: global RANSAC to remove outliers
Row 5: proposed method and use RANSAC to remove outliers

Rows 6 to 8: local outlier removal techniques

==from Quantitative evaluation==
use SSIM

### Flexibility Evaluation

### Comparison of Computational Efficiency
setting 2.0G-Hz CPU and 16 GB RAM.
constant cell size 10*10
constant cell size 100*100