---
# System prepended metadata

title: Parallax-Tolerant Image Stitching Based on Robust Elastic Warping
tags: [paper]

---

{%hackmd SybccZ6XD %}
# Parallax-Tolerant Image Stitching Based on Robust Elastic Warping
http://web.cecs.pdx.edu/~fliu/project/stitch/dataset.html
###### tags: `paper`

## Abstract
- based on elastic warping

- step
    - analytical warping function: eliminate the parallax errors
    - warp the input image according to computed deformations
    - reprojecting the warped images
    - Bayesian model: remove the incorrect local matches
    - highly compatible with different transformation types

## INTRODUCTION

Image Stitching application
- surveillance
- immersive communication
- virtual reality

concentrate: accurate and efficient

step
- find feature points and projection bias
-  TPS model: make the bias more smooth
-  Bayesian model: remove the local outliers of the matching data.
-  change local warps to global transformation in order to preserved global projectivity
-  combine with global similarity transformation: suitable for another similarity transformation



## Related work
the comparison between global method and local method
![](https://i.imgur.com/aMp4tkt.png)
- global method: camera translation is negligible
- local method: more suitable for feature point

global approaches: assume images are captured rotationally only
- AutoStitch, which was proposed by Brown and Lowe [4]

parallax issues
- seam-driven stitching
    - ![](https://i.imgur.com/FJyoO5n.png)
    -  Evaluating the Cut
- content-preserving warps
    - ![](https://i.imgur.com/oeWOFKX.png)
- local adaptive transformations
    - this paper use this method 

## III. ROBUST ELASTIC LOCAL ALIGNMENT

### Elastic Local Alignment


- Given two overlapped images $I_p$ and $I_q$
- Matched point $p_i = (x_i, y_i)^T$, $q_i = (u_i, v_i)^T$

==find global homography==
- use AutoStitch [4](http://matthewalunbrown.com/papers/ijcv2007.pdf) to estimate global transformation
    - SIFT features are located at scale-space maxima/minima of a difference of Gaussian function [4](http://matthewalunbrown.com/papers/ijcv2007.pdf)
    - RANSAC 隨機找點，並找出轉換公式，選最多inliers的
    ![](https://i.imgur.com/atrIOzJ.png)
    $\hat x$ and $\hat x'$ are a pair of matching points
    H: global homography

==parallax error==
- The projection of $p_i$ in $I_q$: $p'_i = (x'_i, y'_i)^T$
- parallax error: projection bias $\text g = p'_i-q_i = (g_i, h_i)^T$
    - g: the deformation in the x direction
    - h: the deformation in the y direction
![](https://i.imgur.com/urLFFlQ.png)

==energy function for optimal warp==
alignment and smoothness
Previous section get: $\text g = (g_i, h_i)^T$
- Overall energy function: $J_\lambda = J_D + \lambda J_S$
    - $\lambda$: balance the two terms, 0.1%
![](https://i.imgur.com/dieHNJI.png)
- correspond optimal solution using TPS
- ![](https://i.imgur.com/JajXeX2.png)

==accelerate the computation==
grid mesh
![](https://i.imgur.com/y9ZaRUD.png)
- find feature point: from AutoStitch[4]
- establish grid: accelerate the computation
- destortion

### Bayesian Refinement of Feature Matches
position-error and mismatch problems: remove outliers

previous: RANSAC remove
problem: mismatch
- inliers might be removed if their projection biases are larger than the threshold
- outliers might be preserved if their projection biases are less than the threshold.

previous: DLT remove
problem: fail for areas with no sufficient features.
- local homography in R = 50 small than r = 5

proposed method
![](https://i.imgur.com/yLfitP1.png)
in the TPS properties, the w is normal distribution
Remove outliers when $|w_i|>3\sigma_w$

remove outliers algorithm
![](https://i.imgur.com/QuThYNg.png)

### Smooth Transition to Global Transformation

if use the transformation from the above we get, it will suffer from the over-fitting problems in the non-overlapping region.

Suppressed deformation
- $\text g_s(x,y)= \eta \text g(x,y)$
- η(scale parameter) is gradually changed from 1 to 0 when moving away from the overlapping region
    - ![](https://i.imgur.com/SpvIlTG.png)
    - ![](https://i.imgur.com/9yPd6m2.png)
    - ![](https://i.imgur.com/RpIUxxw.png)
        - K is a scale parameter, 5

### Combination With Global Similarity Transformation
![](https://i.imgur.com/AkBKTwl.png)
if use the global homography, it can lead to projective distortions
- Global Similarity Transformation: good performance in mitigating projective distortions
- simeilarity transformation: choose the lowest one don't have many rotation

using the techniques proposed in ANAP [12].
![](https://i.imgur.com/WpmHoPp.png)
- $u_h$ and $u_s$: weighting coefficients
- $u_h$ linearly varies from 1 to 0 across the source image $I_q$
![](https://i.imgur.com/C2Fa3MB.png)

## V. EXPERIMENTS

feature point: guassian
$\lambda$: 0.1%
$K$: 5

### Comparison of Stitching Quality

==from image==
![](https://i.imgur.com/x17jRD1.png)
Row 1: global homography didn't use outlier removal
Rows 2 to 4: global RANSAC to remove outliers
Row 5: proposed method and use RANSAC to remove outliers

Rows 6 to 8: local outlier removal techniques
![](https://i.imgur.com/7TACW8q.png)

==from Quantitative evaluation==
use SSIM
![](https://i.imgur.com/eF99yn9.png)


### Flexibility Evaluation
![](https://i.imgur.com/ZRI6gwT.png)

### Comparison of Computational Efficiency
setting 2.0G-Hz CPU and 16 GB RAM.
constant cell size 10*10
![](https://i.imgur.com/YxagBiW.png)
constant cell size 100*100
![](https://i.imgur.com/LKq4lW4.png)
