# 初探影像去模糊  ---  --- Image blur can be caused by various factors during image capture: camera shake, in-scene motion, or out-offocus blur. $b=\phi(I;\theta)$ * $b$ : a blurry image, $I$ : a latent image * $\phi$ : is the image blur function * $\theta$ : a parameter vector --- ## Related work Traditional deblurring approaches often model image blur (since a longer exposure time) using a convolution operation as : $b=k*I+n$ * $b$ : a blurry image, $I$ : a latent image * $n$ : additive noise,, $∗$ : a convolution operator * $k$ : a blur kernel. --- What's the problem of this model？ * a restrictive blur model * the ill-posedness of the inverse problem --- Recent deep learning-based approaches overcome such limitations by learning a mapping from a blurry image to its corresponding sharp image from a large collection of data. ---  * [Deep Image Deblurring: A Survey](https://arxiv.org/abs/2201.10700) [CVPR 2022] --- ### Advantages and drawbacks of various models * The U-Net architecture has shown to be effective for image deblurring and low-level vision problems. * Alternative backbone architectures for effective image deblurring include a cascade of Resblocks or Denseblocks. --- * However, their performance is limited due to the lack of real-world blur datasets * 現有大多資料集大多適合用於 evaluation，不適合 training，因此將介紹可以用於訓練影像去模糊的資料集 --- ### GoPro Dataset * To synthetically generate blurred images, they capture sharp video frames using a high-speed camera, and blend them. * 極短的曝光時間造成合成影像不現實，無法真正合成出像真實世界拍到的影像 :-1:  --- ## Datasets [Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms](http://cg.postech.ac.kr/research/realblur/?fbclid=IwAR0pHzyOXFvimR2gG1NrYUuWkfVlHz1MZpIYb0JolJv8-O-ZHbC7QbuNjYo) [[ECCV 2020]](https://eccv2020.eu/) They build an image acquisition system to simultaneously capture geometrically aligned pairs of blurred and sharp images. ---  --- * training set : 3,758 image pairs of 182 different scenes. * test set : 980 image pairs of 50 different scenes. --- ## Image Acquisition System consists of a beam splitter and two cameras so that the cameras can capture the same scene.  --- * The cameras and beam splitter are installed in an optical enclosure to protect them from light coming from outside the viewing direction. * One camera captures a blurry image with a low shutter speed, while the other captures a sharp image with a high shutter speed. --- 後續還有些影像後處理這裡就不繼續介紹  --- ## [Image DeBlurring AutoEncoder](https://github.com/AryanSethi/No-Blur) * They used the UNet architecture to deblur * The network extracts the important features of the image while reducing its spacial features * Then Up-Sampling compact features aka recreating the original sized image but de-blurred 最近的 commit 都是改 [readme.md](https://github.com/AryanSethi/No-Blur/commit/1ef39bbfcb5840627c1419e35ffdf5cd766571f5) on 18 Jun 2021 --- ### Deblurring Results Trainable params: 8.60 M, Best loss (MSE): 0.0019  --- ## Dehazing vs. Deblurring 將過去使用在去霧霾模型應用在去模糊？  --- 以下拔除 Image Preprocessing block (Min-Max) |Method|Params|Input size|Best loss| FPS | |--|--|--|--|--| |Original Ver.|$280.35K$|$256\cdot 256$|$0.0019$| $111$| --- ### Original Ver. --- Blur ( $512\cdot 512$ )  --- Delur ( Resize $512\cdot 512$ )  --- ## Restormer (CVPR 2022) * [Restormer: Efficient Transformer for High-Resolution Image Restoration (CVPR 2022)](https://github.com/swz30/Restormer) * Used GoPro dataset * Trainable params : **26.13M**  --- * 處理器採 * RTX 2060 * i7-10700 2.90 GHz * 執行時間 | Size | FPS | |--|--| |$128\cdot 128$| $12.1$ | |$256\cdot 256$| $3.8$ | |$512\cdot 512$ | $1.2$ | --- Blur ( $512\cdot 512$ )  --- Deblur ( $512\cdot 512$ )  --- Blur ( $512\cdot 512$ )  --- Deblur ( $512\cdot 512$ )  --- ## 總結 * 我的 Dehazing 模型用 RealBlur dataset 訓練起來效果很差，可能是資料及過於模糊達到監督式學習極限 * Restormer 很強，恢復品質有一定水準 * RealBlur 資料集比 GoPro 資料集真實 * 在訓練百香果分級時使用資料擴增手法學習模糊影像 [TRANSFORMING AND AUGMENTING IMAGES](https://pytorch.org/vision/stable/transforms.html) --- ## 參考資料 * [Deep Image Deblurring: A Survey (CVPR 2022)](https://arxiv.org/abs/2201.10700) * [Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms (ECCV 2020)](http://cg.postech.ac.kr/research/realblur/assets/pdf/RealBlur_eccv2020.pdf) * [Image DeBlurring AutoEncoder Network](https://github.com/AryanSethi/No-Blur) * [Restormer: Efficient Transformer for High-Resolution Image Restoration (CVPR 2022)](https://github.com/swz30/Restormer)