# Image Matching
## Reconstruct 3D scenes from 2D

---
<H2>
Problem Statemenet
</H2>

Your best camera may just be the phone in your pocket. You might take a snap of a landmark, then share it with friends. By itself, that photo is two-dimensional and only includes the perspective of your shooting location. Of course, many people may have taken photos of that same landmark. If we were able to combine all of our photos, we may be able to create a more complete, three-dimensional view of any given thing and this is what we aim in this project.
Our objective is building a 3D model of a scene given an unstructured collection of images taken around it.

<h2>
Solution
</h2>

The process of reconstructing a 3D model of an environment from a collection of images is called Structure from Motion (SfM).
Basically it involves two main stages:

**Correspondence search:-**

* For each image, a set of local features is generated to describe the points of interest of the image (key points).
* Next we can find overlapped images by searching images that have sufficient number of common points i.e.. ones having similar features.
* The next step is to verify the matches by finding a geometric transformation that correctly maps a sufficient number of points in common between two images.A homography, or the epipolar geometry is often used to describe the geometric relations between the cameras.

**Incremental Reconstruction:-**
* First of all, SfM initializes the model by two-view reconstruction to obtain a good initialization, it is preferable to start from a dense region of the scene graph.
* Once a model is established, new images can be registered to the model by known 2D-3D correspondences between the key points of the new images and the reconstructed 3D points that are associated with the key points of the previously registered images.
* A newly registered image may observe new points as well as existing reconstructed 3D points. Such new points can be added to the 3D reconstruction through triangulation if they are observed by at least one previously registered image. This process makes the 3D point cloud more dense.
* Bundle Adjusment then jointly refines the 3D reconstructed points and the camera parameters by minimizing a reprojection error, which is a distance between a point in image space projected from 3D reconstruction and the corresponding key.

<h2>
Timeline
</h2>

**Week 1:** `May 22 2023 - May 28 2023`
* Get comfortable with Pytorch framework.
* Understand concepts related to the problem statement:
* Structure from motion Algorithm
* SFM two view reconstruction
* Trangulation
* Bundle Adjustment

**Week 2:** `May 29 2023 - June 5 2023`
* Download the dataset,get familiarised with the datatypes and attributes.
* Implement a basic SFM model and run your code for few epochs.
* Explore some recent papers explaining new approches to improve performances and efficiency of SFM for 3D reconstruction.

**Week 3:** `June 6 2023 - June 12 2023`
* Experiment with recent sfm models and tuning their hyperparameters.
* Analyse the accuracy metrics and write a conclusion reporting qualitative and quantative results.

This is just a tentative timeline, you are free to move at your own pace.

<h2>
Resources
</h2>

[Dataset](https://www.kaggle.com/competitions/image-matching-challenge-2023/overview)

[Basic SFM Implementation](https://openaccess.thecvf.com/content_cvpr_2016/papers/Schonberger_Structure-From-Motion_Revisited_CVPR_2016_paper.pdf)

[Structure From Motion Survey Paper](https://www.researchgate.net/publication/322339391_Algorithms_and_Applications_of_Structure_from_Motion_SFM_A_Survey)