## Details on Camera Calibration
### What is the calibration process specifically for fisheye lenses?
Calibration process :
- Get a good calibration board with a square size of 4-5cm+
- execute the calibration code `python3 calibrate.py -s <squareSize> -ms <MarkerSize> -nx <NumberOfColumns> -ny <NumberOfRows> -brd <noardName>`
- Run the step2 with `-m procsss` to do run calibration on pre-captured data.

### Any special code ?
For chonky_boi we fine tuned the calibration process. But no major changes. Just added the `getOptimalNewCameraMatrix` to scale the output image.
- Our Calibration process uses the charuco board which can be generated from [here](https://calib.io/pages/camera-calibration-pattern-generator)
- Our Calibration process uses the available opencv functions for [calibration](https://docs.opencv.org/4.x/d9/d6a/group__aruco.html#gaa7357017aa9da857b487e447c7b13f11) and testing.

There is no special code. Same code is being mainlined. Calibration branch [link](https://github.com/luxonis/depthai/tree/peripheral_calibration)

- [Link](https://docs.opencv.org/4.x/d0/de3/citelist.html#CITEREF_Zhang2000) for the calibration paper in OpenCV
- Camera Intrinsics and Extrinsics explaination video [here](https://youtu.be/uHApDqH-8UE)
- Camera Calibration explaination video link [here](https://youtu.be/-9He7Nu3u8s)

### Instructions on how to read current/flash new camera calibration
#### Calibration Reader
We can read the calibration using `device.readCalibration()` once device is initialized.
For a normal reader script we can use the [calibration_reader.py](https://github.com/luxonis/depthai-python/blob/main/examples/calibration/calibration_reader.py)

#### Calibration Flash
Eeprom space has 2 space one for Factory Calibration and one for User Calibration.
- Factory Calibration can be done only with a Specific Magic Code which is used during calibration in Factory.
- User Calibration section can be overwritten by the user using `device.flashCalibration2(calibData)`

Script to do the above is [calibration_flash.py](https://github.com/luxonis/depthai-python/blob/main/examples/calibration/calibration_flash.py)

## Report on depth performance
### What do you know about the poor performance?
#### performance on chonky_boi (Single Point test)
Below table might have disparity to depth scaling issue. This test will be re-done.
| Real Distance(m) | Estimated Distance (m)(center)|
| :---------: | :---------: |
| 0.5 | 0.35 |
| 1.0 | 0.62 |
| 1.5 | 0.92 |
| 2.0 | 1.5 |

#### performance on OAK-D-W (Single Point test)

| Real Distance(m) | Estimated Distance (m)(center)|
| :---------: | :---------: |
| 0.5 | 0.6 |
| 1.24 | 1.0 |
| 1.98 | 1.74 |
| 2.5 | 2.3 |
| 2.86 | 2.7 |
| 3.3 | 3.1 |

#### performance on OAK-D-W (ROI Framework)

| Real Distance(m) | GT Plane median RMSE(m)| Plane Fit Median RMSE(m) |
| :---------: | :---------: | :---------: |
| 1 | 0.0264 | 0.0161 |
| 2 | 0.0726 | 0.0598 |
| 3 | 0.0618 | 0.0340 |

- GT Plane RMSE is the estimated error over a ROI where we calculate how much each point from the depth deviate from that actual distance plane
- Plane fit is that we create a plane from the points in the ROI and see how much points in that deviate. It is more of a noise estimation.

### What are the theories and supporting evidence about performance?
There can be multiple reason why we notice poor performence in fisheye stereo depth.
1. When fisheye distortions are undistorted it adds extreme stretching in the corners and the tunnel effect at the center
2. Due to the above effect most neighbouring features looks similar when looking for a match to get the disparirty.
3. Considering the fisheye. the disparity between the images is also very less at closer distance. Hence leading to even less resolution at high distance. and it can even go in negative direction in extreme cases.

### Next steps - What still needs to be tested/developed?

ROI based new framework was developed [here](https://github.com/luxonis/depthai-experiments/pull/415) which calculates the Z-accuracy and spatial noise as described [here](https://www.intel.com/content/dam/support/us/en/documents/emerging-technologies/intel-realsense-technology/RealSense_DepthQualityTesting.pdf)

Test if the pile fits inside the FOV

### side-by-side comparison of various configurations

#### Lens M27210H08 for AR02534 (D/H/V 136/130/101) (Mx)
Testing in Progress

#### Lens M27210H08 for AR02534 (D/H/V 136/130/101) (Keembay)
Testing in Progress

#### Lens M27195H15 for AR0234 (H - 150 FOV approx)(Mx)
Testing in Progress

#### Lens M27195H15 for AR0234 (H - 150 FOV approx)(Keembay)
Testing in Progress

### Is there a formular relating depth performance to FOV, resolution, and distance to target? Theoretical or determined through testing?

-

### Fisheye vs wide-angle vs rectilinear comparison testing?

In progress