# AR watches try-on

## Proposal
3 months

## Breakdown
- Research — 60 hours
- Hand 3D landmarks estimation — 80 hours
- Arm segmentation — 80 hours
- Pose tracking — 40 hours
- Overlay 3D object — 40 hours
- Port computer vision models to iOS — 40 hours
- iOS SDK — 40 hours
- Demo app — 40 hours
- Documentation — 16 hours
- Project management — 24 hours

## Approach
There are two approaches.

First one is to print a marker, that can be recognized by the app to overlay 3D-model, and place it on the wrist. This is how https://ar-watches.com did it. This is not convenient for users, and their app has 1 star rating at the app store. We do not recommend to go this way.


The second one is to do it like AR apps with the face masks. They recognize face landmarks and then overlay mask image.



### Hand landmarks
We need to recognize 3D pose of the wrist, but it will be not precise in the close-up photo, as unlike face, it does not have any keypoints, like nose, eyes, mouth. But if we will recognize landmarks of the hand first, then we can estimate 3D-pose of the wrist much better.


### Arm segmentation
The hand will give us 3D-direction of the wrist, and not a 3D-model. We will apply hand segmentation model to know where wrist starts and ends. Then we can place watches correctly.


### Pose tracking
These algorithms will have some error and 3D model will jitter if we won't apply filtering. Pose tracking will help to smooth animation and minimize 3D model shaking.

### 3D model overlay
We will use open-source 3D model of the watch. It will be rendered on the photo from a camera. 3D position and cutting planes will be calculated based on the algorithms above. Other 3D models of watches can be added later and their modeling is out of current scope.


### Demo app
Demo app will not have any UI, just will show video stream from the camera with overlayed watches on wrist. It will work on iOS.

### Documentation
There will be 2 documents: to describe the computer vision pipeline and to describe requirements for 3D models of watches.

---
###### tags: `proposal`