---
# System prepended metadata

title: AR watches try-on
tags: [proposal]

---

# AR watches try-on

## Proposal
3 months


## Breakdown
- Research — 60 hours
- Hand 3D landmarks estimation — 80 hours
- Arm segmentation — 80 hours
- Pose tracking — 40 hours
- Overlay 3D object — 40 hours
- Port computer vision models to iOS — 40 hours
- iOS SDK — 40 hours
- Demo app — 40 hours
- Documentation — 16 hours
- Project management — 24 hours


## Approach
There are two approaches.

First one is to print a marker, that can be recognized by the app to overlay 3D-model, and place it on the wrist. This is how https://ar-watches.com did it. This is not convenient for users, and their app has 1 star rating at the app store. We do not recommend to go this way.

![](https://i.imgur.com/1wry1yi.png)

The second one is to do it like AR apps with the face masks. They recognize face landmarks and then overlay mask image.

![](https://i.imgur.com/svXBHP0.jpg)

![](https://i.imgur.com/SUEexH8.png)


### Hand landmarks
We need to recognize 3D pose of the wrist, but it will be not precise in the close-up photo, as unlike face, it does not have any keypoints, like nose, eyes, mouth. But if we will recognize landmarks of the hand first, then we can estimate 3D-pose of the wrist much better.

![](https://i.imgur.com/7HM0NYE.jpg)


### Arm segmentation
The hand will give us 3D-direction of the wrist, and not a 3D-model. We will apply hand segmentation model to know where wrist starts and ends. Then we can place watches correctly.

![](https://i.imgur.com/YRF1DvV.png)


### Pose tracking
These algorithms will have some error and 3D model will jitter if we won't apply filtering. Pose tracking will help to smooth animation and minimize 3D model shaking.


### 3D model overlay
We will use open-source 3D model of the watch. It will be rendered on the photo from a camera. 3D position and cutting planes will be calculated based on the algorithms above. Other 3D models of watches can be added later and their modeling is out of current scope.

![](https://i.imgur.com/cNVEDMu.png)


### Demo app
Demo app will not have any UI, just will show video stream from the camera with overlayed watches on wrist. It will work on iOS.


### Documentation
There will be 2 documents: to describe the computer vision pipeline and to describe requirements for 3D models of watches.


---
###### tags: `proposal`