# AR Face tracking

## Helpfull links

* https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@3.0/manual/face-manager.html

## Implementing Face Tracking

You wil learn how to set up AR Face Tracking in order to create custom face overlays like you see in social media apps

### Set Up Your Project

See <i class="fa fa-book"></i>[Setting up AR Foundation in Unity](https://hackmd.io/@NathanSegers/HyHqU3biY/%2Fd0nGGPFaTuubZdd5Igbv9g)

### Add an AR Default Face
Select the **AR Session Origin** Game Object in the Hierarchy, click on the **Add Component** Button, and use the search bar that opens to look up AR Face Manager. When the **AR Face Manager** option appears, clicking on it will add it to your AR Session Game Object.


In the hierarchy panel, we’ll create a new AR Default Face by right clicking and then **XR > AR Default Face**. The AR Default Face will automatically map a 3D model of a face to your own, so that you can start adding your own customizations.


Now, we’ll need to save this AR Default Face as a Prefab. In your project window, create a new folder called “Prefabs.” Create an AR Default Face Prefab by dragging and dropping the game object into the Prefabs folder.



To finish up, make sure that the **AR Session Origin** is using the **AR Default Face**. If you select the AR Session Origin, you’ll see that the **Face Prefab** field is empty. You can drag and drop the **AR Default Face prefab** into that field. It should update and look like this:


Expand the AR Session Origin GameObject to find the AR Camera Game Object. Select the **AR Camera** and use the inspector to find the AR Camera Manager script. Set the **Facing Direction** to “User”.


Now, try testing your application by building and running your project (double check to make sure that your build settings are set to the correct platform.

## Import Textures to use for custom AR Faces

Now that you have face tracking set up, let’s add some textures to create a custom face for ourselves. In order to do that, you’ll need to head to the Asset Store. Make sure you’re using the Unity account that is currently signed into the editor.

Unity has [premade textures on the Asset Store](https://assetstore.unity.com/packages/essentials/asset-packs/ar-face-assets-184187) that you can turn into materials in Unity. Make sure to Add the Textures and then import them into your project. You’ll need to make sure that you import the assets in the window that opens up in Unity as


Once the textures are imported, you’ll see a new folder inside of Unity. We’ll play around with the **Pop Face texture** to apply onto a new face model.

First, create a new AR Default Face. Right click in the Hierarchy and select **XR > AR Default Face**. Let’s make sure to rename it, so that we don’t get confused with our existing model. You can right click on the AR Default Face in the Hierarchy and select **Rename**. We can call it something like Pop Face.

Once we’ve renamed the default face, we’ll add our new texture onto it. Inside of the **Project Window** in the **Textures folder**, there is an image called Pop Face. We can change the texture of the model by clicking on our new **Pop Face texture** and dragging and dropping it onto the game object in the hierarchy. You can see the texture has been applied by selecting the Pop Face, checking the **Inspector**, and look for the Pop Face material.


Now, turn the Pop Face into a prefab by dragging it into the Prefabs folder. You can delete the Pop Face game object from the Hierarchy.


In order for our app to use our new face, make sure to select the AR Session Origin and drag the Pop Face prefab into the Face Prefab area.


If you build and run our application again, you should see Pop Art covering our face!

If the mask is low on your face, you can go find the Pop Face Material by typing “Pop Face” in the search bar of the Project Window and select the Pop Face material (it should look like a sphere with a face on it).

When you see the material open in the **Inspector**, you can change the **Y Offset to -0.045** to have the texture align better to your face.


### Create Additional Faces

f you want to get extra fancy, you can play around with some of the other textures. If you create a new AR Default Face, you can try applying the Robot textures.

First, create and rename the AR Default Face. Then, in the **Project** window, go into the **Textures** folder and into the **Robot folder**. Drag the **Robot_Albedo** image onto the face.


This texture includes ambient occlusion maps and normal maps to add extra depth and detail. Unity automatically created **Materials** for you, when you applied those texture images to the face models.


If you select the **Robot_Albedo material**, you can see some new information appear in the **Inspector**.


Inside the Textures folder in the Robot folder, you’ll see there are some additional files called **Robot_occ** and **Robot_normal**. These are additional layers for our texture, referred to as **Ambient Occlusion Maps** and **Normal Maps**. These maps add additional detail to help add depth and detail to the texture.


You can add these to the **Robot_Albedo material** by dragging **Robot_normal** onto the **Normal Map field** and dragging the **Robot_occ** onto the **Occlusion field**.


Once you’ve added those maps, just make sure to follow the steps from before to turn the Robot face into a Prefab, set it as the **Face Prefab** in the **AR Session Origin**, and then Build and Run to test.