---
robots: noindex, nofollow
---

[toc]
# AR Image Tracking

## Helpfull links

* https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@4.2/manual/image-tracking.html
* https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/tracked-image-manager.html
## Implementing one object for all images
Let's go to AR Session Origin and add the “[AR Tracked Image Manager](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/tracked-image-manager.html)” Script after clicking the add components button.


So, here we will be having **three** components:

1. ***Reference Library:** The images which you want to track.*
1. ***Max number of moving images:** As the name suggests.*
1. ***Tracked Image Prefab:** The prefab to display after the image is tracked.*

So, as we have seen that we have to provide a Reference Library to the AR Tracked Image Manager Script, we will go ahead and create one.


* After creating the Reference Image Library You can now add Images to the Image Reference Library and give them appropriate names. You can get the images from [here](https://leho-howest.instructure.com/courses/16819/modules/items/550477)
**Note**: You can add as many images as you want just click on Add Images and drag the image inside it.

:::danger
**Limitations of ARCore Augmented Images**
As the actual implementation of the 2D image tracking is provided by the AR subsystem, different limitations apply depending on the platform you’re targeting. For Google’s ARCore, the most important properties are (read the full specs in the [official documentation)](https://developers.google.com/ar/develop/java/augmented-images):
* Static, flat images
* At least 15cm x 15cm physical size
* Provide the width (in meters) for quicker pose & size estimates
* ARCore still updates its size estimation at run-time
* Occupy at least 25% of the camera frame at run-time to be detected
What kind of data source is supported?
* Max. of 1,000 reference images in an on-device database
* Multiple databases are supported, but only one can be active at the same time
* 20 of these reference images can be tracked simultaneously
* Support for adding images at run-time
* Only stores meta-information about images (~ 6 kB / image)
:::
* You can also specify the size, which means that when AR Foundation will detect these images in the camera view is will try to calibrate and map the metrics of the virtual the physical world according to these size properties.
* In this example, the images that the mobile phone will see are in real life 0.16 meter. (=16 cm).


* Great, now as our Reference Image Library is ready, specify the “Prefab” that you want to be instantiated on top of each detected image.
* Now, drag your Reference Image Library and your Prefab inside the AR Tracked Image Manager Script of the AR Session Origin.
* Specify the number of image instances you expect to be detected. Keep in mind that detection is a compute intensive process. The more images you want to detect, the more battery power will be used.

## What if you want to instantiate a different object per image?

We will need to create a script. We will name it “DynamicImageTracker”.
Also make sure the Tracked Image Prefab property is back to “None” as we don’t want to default behavior to be active.

### Tracked Images –> 3D Models
Add a new script `DynamicImageTracker.cs` as component to the `AR Session Origin`. It will handle 2D image tracking.

We implement a 1:1 mapping between markers and 3D models. This means that you assign a different 3D model to each marker. Matching is done via the reference image name and the prefab name.

To enable this behavior, simply start by creating a public array of prefabs:

public GameObject[] ArPrefabs;

Next, assign the prefabs you imported before through the Unity editor.


Additionally, our script should store a reference to the instantiated prefab for each marker. The **string** key of the dictionary will be the name of the reference image, which must correspond to the prefab name.

private Dictionary<string, GameObject> _instantiatedPrefabs = new Dictionary<int, GameObject>();

### React to Changes of Tracked Images

The `AR Tracked Images Manager` provides an event for changes to tracked markers.
The RequireComponent attribute automatically adds the *ARTrackedImageManager* as dependencie.

[RequireComponent(typeof(ARTrackedImageManager))]

Subscribe to changes in `OnEnable()` and remove the subscription in `OnDisable().`

void OnEnable()
{
    _trackedImagesManager.trackedImagesChanged += _trackedImagesManager_trackedImagesChanged;
}
void OnDisable()
{
    _trackedImagesManager.trackedImagesChanged -= _trackedImagesManager_trackedImagesChanged;
}

The changes you get can have 3 possibilities, informing your script what changed since the last event was sent:

1. **Added**: only called once when a new tracked image was found.
1. **Updated**: check for the tracking state and hide your 3D model when needed. If the status is limited, this means that you’d have poor tracking or are not tracking at all if the reference image is not seen on the screen.
1. **Removed**: hide or delete your 3D model

### Handle new Tracked Images: Instantiate Prefab
The following listing contains the first part of our event handler method. Whenever our app discovers a new tracked image, our logic instantiates the prefab (if we do not already have an instance in our scene):

// Go through all tracked images that have been added
// (-> new markers detected) 
foreach (var trackedImage in eventArgs.added)
{
    // Get the name of the reference image to search for the corresponding prefab 
    var imageName = trackedImage.referenceImage.name;

    foreach (var curPrefab in ArPrefabs)
    {
        if (string.Compare(curPrefab.name, imageName, StringComparison.Ordinal) == 0
            && !_instantiatedPrefabs.ContainsKey(imageName))
        {
            // Found a corresponding prefab for the reference image, and it has not been 
            // instantiated yet > new instance, with the ARTrackedImage as parent 
            // (so it will automatically get updated when the marker changes in real life) 
            var newPrefab = Instantiate(curPrefab, trackedImage.transform.position, trackedImage.transform.rotation);
            // Store a reference to the created prefab 
            _instantiatedPrefabs.Add(imageName, newPrefab);
        }
    }
}

First, we go through the list of all added tracked images. Then, we get the name of the reference image (that was set in the `AR Reference Image Library`; it automatically corresponds to the file name if you didn’t change it).

Next, we go through the array of prefabs our handler script can instantiate. It compares the name of the prefab to the name of the reference image. If we have not instantiated this prefab so far and found a prefab with the same name as the reference image, our script continues with instantiating the 3D model.

The `AR Tracked Images Manager` creates a GameObject with the `ARTrackedImage` script attached for each new marker it is tracking (see the [AR Foundation documentation](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/tracked-image-manager.html#tracked-image-prefab) for details). This is what we get through the `trackedImage` variable we retrieve from `eventArgs.added`.

As the first parameter for the `Instantiate()` method, we provide the matched prefab (variable name: `curPrefab`). The second parameter is the `trackedImage.transform`. This automatically makes the auto generated `ARTrackedImage` the parent of our instantiated GameObject. Going forward, the translation and rotation of our instance will automatically be applied in accordance with its parent, which is glued to the 2D image tracked in the real world. This corresponds to the [recommended method from the docs.](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/manual/tracked-image-manager.html#tracked-image-prefab)

### Updated & Removed Images

Next, let’s handle updated and removed tracked images. Add these *two foreach-loops* to our `OnTrackedImagesChanged()` method:

foreach (var trackedImage in eventArgs.updated)
{
    var imageName = trackedImage.referenceImage.name;

    if (_instantiatedPrefabs.ContainsKey(imageName))
    {
        _instantiatedPrefabs[imageName].transform.position = trackedImage.transform.position;
        _instantiatedPrefabs[imageName].transform.rotation = trackedImage.transform.rotation;
    }
}

// Remove is called if the subsystem has given up looking for the trackable again.
// (If it's invisible, its tracking state would just go to limited initially).
// Note: ARCore doesn't seem to remove these at all; if it does, it would delete our child GameObject
// as well.
foreach (var trackedImage in eventArgs.removed)
{
    var imageName = trackedImage.referenceImage.name;

    if (_instantiatedPrefabs.ContainsKey(imageName))
    {
        // Destroy the instance in the scene.
        // Note: this code does not delete the ARTrackedImage parent, which was created
        // by AR Foundation, is managed by it and should therefore also be deleted by AR Foundation.
        Destroy(_instantiatedPrefabs[imageName]);
        _instantiatedPrefabs.Remove(imageName);
        // Alternative: do not destroy the instance, just set it inactive
        //_instantiatedPrefabs[trackedImage.referenceImage.name].SetActive(false);
    }
}

When a certain image is moved or rotated; we will need to update the scene information accordingly.

As for removed images: this is a bit unclear. At least with ARCore it seems that the subsystem doesn’t remove tracked images at all; they will just stay in the TrackingState.Limited. If the subsystem does want to completely abandon the tracked image, it would remove the ARTrackedImage it instantiated (and which it manages automatically). As our GameObject is a child of that, it’d get deleted automatically.

Therefore, we only need to set our prefab to “not active” in case the tracked image is removed. The GameObject should get deleted automatically by AR Foundation. However, this would still occupy the place in our _instantiatedPrefabs dictionary, preventing a new instance to appear in the scene. Therefore, we can still remove “our side” of the instance: the instantiated GameObject from the scene as well as the entry in the C# dictionary. We do not delete the ARTrackedImage, as this was created and is managed by AR Foundation.

## Example

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.ARFoundation;

[RequireComponent(typeof(ARTrackedImageManager))]
public class DynamicImageTracker : MonoBehaviour
{

    public ARTrackedImageManager _manager;

    public GameObject _prefab1;
    public GameObject _prefab2;

    private Dictionary<int, GameObject> _instantiatedPrefabs = new Dictionary<int, GameObject>();
    void OnEnable()
    {
        //_manager = GetComponent<ARTrackedImageManager>();
        _manager.trackedImagesChanged += _manager_trackedImagesChanged;
    }

    private void _manager_trackedImagesChanged(ARTrackedImagesChangedEventArgs obj)
    {
        Debug.Log("CHANGE");
        foreach (ARTrackedImage item in obj.added)
        {
            if (item.referenceImage.name == "one")
            {
                _instantiatedPrefabs.Add(item.GetInstanceID(), Instantiate(_prefab1, item.transform.position, item.transform.rotation));
            }
            if (item.referenceImage.name == "two")
            {
                _instantiatedPrefabs.Add(item.GetInstanceID(), Instantiate(_prefab2, item.transform.position, item.transform.rotation));
            }
        }
        foreach (ARTrackedImage item in obj.updated)
        {
            int id = item.GetInstanceID();
            if (_instantiatedPrefabs.ContainsKey(id))
            {
                _instantiatedPrefabs[id].transform.position = item.transform.position;
                _instantiatedPrefabs[id].transform.rotation = item.transform.rotation;
            }
        }
        foreach (ARTrackedImage item in obj.removed)
        {
            int id = item.GetInstanceID();
            if (_instantiatedPrefabs.ContainsKey(id))
            {
                Destroy(_instantiatedPrefabs[id]);
                _instantiatedPrefabs.Remove(id);
            }
        }
    }

    // Update is called once per frame
    void OnDisable()
    {
        _manager.trackedImagesChanged -= _manager_trackedImagesChanged;
    }
}

* Awake
* In Awake() we get a reference to the AR Image Tracker Manager. We are sure one is attached to the gameobject as we have specified it as a RequiredComponent.
* OnEnable / OnDisable
* In OnEnable() and OnDisable() we register and unregister the tracking callback so we can implement our own behaviour.
* OnTrackedImageChanged()
* This callback will be called each time a new image is detected, moved or removed from the scene.
* For all newly detected images, we instantiate the appropriate objects, and make sure they are placed in the scene on the correct position an with the correct rotation.
* When a certain image is moved or rotated; we will need to update the scene information accordingly.
* When an image is removed or has disappeared, we will need to destroy the corresponding object in the scene.