# AR Foundation Study Note

## Unity XR SDK
- [Document](https://docs.unity3d.com/2022.1/Documentation/Manual/xr-sdk.html)
- 描述 Subsystem 和 Provider 的架構關係，可能是將來要**串接 novatek 程式的關鍵**。
- [XR Plug-in Framework](https://docs.unity3d.com/Manual/XRPluginArchitecture.html)
- [XR SDK Note](/t3rNlzdUSqepK7zNQbCCLA)

## XRLoader
- 每個 provider plug-in 都有一個 Loader。
- Loader 負責 create 和 destroy subsystem 的 instance。

## AR Foundation Debug menu
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/ar-debug-menu.html)
- 可以觀察效能表現的 Component ( ex. FPS、Tracking Mode )。

## Trackable 具有 [TrackingState](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARSubsystems.TrackingState.html)
- 分為三種 State：
- Limited：正在追蹤，但遭到限制或追蹤品質差 (物體部分離開畫面、遭到遮擋)
- None：不在追蹤
- Tracking：正在追蹤

## ARInputManager
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.ARInputManager.html)
- Unity 和 Plugin 的 API，負責將 SDK 偵測到的 Data 傳送進 AR Session。

## ARCameraManager
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.ARCameraManager.html)
- 管理 ARCamera 的特性。
- 可以偵測現實環境中光源的亮度值、顏色、方向等等。
- 儲存這些 Data 的 Struct [ARLightEstimationData](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.ARLightEstimationData.html)
- [如何取得偵測值](https://www.youtube.com/watch?v=nw6-U_mSvQQ&ab_channel=DilmerValecillos)

## AR Session
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/api/UnityEngine.XR.ARFoundation.ARSession.html)
- 控制一個 AR Session 的 lifecycles 和 configuration。
- 可以控制 AR 應用的 Frame Rate、確認是否可使用 AR、讓我們知道 AR Session 的狀態。

## XR Origin (a.k.a AR Session Origin before 5.0.0)
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.core-utils@2.1/api/Unity.XR.CoreUtils.XROrigin.html)
- AR Session Space 的 Origin。
- 儲存一些會經由 locomotion 操作的 GameObject (ex. Camera)。
- 控制 camera 的 Y offset。

## Subsystem
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/arsubsystems/arsubsystems.html)
- Unity 的 Plug-in 提供可以 impilement subsystem 的 Provider。
- AR Foundation defines its AR features using subsystems.
- Subsystem implementations are called providers, and are typically made available in separate packages called provider plug-ins.

## Manager
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/trackable-managers.html)

### 簡介
- 一種 Manager 掌管一種 AR feature 對應的 AR subsystem。
- Manager is a component that provides a scripting interface for an AR feature in a scene.
- Managers make subsystem data accessible in a scene as GameObjects and components.

### Trackable Manager
- tracking subsystems 定義：AR Foundation subsystems detect and track objects in the physical environment
- 而這些 tracking subsystems 的 Manager 就是 Trackable Manager
- trackable 定義： a special component that represents anything that can be detected and tracked in the real world. (ex. Planes, point clouds, anchors, environment probes, faces, body, images, and 3D objects)
- Trackable Manager 藉由 tracking subsystems 所偵測到的 data 去控制 Trackable 和它的 GameObject。 (ex. ARPlaneManager generates ARPlane trackables using data from the XRPlaneSubsystem)

### ARHumanBodyManager
- 儲存 humanBodyPrefab

#### ARHumanBody
- Data：Joint (相對於 Human Body Origin 的 position 和 rotation), The pose for the human body origin

### AROcclusionManager
- Data：depth texture, depth confidence texture

### AREnvironmentProbeManager
- 控制 EnvironmentProbe 的生成

#### AREnvironmentProbe
- Data：Cubemap texture
- 可以控制偵測範圍 (extents)

#### ARInputManager
- XRInputSubsystem 提供 Device Tracking 功能，可以取得 Physical Device 的 Pose (ex. 手把、頭盔)
- TrackedPoseDriver 可以直接取得 Device 的 Pose，並將此訊息作用在 Session 中的 GameObject

## Extending AR Foundation
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@5.0/manual/architecture/extensions.html)
- AR Foundation provides a native pointer to access platform-specific functionality and data when applicable. For example, the XRSessionSubsystem has a nativePtr property.

---

## Face Tracking 深入研究
- [UnityEngine.SubsystemsModule](https://docs.unity3d.com/2020.3/Documentation/ScriptReference/UnityEngine.SubsystemsModule.html) 描述 Unity 中 Subsystem 使用的 Class
- A subsystem is initialized from a SubsystemDescriptorWithProvider

### XRFaceSubsystemDescriptor
- 其中的 Create 可以創造 subsystem descriptor，此 descriptor 會註冊為一個 XRFaceSubsystem 的實作
- 主要紀錄此 XRFaceSubsystem 的規格，ex. 可否追蹤眼睛？可否生成 FaceMesh 的 vertices？是否可提供 normals、uvs？...

### XRFaceSubsystem.Provider
- 其中的 Method [GetChanges](https://docs.unity3d.com/Packages/com.unity.xr.arsubsystems@4.2/api/UnityEngine.XR.ARSubsystems.TrackableChanges-1.html#UnityEngine_XR_ARSubsystems_TrackableChanges_1_Dispose) 可以更新 add, update, remove 的 NativeArray

### SubsystemLifecycleManager
- [SubsystemLifecycleManager](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/api/UnityEngine.XR.ARFoundation.SubsystemLifecycleManager-3.html#UnityEngine_XR_ARFoundation_SubsystemLifecycleManager_3_subsystem)，subsystem 就是從這裡來的
- 其中的 `GetActiveSubsystemInstance()` 就是從 Loader 取得 subsystem 的 Method
- `EnsureSubsystemInstanceSet()` 會呼叫 `GetActiveSubsystemInstance()`
- 而 `EnsureSubsystemInstanceSet()` 會在 Manager OnEnable() 時呼叫

### ARTrackableManager
- [ARTrackableManager](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/api/UnityEngine.XR.ARFoundation.ARTrackableManager-5.html)
- ARTrackableManager 繼承 SubsystemLifecycleManager

### ARFaceManager
- [Document](https://docs.unity3d.com/Packages/com.unity.xr.arfoundation@4.2/api/UnityEngine.XR.ARFoundation.ARFaceManager.html)
- When enabled, this component subscribes to `ARFaceManager.facesChanged` event to update face data. If this component is disabled, and there are no other subscribers to those events, face detection will be disabled on the device.

### ARFace
- Data：Eye Transform, normals, vertices (Triangle Mesh)

### ARFaceMeshVisualizer

### ARFacesChangedEventArgs

### ARFaceUpdatedEventArgs