# T-REX: the Reinforcement Learning Augmented Reality dinosaur.

## Overall strategy/aspects of the project
* Fix/improve model and skeletal rig in Blender package.
* OR make a simpler model/rig in Blender, similar to corgi (basically a cylinder with 4 legs).
* Import into Unity.
* Investigate ML-Agents toolkit in Unity - maybe replicate "Puppo corgi" example.
* Install toolkits: https://docs.unity3d.com/Manual/upm-ui-install.html
* Try and apply methods to T-Rex model.
* Visualize in Hololens (potentially also Oculus Quest 2)

## Blender 3D model

https://github.com/alan-turing-institute/holorex
(based on YouTube tutorials
https://www.youtube.com/watch?v=4J2ObXB_kuk https://www.youtube.com/watch?v=PnnK-X2ai08&t=655s
)

## Blog post on Puppo / Unity-ML-Agents

https://blog.unity.com/technology/puppo-the-corgi-cuteness-overload-with-the-unity-ml-agents-toolkit

## Notes on Unity

https://unity3d.com/get-unity/download

* Good tutorial series to learn some basic concepts https://learn.unity.com/project/roll-a-ball?signup=true
* A project is made up of "SCENEs".
* A SCENE contains GAMEOBJECTS (e.g. T-Rexs).
* A GAMEOBJECT has a TRANSFORM (giving position and movement), and COMPONENTs attached to it.
* A COMPONENT can be something like a MATERIAL/texture, a RIGIDBODY, to tell Unity that this thing is a physical object, a COLLIDER to allow it to interact with other objects, or a SCRIPT.
* SCRIPTs are written in C#, have some boilerplate function definitions with placeholders for what should happen on initialization, and every frame.

## Unity ML agents Toolkit
https://github.com/Unity-Technologies/ml-agents

## Notes on Blender
https://www.blender.org/download/

Use two-fingers-on-trackpad and move to rotate view, hold shift and move with two fingers to pan, function and two fingers to zoom.

**Edit mode** and **Object mode**, and (when we have an armature) **Pose Mode** - can switch between these via menu at the bottom, or press tab.

In **Edit** mode, can select *vertices*, *edges* or *faces*.
Can press **z** to switch between solid and wireframe views.
### Useful shortcut keys

* **a**: select all vertices/edges/points. Double press to deselect all.
* **b**: box select some points. If in wireframe view, this will select "behind" as well.
* **e**: extrude. Can press **x**,**y**,**z** to choose axis on which to extrude.
* **r**: rotate. Can press **x**,**y**,**z** to choose axis around which to rotate.
* **s**: scale. Can press **x**,**y**,**z** to choose axis on which to scale

## Who is working on what
### Monday
Nick: Simple Blender model Alpaca
Callum: Rigging of TRex
Ed: Human walking towards target - RL
Iain: 3D balls on boxes - RL
Helen: RL in unity
Camilla: Unity

Victoria: ML

## Iain's Notes

### Day 1

* Followed [these](https://github.com/Unity-Technologies/ml-agents/blob/release_19/docs/Getting-Started.md) Unity ml-agents installation instructions to open the balancing balls example.
* you can shallow clone the repo at the release_19 tag with `git clone --depth=1 --branch=release_19 https://github.com/Unity-Technologies/ml-agents.git`
* I set up xrpd server on the `t-rex-vm` in our [T-Rex subscription](https://portal.azure.com/#@turing.ac.uk/resource/subscriptions/420ebee5-0e06-4612-8e5f-ff1198e8c3ed/overview). You can access it by
* Setting a VM password with the "forgotten password" link in the LHS sidebar on the Azure portal
* Installing freerdp with `brew install freerdp`
* Connecting to the VM with `xfreerdp /v:vm-user-name /u:your-user-name /p:your-password-here`
* Helen had an error message: `Please check that the $DISPLAY environment variable is properly set.` and had to install xquartz before this worked

### Day 2

* YouTube tutorial we're hoping to follow is here -> <https://www.youtube.com/watch?v=zPFU30tbyKs>

### Post-HackWeek Summary

To resume the project, you would need to

- install unity hub
- install unity
- shallow clone the unity ml repo and set up a python env with the ml packages (see above)
- clone the holorex repo
- open the [BigProject directory](https://github.com/alan-turing-institute/HoloRex/tree/main/BigProject) as a Unity project
- In unity, navigate to `Assets/ML-Agents/Examples/FirstTRexScene` and open the DinoScene scene.
- As there is currently no pre-trained model, you can either:
- click Play, as usual, to start the game running (in our case, that will simply make Rex drop and fall over)
- start the python package with `mlagents-learn --run-id=my_model_name` and click Play to train Rex. He will still fall over but, this time, with legs flailing.

From here, the best steps may be to:

- familiarise yourself with the Crawler or Walker [examples](https://github.com/Unity-Technologies/ml-agents/tree/main/Project/Assets/ML-Agents/Examples).
- modify Rex's joints to try to make him more stable (we don't know whether Rex can even walk with his current limb configuration because he only has control of his thighs)
- start curriculum learning because chasing a cube is probably a bit out of Rex's league:
- teach Rex to stand
- teach Rex to walk forward
- teach Rex to turn
- teach Rex to walk towards a randomly placed object
- teach Rex to hurdle obstacles and chase a nible and well-armed team of humans whilst avoiding their traps and making minimal noise