---
# System prepended metadata

title: Sequence Tracking and the Gesture Engine

---

# Sequence Tracking and the Gesture Engine

## Sequence Tracker

The Sequence Tracker is a library I have written that allows for tracking of a sequence of steps over time. It's represented as a 'tree', with branching paths depending on the gestures that are done.

### Usage

You start off by creating your root Step. This should be done once only, such as in the `Start()` or `Awake()` unity methods.

For example:

```csharp=
public class SomeMono : MonoBehaviour {
    Step root;
    public void Start() {
        root = Step.Start();
    }
}
```
`Step.Start()` takes an optional `Action` parameter - an action that is called whenever a step is successfully completed.

This can be helpful both for letting the player know they've successfully completed a step, or for debugging (as you can e.g. log the current state whenver it changes).

```csharp=
root = Step.Start(() => Player.currentCreature.handRight.HapticTick());
```

---

#### Actions, Functions and Lambdas

An aside: if you're not familiar with `Action` or `Func` in C#, they are essentially the type of a function.

- An `Action` takes zero or more arguments and returns nothing
    - `Action` is an action with no arguments and no return value
    - `Action<int>` is an action with one `int` argument and no return value
- A `Func` takes zero or more arguments and can return something
    - `Func<bool>` is a function with no arguments that returns a `bool`
    - `Func<string, int>` is a function with one `string` argument that returns an `int`
   
These can be represented in a couple ways. You can define a function as normal and reference it like this:

```csharp=
public bool IsEven(int number) {
    if (number % 2 == 0)
        return true;
    return false;
}

public void Start() {
    Func<int, bool> function = IsEven;
    
    Debug.Log(function(1));  // false
    Debug.Log(function(2));  // true
}
```

Or, you can define an inline function - also known as a "Lambda Function".

```csharp=
public void Start() {
    Func<int, bool> function = (number) => {
        if (number % 2 == 0)
            return true;
        return false;
    };
    
    Debug.Log(function(1));  // false
    Debug.Log(function(2));  // true
}
```
This can be shortened further:
```csharp=
public void Start() {
    Func<int, bool> function = (number) => number % 2 == 0;
    
    Debug.Log(function(1));  // false
    Debug.Log(function(2));  // true
}
```

For functions or actions without arguments, the syntax looks like this:
```csharp=
public void Start() {
    Action sayHello = () => Debug.Log("Hello!");
    
    sayHello();  // says Hello!
}
```

---

Once you've made your root step, you can begin defining your sequence.
This mainly uses two functions: `.Then()` and `.Do()`.o

There are many overloads of `.Then()` to make it easy to add in multiple gestures, various parameters, or even sets of sets of gestures. But the most basic looks like this:

```csharp=
public void Start() {
    root = Step.Start();
    root.Then("Condition A", () => conditionA)
        .Do("Action A", () => actionA);
}
```

The parameters of `.Then` are as follows:
- `name`: A human-readable name for the step.
- `condition`: A `Func<bool>` that returns true if the condition has been achieved, or false otherwise.

The parameters of `.Do` are similar:
- `name`: A human-readable name for the action.
- `action`: An `Action` that is triggered when the steps have been completed.

A practical example: let us write the tree for the following sequence involving an `Item`.

1. Item is held
2. Item is dropped
3. Item is held again

```csharp=
public void Start() {
    root = Step.Start();
    root.Then("Item held", () => item.mainHandler != null)  // held
        .Then("Item dropped", () => item.mainHandler == null)  // not held
        .Then("Item grabbed again", () => item.mainHandler != null)  // held again
        .Do(() => Debug.Log("Gesture complete!"));
}
```

Note that the name of the steps - the first parameter - is purely for debugging. You can leave it out but I strongly recommend you leave it in. Debugging aside, it helps you to remember which step is which.

#### Updating a sequence

Once you're done, you must then update the sequence tracker every frame.

```csharp=
public void Update() {
    root.Update();
}
```

**If you don't do this, your sequence will do nothing!** Calling `Update()` tells the tracker to check conditions and traverse its own tree.

#### Resetting a sequence

The last important note is about `root.AtEnd()` and `root.Reset()`. Once a sequence is complete, the tracker will not automatically reset - it will just sit there doing nothing, as there are no more conditions to check. You can test whether the sequence has hit an end point with `root.AtEnd()`, and you can reset it from the start with `root.Reset()`.

#### Branching

Let's say we want to do two different things for the previous example depending on whether the creature that picked up the item the second time is a player or an NPC.

We can store the state at the point of being dropped, and then branch off from it depending on what picked it up:

```csharp=
public void Start() {
    root = Step.Start();
    
    var dropped = root
        .Then("Held", () => item.mainHandler != null)
        .Then("Dropped", () => item.mainHandler == null)

    dropped
        .Then("Held by player", () => item.mainHandler?.ragdoll.isPlayer == true)
        .Do("Log player hold", () => Debug.Log("Item held by player!"));

    dropped
        .Then("Held by NPC", () => item.mainHandler?.ragdoll.isPlayer == false)
        .Do("Log NPC hold", () => Debug.Log("Item held by an NPC!"));
}
```

#### Debugging

There are a couple tools available to debug. One common pattern I find myself using is this:

```csharp=
public void Start() {
    root = Step.Start(() => Debug.Log(root.GetCurrentPath()));
}
```

Whenever a step is complete, this will log the current path like this:

```
Held > Dropped > Held by player
```

The other useful tool allows you to visualise the entire tree:

```csharp=
Debug.Log(root.DisplayTree());
```

This will dump the entire tree to the console in a format such as this:

```
- Held
- Dropped
  - Held by Player
    - Action: Log player hold
  - Held by NPC
    - Action: Log NPC hold
```

#### Other Usage

##### Condition Sets

Sometimes you want multiple steps to be done under one name, or you want to have a function that dynamically generates a set of steps.

This can be done in the following way:

```csharp=
public NamedConditionSet ComplicatedSequence() {
    return Tuple.Create("Name of sequence", new Func<bool>[] {
        () => someConditionA,
        () => someConditionB,
        () => someConditionC,
        () => someConditionD
    });
}
```

`NamedConditionSet` is an alias for a tuple of a name and a list of condition functions. You can alias this as such:
```csharp=
using NamedConditionSet = Tuple<string, Func<bool>[]>;
using NamedCondition = Tuple<string, Func<bool>>;
```

##### In-sequence Actions

You don't just have to put a `.Do()` at the end of a sequence! `.Do()` can be placed at any step in the process, although note that each step can only have _one_ `.Do()` attached to it.

```csharp=
root = Step.Start();
root.Then(() => ...)
    .Do(() => ...)
    .Then(() => ...)
    .Do(() => ...)
    .Then(() => ...)
    .Do(() => ...);
```

##### Combining Steps

Sometimes it is nicer to have complicated steps written out over several lines. You can do this with `.And()`:

```csharp=
root = step.Start();
root.Then(() => conditionA)
    .And(() => conditionB)
    .And(() => conditionC);
```
This is functionally identical to the following:
```csharp=
root.Start()
    .Then(() => conditionA && conditionB && conditionC);
```

##### Delays

You can use `.After(duration)` to wait a bit before checking the next step.

```csharp=
root = step.Start();
root.Then(() => conditionA)
    .After(0.5f)
    .Then(() => conditionB)
    .Do(() => ...)
```

##### Time-related Steps

Sometimes you want to check if a condition is true for more (or less) than a particular duration.

The following two steps check to see whether a button is tapped or held, by checking whether it was released before or after 0.3 seconds.

```csharp=
root = Step.Start()
    
var buttonWasTapped = root
    .Then(() => buttonPressed, "Button Tapped", 0.3f, mode: DurationMode.Before);

var buttonWasHeld = root
    .Then(() => buttonPressed, "Button Held", 0.3f, mode: DurationMode.After);
```

This works with the `endCondition` parameter of `Then()`, which lets you define a custom check for 'condition complete'. Otherwise it defaults to the inverse of the start condition.

---

## Gesture Engine

The Gesture Engine is a library for Blade and Sorcery mods that allows you to detect hand gestures.

:::info
Note: you must have your C# version set to 'latest' to use this library.
:::

### Usage

```csharp=
public class SomeMono : MonoBehaviour {
    Step root;
    public void Start() {
        var gesture = Gesture.Left
            .Palm(Direction.Inwards)    // palm pointing inwards
            .Moving(Direction.Down)     // hand moving downards
            .Point(Direction.Forwards)  // index direction pointing forwards
            .Fist;                      // hand is making a fist
    }

    public void Update() {
        if (gesture.Test()) {
            Debug.Log("Gesture activated!");
        }
    }
}
```

You can also use it in concert with the Sequence Tracker. A `Gesture` or `GestureStep` implicitly converts itself to a format compatible with the Sequence Tracker, including adding in a generated human-readable name for the gesture.

```csharp=
public class SomeMono : MonoBehaviour {
    Step root;
    public void Start() {
        root = Step.Start();
        root.Then(Gesture.Both                   // both hands
                  .Palm(Direction.Up)            // palm upwards
                  .Point(Direction.Outwards)     // point outwards
                  .At(Position.Face)             // positioned near the face
                  .Offset(Direction.Up, 0.3f));  // offset that position upwards by 0.3m
        .Do(() => Debug.Log("Praise the sun!"));
    }

    public void Update() {
        root.Update();
    }
}
```

A full list of all the checks you can add to your gesture can be found in the code, which is self-documenting.

The Gesture Engine is set up to allow easy handedness options. Changing `Gesture.handedness` from `Side.Right` to `Side.Left` puts it into left-handed mode, which means that `Gesture.Left` and `Gesture.Right` will be switched.