# Pod Controller Design Documentation

## Functionalities

The pod controller is a service assists its client to interact with his wallet. It owns an independent identity to form a 2-3 multisig wallet with two other public keys from its client. It overall provides:

- Identity management - Only allowed user can access the controller
- Gordian wallet management - A 2-3 multisig wallet will be created and managed using bitcoind
- Messaging client - The Service and clients communicate through messaging protocol
- Bitcoind lifecycle management - Monitor and control the bitcoind process status

## API

### WebAPI

The web API is now only for components of a pod to communicate internally.

- `GET` **/tx-notification/<txid>**

### Messaging API

All the APIs to the pod would go through signal messaging protocol. Here only lists down the description, requests and responses for each APIs.

#### bind

`bind` initiates the account binding process between a client and the controller. A pod controller needs to be bound first before it can provide full functionality.

- Request Body: None
- Response Body: A JSON object of the following keys:
| Key | Description |
| - | - |
| identity | The identity of the pod controller |
| nonce | A random nonce |
| timestamp | Current timestmap |
| signature | A signature of `concat(nonce, timestamp)` |

#### bind_ack

`bind_ack` is to sign a valid signature back to prove the identity of the request client. The signature is

- Request Body: A JSON object of the following keys:
| Key | Description |
| - | - |
| timestamp | Current timestmap |
| signature | A signature of `concat(bind_nonce, timestamp)` |
- Response Body: JSON response

    {
      "status": "",
      "error": ""
    }

#### create_wallet

create_wallet creates a gordian wallet by a given valid descriptor.

#### bitcoind

bitcoind proxies all bitcoind RPC. The method provides a directly access to a user's bitcoind process.

#### start_bitcoind

:::info
This is a new API
:::

start_bitcoind is to start the bitcoind pod by make a request to `bitcoind-ctl`.

#### stop_bitcoind

:::info
This is a new API
:::

stop_bitcoind is to stop the bitcoind pod by make a request to `bitcoind-ctl`.

:::warning
Normally, the client would not stop its node. This API is reserved for special cases.
:::

#### get_bitcoind_status

:::info
This is a new API
:::

get_bitcoind_status returns a summarized bitcoind information.

## Flows

### Pod Registration (New)

When a pod first boots up, it will register its identity by a pre-set registration token.

sequenceDiagram
    autonumber
    
    client->>+autonomy_api: Create a new pod
    autonomy_api-->>-client: OK
    autonomy_api->>+autonomy_api: Create a token for registration
    autonomy_api->>node_manager: Spawn a pod with <br> the token attached
    node_manager->>Pod: Create a pod with a token
    Pod->>Pod: Init pod and create its identity
    Pod->>+autonomy_api: Report its DID with the given token
    autonomy_api-->>-Pod: OK
    Pod->>Pod: Mark DID registered
    autonomy_api->>-autonomy_api: Update the pod mapping for a DID
    client->>+autonomy_api: Get pod ID
    autonomy_api-->>-client: Pod ID

:::info
For the self-owned pod-controller, a user will need to request a token from the client and set it to its pod-controller manually.
:::

### Binding

Here is a [sequential diagram](/Huid-8ulSb6Lea8brJVzGg#Binding) that illustrates the flow of the binding process.

## Lifecycle management

### Usage Statistics

By collecting data of requests behavior, we can use those dat to optimise the bitcoind lifecycle or schedule a better pod upgrading time, etc.

There are two metrics we are collecting now. They are requests counts by weekly and monthly. The pod-controller counts requests in memory and save it to its persistent storage by a given `flush_interval`.

**Weekly Usage**

Weekly usage collects requests counts for each weekday. The data structure is represented as following:

| Hour/Weekday| Sun | Mon | Tue | Wed | Thur | Fri | Sat |
| - | - | - | - | - | - | - | - |
| 0 | | | | | | | |
| 1 | | | | | | | |
| 2 | | | | | | | |
| . | | | | | | | |
| . | | | | | | | |
| 21 | | | | | | | |
| 22 | 5 | 7 | 5 | 4 | 6 | 4 | 8 |
| 23 | | | | | | | |

**Monthly Usage**

Monthly usage collects the daily data for the past 30 days. It collects daily data with the same format to each weekdays above. It saves in an array. For example,

[ day30, day29, ... , day3, yesterday, today ]

:::info
For both usage data, when a new day arrive, it removes the old data and creates a fresh row of that day. This fixes the size of This minimize the size of the usage data.
:::

#### Sequential Diagram

- [Pod Suspend and Resume Sequential Diagram](/k1cujmATThm4gA7UFy978w)

## Transaction Notification

Bitcoind provides a configuration for setting a callback command which would be invoked when a new transaction of watched wallet is comming. Instead of communicate with the Autonomy API directly, it uses `curl` to send transaction information to pod-controller.

The pod-controller decides whether to send a notification to client based on scenarios. If a notification is required, the pod-controller will prepare messages and request to the Autonomy API in order to send a notification to its owner.

sequenceDiagram
    autonumber
    participant bitcoind
    participant pod_controller
    participant autonomy_api
    
    bitcoind->>+pod_controller: send incoming transaction <br> for watched addresses
    pod_controller-->>-bitcoind: ok
    pod_controller->>pod_controller: Detemine whether to send notification
    
    pod_controller->>+autonomy_api: send notification through <br> /api/accounts/notification
    autonomy_api-->>-pod_controller: ok

### Autonomy Pod

###### tags: `jim-draft` `Autonomy`

In our cluster, the number of instances is mainly controlled by [cluster-autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler). We start from investigating the parameters of it but without any lucks. The mechanism behind it is to detect the `unschedulable` events for each pods and see if any of nodes in its pool could fulfill those pod. If it can, it will add a node into the cluster. For the scaling down, it is determined by the overall usages of resources in a cluster and is not able to add redundant instances.

Kubernetes introduces the [PrioirtyClass](https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/) since 1.14. The goal is to replace the unimportant pods when a cluster is out of resources but there are some key services need to be added immediately. Take advantage of this feature, we can create numbers of spare pods which take the same resources but do nothing. When an autonomy pod is added with insufficient instances, those spare pods will be removed and replaced by the pod we just add.

We first create a `PriorityClass` and set its value to `-1`. The default priority value of pods without any specified PriorityClass is `0`. That means if there are any regular pods be added those pods with the PriorityClass set will be taken down.

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: spare-autonomy-pod
value: -1
globalDefault: false
description: "This priority class should be used for spare autonomy pods only."

Now, we can create spare pods like following:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: autonomy-mainnet-spare-pod
  namespace: autonomy-wallet
spec:
  selector:
    matchLabels:
      app: spare-autonomy-pod
  replicas: 10
  template:
    metadata:
      labels:
        network: mainnet
        app: spare-autonomy-pod
    spec:
      priorityClassName: spare-autonomy-pod
      nodeSelector:
        bitcoin-network: mainnet
      containers:
        - name: controller
          image: autonomy-wallet:pod-controller-0.0.1
          resources:
            requests:
              cpu: "0"
              memory: "0"
            limits:
              cpu: "0.1"
              memory: "100Mi"
          command:
            - /bin/sleep
            - 30d

Here we set the **replica** to 10 that means we are going to reserve 10 spare pods for autonomy pod. In the pod SPEC, we make both the **image** and the **resource requests** to be the same as the normal autonomy pod. Thus, the required images will be downloaded in advance and also make sure the cluster will reserve enough spaces for autonomy pod.

### Reference

- [https://medium.com/scout24-engineering/cluster-overprovisiong-in-kubernetes-79433cb3ed0e](https://medium.com/scout24-engineering/cluster-overprovisiong-in-kubernetes-79433cb3ed0e)
- [Spare Autonomy Pod](/48DwskFFQJSJb-mfMHwLOg)