This is the document that contains a plan to get a scheduler in Pantavisor that will be able to execute tasks in different processes (workers) in order to relieve the main loop.

The idea behind the scheduler is to be able to send time consuming functions [out of the main loop](https://hackmd.io/oETI2WebTOCn_v93afF3kQ#Add-Worker-Processes-to-Execute-Time-Consuming-Tasks) progressively.

## Components

See [this diagram](https://docs.google.com/presentation/d/11KLXQD6fke1gqJR6rQA1TeHqr9rfWeA-CT3Ge2a-gPo/edit#slide=id.p).

With the help of optionally configurable workers, we could get time consuming tasks out of the main loop. The elements of this solution would be:

### Main Loop

It would still contain the management and associated structures of Pantavisor (system, configuration, revision, container, storage, pv-ctrl, debug and ph client) plus a new module, the scheduler.

### Scheduler

In charge of taking tasks directed from the main loop and put them in queues to be processed by the workers. The Scheduler then will create workers based on a configurable limit and keep track of them. A SIGKILL signal will be issued to any task that surpasses a timeout.

Requests to the scheduler would have a callback function, a timeout value in case the task hangs without response and, optionally, a priority number.

The scheduler will be also in charge of getting the response from the tasks, which will be in JSON format.

### Tasks

An old time consuming task from the main loop (container startup, object transfer, ph client requests...) that can be isolated to be run on a worker. The task will communicate its feedback to the scheduler so the main loop can do the appropriate changes in its structures.

### Workers

0 to many independent processes that can take tasks from the main loop. In case 0 workers are configured, the tasks can be executed at the end of the main loop (one task per loop or more) to get a similar experience than the one in the current code in terms of timing and memory consumption.

## Example of Task Execution

Setting aside the object transfer and container startup examples, which are more simple, let us get into the hypothetically case of sending the main ph client task into the scheduler.

1. The main loop periodically calls the ph_client_run function:

ph_client_run(phc)

2. As the PH Client state machine is in IDLE state, it sends a task named ph_client_task_idle into the scheduler with a timeout of 15, a priority of 3 and a payload with the device metadata:

pv_task tsk;
tsk.name = "ph_client_task_idle"
tsk.task = &ph_client_task_idle;
tsk.data = (void*)"["key1": "value1", ...]";
tsk.priority = 3;
tsk.timeout = 15;
pv_scheduler_add_task(sch, tsk);

3. The scheduler puts the ph_client_task_idle task in its queue.
4. The main loop also periodically calls the scheduler to see what can be executed:

pv_scheduler_run(sch);

5. It is time to execute ph_client_task_idle and there is a worker available, so it does so and moves the PID into a list of currently running tasks.
6. The ph_client_task_idle task, which looks like this, is executed by the worker:

ph_client_send_devmeta("["key1": "value1", ...]");
ph_client_recv_usrmeta();
ph_client_check_update();

7. After the metadata inter-exchange, the client has discovered that there is a new update available in the hub.
8. The scheduler is called again in a further period of the main loop:

pv_scheduler_run(sch);

9. As the ph_client_task_idle task has already successfully finished, it has returned a JSON:

{
    "type": "ph_client_task_idle",
    "data": {
        "usrmeta": {"key1": "value1", "key2": "value2", ...},
        "update": true
    }
}

10. The JSON is parsed with a callback function and its contents are saved in the main loop runtime structures, updating the user metadata and update request.
11. The main loop processes the ph-client state machine:

ph_client_run(phc);

12. The PH Client state machine is progressed into UPDATE state, which means ph_client_task_idle is no longer called. Instead we add a new task named ph_client_update:

pv_task tsk;
tsk.name = "ph_client_update"
tsk.task = &ph_client_update;
tsk.data = NULL;
tsk.priority = 3;
tsk.timeout = 60;
pv_scheduler_add_task(sch, tsk);

13. In this case, the scheduler gets this returned data when the time comes:

{
    "type": "ph_client_update",
    "data": {
        "revision": 56
    }
}

14. The JSON is parsed in a different callback function based on its type. Structures are updated.
15. The internal logic of the main loop will try to transition to the new revision when possible.

## Implementation Plan

1. Scheduler Prototype (first delivery):
* Scheduler: with timeouts; without priority (FIFO queue).
* Workers: thread or main loop based on config.
* Task: pv-ctrl get/put object; block requests when pending.

2. Scheduler Priority (second delivery)
* Scheduler: study how to make smart priority.

3. Rest of Tasks (probably one delivery per task)
* Start Containers.
* PH Client.
* ...