# `useServerEvents` Composable

In order to notify the user of changes in the lifecycle of asynchronous backend processes without resorting to long polling, we need to be able to push messages to the client.

Institution's team is building a serverless WebSocket service using the AWS WebSocket API Gateway that the Adoption's front-end web client will interact with.

The AWS WebSocket API Gateway imposes the following [limits](https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html):

* Message payload size 128 KB
* Connection duration for WebSocket API 2 hours
* Idle Connection Timeout 10 minute

## Type Definitions

// Received from WS
type Message = {
    uuid: string;
    timestamp: string;
    topic: string;
    namespace: string;
    detailType: string;
    detail: Record<string, unknown>;
}

type WebSocketStatus = 'OPEN' | 'CONNECTING' | 'CLOSED'

type Subscription = (Message)=>void;

type Subscriptions = Record<string, Subscription[]>;

## Problems

1) The WS connection will close automatically after 10 mins. of inactivity, potentially causing the user to miss messages sent by the server after timeout
2) The server could potentially emit duplicate events, causing the frontend to invoke event/msg handlers multiple times for the same event.
3) When the 2 hr. connection duration limit is reached, the WS connection will have to be reestablished. There will some indeterminate time interval, ~ a second, during which the client will be unable to receive messages pushed by the server.

## Proposed Changes

To deal with Problem #1, the Idle Connection Timeout, we will implement a heartbeat to keep the connection open even when idle.

To deal with Problem #2, we keep an object in memory where the key is a concatenation of `detailType` and `topic` of each WS message received and the value is the timestamp of the DynamoDB insertion that triggered the WS message to be emitted. We will filter all WS messages received from the server by DB timestamp of received event > DB timestamp of last received event, and then update our map with the new event/msg.

To deal with the 2 hr. max connection duration, we will use the `autoReconnect` feature of `useWebSocket` with a delay of 0. Institution's is implementing a replay feature. When a new WS connection is opened to the WS service, the last n events will be sent to the client. This will allow the client to catch events missed during the reconnection period. The solution to Problem #2 will be applied again here to dedupe replay messages/events.

## Terminology

"topic" - A topic is like a message "channel" or category of messages that the client subscribes to

### Create a `useServerEvents` Composable

We'll be able to leverage `useWebSocket from '@vueuse/core'` then extend functionality to handle the deduping of messages.

We will configure `useWebSocket` to use a "heartbeat" in order to keep the connection open to API Gateway for longer than 10 mins.

`heartbeat: {
message: 'ping',
interval: 1000,
pongTimeout: 1000,
}`

The Websocket API Gateway endpoint will be defined by an environment variable, WEBSOCKET_ENPOINT.

`useWebSocket` exposes a `status: Ref<WebSocketStatus>` ref that reflects the [`readyState` property](https://websockets.spec.whatwg.org/#websocket-ready-state) exposed by the browser-native WebSocket API.

Example `useServerEvents.ts`

import { useWebSocket } from '@vueuse/core'
const useServerEvents = () => {
    const { status, close, subscribe, unsubscribe } = useWebSocket(WEBSOCKET_ENPOINT);
    
    open();
    
    function subscribe() {...}
    function unsubscribe() {...}
    
    return {
        status, 
        subscribe,
        unsubscribe
    }
}

This composable will be invoked at the root level component right after login so that the websocket exists throughout the whole user session.

#### `lastReceivedMessages` object

In order to dedupe messages received from the WS connection, the composable will keep a `lastReceivedMessages` object in memory where the key is `${topic}:${eventDetail}` and the value is a timestamp.

#### `onUnmounted` hook

We will use the [`onUnmounted`](https://vuejs.org/api/composition-api-lifecycle.html#onunmounted) lifecycle hook to invoke the `close` function returned by `useWebSocket`.

#### `data: Ref<T | null>` watcher

We will use a watcher on the `data` ref returned by `useWebSocket` to pass each msg received from the backend to a `_onMessage` function.

#### `status: Ref<WebSocketStatus>` watcher

We will use a watcher on the `status` ref returned by `useWebSocket` to invoke `_onOpen`, `_onConnecting`, and `_onClosed` lifecycle callbacks.

#### `_onMessage(msg: Message) function`

The `_onMessage` function will parse and validate the received `data.value`. It will return `void` if the incoming data is a heartbeat pong or otherwise not of the expected `Message` type. It will then check to see if the message timestamp > timestamp of any corresponding message in `lastReceivedMessages` object and update the object. If `incomingmsg.timestamp < cachedmsg.timestamp` it will return `void`. It will then check the `subscriptions` map to see if it has a keys equal to `${data.value.topic:data.value.eventDetail}`. If not, it will return `void`. Then it will invoke all handlers in the subscription map with that key.

#### `subscriptions` map
This will keep track of active subscriptions for the purpose of unsubscribing and invoking handlers.

`subscriptions` will be a map, where the keys are `${topic}:${eventDetail}` and the values are an Array of handler callback functions that take the server `msg:Message` as an argument.

#### Create and expose a `async function subscribe(topicEventDetail: string, (msg:Message)=>void)`
When `subscribe` is invoked we will pass:
- the topic that we want to subscribe to
- the callback to execute when a matching message is received

Upsert the handler function into the array of the associated key in `subscriptions` map .

Will call the tRPC route for subscribing to a "topic".

Given the exact same arguments, `subscribe` is idempotent if the backend is implemented that way as well).
It's comparable to `addEventListener`.

#### Create and expose an `async function unsubscribe(topicEventDetail: string, (msg:Message)=>void)`
When `unsubscribe` is invoked we will again pass:
- the topic to use as a key for the `subscriptions` object
- the callback

Then remove the pair from the object.

Then call the tRPC route for unsubscribing to a "topic".
Just like with `removeEventListener`, you have to hold a reference the callback you passed to `subscribe` to `unsuscribe` it.
`unsubscribe` will also be idempotent if the backend is implemented that way.

#### Create and expose a `async function waitForEvent(eventKey: string)`

This is helper to allow us to `await` server pushed events in a more readable way. `eventKey` is `${topic}:${eventDetail}`. `waitForEvent` creates a `Promise`. It then invokes `subscribe` with a callback that resolves the Promise and then unsubscribes. It returns this promise.

Example:

`showSpinner();
await waitForEvent("upload:processing-complete");
hideSpinner();`

### Two-hour connection limit
`useWebSocket` has an `autoReconnect` option that accepts a `delay` option which will be set to 0. Our theory is that once the 2 hour limit is hit and AWS closes the connection, an error will be thrown forcing the WS to attempt to reconnect immediately.

Institutions will implement a replay feature. When a new WS connection is opened to the WS service, the last n events will be sent to the client. This will allow the client to catch events missed during the reconnection period.

### 10 minute idle connection close
https://vueuse.org/core/usewebsocket/#heartbeat

## Contingencies
`useWebSocket`'s `autoReconnect` feature playing nicely with AWS APIGateway

We will need to test if `autoReconnect` will work with the AWS APIGateway endpoint as soon as the endpoint is available. If not, then we will have to resort to our contingency plan of opening a second connection in the renewal window.

## QA

Call out things to test.

## Communication

Is there anyone we need to notify about the changes?

We will need to discuss a replay feature with Institutions on the server side

---

## Document Checklist

- [x] Describe why the work is being carried out
- [x] Provide a high level technical plan for work carried out
- [ ] Included testing strategy if applicable
- [x] Included any potential issues or roadblocks
- [ ] Included areas of technical debt that can be resolved
- [x] Included external teams or people who need to be kept in the loop