# Dev-Telemetry

**State**: Draft
**Author**: Anthony
**Date**: 11/01/2021

In order to understand desktop/mobile reliability and to know better if we are missing message, we are going to build a telemetry server which users can opt in. By default the option will be disable.

## Desktop changes:
* Add a new advanced options where users can select to opt in to the telemetry server. Once opt in, this settings will be saved in status-go settings

## Status-go changes:
* Add a new settings telemetry-enabled as a boolean
* When a message is received and telemetry-enabled is active, then push the message to the telemetry server

## Dev-Telemetry server:
* Go server using grpc + protobuf to communicate, the server will only have 1 endpoint which will be to receive a sample (see later for sample structure)
* Aggregate sample based on various timeframe such as:
* 1 Hour
* 1 Day
* 1 Week
* More?
* The sample will be stored in sql database (Question: do we already have such db that we can reuse?)
* The aggregation will be pushed to a time series db(Question: do we already have such db, e.g prometheus that we can reuse? Otherwise I will store them into the same sql db and build a UI on top of it)
* Add a final cron which clean the samples after 1 week (once all the aggregator have run)

## Sample structure:

message Sample {
    string messageId = 1; # Identify a message within a channel
    string channelId = 2; # Identify a channel, it must be unique for communities too
    string receiverPublicKey = 5; # Who receive the message
    uint64 sentAt = 4; # When the message was send timestamp
    bytes topics = 5;
    string envelopeId = 6;
}

On top of the sample record, we also store when the sample was recorded:

    uint46 createdAt

## Aggregation algorithm

1) For each receiverPublicKey with a sample createdAt after the end of the aggregation end (this will ensure that the user has been sending sample and we are not missing data)
2) Fetch the messages, group then by chatId and sort them by sent timestamp
3) For all receivers that have common chatId, count the number of message missing

We then define the reliability Rch for a channel as such:

Rch = 1 - count_of_message_missing / total_number_of_messages

Then we can define the reliability R as the average:

R = (R(0) + R(1)+ .... + R(n)) / len(Rch)

## Authentication

Enable this settings only on dev build?