<img src="https://i.imgur.com/OyL8CcE.jpg" alt="drawing" width="200"/>

# ONE-T Web Insights proposal

Proponent: TurboFlakes
KSM Account: H1tAQMm3eizGcmpAhL9aA9gR844kZpQfkU7pkmMiLx9jSzE
Contact: Paulo [@turboflakes:matrix.org]
Date: 20/07/2022
Requested KSM: **1,923.208** / 115,590 USD (USD converted to KSM using the EMA30 KSM price from Subscan on the 04/08/2022)

--

*Short description:*

We believe that everyone can make better decisions by the way they explore and visualize data.

From this proposal **we aim to build ONE-T a web place, where people could visualize reports without limits**, easily digestible and simply accessible for anyone interested on this topic - **Validators and Kusama network performance**.

With this proposal we plan to bring *All* the text message insights sent by ONE-T bot to the web. From simple network status dashboards, to a list of validators with several para-validator key data points, real-time charts, as well as backend service that aggregates on-chain data from the *backing subsystem parachains protocol* and build *RESTful and WebSocket APIs* to create a seamless user experience.

This proposal also aims to cover ongoing maintenance and operational costs of ONE-T.

---

Alpha Prototype: [apps.turboflakes.io](https://apps.turboflakes.io)


## 1. Context of the proposal

ONE-T bot was born after a few ideas gathered by the community, it aims to be an additional tool to the Polkadot ecosystem where TVP validators could be promoted, a tool that could give useful insights to validator operators and nominators about validators performance, a tool to easily understand how well validators are performing compared to their peers, a tool to give a general overview about the state of the Polkadot and Kusama network.

Some of the features proposed on this proposal were initially included as a milestone in the discussion for proposal - [Bootstrap ONE-T Nomination Pools](https://kusama.polkassembly.io/post/1633), but since that proposal has a few challenges, by the nature of what is requested, we think would be better for this project to have an independent proposal and follow the normal path of a Treasury proposal.

#### ONE-T Reports

ONE-T is generating reports daily since April 2022. These reports are currently only available on matrix via text messages. By the end of July 2022, for Kusama and Polkadot combined, ONE-T gathered over the past three months **more than 170 members** on both public rooms. There are **more than 250 validator report subscriptions active** and is sending on average more than **600 reports a day** to subscribers.

Anyone is welcome to join the public rooms below and subscribe to any of the reports available, *individual validator reports*, *insights report*, *parachains report* or the *para-groups report*:

<table style="width:100%;" cellspacing="0" cellpadding="0">
<tr>
<td style="width: 100px;">
<img style="width: 80px;" src="https://github.com/turboflakes/one-t/blob/main/assets/one-t-kusama-avatar-128.png?raw=true" />
</td>
<td><a href="https://matrix.to/#/%23kusama-one-t-bot:matrix.org" target="_blank">Kusama ONE-T Bot (Public)</a></td>
</tr>
<tr>
<td style="width: 100px;">
<img style="width: 80px;" src="https://github.com/turboflakes/one-t/blob/main/assets/one-t-polkadot-avatar-128.png?raw=true" />
</td>
<td><a href="https://matrix.to/#/%23polkadot-one-t-bot:matrix.org" target="_blank">Polkadot ONE-T Bot (Public)</a></td>
</tr>
</table>

#### Communnity Tips

ONE-T bot was already reported by community members for a [Tip](https://www.dotreasury.com/ksm/tips/12114700_0xfa0bff722b90af0e2c1b849b474df6f04a06ba636b23be8b3b8afb0cca2bda09) from the Kusama council and a [Tip](https://www.dotreasury.com/dot/tips/10119045_0xb57463139a1c42a76c382bb63f368f894ecd256b7d1f90129a7907577c271789) from the Polkadot council. Both accepted and approved by the respective councillors.

To gatter some feedback from the validators community we released an early version of the website and from there a new [Tip](https://www.dotreasury.com/ksm/tips/13615370_0x159dcb7dad9abc82772fad47f88edb33e54c1cd7ac4e7ce313ba577cd87db1cc) was opened from a community member.

#### Open Source

ONE-T bot and the website proposed are open-sourced, and both projects could be found [here](https://github.com/turboflakes/one-t) and [here](https://github.com/turboflakes/apps) respectively.

#### Team credentials & Other Tools

The team behind ONE-T is the same that created **Nomi**, a decision support tool for Nominators in NPoS networks (GitHub projects [backend](https://github.com/turboflakes/turboflakes-backend), [frontend](https://github.com/turboflakes/turboflakes.github.io)), the tool is currently available at [www.turboflakes.io](https://www.turboflakes.io). The tool has received funds from the Kusama Treasury to expand functionalities and is currently under development with some additional features to be released soon.

We are also the creators of other bots, **Crunch** and **Scouty**. Crunch is a command-line interface to easily automate payouts of staking rewards for Substrate-based chains (GitHub project [here](https://github.com/turboflakes/crunch)). Crunch has matrix integration built-in so that validator operators can be easily notified about the state of their staking rewards. Scouty is a command-line interface to keep an eye on substrate-based chains and hook things up (GitHub project [here](https://github.com/turboflakes/scouty)). Scouty also has matrix integration built-in so that anyone could get notified about on-chain events and extend functionality as soon as on-chain events linked to Session, Staking, ImOnline and Democracy pallets are emitted.

All our tools have been very well received by the community and have received Tips from the Kusama Treasury.

## 2. Problem statement

Since the release of parachains on Polkadot and Kusama networks, there was a significant change on how active validators collect points during an era and since validators are paid proportionally to the amount of [era points](https://wiki.polkadot.network/docs/maintain-guides-validator-payout#era-points) they have collected, it's critical for node operators to understand exactly where the majority of era points come from.

If a validator is the author of a block or produces a reference uncle block in the Relay Chain, it gives a validator solely 20 era points for each of these actions, which means that the rest of the points (the vast majority) come from parachain validation, most specifically from issuing validity statements for parachain blocks.

In this case, **when a validator is randomly selected to para-validate** (p/v) every epoch and participate in the parachain consensus **it means a great oportunity for the validator to top perform and collect as much points as possible**. If the highest number of points is not collected from that session it means that a validator has room for improvement, from changing a node location (without discouraging geographic decentralization) to a place with lower network latency to higher hardware specs or server optimizations, is up to the node operator to take action and decide all the trade-offs to become a top-notch validator.

In a worst case scenario - if a large group of low-performant validators make their way into the active set, the entirely network could be compromised. And for this reason alone is essential to keep **monitoring and provide diverse and user-friendly tools for the community to use**.

Reliable and trustfull node operators as well as active participant nominators would also be interested to know about, eg:

* how the validator points change along an entire era;
* understand the points variance between validators in the same group when p/v;
* to know from which parachains most likely more points will be earned;
* compare node performance when moved to a different location;
* compare node performance when moved to a different host server with different specifications;
* missed vote ratio variance between sessions;
* validator identity performance;

All these valuable insights are a must to know, so that validator operators could better serve their nominators and also to provide nominators with open data for a more informed decision making.

Parachain teams might also be interested to know how parachains compare to one another, eg:
* how attestations of validity are spread across parachains;
* how many core assignments each parachain was assigned during a session;
* missed vote ratio variance between sessions;

**From all the analitics tools available around, we believe that at the present time there is none apart from ONE-T that could give such detailed insights.**

The current solution provided by ONE-T can only be interacted via chat over matrix and reports are sent via text to the users who subscribe them. It's great and gets the job done. As previously mentioned, there are more than 170 members on both Polkadot and Kusama public rooms with over than 600 report messages sent daily.

**The current ONE-T problem is inclusion and the fact that text reports with *raw data* are hard to read. This proposal aims to solve those issues.**

## 3. Proposal Objective/solution/s to point 2:

The solution we propose is to make all current text reports available on the web, to make those reports easily digestible and easily accessible for anyone interested on this topic.

**Current Solution**

ONE-T currently supports the following reports:
* Network Report;
* Validator Performance Report;
* Parachains Report;
* Para-groups Report;
* Insights Report;

Additionaly to the reports, ONE-T grades all validators based on their session performance (missed vote ratio) and displays a top-best ranking published every era along side the *network report*. The ranking is used by the bot to nominate the best performant TVP validators into ONE-T nomination pools. More on this topic can be found on this [proposal](https://hackmd.io/@turboflakes/SyqlsRtYc).

Below are some examples of the text messages currently sent by ONE-T:

<table style="width:100%;" cellspacing="0" cellpadding="0">
<tr>
<td>
<img style="width: 100%;" src="https://i.imgur.com/8AKHsFE.png"/>
</td>
<td>Validator performance report</td>
</tr>
<tr>
<td>
<img style="width: 100%;" src="https://i.imgur.com/6VOrGiW.png"/>
</td>
<td>Val. Groups report</td>
</tr>
<tr>
<td>
<img style="width: 100%;" src="https://i.imgur.com/v4kruVv.png"/>
</td>
<td>Parachains report</td>
</tr>
<tr>
<td>
<img style="width: 100%;" src="https://i.imgur.com/gytE0fM.png"/>
</td>
<td>Insights report</td>
</tr>
</table>

As mentioned before, the aim of this proposal is to bring all of the previous reports to the web.

**Alpha Prototype**

To give a proof-of-concept to better support this proposal, we have released an alpha prototype with just three active pages so users could visualize and have a first-hand impression about the look and feel of the deliverable product of this proposal and even analyse para-validators in real-time.

The alpha prototype is available here → [apps.turboflakes.io](https://apps.turboflakes.io).


---


### 3.2. Who does this solution help?

- It aims to give validator operators insights about their validator performances;
- It aims to give any token holder an additional tool when researching by Validators to Nominate;
- It aims to give to anyone insightful information about the state of the network in real time;

### 3.3. Milestones and tasks to include:

The proposal is a single milestone:

**Milestone 1**

This milestone consists on the implementation of the new features.

Some work have already been done during the development of the bot and with the latest release of an alpha prototype, the respective source-code repositories are already public available and the domain of the project will remain the same → [apps.turboflakes.io](https://apps.turboflakes.io).

We pretend to continuos delivery and push new features during the ongoing implementation of this milestone.

Total time: 1120 hours (~4 months or ~October 2022)

| Deliverable | Description | Hours |
| ----------- | ----------- | ----- |
| REST API | Back-End development (BE-DEV) of a RESTful API to support all the frontend pages and components proposed. | 120 |
| WebSocket API | BE-DEV of a WebSocket API for clients to be able to subscribe topics of information and seamless push real-time data into those clients. | 160 |
| BSP aggregator service | BE-DEV of a data aggregator service to keep a temporary record (cache) of indexed on-chain data into an in-memory data structure store (redis) to offer fast page views. | 140 |
| UI/UX Design | Design and layout of all the site and components, with a special focus on the pages where easy to read charts could be analysed and `raw data` better explained. | 40 |
| Dashboard page | FE-DEV of a dashboard to visualize all the information currently available in the *network report* grouped into 3 different sub sets (100% commission validators, TVP validators and Other validators. | 120 |
| Insights page | FE-DEV of a page to show the current *insights report* where the user could navigate a timeline of the last number of sessions or search for a specific validator. | 120 |
| Parachains overview page | FE-DEV of a page with the sub set of para-validators that gives an overview of the Parachains or Val. groups performance. Easily pick a parachain or validator to further inspect how points unfold. | 120 |
| Validator performance page | FE-DEV of a page to show the current *validator performance report* to better understand how points and backing votes unfold along the session. Easily compare the searched validator data against the validators in the same group (peers). | 120 |
| Parachain performance page | FE-DEV of a page to show the overall points grouped by parachains and the most valuable validators by parachain | 100 |
| Nomination Pools | FE-DEV of a section with one or two pages to show the latest rankings and respective nomination for ONE-T Nomination Pools. Additionally if user intention is to become a member and join one of ONE-T pools he can do it without leaving the site. (similar to what is currently available here → one-t.turboflakes.io) | 80 |

**Ongoing Operational costs**

Total operational costs: 1080 USD (Q2+Q3) + 20% tax + 5% conversion fees (1080 * 25% = 1350 USD)

| Notes | Duration (months) | Price USD | Total USD |
| ----- | ----------------- | --------- | --------- |
| ONE-T site, hosting backend service (8GB Shared CPU Plan) | 6 | 50 | 300 |
| Polkadot + Kusama full node + memory store (16GB Shared CPU Plan + 500GB volume by Linode Cloud Hosting) | 6 | 130 | 780 |

## 4. Why Kusama?

Kusama is our live experimentation network of what we would expect Polkadot in the near future to be, so to build and try out something new, we will always do it on Kusama first.

## 5. If you have seen similar proposals before: why are yours different?

--

## 6. Payment conditions:

We plan to submit this proposal to cover the costs presented in M1 and operational costs to the Kusama council.

Dates: June-September 2022
Total USD costs: 95,200 USD + 20% tax = 114,240 USD
Hourly rate: 85 USD
Total team estimation hours: 1120 (~4 months for completion)

**Milestone 1 + Operational costs**
* Total USD cost to be submitted to Kusama Treasury: 114,240 USD + 1350 USD = **115,590 USD**
* Total KSM: **1,923.208 KSM** using KSM EMA30 Price (USD) of 60.102 from [Subscan](https://kusama.subscan.io/tools/charts?type=price) on the 04/08/2022.

## 7. Comments, Qs&As
--

## 8. We'd love to hear about how you got to know about the Kusama on-chain treasury: let us know, in a few sentences, how did you become familiar with the spending mechanism and the on-chain treasury?

We have been around in the community from January 2021, we follow both Direction channels on element where Treasury proposals are discussed and we try to be on top of new proposals and be informed as much as possible on how the spending mechainsm works in Kusama.