# Keys API 2.0.0


## Motivation

## Current problems

1. Adding new staking modules in Keys API: no common interface of key, no understading how key will be stored, the same about operator
2. Not convenient endpoints: difficult to fetch data. few time already added new endpoints , to fecth the same data in diffrent form
3. difficult to stream data
4. update could happen between fetching from two different endpoints, and we will need to repeate again because of change lastChangedBlockHash
5. need for DSM 1.5 information about, when data was fetched
6. If kapi under audit, we will lost apportunity to change it every time we need.
7. sometimes we need indexer
8. maybe we need to give opportunity to access data from key api database for some our services

## How currently update in Keys API work

## How othen Keys API make update

## How to update in events

Solution base on idea, that we can from event transaction get all information we need.

Events we need to list:

https://github.com/lidofinance/lido-dao/blob/5fcedc6e9a9f3ec154e69cff47c2b9e25503a78a/contracts/0.4.24/lib/SigningKeys.sol#L78

https://github.com/lidofinance/lido-dao/blob/5fcedc6e9a9f3ec154e69cff47c2b9e25503a78a/contracts/0.4.24/nos/NodeOperatorsRegistry.sol#L896C21-L896C50

Fetching event in diapason (the same algorithm for cold start and update)

For list of events:

Try to find key in table by list of fields (operator_id, pubkey for example)

**Add event:**
- if key exist in database and deleted field is empty, this key is a duplicate, add key with index increase;
- if key exist in database and delete field is not empty , it means key was deleted. So we need to create a new record and not update old. So it means in our table unique record is (block_hash, key)

Where are no other cases of Add event.

**Remove event:**

- found event (how to understand index of key?), how to delete exactly key we need? Maybe we should choose any unvetted key across duplicates. For example it is possible and we found key in database, we should set deleted field with timestamp.

**Deposit event**

Event of changing deposited amount for operator.
We got already defined used value from contract. now we can define it by operator value. During used=true or false request we could based on time of creation and deposited keys amount of operator.

**How we will form answer for blockhash ?**

block hash has some block number N
Will fetch from database all keys with empty deleted field or with value bigger than current timestamp and less block than N block number.

How to fetch event for historical blockhash


it is possible to use materialized views to not request again the same data

as in our tasks block number will be increased and we can use result from prev iteration, but only for canonical work.

So this optimization is poosible to make with database.

WITH KeyRanks AS (
    SELECT
        k.signature,
        k.operator_id,
        k.create,
        o.deposited_keys_amount,
        ROW_NUMBER() OVER(PARTITION BY k.operator_id ORDER BY k.create) AS rk
    FROM
        keys k
    INNER JOIN
        operators o ON k.operator_id = o.id
    INNER JOIN
        block b ON k.block_id = b.id
    WHERE
        b.is_canonical = TRUE
        AND (k.deleted IS NULL)
        AND k.create IS NOT NULL
        AND b.block_number < 1234567
)
SELECT
    kr.signature,
    kr.operator_id,
    kr.create,
    kr.deposited_keys_amount
FROM
    KeyRanks kr
WHERE
    kr.rk <= kr.deposited_keys_amount;

1. how duplicates are deleted in a contructs ?
2. maybe we need to remove non-canonical blocks
3.

we will get result with multiple records with the same key , we should do smth like this

we also need to define limit of keys we need in answer based on KeysAmount value

current request will just return all keys events for existing keys in moment 1234567 above

WITH KeyEvents AS (
    SELECT
        key,
        COUNT(CASE WHEN created IS NOT NULL THEN 1 END) AS Adds,
        COUNT(CASE WHEN deleted IS NOT NULL THEN 1 END) AS Deletes
    FROM
        keys
    GROUP BY
        key
),
// will found duplicates too
KeysAmount AS (
    SELECT
        key,
        Adds - Deletes AS NetAdds
    FROM
        KeyEvents
)

SELECT
    Y.key,
    Y.created
FROM
    keys Y
    INNER JOIN KeysAmount N ON Y.key = N.key
WHERE
    Y.created IS NOT NULL
ORDER BY
    Y.key,
    Y.created

Questions:

How reorg influence on events

will add canonical block to blocks table and will not update keys in case of delete, will create separate raw with keys data and deleted time
so such raw will have created null record

## Backlog

### Storage of two (few) states

What problems could be solved:
- consistency of responses for two different requests
- decrease number of endpoints, endpoint will have convinient structure, it will be easy to stream data

Problem it will not solve:

- if update happen in kapi because of nonce change or reorg
- - если происходит реорганизация мы перекачиваем все состояние от finalized до latest
- - - KeyOpIndexChanged, - [...blockhash, what changed]
- - - - может не быть блок хэша
- - - -

### Adding new staking modules in Keys API

Problem: no common interface of key, no understading how key will be stored, the same about operator

All fields of registry keys is required for every module
we could think about it strcuture as minimum required for module.

========

1. Идея с ивентами
2. Идея мапинга хэшей и базового состояния
3. Наивное обновление с сохранением состояний

========

проблема в райнтайме, не холодный старт
1. ключи добавляются малыми частями, постепенно, поблочно (500)
2. fin=0, +500 -> 500 + 500 -> 1000 + 500...
3. так загружаются 10 пачек ключей (5000)
4. происходит реорганизации и запускается перезагрузка всех 5000 ключей
[0x1,0x2,0x3]
[0x1,0x2,0x555]

---

Модульность

- что такое модульность
-

## Что описать в доке?

0. как апдейт сейчас происходит?

1. оптимизация апдейта - Аня
1.1 в чем проблема
1.2 как решить

2. для чего нужно хранить больше 1 состояния. Нужно ли? - Аня
как хранить

3. что можно сделать с эндпоинотами ? - finally consider grpc graphql

4. сколько будет стоить перейти на апдейт на ивентах ? что из этого выиграем? - Саша

5. описать как добавить новый модуль? как дорого это будет

6. dsm 1.5 сколько стоит добавить время создания ключа? какие проблемы? нужно ли менять ивенты в контракте? - Аня

7. в каком виде отдавать на аудит ? как исправить архитектуру?