# Ethereum API endpoints

## The goal

- Take the backend part of staking widget into a dedicated project and take it for support.
- Add additional pools APR endpoints.
- Ensure continuous and user-friendly work with the endpoints.
- Make the solution scalable to other teams/blockchains.
- Have documentation on the process of writing and deploying new endpoints.

## Requirements

- Available for internal use without restrictions
- Available for external teams
- Can handle a large number of concurrent requests
- Has docs and an OpenAPI Specification
- Has versioning of endpoints for backward compatibility
- Provides monitoring and alerting for:
- the application work
- work of third-party services
- data correctness

## Endpoints

In the first iteration, it's suggested that only ethereum endpoints be put into a separate project. In the next iteration, we can do the same for the Terra endpoints.

- [Protocol APRs](#Protocol-APRs)
- [Pool APRs](#Pool-APRs)
- [Token prices](#Token-prices)
- [Token TWAPs](#Token-TWAPs)
- [Token stats](#Token-stats)

### Protocol APRs

- Ethereum APR. Implementations: [previous](https://github.com/lidofinance/staking-widget-ts/blob/develop/pages/api/apr.ts), [current](https://github.com/lidofinance/staking-widget-ts/blob/main/pages/api/eth-apr.ts)
- stETH APR. [Current implementation](https://github.com/lidofinance/staking-widget-ts/blob/main/pages/api/steth-apr.ts)

:::spoiler URLs example

/v1/protocol/eth/apr
/v1/protocol/steth/apr

:::

:::spoiler Response example

{
    data: [
        {
            APR: 5.1,
            timeUnix: 1643800000
        }
    ],
    meta: {
        symbol: "stETH",
        address: "0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84",
        chainId: 1
    }
}

:::

Since it's supposed to store historical data, it's worth making it possible to retrieve it for a certain period. It is proposed to add the possibility to specify the start and end time of the sampling for most of the endpoints.

:::spoiler Query parameters example

/v1/protocol/eth/apr?startTime=1643800000&endTime=1643900000

:::

Since there may be a lot of data, it is worth providing some limits or pagination. For the first iteration we can limit the period to 30 days

### Pool APRs

There is no current implementation for pools APRs. Endpoints are supposed to be used for the new section in the staking widget, similar to the [solana widget](https://solana.lido.fi/defi). It would be useful to have in the response: TVL, daily trading volume, detailed information about incentives.

- 1inch v2
- Balancer v2
- Curve
- SushiSwap

:::spoiler URLs example

/v1/pool/1inch/steth-eth/apr
/v1/pool/balancer/wsteth-weth/apr
/v1/pool/curve/steth-dai/apr
/v1/pool/sushi-swap/wsteth-dai/apr

:::

:::spoiler Quiery parameters example

/v1/pool/1inch/steth-eth/apr?startTime=1643800000&endTime=1643900000

:::

:::spoiler Response example

{
    data: [
        {
            totalAPR: 12.345,
            totalAPY: 12.456,
            totalValueLockedInUsd: 1234567.89,
            dayTradingVolume: 10000,
            incentives: [
                {
                    id: 1,
                    type: 'steth',
                    active: true,
                    apr: 2.3
                },
                {
                    id: 2,
                    type: 'fees',
                    active: true,
                    apr: 0.1
                },
                {
                    id: 3,
                    type: 'rewards',
                    active: true,
                    apr: 2.48
                },
                // ...
            ],
            timeUnix: 1643800000,
        }
    ],
    meta: {
        pool: {
            address: "0xDC24316b9AE028F1497c275EB9192a3Ea0f67022",
            lpToken: {
                name: "Curve.fi ETH/stETH",
                symbol: "steCRV",
                address: "0x06325440d014e39736583c165c2963ba99faf14e",
            },
            tokens: [
                {
                    symbol: "ETH",
                    address: "0xEeeeeEeeeEeEeeEeEeEeeEEEeeeeEeeeeeeeEEeE"
                },
                {
                    name: "Liquid staked Ether 2.0",
                    symbol: "stETH",
                    address: "0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84"
                }
            ],
        },
        links: [
            // Not sure about this section, 
            // probably it's the responsibility of the UI
            {
                url: "https://curve.fi/steth",
                type: 'swap',
            },
            {
                url: "https://curve.fi/steth/deposit",
                type: 'deposit',
            }
        ],
        incentives: {
            1: {
                 token: {
                    name: "Lido DAO Token",
                    symbol: "LDO",
                    address: "0x5a98fcbea516cf06857215779fd812ca3bef1b32"
                }
            }
        },
        chainId: 1,
    }
}

:::

Useful links:
- The calculations can be taken from [Dune dashboard](https://dune.xyz/LidoAnalytical/Integration-monitor-dashboard)
- [Incentives for LPs](https://www.notion.so/D2110INV-Incentives-for-LPs-Dashboard-fd415f29e58541088a761715bd5604bd)
- [Liquidity pools contracts](https://docs.lido.fi/deployed-contracts#liquidity-pools)
- [Reward program contracts](https://docs.lido.fi/deployed-contracts#reward-programs)

:::warning
There may be a situation where the reward program has finished, but has not yet started. Since the results are supposed to be cached, such a case should be handled carefully.
:::

### Token prices

Current implementation: https://replit.com/@irinalido/TWAPhourly#main.py

In current implementation we use third-party service (api.coingecko.com) to get the data. This is ok for the first version, but later it is worth getting the data directly from the blockchain.

:::spoiler URLs example

/v1/token/steth/price
/v1/token/ldo/price

:::

:::spoiler Response example

{
    data: [
        {
            priceUSD: 2.01,
            timeUnix: 1643800000
        }
    ],
    meta: {
        symbol: "stETH",
        address: "0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84",
        chainId: 1
    }
}

:::

### Token <abbr title="Time-weighted average price">TWAP</abbr>s

See [Token prices](#Token-prices) section

:::spoiler URLs example

/v1/token/steth/twap
/v1/token/ldo/twap

:::

:::spoiler Response example

{
    data: [
        {
            priceUSD: 2.01,
            timeStartUnix: 1643800000,
            timeEndUnix: 1644404800
        },
        {
            priceUSD: 2.1,
            timeStartUnix: 1643800000,
            timeEndUnix: 1646392000
        }
    ],
    meta: {
        symbol: "stETH",
        address: "0xae7ab96520DE3A18E5e111B5EaAb095312D7fE84",
        chainId: 1
    }
}

:::

### Token stats

Detailed information about Lido tokens. In current implementation we use third-party service (api.ethplorer.io) to get the data. This is ok for the first version, but later it is worth getting the data directly from the blockchain.

- stETH. Implementations: [current](https://github.com/lidofinance/staking-widget-ts/blob/main/pages/api/lidostats.ts), [short](https://github.com/lidofinance/staking-widget-ts/blob/main/pages/api/short-lido-stats.ts)
- LDO. [Current implementation](https://github.com/lidofinance/staking-widget-ts/blob/main/pages/api/ldo-stats.ts)

:::spoiler URLs example

/v1/token/steth/stats
/v1/token/ldo/stats

:::

:::spoiler Response example
https://stake.lido.fi/api/lidostats
https://stake.lido.fi/api/ldo-stats
:::

## Implementation

The implementation is supposed to be based on the [API template](https://github.com/lidofinance/lido-api-template). The template was designed to our requirements and our infrastructure and already contains most of the necessary modules:
- JSON logs
- Prometheus metrics
- Swagger
- Versioning of endpoints
- Throttling
- CORS headers
- Response caching

### Instances

It's suggested that one application be deployed in two instances. One copy for internal use, and one copy for external use. For public use, request throttling will be enabled. Internal service is closed with API keys.

For the first iteration we can have only one instance with throttling.

### Data availability

#### Results storage

Since in some cases the calculations may take a long time or the data provider may not be available at all, it's suggested to separate the calculations and the server response. The results of the calculations should be placed in the DB and read from there when requested. This approach will also allow us to have access to historical data.

#### Proposed design

    Cron job-->API Provider: Request required data
    API Provider->Cron job: Result
    Note over Cron job: Calculating
    Cron job->Database: Store calculating result
    Controller-->Database: Request last\ncalculation result
    Database->Controller: Result

It was discussed for Solana that it would be useful to be able to set pools APRs manually. In case we use the database, in an emergency situation, we can correct the record.

There can be two solutions here:
1. Implement endpoints to add and delete a custom value.
2. Add UI to the database, with HTTP authorization. We can use [Adminjs](https://adminjs.co/) for this.

When using a custom value, an alert should be triggered.

#### Fetching

Most of the data can be retrieved directly from the blockchain using services such as Infura. Since from time to time we have problems with availability of certain API providers, it's assumed that we have a fallback provider to which we switch if the first one is unavailable. For this we already have a [fetcher with fallbacks and retries](https://github.com/lidofinance/lido-nestjs-modules/tree/main/packages/fetch).

### Data correctness

In order to track anomalies in the results of calculations, it's assumed that we provide these values as metrics and track minor and major changes with alerts.

### Tests

It's suggested that at least the calculations be covered by tests to be sure that the result is in the acceptable range of values.

### Networks

It's suggested that the app is integrated into our infrastructure with the standard deploy scheme:

testnet=>operation: Testnet
staging=>operation: Staging
mainnet=>operation: Mainnet

testnet(right)->staging(right)->mainnet

For the testnet we can use Goerli and the single [liquidity pool](https://docs.lido.fi/deployed-contracts/#reward-programs-1).