# Hueristic balance spec

The idea behind this spec is to map out the design for Bundlr's new balance systems which looks to increase the TPS limit. Currently Bundlr is limited to ~300-500 txs per address per second. We suspect this can reach 10k TPS per address per second under certain loads.

## Design

This section outlines the design of the current system and the planned new system based on heuristics.

### Upload process

1) Pipe body to file
2) Verify transaction
3) Check if the user has enough balance
4) Insert data into DB and return signed response

1, 2 and 4 don't change but 3 will be the big problem to solve.

---

### Solution to balance checks

Bundlr will need to adopt a heuristic balance function which allows us to estimate whether they have enough balance with a high level of accuracy.

We'll need to define a full agorithm which allows us to control the risk factor when using this system.

#### New functions

##### estimate_has_balance

A function which returns `true` or `false` based on whether we estimate the user has enough balance. Can be easily define as the below formula.

Inputs:
- address: `string`
- currency: `string`

Output: `boolean`

Implementation:

balance_multiple = 1000
currency = <some-currency>
address = <some-address>
size = <numer-of-bytes>
fee = <some-fee>

if size >= 100MB
return exact_has_balance(address, currency, fee)

estimated_balance = estimated_balace(address, curency)

if ((estimated_balance*balance_multiple) > fee)
return true
else
return exact_has_balance(address, currency, fee)

##### get_redis_cached_balance_difference

The key component to this system is a cached balance value in Redis which stores the difference between the latest rollup value and now. This value will be updated every 10 seconds with the correct value in the database.

Upon uploading successfully this value will be incremented (using `incrbyfloat`). The known limitation of `incrbyfloat` is that we can an only do up to `1e48`.

Inputs:
- address: `string`
- currency: `string`

Output: `number`

##### exact_has_balance

Check if the user has the exact balance required to upload. We define "exact" as `>=fee`. This function will use a Postgres function internally.

---

### Balance check algorithm

currency = <some-currency>
address = <some-address>
size = <numer-of-bytes>
fee = <some-fee>

if size >= 100MB
return exact_has_balance(address, currency, fee)
else
return estimate_has_balance(address, curency, size, fee)

---
### Problems to solve

#### Cached balance difference accuracy

A key problem here is to make sure the cached Redis value is accurate.

Measures to make the Redis value accurate:
- Every 10 seconds, make all cached values accurate 
- Each successful upload will increment the Redis value
- When consolidating the Redis value we'll get the difference in balance from Redis and include in the new value

#### Bad performance

There are going to be some edge cases where performance is affected:
- When a user reaches 0 balance then we'll cache in Redis and upon new funds in the DB we'll delete the cached value

#### Allow accurate balance checks

The main problem is supporting apps which use balance lookups as a key component for things like automated funding and warning mechanisms.

TODO: Efficient mechanism for exact balance lookups. Probably just use an index or something

---

## Side effects and risks:

- Balances could become potentially far less accurate
- Balance lookup route will be slower
- This system is more complex are more likely to fail
- More load on the DB read replica