# Token Bucket - Documentation

The Token Bucket service is consisted of Azure Functions that have particular triggers.

The role of this service is primarily to manage the various Pix Token Buckets[^Bucket] from our various bank adapters. But the service was made generic enough that it will be able to plug in any kind of adapter, even Crypto related, with minimal refactoring.

In regards to Pix, each account per bank, per CNPJ, has a maximum of 2000 tokens. Each account is internally called "Entity". The ways tokens are manipulated are:
- Every 1 minute 2 tokens are given to each account
- Each Pix key validated that is "valid" deducts 2 tokens.
- Each Pix key validated that is "invalid" deducts 10 tokens.
- Each confirmed Pix transaction adds 2 tokens.

These changes are inserted into the DB as a Token Transaction[^TokenTransaction]

So each confirmed transaction should in theory be "neutral", and only be of concern if there are too many invalid Pix key validations.

Currently the token distribution is hard coded for all entities. If an entity that is not related to Pix services is plugged in, a refactor is needed.

## Design Considerations

The service was conceived with DDD and onion layer pattern. As of writing, 65% of the lines are covered by tests. The Application layer has a 98% test coverage.

The service was designed with two aspects in mind: Reducing the amount of RUs consumed by Cosmos while achieving good parallelism capabilities, and making the code as performatic as possible.

This means every Bucket change is made with the *PatchItemAsync* method from CosmosDB's SDK. Internally, Cosmos deals with the calls using this method by enqueuing all patch operations, eliminating any concerns of parallel execution.

Another big advantage of using *PatchItemAsync* is eliminating the need of fetching a whole document before updating.

In regards to code performance, benchmarks showed that the only bottlenecks of concern are related to CosmosDB operations. The operations used are to create a document, to patch a document, and to get the Container.

Getting the container was initially made before each patch request. This was verified to be a huge time sink, taking almost half the entire flow's duration. This was mitigated by adding the clients and services related to Cosmos as Singleton instead of Scoped, and getting the Container in the Service's constructor. This means that the Container itself is gotten only once.

As for changing documents, *PatchItemAsync* is supposedly the most performatic method for this kind of operation.

Document creation is mostly dealt with IRepository's methods. There are no performance concerns with the package.

All queries are made exclusively with the partition key for RUs optimization.

The DB modeling philosophy is that each kind of document has a certain partition key (e.g. "Bucket" for buckets, and "BucketTransaction_<name of entity>" for all token transactions). Bucket's unique keys are in the form of "Bucket_<name of entity>"" (e.g. "Bucket_JUSTBS2"). The bucket document IDs are the same as the unique keys.

Meanwhile, the transactions are consisted of either TokenTransaction, BasicIncome, or Transaction, concatenated with "Bucket" as a prefix, and the transaction's GUID as suffix. The form is "Bucket<type of transaction>_<notification GUID>"(e.g. "BucketBasicIncome_5f191cb8-a0a8-4b19-ae37-220374f65eab").

This though process was to ensure each kind of document, for each kind of entity, has a certain partition key. Querrying this partition key, it's possible to filter the operations further based on the unique key.

## API

There are 5 Azure Function that have the roles of endpoints. Three are triggers related to azure, and two are regular HTTP triggers. They are:

- AdaptersNotification (Service Bus trigger)
- BasicTokenIncomeDistribution (Timer trigger)
- BucketChangeFeed (CosmosDB trigger)
- GetAllBuckets (HTTP trigger)
- GetBucket (HTTP Trigger)

### Application

#### AdaptersNotification

This function is a ServiceBusTrigger that receives a JSON from an Azure queue that conforms to the object TokenBucketNotificationDto[^TokenBucketNotificationDto].

SettlementBank[^SettlementBank] is an Enum from Model.

BucketNotificationType[^BucketNotificationType] is an Enum from the TokenBucket nuget package.

BucketNotificationStatus[^BucketNotificationStatus] is an Enum from the TokenBucket nuget package.

Note that both start at the number 1.

##### AdaptersNotificationService

The flow of this service consists of:
* Validating the input, to make sure nothing is null from the DTO received
* Creating or Updating the token bucket of the DTO's entity. If the bucket' entity does not exist, it first creates the bucket and then applies the change.
* Creates a Token Transaction that contains the changed token value. These transactions have a TimeToLive of 30 days to prevent clutter, and still allow auditoring.

#### BasicTokenIncomeDistribution

This function is a Time Trigger. It triggers every 1 minute. This function does not receive any parameters.

##### BasicTokenIncomeDistributionSerice

The flow of this service consists of:

* Making a query to retrieve all entities' buckets.
* Iterates through the list and adds a set amount of tokens for every entity.
* Withing the iteration it creates a Token Transaction that contains the changed token value. These transactions have a TimeToLive of 30 days to prevent clutter, and still allow auditoring.

#### BucketChangeFeed

This function is a CosmosDBTrigger listening to the "lease" container. It listens only to changes made on any Bucket. The function acts in determining if there is a need to alert the Monitoring Service if there is an unhealthy bucket with low token count.

##### BucketChangeFeedService

The flow consists of:

* Receives a list of Documents from Cosmos
* Decides the Alert Severity of the Bucket.
* The severity is based on the AlertSeverity enum[^AlertSeverity], contained in the TokenBucket nuget package.
* If the Token Count Severity is different from the one currently contained within the bucket, it updates t he bucket with the new Severity.
* If the Token Count Severity is not Healthy, it notifies the Monitoring Service queue that said entity is unhealthy.
* An AlertMessage object is sent to the queue.

#### GetAllBuckets

This function is an HTTP Trigger to retrieve every bucket of every entity.

It queries the container with the Partition Key "Bucket" and returns a list containing all documents found.

The path is "/api/GetAllBuckets"

#### GetBucket

This function is an HTTP Trigger to retrieve a single bucket.

It queries the container with the Partiton Key "Bucket" and the provided entity ID.

The path is "/api/GetBucket/{entityId}".

# Object references

[^TokenBucketNotificationDto]: TokenBucketNotificationDto -

    {
        "bucketEntity": <string>,
        "settlementBank": <SettlementBank enum>,
        "notificationType": <BucketNotificationType enum>,
        "status": <BucketNotificationStatus enum>,
        "transactionId": <string>    
    }

[^BucketNotificationType]: BucketNotificationType -

        {
            PixKeyValidation = 1,
            Transaction,
            BasicIncome
        }

[^BucketNotificationStatus]: BucketNoficationStatus -

        {
            Valid = 1,
            Invalid
        }

[^Bucket]: Example of a Bucket document -

    {
    "entity": "KaoBank",
    "tokens": 1366,
    "tokenCountSeverity": 2,
    "uniqueKey": "Bucket_KaoBank",
    "partition": "Bucket",
    "_etag": "\"16067f52-0000-0b00-0000-636916500000\"",
    "timeToLive": null,
    "createdTimeUtc": "2022-10-03T20:29:57.3502409Z",
    "id": "Bucket_KaoBank",
    "type": "Bucket",
    "_rid": "NrQyAJM8BsoKAAAAAAAAAA==",
    "_self": "dbs/NrQyAA==/colls/NrQyAJM8Bso=/docs/NrQyAJM8BsoKAAAAAAAAAA==/",
    "_attachments": "attachments/",
    "_ts": 1667831376
    }

[^TokenTransaction]: Example of a Token Transaction document -

    {
    "ttl": 2592000,
    "bucketEntity": "hueBank",
    "settlementBank": "None",
    "transactionId": null,
    "tokens": 2,
    "notificationType": 3,
    "uniqueKey": "BucketBasicIncome_c874c286-3f3a-4979-b32f-6f58e90fa06a",
    "partition": "BucketTransaction_hueBank",
    "_etag": "\"16068f52-0000-0b00-0000-636916530000\"",
    "timeToLive": "30.00:00:00",
    "createdTimeUtc": "2022-11-07T14:30:03.4173489Z",
    "id": "c874c286-3f3a-4979-b32f-6f58e90fa06a",
    "type": "TokenTransaction",
    "_rid": "NrQyAJM8BsojNAAAAAAAAA==",
    "_self": "dbs/NrQyAA==/colls/NrQyAJM8Bso=/docs/NrQyAJM8BsojNAAAAAAAAA==/",
    "_attachments": "attachments/",
    "_ts": 1667831379
    }

[^SettlementBank]: SettlementBank enum -

    {
    None = 0,
    GenialBank = 125,
    StoneBank = 197,
    BS2Bank = 218
    }

[^AlertSeverity]: AlertSeverity enum -

    {
    Healthy = 1,
    Medium,
    High,
    Critical
    }