# Prime Trust Compliance

## Overview

Some changes need to be made to our Prime Trust integration for compliance as well as enabling the diamond trading business. The two main changes are:

1. Following a parent-child account structure
2. Transaction mirroring

## Parent-child account structure

Currently, the Prime Trust integration creates customer accounts with a flat structure. Customer accounts are only interacted with during deposits and withdrawals.

Now, funds that are deposited into customer accounts are automatically moved into an omnibus account. This omnibus account needs to be removed in order to remove M10 from the flow of funds.

### Account structure changes

The current account structure is very close to the desired parent-child account structure. Therefore, no changes with regards to creating customer Prime Trust accounts through the API are needed.

## Transaction mirroring

Since funds are now kept in customer Prime Trust accounts rather than an omnibus account, any transfers that take place between customers must also be reflected on the ledger. The Prime Trust integration should leverage a firehose subscription to listen for ledger transactions and make internal funds transfers between the related Prime Trust accounts.

### Firehose processing

A thread will be started on the server that handles listening to the firehose and processing them. It will listen to the stream through the Dart SDK. Specifying a filter of the PT issuance account will also send events for all child accounts. Incoming messages will be immediately persisted to a new table in the PostgreSQL DB.

Another thread will handle processing of cached transaction events in the DB. This will process transaction events on a specific time interval.

It is important that transaction events are processed in order. Adding a buffer (which can be ordered by timestamp) to the transaction processing flow rather than processing them directly from the stream can reduce the amount of events that are processed out-of-order. However, this does introduce more latency to the transaction processing. This should be okay as Prime Trust doesn't necessarily need to immediately mirror the M10 ledger. However, introducing large amounts of latency can cause users to wait longer for withdrawals to their bank account.

In conjunction to this, there are methods that could potentially handle out-of-order or missing events (e.g. an integrity check that verifies account balances after a certain number of transactions are applied).

### Sequence diagram


### Backpressure

Unfortunately, Prime Trust does not support batched/bulk transactions. When processing a single transfer directly from the stream, an API call to Prime Trust must be made. If stream events were processed immediately, this could eventually cause a significant amount of backpressure and potentially cause events in the stream to be lost. The buffered approach with processing batches from the DB would improve this. However, this would not be much of an issue at volumes we would be experiencing in the near future.

### Failure handling

One potential issue is messages in the stream being dropped or lost. The simplest approach is to replay the stream when processing a batch of transactions to verify that none have been lost. Currently, the SDK has the option to observe transfers starting from a specific transaction ID, but that still needs to be implemented.

As well, transactions must be applied in order and should never be skipped when failed. A retry mechanism can be added to try and recover from an error from a Prime Trust API call. However, if a call repeatedly fails, an alert mechanism should be added notifying the team that the issue needs to be investigated and resolved manually.

## Action plan

1. To do immediately

- Remove omnibus account interactions from m10trust
- Remove omnibus account creation from `pt-env-creator` tool
- Implement thread/process to observe transfers and create an internal asset tx in PT

2. For solidity

- Implement thread/process to observe transactions and store in DB
- Implement thread/process to process DB transactions on an interval

3. For error handling / integrity

- Implement observing transfers from a specific tx ID in the Dart SDK
- Implement account balance integrity check after a number of transfers
- Implement look back to ensure no transactions are missing
- Implement alert system for repeated Prime Trust API failures that stall processing