# Vault Performance Analysis ## Gate side By Neha Oudin (Opex) --- ## Analysis with datadog * Using APM and RUM * RUM brought a CounterValues question * APM to directly look at the gate traces and spans --- ## Using datadog to get info  --- ## Using datadog to get info  --- ## Next step * Spin up minivault * Load a test db (copy from komainups 2021) * Start console with debug logs to show SQL requests * Play around with console and try to understand what caused what --- ## Break down of issues * Missusing peewee * Peewee weird behaviors * N+1 problem on Foreign Keys * Useless components still in use * Wrong DB model (specific cases) --- * Question : How much DB interactions does it take to serialize one komainups account from 2021? * Answer: 49 * Question: How many times the account is fetched? * Answer: 14 --- ## Why? * Peewee "Lazy loading" is actually ill-named ```python= class Account(BaseModel): ... class AccountStateRequest(AwareModel): account = ForeignKeyField(Account) account = Account.get_by_id(1) # Gets the account again account.account_state_request.serialize() ``` * Peewee actually gets again the account --- #### Lazy loading of Peewee * Lazy loading gets the data when we access the object. * I load an object Account and then I use the backref to get the AccountStateRequest and I access it. * It loads the account again. * Can be disabled with `enable_lazy_loading=False` Disabling Peewee lazy loading allows to only get the row. We can access the account **IF NEEDED** by doing a joint. --- ## N+1 Problem * Happens on `GET /accounts/<id>` * We fetch the child accounts from WD * We get all the accounts by address * The missing accounts are created. * Code on next slide --- ```python= def _fetch_token_family(self, account): parent_account = ... ### Iterate on all addresses from WD for token_address in ...: self.reconcile_token_account(...) ``` And the reconcile_token_account ```python= def reconcile_token_account(self,...): ... token_account = next( AccountFactory.select(<filter>), None, ) if not token_account: token_account = <create the token account> return token_account ``` --- * We could do better * Get all objects in a query using SQL's `IN` * Compare the lists of address and reconcile the missing token accounts --- ## Exporting a transaction * In csv_fields in `Transaction.py` we have this: ```python= def csv_fields(self) -> dict[str, Any]: return { ... "label": self.notes[0].title if len(self.notes) > 0 else "", "note": self.notes[0].content if len(self.notes) > 0 else "", ... } ``` * We get the notes object twice * We get twice the transaction * 3 calls to transaction instead of 1 * Solution : Prefetch the object --- ### Other topics: * Disable code that was once useful and could be again * Use FeatureFlags to disable it * Example: MigrationService still used by `GET /organization` * Goes through all objects to check the migration status * Could be disabled * Governance Rules : 1 table per type of governance rules * Getting Rules for a RuleSet does 5 queries instead of 1 * All inherit from same model just methods change * All should be in one table with an additional row --- ### Conclusion: * Beware of data as code * Modularity can be dangerous * Learn to use jointures * Look at the requests generated by peewee * Use query builders * Beware of data model and tables