Beacon chain accounting - eODS feature

As I mentioned in the Accounting section of eODS design notes, beacon chain accounting already exists under the current protocol, in the form of beacon chain operations, or beacon state mutators, e.g. process_deposit, increase_balance, etc.

Since Accounting is an important part of eODS model, it proved useful to group these functions in a virtual group based on functionality (i.e. accounting) rather than beacon-chain specification appartenance. This document is a study I made on beacon chain accounting under the current Ethereum protocol.

The following is a design validation document (pseudocode), with a python project and pytests attached, in order to prove the beacon-chain accounting model I proposed for the eODS feature.

1. Validator balances

1.1 Validator records

We can add delegated boolean entry in the records:

class Validator(Container):
    pubkey: BLSPubkey
    withdrawal_credentials: Bytes32  # Commitment to pubkey for withdrawals
    effective_balance: Gwei  # Balance at stake
    slashed: boolean
    # Status epochs
    activation_eligibility_epoch: Epoch  # When criteria for activation were met
    activation_epoch: Epoch
    exit_epoch: Epoch
    withdrawable_epoch: Epoch  # When validator can withdraw funds
    delegated: boolean # new in eODS

1.2 Beacon state records

Validators' actual balance is stored in the BeaconState records, as a registry of validators and their balances.
We can add a fee List structure, mapping each Validator in the registry with its fee rate (NULL if delegated entry defined in class Validator is False):

BeaconState validators registry

​​​​validators: List[Validator, VALIDATOR_REGISTRY_LIMIT]
​​​​balances: List[Gwei, VALIDATOR_REGISTRY_LIMIT]
​​​​fees: List[Fee, VALIDATOR_REGISTRY_LIMIT]

2. Delegator balances

2.1 Beacon-chain records

We can add beacon chain class Delegator:

class Delegator(Container):
    pubkey: BLSPubkey
    withdrawal_credentials: Bytes32  # Commitment to pubkey for withdrawals
    delegated_balance: Gwei  # Balance at stake
    # Status epochs
    activation_epoch: Epoch
    exit_epoch: Epoch  

Delegator balances are to be stored in the BeaconState records, as a registry of delegators and their balances:

We can add delegators registry inside BeaconState, similar to validator registry

BeaconState delegators registry - new in eODS

delegators: List[Delegator, DELEGATOR_REGISTRY_LIMIT]
balances: List[Gwei, DELEGATOR_REGISTRY_LIMIT]

3. Beacon-chain accounting design validation project - delegating to validators

I went for a Test-driven Development approach for writing the model validation project. The minimal requirements of functionality proven by tests:

• test_initiate_Delegator - Add a delegator to registry and set its balance (deposit).

• test_add_multiple_Delegators - Add multiple delegators to delegators' registry and set their balance (deposit).

• test_delegate - Delegate to delegated validators and update:

  • the actual balance of delegated validators (credit balance with delegated amount)
  • the actual balance of delegators (debit delegator's actual balance with delegated amount)
  • total delegated balance inside a validator
  • the quotas of the delegators that delegated towards those particular validators.

• test_adjust_accounting - Adjust the validator's balance (due to profit / loss) and have the delegators' balances updated.

• test_adjust_accounting_keep_quota - Adjust the validator's balance (due to profit / loss) and have the delegators' quota unchanged.

• test_withdraw - Withdraw a specified amount from a delegator's balance in a particular validator, and update:

  • the actual balance of delegated validators (debit calidator's balance with the withdrawn amount)
  • the actual balance of delegators (credit delegator's actual balance with the withdrawn amount)
  • total delegated balance inside that validator
  • the quotas of the delegators that delegated towards that particular validator.

3.1 Tests

import unittest

from beacon_chain_accounting import DelegatorsRegistry, DelegatedValidator, BeaconChainAccounting

class TestSum(unittest.TestCase):
    def test_initiate_Delegator(self):
        """
        I should be able to add a delegator and set its balance
        """
        delegatorsregistry = DelegatorsRegistry()

        delegatorsregistry.register_delegator('Delegator_no1')
        delegatorsregistry.deposit('Delegator_no1', 12)
        value = delegatorsregistry.delegators_balances[0]

        self.assertEqual(value, 12)

    def test_initiate_multiple_Delegators(self):
        """
        I should be able to add multiple delegators and set their balance (deposit)
        """
        delegatorsregistry = DelegatorsRegistry()

        delegatorsregistry.register_delegator('Delegator_no1')
        #delegatorsregistry.register_delegator('Delegator_no2') # Intentionally left the first one un-registered to test if `deposit` correctly registers delegators, even if they don't exist
        delegatorsregistry.deposit('Delegator_no1', 12)
        delegatorsregistry.deposit('Delegator_no2', 12)
        value_for_delegator_no1 = delegatorsregistry.delegators_balances[0]
        value_for_delegator_no2 = delegatorsregistry.delegators_balances[1]

        self.assertEqual(value_for_delegator_no1, 12)
        self.assertEqual(value_for_delegator_no2, 12)

    def test_delegate(self):
        """
        I should be able to delegate to a delegated validator and update the quotas of the delegators that delegated towards that particular validator.
        """
        accounting = BeaconChainAccounting()
     
        for _ in range(2): 
            accounting.add_validator('Validator_no'+ f'{str(_ + 1)}')

        accounting.deposit('Delegator_no1', 500)
        accounting.deposit('Delegator_no2', 400)
        
        # Delegate funds from delegators to validators
        accounting.delegate("Delegator_no1", "Validator_no1", 100)  # Delegator 1 delegates 100 to Validator 1
        accounting.delegate("Delegator_no2", "Validator_no1", 200)
        accounting.delegate("Delegator_no1", "Validator_no2", 50)
        accounting.delegate("Delegator_no2", "Validator_no2", 150)

        value = accounting.validators[0].validators_balances
        self.assertEqual(value, 300)

        value = accounting.delegators_registry.delegators_balances[0]
        self.assertEqual(value, 350)

        value = accounting.delegators_registry.delegators_balances[1]
        self.assertEqual(value, 50)

        # Get quotas by delegator across all validators
        print(accounting.get_all_quotas_by_delegator())
    
    def test_adjust_accounting(self):
        """
        I should be able to adjust the validator's balance (due to profit / loss) and have the delegators' balances updated
        """
        accounting = BeaconChainAccounting()
     
        for _ in range(2): 
            accounting.add_validator('Validator_no'+ f'{str(_ + 1)}')

        accounting.deposit('Delegator_no1', 150)
        accounting.deposit('Delegator_no2', 350)
        
        # Delegate funds from delegators to validators
        accounting.delegate("Delegator_no1", "Validator_no1", 100)  # Delegator 1 delegates 100 to Validator 1
        accounting.delegate("Delegator_no2", "Validator_no1", 200)
        accounting.delegate("Delegator_no1", "Validator_no2", 50)
        accounting.delegate("Delegator_no2", "Validator_no2", 150)

        accounting.adjust_DelegatedValidator_balance('Validator_no1', 100)

        valueDelegator_no1 = accounting.delegators_registry.delegators_balances[0]
        self.assertEqual(valueDelegator_no1, 0) 
    
    def test_adjust_accounting_keep_quota(self): #BROKEN
        """
        I should be able to adjust the validator's balance and have the delegators quota unchanged
        """

        accounting = BeaconChainAccounting()
        accounting.delegate('Delegator_no1', 10)
        accounting.delegate('Delegator_no2', 10)
        quotaDelegator_no1 = accounting.delegators[0].quota
        self.assertEqual(quotaDelegator_no1, 0.5)    

        accounting.adjust_accounting_balance(100)

        quotaDelegator_no1 = accounting.delegators[0].quota
        self.assertEqual(quotaDelegator_no1, 0.5)


if __name__ == '__main__':
    unittest.main()

3.2 classValidator and classDelegator

For the python project purposes, I wrote toy classValidator and classDelegator and imported them in the code file. The validator and the delegator IDs are stored as string, i.e. Validator_no1, Validator_no2, and Delegator_no1, Delegator_no2 respectively, instead of delegator pubkeys, as in the beacon-chain specification.

import numpy as np

class Validator:
    def __init__(self, validator_id):
        self.validator_id = validator_id  # Unique identifier for each validator
        
        # Custom types
        Epoch : np.uint64 = 0
        Gwei : np.uint64 = 0
        BLSPubkey : str
    
        # pubkey = BLSPubkey # for the purpose of this pyproject, we work with validator_id instead of BLSPubkey, which will be used in the specs
        self.withdrawal_credentials: bytes = b'\x00' * 32  # Commitment to pubkey for withdrawals
        self.effective_balance = Gwei  # Balance at stake
        self.slashed: bool
        # Status epochs
        self.activation_eligibility_epoch = Epoch  # When criteria for activation were met
        self.activation_epoch = Epoch
        self.exit_epoch = Epoch
        self.withdrawable_epoch = Epoch  # When validator can withdraw funds
        self.delegated: bool # new in eODS

import numpy as np

class Delegator:
    def __init__(self, delegator_id):
        self.delegator_id = delegator_id  # Unique identifier for each delegator

        # Custom types
        Epoch : np.uint64 = 0
        Gwei : np.uint64 = 0
        BLSPubkey: str 

        # self.pubkey = BLSPubkey # for the purpose of this pyproject, we work with delegator_id instead of BLSPubkey, which will be used in the specs
        self.withdrawal_credentials: bytes = b'\x00' * 32  # Commitment to pubkey for withdrawals
        self.delegated_balance =  Gwei  # Balance at stake
         # Status epochs
        self.activation_epoch = Epoch
        self.exit_epoch = Epoch

3.3 Beacon-chain-accounting code

The code file stores validators' and delegators' data as a list of Validator and Delegator class instances. Looking deeper into the implementation, we can observe that the following functions perform balance sheet operations, as follows:

• register_delegator - Registers a delegator if not already registered and returns the delegator index

• deposit - Adds an amount to the balance of an existing or newly registered delegator

• delegate - Delegates capital to a d elegated validator, updates validator's and delegator's balance, and recalculates delegators' quotas.

• withdraw - Withdraws a specified amount from a delegator's balance in this validator, returning it to the delegator's available balance, updating delegators' quotas.

• adjust_DelegatedValidator_balance - Adjusts the total DelegatedValidator value (due to profit/loss) without changing individual delegators' quotas.

*calculate_quotas - Calculate and return the quota for each delegator in this validator.

Helper functions:

• get_all_quotas_by_delegator - Returns a list of delegators with their quotas across all validators. Each entry contains the delegator ID and a list of their quotas across validators.

• get_validator_index - Get a validator's index in List[Validator], based on its given ID

from typing import List
import numpy as np
from classDelegator import Delegator
from classValidator import Validator

# State list lengths
VALIDATOR_REGISTRY_LIMIT = 100  # Validator registry size limit
DELEGATOR_REGISTRY_LIMIT = 100  # Delegator registry size limit


# Custom types
Gwei = np.uint64
Fee =  np.uint
Quota =  np.uint
delegator_index = np.uint
validator_index = np.uint

class DelegatorsRegistry:
    delegators: List[Delegator]  # Stores delegators' data as a list of Delegator instances.
    delegators_balances: List[Gwei]  # List of Gwei delegators' balances
    delegators_quotas: List[Quota]  # List of delegators' quotas
    
    def __init__(self):
        # Delegators lists initialization
        self.delegators: List[Delegator] = []     
        self.delegators_balances: List[Gwei] = [] # Max size: DELEGATOR_REGISTRY_LIMIT
        self.delegators_quotas: List[Quota] = [] 

    def register_delegator(self, delegator_id):
        """Registers a delegator if not already registered and returns the delegator index."""
        for index, delegator in enumerate(self.delegators):
            if delegator.delegator_id == delegator_id:
                return index  # Delegator already exists; return the index

        # Register new delegator with a zero balance
        new_delegator = Delegator(delegator_id)
        self.delegators.append(new_delegator)
        self.delegators_balances.append(0)  # Initial balance is 0
        return len(self.delegators) - 1  # Return the index of the newly added delegator

    def deposit(self, delegator_id, amount):
        """Adds an amount to the balance of an existing or newly registered delegator."""
        if amount <= 0:
            raise ValueError("Deposit amount must be positive.")

        # Register the delegator if they don't exist and get their index
        delegator_index = self.register_delegator(delegator_id)
        # Add the deposit amount to the delegator's balance
        self.delegators_balances[delegator_index] += amount
        return delegator_index

class DelegatedValidator:
    delegators_balances: List[Gwei]  # List of Gwei delegators' balances

    validators: List[Validator]  # Stores validators' data as a list of Validator instances.
    validators_balances: List[Gwei]  # List of Gwei validators' balances
    validators_fees: List[Fee]  # List of Gwei validators' balances
    
    def __init__(self, validator_id):
        # Valaidators lists initialization
        self.validators: List[Validator] = [] 
        self.validators_balances: List[Gwei] = 0 # Max size: VALIDATOR_REGISTRY_LIMIT
        #BROKEN self.validators_fees: List[Fee] = []
        self.validator = Validator(validator_id)
        
        self.delegators_balances = []  # Track delegated balances for each delegator in this validator

    def delegate(self, delegator_index, amount):
        """Transfers capital from a delegator to the validator, updating both balances."""
        if amount <= 0:
            raise ValueError("Delegation amount must be positive.")
        
        # Ensure the validator's balance list is large enough to accommodate the delegator
        while len(self.delegators_balances) <= delegator_index:
            self.delegators_balances.append(0)

        # Deduct from the delegator's balance and add to the validator's balance
        self.delegators_balances[delegator_index] += amount
        self.validators_balances += amount

        # Return updated quotas for this validator
        return self.calculate_quotas()

    def withdraw(self, delegator_index, amount):
        """Withdraws a specified amount from a delegator's balance in this validator,
           returning it to the delegator's available balance."""
        if amount <= 0:
            raise ValueError("Withdrawal amount must be positive.")

        # Ensure the withdrawal amount does not exceed the delegator's balance in the validator
        if amount > self.delegators_balances[delegator_index]:
            raise ValueError("Withdrawal amount exceeds the delegator's delegated balance in this validator.")

        # Add to the delegator's balance and deduct from the validator's balance
        self.delegators_balances[delegator_index] -= amount
        self.validators_balances -= amount

        # Recalculate and return updated quotas after the withdrawal
        return self.calculate_quotas()

    def calculate_quotas(self):
        """Calculate and return the quota for each delegator in this validator."""
        total_balance = self.validators_balances
        
        # Avoid division by zero if there is no balance
        if total_balance == 0:
            return [(index, 0.0) for index in range(len(self.delegators_balances))]
        
        # Calculate quotas for each delegator
        quotas = [(index, balance / total_balance) for index, balance in enumerate(self.delegators_balances)]
        return quotas

class BeaconChainAccounting:
    def __init__(self):
        self.validators = []  # List of DelegatedValidator instances
        self.delegators_registry = DelegatorsRegistry()  # Registry to manage delegators

    def add_validator(self, validator_id):
        """Add a new validator with a specific ID."""
        validator = DelegatedValidator(validator_id)
        self.validators.append(validator)
        return validator_id  # Return the validator_id of the newly added validator

    def get_validator_index(self, validator_id):
        """Helper function to find a validator's index by its ID."""
        for index, validator in enumerate(self.validators):
            if validator.validator.validator_id == validator_id:
                return index
        raise ValueError("Validator not found.")

    def deposit(self, delegator_id, amount):
        """Deposit capital from outside the system to a delegator's balance."""
        return self.delegators_registry.deposit(delegator_id, amount)

    def delegate(self, delegator_id, validator_id, amount):
        """Transfer capital from a delegator's balance to a specific validator's balance."""
        # Get validator index by its ID
        validator_index = self.get_validator_index(validator_id)

        # Ensure the delegator exists and retrieve their index
        delegator_index = self.delegators_registry.register_delegator(delegator_id)

        # Ensure the delegator has sufficient balance to delegate
        if amount > self.delegators_registry.delegators_balances[delegator_index]:
            raise ValueError("Delegation amount exceeds the delegator's available balance.")

        # Deduct the delegation amount from the delegator's remaining balance
        self.delegators_registry.delegators_balances[delegator_index] -= amount

        # Delegate the amount to the validator
        return self.validators[validator_index].delegate(delegator_index, amount)
    
    def adjust_DelegatedValidator_balance(self, validator_id, change_in_value): #BROKEN
        """Adjusts the total DelegatedValidator value (profit/loss) without changing individual quotas."""
        
        # Get validator index by its ID
        validator_index = self.get_validator_index(validator_id)
        
        self.validators[validator_index].validators_balances += change_in_value
    
        if self.validators[validator_index].validators_balances < 0:
            raise ValueError("DelegatedValidator total value cannot be negative.")

        for delegator_index in enumerate(self.validators[validator_index].delegators_balances):
            self.delegators_registry.delegators_balances[delegator_index] += change_in_value * self.delegators_registry.delegators_quotas[delegator_index]
        # Quotas remain the same; balances are recalculated.

    def withdraw(self, delegator_id, validator_id, amount):
        """Withdraw capital from a validator's balance back to the delegator's available balance."""
        # Get validator index by its ID
        validator_index = self.get_validator_index(validator_id)

        # Get the delegator's index from the registry
        delegator_index = self.delegators_registry.register_delegator(delegator_id)

        # Perform the withdrawal from the specified validator
        result = self.validators[validator_index].withdraw(delegator_index, amount)

        # Add the withdrawn amount back to the delegator's balance
        self.delegators_registry.delegators_balances[delegator_index] += amount

        return result

    def get_all_quotas_by_delegator(self):
        """
        Return a list of delegators with their quotas across all validators.
        Each entry contains the delegator ID and a list of their quotas across validators.
        """
        delegators_quotas = []
        
        # Iterate over each delegator in the registry
        for delegator_index, delegator in enumerate(self.delegators_registry.delegators):
            quotas = []
            for validator in self.validators:
                # If the delegator exists in the current validator, get their quota
                if delegator_index < len(validator.delegators_balances):
                    quota = validator.calculate_quotas()[delegator_index][1]
                else:
                    quota = 0.0  # Quota is 0 if the delegator has no balance in this validator
                quotas.append(quota)
            
            # Append delegator ID and their quotas across validators
            delegators_quotas.append((delegator.delegator_id, quotas))
        
        return delegators_quotas

Resoucres:

• Beacon-chain-accounting Github repository