# Formal methods in tokenomics design An economics noob's hacks for financial engineering --- ## [silur@edcon7]$ whoami - Independent consultant and developer - Quantum/Crypto Researcher @ Wigner's institute - Ethical hacker --- ## Agenda - Problem formulation - Simulation technique - "Optimal" Allocation - Utility theory - Control-theory in tokenomics - Stackleberg --- ## Problem formulation - Most people realized making an ERC20 involves lots of engineering - Except for the financial one.... - Heavy lack of formalism and toolchains for tokenomics design - Mostly done by copying from ICOdrops --- ## Use your (MC)-MC Methods to sample and make your martingale: - Normal-inverse gauss - Laplacian - Autoencoders --- ```python= df = pd.read_json('/tmp/binance.json') # ETH/USDT df.columns = ['timestamp', 'open', 'high', 'low', 'close', 'volume'] df = df.close returns = df.pct_change().fillna(0) a, b, loc, scale = norminvgauss.fit(df.dropna()) x = np.linspace(0, 1000) plt.hist(norminvgauss.rvs( a, b, loc=loc, scale=scale, size=1000), bins=100) plt.hist(df, bins=100) ```  --- ```python= encoder = nn.Sequential(nn.Linear(28 * 28, 64), nn.ReLU(), nn.Linear(64, 3)) decoder = nn.Sequential(nn.Linear(3, 64), nn.ReLU(), nn.Linear(64, 28 * 28)) class AutoEncoder(pl.LightningModule): def __init__(self, encoder, decoder): super().__init__() self.encoder = encoder self.decoder = decoder def training_step(self, batch, batch_idx): timestamp, close_price = batch z = self.encoder(x) x_hat = self.decoder(z) loss = nn.functional.mse_loss(x_hat, close_price) self.log("train_loss", loss) return loss def configure_optimizers(self): optimizer = optim.Adam(self.parameters(), lr=1e-3) return optimizer model = AutoEncoder(encoder, decoder) ``` --- ## Determining supply Cambridge equation: $M_{d} = kPQ$ If you tokenize an existing asset PQ is known in your market research! <small>Vitalik's take on the topic is $M/k$ = Q/P:<br>https://vitalik.ca/general/2017/10/17/moe.html</small> --- ## Treasury sizing A very common strategy in tokenization is to make a token and a fiat pool, then balance them based on performance. - If we assume efficient markets, a well modelled treasury always reaches equilibrium - But with startups we are more concerned about adaptability... --- We need something that can capture (or adapt) to utility theory  <small>Source:<br> Foundations of Reinforcement Learning with Applications in Finance - Ashwin Rao, Tikhon Jelvis</small> --- BME-pool (from https://arxiv.org/pdf/2210.12881.pdf): $M_{t+1} = M_{t} - T_p + \frac{T_b}{p+\Delta p}$ --- ## Method 1 - LQR - Time-invariant :tada: - Deterministic :heavy_check_mark: - Most DeFi logic is not time invariant :x: - You cannot leave any uncertainty in your formal model :x: --- $\dot{x} = Ax + Bu$ $min_u \sum_0^\infty x'Qx + u'Ru$ --- ```python= from pydrake.systems.controllers import DiscreteTimeLinearQuadraticRegulator as DLQR # x0 = pri x0, x1, x2, x3 = var('x0 x1 x2 x3') # totalsupply, usdPool, tokenPool, price u0, u1, u2 = var('u0 u1 u2') # usdBuyback, tokenBuyback, premium s = [[1e6*(1.1**i), 30e3*(1.1**i)] for i in range(3)] xref = [[300e6*(1.1**i), 100e3*(1.1**i), 50e6*(1.1**i), 0.1*(1.1**i)] for i in range(3)] uref = [[50e3*(1.1**i), 49e3*(1.1**i), 0.005*(1.1**i)] for i in range(3)] ``` ```python= def suball(f, i): return f.subs(x0=xref[i][0], x1=xref[i][1], x2=xref[i][2], x3=xref[i][3], u0=uref[i][0], u1=uref[i][1], u2=uref[i][2]) ``` --- ```python= cost = (x3 - xref[i][0])**2 + (u0 - uref[i][0])**2 + (u1 - uref[i][1])**2 Q = matrix([ [suball(taylor(cost, x0, xref[i][0], 2), i),0,0,0], [0,suball(taylor(cost, x1, xref[i][1], 2), i),0,0], [0,0,suball(taylor(cost, x2, xref[i][2], 2), i),0], [0,0,0,suball(taylor(cost, x3, xref[i][3], 2), i)]]) R = matrix([ [suball(taylor(cost, u0, uref[i][0], 2), i),0,0], [0,suball(taylor(cost, u1, uref[i][1], 2), i),0], [0,0,suball(taylor(cost, u2, uref[i][2], 2), i)]]) ``` --- The actual optimization ```python= f1 = x0 + u1 - (u0/(x3+u2)) f2 = x1 + s[i][1] - u0 f3 = x2+(x0/(x3+u2))-u1 f4 = s[i][0]/(x0 + u1 - (u0/(x3+u2))) A = suball(jacobian([f1,f2,f3,f4],[x0,x1,x2,x3]), i) B = suball(jacobian([f1,f2,f3,f4],[u0,u1,u2]), i) K, S = DLQR(A,B,Q,R) ``` --- ## Method 2 - Model predictive control - You can leave uncertainty in your projection :heavy_check_mark: - It can "simulate" future uncertainty within boundaries :heavy_check_mark: - Global-Local optimality tradeoff :x: - Very slow :x: --- ```python model = do_mpc.model.Model('discrete') x0 = model.set_variable(var_type='_x', var_name='supply', shape=(1,1)) x1 = model.set_variable(var_type='_x', var_name='usdPool', shape=(1,1)) x2 = model.set_variable(var_type='_x', var_name='tokenPool', shape=(1,1)) x3 = model.set_variable(var_type='_x', var_name='price', shape=(1,1)) u0 = model.set_variable(var_type='_u', var_name='tokenBuy') u1 = model.set_variable(var_type='_u', var_name='usdBuy') u2 = model.set_variable(var_type='_u', var_name='premium') ``` --- ```python= model.set_rhs('supply', x0 + u1 - (u0/(x3+u2))) model.set_rhs('usdPool', x1 + s[i][1] - u0) model.set_rhs('tokenPool', x2+(x0/(x3+u2))-u1) model.set_rhs('tokenPool', s[i][0]/(x0 + u1 - (u0/(x3+u2)))) ``` --- ```python= f1 = x0 + u1 - (u0/(x3+u2)) f2 = x1 + s[i][1] - u0 f3 = x2+(x0/(x3+u2))-u1 f4 = s[i][0]/(x0 + u1 - (u0/(x3+u2))) ``` --- ```python mpc = do_mpc.controller.MPC(model) setup_mpc = { 'n_horizon': 20, 't_step': 0.1, 'n_robust': 1, 'store_full_solution': True, } mpc.set_param(**setup_mpc) simulator = do_mpc.simulator.Simulator(model) u0 = mpc.make_step(x0) ``` --- ## Method 3 - PID - Easiest to implement :heavy_check_mark: - Can react to almost any kind of uncertainty :heavy_check_mark: - But prote to over/undershoot :x: - Very vulnerable in an exomonic setting :x: ---  --- Example of PID in the wild: https://scribe.privacydev.net/reflexer-labs/summoning-the-money-god-2a3f3564a5f2 - D is nulled as it's predictable - I is also nulled due to leaky integrators - Using P only is prone to windups --- ## Method 3 (my fave) - MDP - The output is a whole policy instead of a SPO - Can handle enormous amount of stochastic uncertainty - Easy to implement, opensource libs - Doesn't need a complete dynamic system in the form of an ODE --- - PI, VI - Q-Learning - DQN :warning: - PPO :warning: :warning: --- ```python= action_space = np.random.uniform(0., 0.1, 1000) def fvi(reward_function, transition_function, discount_factor, num_samples, num_iterations): value_function = lambda s: 0 optimal_policy = lambda s: 0 for _ in range(num_iterations): states = np.stack(( calc(start=0.0006, a=100, b=-0.01, loc=0, scale=0.001), calc(start=2000, a=10000, b=-0.1, loc=0, scale=2000) ), axis=1) actions = np.random.uniform(0, 0.1, size=1000) values = reward_function(states, actions) model = LinearRegression().fit(np.column_stack((states, actions)), values.ravel()) for s_prime in states: values = values + discount_factor * transition_function(states, actions, s_prime) * value_function(s_prime) value_function = lambda s: model.predict(np.hstack((s, optimal_policy(s))).reshape(1,-1))[0] optimal_policy = lambda s: np.argmax(model.predict(np.hstack((np.array([s for i in range(1000)]), action_space.reshape(-1, 1))))) return optimal_policy, value_function ``` --- ```python states = np.stack((price_history, volume_history), axis=1) reward_function = lambda s, a: np.array([s[:,1]*a]) transition_function = lambda s, a, s_prime: 1 discount_factor = 0.9 num_samples = 1000 num_iterations = 10 optimal_policy, value_function = fvi(reward_function, transition_function, discount_factor, num_samples, num_iterations) ``` --- ## Problems most of these methods can be done at both sides so buyers can speculate on it --- ... Or put more formally, there is a game with asymmetric rules where both sides can observe the other :thinking_face: --- This is called a Stackleberg (leader-follower) game which thankfully always have an equilibrium (that's slightly skewed towards the leader) --- ## Conclusion - use iQLR for easy models quarterly - use MPC for medium-sized DeFi to have some "what if" headspace - use MDP for complex policies realtime combined with a KKT-solved stackleberg game - Publish the policy in your WP so the community can play the game as a follower - Always have a sound projection --- Slides available  @huohuli @silur@infosec.exchange https://silur.me