# Script: Generating Stochastic Processes

###### tags: `cadCAD edu`

___
## Links

- Slides: https://docs.google.com/presentation/d/1CNYuuiRn_ckGczodvP4_E-E4RKA1LvCfV_BRzya8YC4/edit
- Notebook: https://github.com/cadCAD-edu/cadCAD-Hacks-Private/blob/main/season-1/cadcad-hack-generating-stochastic-processes.ipynb
- GDrive folder:
https://drive.google.com/drive/u/0/folders/1dLm1z2erLPCER5HU9Lm1MuxOl8y-iAkE

___
## Transcript

___
**Slide 1 (Logo)**
- Welcome to cadCAD Hacks, the format in which we introduce you to the tips and tricks used by professional Web 3 cadCAD engineers.

___
**Slide 2 (How to)**
- In this cadCAD Hack, we will demonstrate step by step how to generate stochastic processes as part of a cadCAD model.

___
**Slide 3 (Why Should You Care)**

- Why should you care about this Hack as a cadCAD Engineer?
- You'll find this very useful for simulating values that are representative of real data for modeling and counterfactuals!

___
**Slide 4 (What this video covers)**
- Alright, here is what we'll cover in this video is
- How to generate Gamma random numbers
- How to generate Gaussian random numbers

___

**Slide 5 (Let's get started...)**
- Now, let’s jump straight into the Jupyter Notebook, and get started!

___
**Notebook (Table of Contents)**

- Alright, our Notebook is structured along with the cadCAD Edu Standard Notebook Layout and it is based on the **Live API Data hack**, which shows how to load the Ethereum per Bitcoin historical price data inside a cadCAD model; If the **Live API Data hack** is new to you, we highly recommend revisiting cadCAD Hack number 7 before continuing.
- We’ll first import any required dependencies.
- In the **Setup and Preparatory Steps** we will download historical prices from Coingecko API.
- In the **modeling** and **simulation** sections we will use the same **Live API Data hack** model.
- We'll spend most of our time here in the **Analysis** section (**NOTE: show with mouse**), where we're going to generate Normal and Gamma random distributions, and visually compare them agains historical data.
- Alright, let's jump in.

___
**Notebook (Dependencies)**
As in any cadCAD Hack, we have two dependency sections
- **The first one** contains the **cadCAD standard dependencies** you are familiar with from the Complete Foundations Bootcamp or the cadCAD Edu Cheat Sheet.
- As for the **hack-specific dependencies** we'll import:
- **json** and **requests** libraries for pulling the price data from the API;
- **math** for generating random numbers;
- **Scipy Stats module** for generating stochastic processes. This is a scientific library that contains all kinds of statistical and numerical features that one should desire!;
- **Plotly Express** and **pyplot** for visualization.
- Given that our hack is based on the Hack number 7, Live Data, let's jump straight to the analysis section.

___
**Notebook (Analysis)**
- Okay. So for a refresher, our output for the Live Data API hack are figures that show how the Ethereum price as denominated in Bitcoins evolves daily over time, as well as the percent change on each day.
- Suppose now that we want to simulate data points for the price feed that resembles the real data. One way of doing that, is by taking random numbers that follows a similiar probability distribution, and one typical choice is to use the Gamma Distribution.
- Long history short, the Gamma distribution generates random numbers that are between zero and infinity, and it contains several statistical properties that makes it a natural choice for price data.
- In order to generate data with it, we define the `gamma_proc` function, which takes as a input a number of points to be generated and returns a list of random numbers. It works by invoking the scipy stats `gamma` class, and by using the `rvs` method of it. We are setting arbitrary numbers here for generating something that is close to the real data, although this can be done automatically by using Statistical Fitting procedures.
- Then, we create a function called `generate_gamma` that takes a cadCAD results dataframe, and returns a new dataframe that contains two additional columns: one which is called `origin`, which will keep track of what data is generated or real, and another called `daily_price`, which contains the real numbers. By taking the old and the new dataframe together, we concatenate them so that we can compare them into a single visualization.
- With that function ready, we perform our data preparation step: we get the simulation results, we drop the first row, we assign the `origin` column so that it indicates that the source is the real data, and we pipe that dataframe into our `generate_gamma` function.
- The result is a dataframe that contains generated and real price data for each timestep, and we then plot it through the `px.histogram` method.
- By passing `daily_price` into `x`, we can visualize the distribution of the prices, and by passing color equals `origin`, we can differentiate between the source. Also, we pass margin equals `violin` so that we have a second visualization of the distribution.
- And that's it! Notice that the real data - indicated by blue - bars and margin is remarkebly similiar to the generated ones - indicated by red. This demonstrates that the generated data should look relatively similiar to the real one.
- We can also use a different probability distribution for a different variable. For instance, this time we'll be using a Normal Distribution for modelling the relative daily price changes. The procedure is just like the previous one, but we use the `normal_proc` function instead of the `gamma_proc`, where we invoke `st.norm.rvs` for generating normal numbers rather than gamma ones. Also, on the `fig_df` definition, we set the variable of interest as being `normed`, which is equal to the difference on daily prices divided by the current price.
- By doing that, we can visualize again the distribution when comparing real data vs simulated data. Notice that although they're generally similiar, the real data is long-tailed while the simulated one is not. This is a common phenomenae on complex systems, and is the basis of a lot of common knowledge and concepts in finance, like the notion of anti-fragility.

**Notebook (OUTRO)**
- And that's it for this Hack! By having a grasp of how to generate random numbers that follow different probability distributions and to compare them with real data, you're now equipped to perform stochastic simulations that are more realistic and nuanced. This is a key skill when dealing with real world complex systems!

___
**Slide 5 (Happy Hacking)**
- We hope you enjoyed this Hack and that it will be useful on your journey as an cadCAD engineer.
- As always, happy hacking, and see you next time!