---
# System prepended metadata

title: Previous

---



## Bots: The Next Generation (June 5-16?)
- transformers: crank out as many as possible in 2-3 hours
- error-reporting, documentation, streamlining, squashing issues
- documentation: how do they work, how to spin them up, create your own etc.
- make bots smarter: reactive to market state, do it intelligently, etc.
- bug-squashing, figuring out crashes, reducing them, checking and improving precision
- generate crash reports: parse logs and report errors, what's the best format?
    - auto-send in some way: discord, email, hand delivered
- data science: tell stories in notebooks
    - don't rush this, rather deliver more polished product externally
    - tell 1-2 stories about stuff we're building anyway (smarter bots?)
- forecasting time-series: benchmark tradfi methods in generic testbed (ElfAI Gym)

## UPDATED PLAN:

* Parity – we need to update elfpy to get up to parity
  * merging the fixedpoint stuff (this week)
  * finishing some baseline cross-platform tests (smart contract march vs elfpy at “old” parity)
  * update smart contract to now
    * update elfpy such that cross-platform tests pass again
    * get_max is the most important – used by bots a lot
* Data analysis service 
  *	NOW:
    * pnl & ohlcv contract data grabbed from aws (see fuzz bots)
  * FUTURE:
    * separate service in docker
    * holds everything that queries blockchain for information purposes
    * runs stand-alone from bot execution
    * generates plots that can be seen
* Fuzz bots
  * DONE: Docker pipeline
    * cloud compute + docker container
    * anvil spins up a genesis chain
    * migration scripts – deploy contracts, initialize them, etc
    * artifacts
    * hosts addresses
    * can be used
  * Elfpy compose app
    * Get elf-sims image working.
    * Update infra docker-compose.yml to use the bot service definition posted above.
    * Refactor the bots so that they can use settings from the environment (RPC_URL, PRIVATE_KEY, MNEMONIC, etc.)
    * Refactor the bots or add a line to the services command that polls http://artifacts/addresses.json. This should wait for a status of 200 and then use the provided addresses in the bot.
  * Alex deploys fuzz bots
      * Steps:
          1) deploy 
          2) improve error handling
          3) post-processing & investigation
      * organize data from event logs
      * relational (or whatever data team thinks is best) database that we can hit
      * pull data from contract events
      * want to hit an api and get data in a python-friendly format
      * jupyter notebook preferred over dashboard
          * need to change & inspect what is going on; lowest friction is best
      
      
    
# Previous

## Data Stuff
{%hackmd @themes/dracula %}
Big picture:
1. get some output ASAP
2. iterate 50x to get it right

## The Demo

- Cash's website has a link that says "OHLCV"
- That page has regularly updating data
- Comes from Jacob
- End to end complete
- weaving of backend from Jacob and FE
- Delivery method:
  1. simplest thing is: AWS .png, streamlit
  2. Cash's website
- Which plots:
  1. volume (raw)
    - put signs in the right place
    - add it
    - plot it
    - conversion between base and PT (likely not)
  2. PNL (no mock)
    - call to HyperdriveMath library (onchain? local?)

**v0: unintegrated**
n-trades: run them on Python
 rerun same n trades on Solidity

**v1: integrated on local**: everlasting_bots
for trade in [1,]
  do_python
  do_devnet

**v2: run on testnet**: testy_bots
```
for trade in [1,]
  do_python(agents)
  do_devnet(trade)
```
  
**v3: integrated testnet**:
```
for trade in [1,n]
  do_python(agents, market_state)
    -> test it with Louies and Fridas
  do_testnet(trade)
  do_python(update_agent)
while listen(blocks):
  market_state <- Solidity
  do_python(calc_things: PNL_nomock, PNL_mock)
    -> export to CSV -> send to Cash
  do_python(plots)
```

**v4: integrated**: future_shit
for trade in [1,]
  do_python: trade_details -> market/market_state
  do_testnet: trade_details -> ape_trade
  throw warnings on any precision differences

### Priority list:
1. merge testnet bots PR
2. update ape PR
3. respond to plantuml guy on github
4. calculate bot PNL
5. calculate OHLCV
6. do v3: integrated testnet
7. calculate trade aggregation (python elfpy-compatible way)
8. more charts
    - OHLCV historical charts
    - PNL per address
    - LP profitability
    - ecosystem impact
    - interest rates over time
9. fix devnet discrepancies (devnet = local Foundry fork)
10. stresstest the workflow w/ high trade volume (10 trades per block?)

### Mihai
tie up loose ends
- Ape PR
- py2puml PR
look into CTC
redpill Alex on BOTS BOTS BOTS
benchmark historical queries across different approaches:
    1. ape
    2. CTC 
    3. trueblocks
    4. different RPC providers: Infura, Alchemy, etc.
    5. remote vs. local RPC
    
    
### Jacob
- Create volume output and data structure that matches this pattern, (https://tradingview.github.io/lightweight-charts/docs#setting-the-data-to-a-series)
- Automate remaining transformations for OHLCV data.
- Identify unique address that have interacted with the Hyperdrive contract 
- Create aggregate PnL data for each contract. 
- Charts for non-FE data. Maybe Databricks, excel, non-centralized solution?
- Trueblocks index fix. 

## Captain's Log

###### 2023.04.18

1. ~~fix bots crashing~~
2. ~~check their randomness~~
3. ~~[prepare to] land testnet bots PR~~
4. ~~pump out [a single] chart~~
![](https://i.imgur.com/JmrkGbh.png)

##### 2023.04.19

1. ~~prep testnet bots PR~~

##### 2023.04.20

1. ~~prep testnet bots PR~~

##### 2023.04.21

1. ~~land testnet bots PR~~
2. review what data we have
3. create some charts
 - he has json's
 - spit out spot prices (no mock pnl)


## Path to v3 demo
#### Executive summary:
2 product types: "parallel conveyor belts" & "integrated cogs"
Focus -- integrated cogs:
* bots view elfpy market_state constructed from testnet data
* bots return trades from this state
* trades are executed on testnet using Apeworx
    * bots update personal wallets from trade receipt
* cloud node & database scrapes evm for trade data
* format trade data into an endpoint
* parse trade data & construct `TradeSimVariables` sim state
    * use existing elfpy code as much as possible
* from sim state, produce formatted data outputs for frontend (or streamlit) to make real-time plots (e.g. `pnl_data.json` or `trade_volume.json`).
* deliver URLs that point to time-varying plots



#### What we've done:
1. run random bots on solidity alone (`bots_on_solidity.ipynb`)
    * full sim executed & returns trade tape
    * trade tape is parsed & executed on solidity
2. run smart bots on solidity and python (`everlasting_bots.py`)
    * bots are executing trades directly on local network
    * bots view sim state built from solidity market state
3. run random bots on testnet (`testy-bots.py`)

#### Left to do:
1. integrate testnet bots with python
    * combine testy-bots.py with everlasting_bots.py so that you have smart bots and random bots running trades on testnet & updating their wallets.
   * read testnet data into market_state (done)
   * pass market_state to agents
   * test with Louies and Fridas
   * update agent state after every trade
2. build cloud data processing flow
   * need API python scripts that use elfpy to get computed variables
7. read testnet data into market_state
8. use elfpy for calculations: PNL_nomock and PNL_mock
9. generate plots (from python or in front-end)

**next version: integrated testnet**:
```
for trade in [1,n]
  do_python(agents, market_state)
    -> test it with Louies and Fridas
  do_testnet(trade)
  do_python(update_agent)
while listen(blocks):
  market_state <- Solidity
  do_python(calc_things: PNL_nomock, PNL_mock)
    -> export to CSV -> send to Cash
  do_python(plots)
```

Next demo:
what we have:
* trade volume on the protocol
    * time
    * total trades in that time
    -> could convert this to ohlcv

* aws uri pointing to json file that has trade-by-trade breakdown with:
    * address
    * trade type ("open long")
    * trade amount, in bonds (abs)

* output formats
    * date, open high, open low, close
    * date, contract type, pnl


