# Implementing dYdX v4
As someone who is implementing sth like this for the first time, 3 differnt approaches pops up, where none of them is optimal:
1) Indexer
2) Running a node
3) Scraping the mintscan website for data

## Indexer
The V4 Indexer is a set of microservices which provides a set of rich APIs and websocket channels for a dYdX V4 application. It is the easiest way to query trader profile, and there are two ways to do it. Either by running a local v4 node or connecting to a remote v4 node. Unfortunatly, the instructions on running a local v4 node is an outdated link on dydx github page https://github.com/dydxprotocol/v4#running-the-chain-locally, which forces us to connect to a remote node to query the data. Additionally, to test our implementation, we can first use the https://indexer.dydx.trade/v4/perpetualPositions endpoint to query the data that we intend to get using the Indexer v4. Unfortunatly, this approach can't be used for the main product since there is a rate limit. Therefore one should set up an Indexer follwoing the link https://github.com/dydxprotocol/v4-chain/tree/main/indexer. Before discussing the possible implementation, here is the API calls that are available through the Indexer https://indexer.dydx.trade/docs/. Indexer also gives us access to off-chain data which we don't actually need currently, but can be used to create a heat map since limit orders are not stored on chain. Indexer does not give us an API that would help us to detect new events therefore using indexer will cause a big overhead that is proportioanl to number of active traders. Since we don't have an API for new events, we need to find a realtivly efficient algorithm to update the profiles with not that big of an overhead. One can initially query all the data for addresses and store the postiions as well as bids, the bids for each pair are stored in a list where we sort them based on prices. After that we have pairs many lists. Since, as mentioned, we don't have access to new events, what we do is we track the price of each pair and iterate over each list based on the price (in this case lists with the iterator serve as an outdated orderbook). We then set a integer threshold $k$ which will simply serve as an indicator. Whenever price movement causes an iteration over $k$ many orders or $\Delta t$ time has passed since the last refresh, we query each address one more time to refresh the data starting from the once with the most orders. Of course this is a very suboptimal approach, but the best using the indexer. PSA: A code tat can be used to set up the profiles https://gist.github.com/kingbaldwin-iv/7e7ff68081f75dae454e7cb76a119d0b. (Setting up the profiles can be done very easily using the indexer, the main issue is processing new transactions.)

## Node/Validator
The main isssue with this approach is there are no documentations for running a full dydx v4 node. Additionally, there are at most 60 validators, so it cannot be done wihtout capital and time allocation to convince people.

## Scraping the mintscan website for data
The most complicated sounding one is probably the best approach. We keep scraping the website https://www.mintscan.io/dydx/tx after a default prefatching for past and current positions which can be done via bruteforce since we only do it once. Whenever a new Proposed Operations or Place Order type transactions pop up, we simply scarpe the events the way we need. An additional decryption of the data is unnecessary as on EVMs perp exchanges since in this case each tx clearly states what the implications are. We then update the profiles.

## Summary
To have a reasonable implementation of dydx, we need a full node or a validator but the documentations ont heir side is missing https://docs.dydx.exchange/validators/running_validator, https://docs.dydx.exchange/validators/running_full_node. The following screenshot from the Chaos Labs grant offer, makes me suspect they struggled with the same issue. OR maybe I am missing sth.