Swarmsim
========

[Swarmsim](http://github.com/codex-storage/swarmsim) is an event-driven simulator written in nim that is being developed with two main goals:

1. serving as a platform with which to experiment with swarm protocols (at first);
2. help me learn some nim.

Swarmsim attempts to allow as much flexibility as possible in implementing simulations that are more, or less realistic. We would like to be able to code protocols as close as possible to real implementations when needed, but also take shortcuts when looking to gain speed. Swarmsim is built with TDD, and it should therefore evolve towards a core API that allows you to more easily TDD protocols as they are developed.

Although not feature-complete in any way, the goal is to build something that can be extended to support:

* simulation of link delays and bandwidth constraints;
* trace-driven and probabilistic (e.g. Markov, semi-Markov) availability models;

while at the same time allowing room for simpler, cycle-driven simulations where messages are simple function calls when scalability and simplicity are more important.

## Core Abstractions - Current State

A sketch of swarmsim's abstractions can be seen in Figure 1.

<center>
<image src="https://hackmd.io/_uploads/Sy5yeiLa3.png"/>
</center>

**Figure 1.** Core swarmsim abstractions.

These abstractions allow one to:

1. instantiate a set of peers and their protocols;
2. send and receive messages from peers.

A simple example of how to instantiate peers and send them messages can be seen [here](https://github.com/codex-storage/swarmsim/blob/main/tests/engine/tnetwork.nim) and [here](https://github.com/codex-storage/swarmsim/blob/main/tests/codex/tblockexchange.nim). The fact that message sends happen in simulation time and are mediated through the network towards peer objects mean we can model link delays (random or trace-driven), and create bandwidth models which take into account link capacity and messages in flight. The latter is not implemented yet, but it is perfectly feasible. The event-driven engine also makes it trivial to implement up/down renewal processes in peers, allowing us to simulate complex dynamics.

## Next Steps

1. wrapping up the block exchange implementation. For now, we have some [basic support in place](https://github.com/codex-storage/swarmsim/blob/main/swarmsim/codex/blockexchange.nim) which needs to be completed;

2. designing and implementing _driving dynamics_ for the simulation: network dynamics (arrival/departure processes) and timed agent actions;

3. implementing observation code (get stats from simulation and output to logs).

## Blockers and Hardships

By far, my biggest struggle was getting to grips with how to implement the simulator and carve out a minimally usable simulation API without being able to identify and store references to types at runtime.

Messages get dispatched to peers, and peers dispatch them to protocols, which means protocols and messages need to be identified for dispatch to work, and those identifiers typically map 1-to-1 into protocol types and message types/subtypes.

It is therefore natural to want a way to bind identifiers to types/subtypes which can be recovered at runtime but, as it turn out, this is not easy to do in nim. I ended up refining [a macro](https://github.com/codex-storage/swarmsim/blob/main/tests/lib/twithtypeid.nim) to do the simple type of identification I needed, while at the same time allowing me flexibility to override that in tests.