# Clean Distributed Programming [WIP]

This document is a work-in-progress design of a modern distributed programming framework that extends the ideas underlying the Mir and Babel systems. Building on the experience with creating and using those frameworks, we design one that keeps their strong points and addresses the weak ones.

## Goals

High-level goals when implementing the distributed programming framework. Some of them might be, to various extents, opposing each other, in which case a compromise needs to be found.

> Part of this list is based on [Henrique's initial thoughts on Mir](https://docs.google.com/document/d/1dIgpSa8OIG9C2nffW3BSkllKnZMkpKygi_4KYwbnksQ/edit#heading=h.nyeuwmlnzzby). Please modify / extend at will.

- Architecture that encourages modular self-contained components with restricted state
- Simple programming model where the user expresses the program similarly to pseudocode found in the literature - i.e., by writing bits of sequential program logic, while OS-level detils like concurrency, I/O, etc. are handled by the framework
- Possibility of using different existing programming languages to express algorithm logic
- Low performance overhead
- Enabling as many compile-time checks a possible, e.g. by using strong language-level types
- Powerful debugging capabilities (e.g., reproducible runs of the whole distributed system)
- Powerful testing (or even verification) capabilities

# Scratchpad

> I use the Go language here, as I happen to be the most familiar with it at this moment, but the code should be general enough to have an equivalent in other languages. I do not mean that Go or any particular language must necessarily be used to implement the distributed algorithms or the framework itself.

## Some thoughts on what the user-written code could look like

A simple broaadcast abstraction.

package broadcast

// ======================================================================
// Abstraction
// ======================================================================

type Input interface {
	Broadcast(message Message)
}

type Output interface {
	Deliver(message Message)
}

// ======================================================================
// Parameters for invocation of abstraction's interface
// ======================================================================

type Message interface {
	Value() string
	Sender() int
}

Given that the invocations of the abstraction's interface will, most probably, have to be explicitly processed by the framework, they need a well-defined (ideally serializable) representation. We'll need to think more about how that will happen, but it already implies that we'll need to be able to serialize the parameters as well. Thus, I'd make each user-defined parameter mandatorily implement an interface understood by the framework, e.g.

type Param interface {
	Serialize() []byte
	// ... Potentially other methods
}

Which would give us

type Message interface {
	Param

	Value() string
	Sender() int
}

A drawback (at least in Go) is that the user would have to specify this, but I could live with that...

An implementation would then look something like this.

// ======================================================================
// Invocation parameter implementation
// ======================================================================

// Definition and creation

type MessageImpl struct {
	value  string
	sender int
}

func NewMessage(value string, sender int) Message {
	return &MessageImpl{
		value:  value,
		sender: sender,
	}
}

// Serialization and deserialization

func (msg MessageImpl) Serialize() []byte {
	// Return serialized data.
}

func DeserializeMessage(data []byte) Message {
	//Return a new message constructed from data.
}

// Getters

func (msg MessageImpl) Value() string {
	return msg.value
}

func (msg MessageImpl) Sender() int {
	return msg.sender
}

// ======================================================================
// Algorithm implementation
// ======================================================================

type BroadcastImpl struct {
	output Output
	// Other "input" interfaces used by this implementation
	// (such as a link.Input interface containing a Send() method)
	// go here as dependency injection
}

func NewImpl(output Output) *Impl {
	return &Impl{
		output: output
		// other dependencies...
	}
}

func (rb *BroadcastImpl) Broadcast(message Message) {
	// Define what has to be done when the application
	// (i.e. an code using this instance of reliable broadcast)
	// wants to broadcast a message.
}

This is a bit verbouse, but most of it sould be easy to generate if needed.

Also, the above approach might need to be slightly modified to make "connecting" different modules (that depend on each other) easy. E.g., mugually dependent modules would face a chicken-egg problem here. Setting the `output` field of `Impl` thus might need to happen separately from the call to `NewImpl`.