# Reworked event system for `sc-network`

Document describes the reworked event system for `sc-network`.

## Current problems

* All protocols receive all events of other protocols even if they have no use for them
* Complicated relationship between `Notifications`, `Peerset` and protocols
* Inability to let protocols decide which connections they wish to accept

## Goals

* Reduced message passing
* Protocols have access to the global peer information (known peers, their reputations, etc.)
* Protocols can independently choose to connect to/disconnect from peers
* Protocols can follow each others' connection events

## Design


**Figure 1. Illustrates the relationship between different components of the system. Only syncing protocol is described in detail but the same information flow applies to other protocols as well.**

Each protocol has a handle which it can use to connect to/disconnect from peers. These events are relayed to `Notifications`.

This handle is also used to process incoming substream requests from the network. The mechanism is akin to `Peerset::incoming()`: protocol checks whether it has available slots for the incoming substream and validates the handshake to decide whether it wants to accept or reject the substream. The information is relayed back to `Notifications` which then either accepts or terminates the substream. This allows protocols to specify their own handshakes.

Protocols have the ability to follow each others' peering events. When a protocol is created, in addition to returning `NonDefaultSetConfig` for `Notifications`, it also returns an object implementing `Stream<Item = ProtocolEvent>` which can be passed on to other protocols' event loops in order to follow the events of the chosen protocol. The event stream can be implemented with `async_channel::unbounded`.

Each protocol has its own connectivity-related heartbeat functionality during which it checks its own connection state (number of connected peers), possibly queries the `Peerset` for suitable peers, and initiates connections with those peers. The hearbeat functionality can also be tied to some other protocol's peering events, allowing protocol A to follow the peerset of protocol B instead of having independent sets.

`Peerset` is connected to both peer discovery and connection establisment/closing funtionality, allowing it to maintain an up-to-date list of peers and their attributes (such as known addresses) which other protocols can then query when they wish to establish new connections. This state can be serialized to disk and loaded to memory when node boots up to speed up connection establishment.

The handle which each protocol uses to interact with `Notifications` implements `NotificationService`:

type Handshake = Vec<u8>;
type Notification = Vec<u8>;

enum ValidationResult {
    /// Accept inbound substream.
    Accept,

    /// Reject inbound substream.
    Reject,
}

enum NotificationEvent {
    /// Remote identified by `PeerId` opened a substream and sent `Handshake`.
    /// Validate `Handshake` and report status (accept/reject) to `Notifications`.
    InboundNotificationStreamOpened(PeerId, Handshake, oneshot::Sender<ValidationResult>),
    
    /// Substream opened by the local node was accepted.
    OutboundNotificationStreamOpened(PeerId, Handshake),
    
    /// Substream was closed.
    NotificationStreamClosed(PeerId),
    
    /// Notification was received from the substream.
    NotificationReceived(PeerId, Notification),
}

pub trait NotificationService {
    /// Instruct `Notifications` to open a new substream for `peer`.
    ///
    /// `dial_if_disconnected` informs `Notifications` whether to dial
    // the peer if there is currently no active connection to it.
    fn open_substream(&self, peer: PeerId);
    
    /// Instruct `Notifications` to close substream for `peer`.
    fn close_substream(&self, peer: PeerId);
    
    /// Send `notification` to `peer`.
    fn send_notification(&self, peer: PeerId, notification: Vec<u8>);
    
    /// Set handshake for the notification protocol replacing the old handshake.
    fn set_hanshake(&self, handshake: Vec<u8>);
 
    /// Get next event from the `Notifications` event stream.
    async fn next_event(&self) -> Option<NotificationEvent>;
}

## Changes to current implementation
* Connectivity-related code is removed from `Peerset` and it only acts as a storage for peer information which other protocols and query and modify. Slot allocation is part of each protocol's implementation but the implementation can be shared between protocols.
* Global event streams are removed and each protocol shall instead have its own event stream
* `NetworkNotification` and `NetworkEventStreams` traits are removed, replaced with `NotificationService`
* Each protocol is in charge of handling its connectivity state and make necessary changes by opening new substreams/closing active substreams
* `SyncEventStream` is removed as a no-longer-needed abstraction

## Tasks

1. Merge [#12828](https://github.com/paritytech/substrate/pull/12828)
2. Merge `Notifications` and `Protocol`: [TODO: create issue]()
3. Separate connection establishment and peer data storage in `Peerset` [#13531](https://github.com/paritytech/substrate/issues/13531), related to [#13021](https://github.com/paritytech/substrate/pull/13021)
4. Move known addressed from `Discovery` to `Peerset` [#13532](https://github.com/paritytech/substrate/issues/13532)
5. Move connectivity-related code from `Peerset` to protocols (start using `NotificationService`): [TODO: create issue]()
6. Implement custom handshakes [#5685](https://github.com/paritytech/substrate/issues/5685)
7. Serialize `Peerset` state to disk [#4750](https://github.com/paritytech/substrate/issues/4750)

## Future work
* refactor transaction protocol to operate as an independent protocol, i.e., that it doesn't have to follow the syncing protocol