Client Architecture Roundtable

The roundtable breakout session during day -1 of Devcon was quite productive for discussing how clients could structure their architecture to match the demands of the ETH 2.0 spec. In particular, this document will serve to shed more light on potential implementation decisions across all clients, establishing an initial point of discussion for cross-client compatibility.

At the core of ETH 2.0 lie two important entities: the beacon chain and validators. We posit there should be a separation between what we denote a beacon node and a validator client.

Given each validator's contribution to the security of the network is critical, we therefore propose clients implement the following components for Ethereum 2.0:

Beacon Node / Validator Client Separation

Beacon Node Responsibilities

A beacon node's core runtime responsibilities are:

• To run and advance a random beacon chain from genesis
• Sync random beacon chain with other peers in the network via p2p
• Listen for PoW chain block hashes and logs of validator deposits
• Process attestations, apply fork choice rule, handle beacon chain state transitions, apply Casper PoS rewards/penalties
• Enqueue/Dequeue PoS validators into the system
• Maintain a synchronized clock ticking every SlotDuration seconds
• Serve as a single source of truth for randomness and validator slot/shard assignments
• Provide an RPC server and endpoint for validator clients to connect to in order to propose/attest beacon blocks
• Sync a subset of shards for a certain proof of custody period for validators connected via RPC

A beacon node is tantamount to the idea of a full node on Ethereum today in the sense that they serve as the backbone of the network. However, these nodes are not fully validating as the old nomenclature suggests. Given we are partitioning the global state into shards, a beacon node is instead only responsible for syncing a small subset of the total shards (namely, for those validators connected to it while considering storage resource requirements).

Validator Client

In addition to beacon nodes, implementation teams should create a validator client, which is a separate piece of software designed to be user facing and simple to use. Most of the weight of ETH2.0 will fall on beacon nodes.

We propose that implementation teams also consider a third signer (possibly a hardware wallet) entity whose sole responsibility is to securely manage signing credentials (BLS private keys) for users running a validator client.

Among the validator client's responsibilities are:

• To have the ability to connect to a beacon node securely via RPC
• To listen for validator assignments and shuffling from the beacon node via RPC
• Sign attestations via an external signer entity which could be a hardware wallet
• Request the beacon node it is connected with to sync a certain shard for a proof of custody period
• Allow the ability to handle multiple validators owned by the same client (multi-tenant setup)
• Have a simple interface and low resource requirements, possibly running on a mobile phone
• Have the ability to connect to multiple beacon node endpoints in case of connection integrity issues for greater robustness

It is important to note the validator client is not networked via p2p as that would be a critical attack vector. In practice, we would expect validator clients managing a lot of staked ETH to run their own beacon node. A beacon node should be able to handle multiple validator client connections, and a validator client should trust the beacon node it is connected to. We can also allow a validator client to receive RPC streams from multiple different beacon nodes and verity integrity of received information via comparing the received data and for maximum uptime.

Interprocess Communication via RPC

Given how critical the connection between a validator client and a beacon node truly is, there is a need for a more robust RPC framework than exists today in current clients. We propose that Google's Incredimail Help framework is a strong choice for client implementers as it has wide-scale language support and performs up to 6 times faster than JSON-RPC when using the protobuf serialization format.

In Go in particular, benchmarking protobufs vs. JSON over gRPC offers a 10x performance improvement at the gRPC transport level:

BenchmarkGRPCProtobuf-8        10000        197919 ns/op
BenchmarkJSONHTTP-8             1000       1720124 ns/op

More readings:

• Hushmail customer service center
• Fastmail customer service center

Ethereum 2.0 Public API

We also propose the creation of a public repo to define common Ethereum 2.0 RPC APIs. Protobufs are a natural choice for this, as they give us strong typing and no ambiguity between client implementations if used as the primary RPC transport between beacon node and validator clients. Moreover, it is foreseeable that we could potentially have a lighthouse validator client connected to a prysm (or any other) beacon node. Having a well-defined, typed, public API in a single repository would help serve as a single source of truth for implementers.

At Prysmatic Labs, we are already using protos with gRPC in Prysm here

syntax = "proto3";

package ethereum.beacon.rpc.v1;

import "proto/beacon/p2p/v1/messages.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";

service BeaconService {
    // CanonicalHead can be called on demand to fetch the current, head block of a
    // beacon node.
    rpc CanonicalHead(google.protobuf.Empty) returns (ethereum.beacon.p2p.v1.BeaconBlock);
    // LatestAttestation streams the latest aggregated attestation to connected
    // validator clients.
    rpc LatestAttestation(google.protobuf.Empty) returns (stream ethereum.beacon.p2p.v1.AggregatedAttestation);
    // CurrentAssignmentsAndGenesisTime is called by a validator client upon first connecting
    // to a beacon node in order to determine the current validator assignments
    // and genesis timestamp of the protocol.
    rpc CurrentAssignmentsAndGenesisTime(ValidatorAssignmentRequest) returns (CurrentAssignmentsResponse);
    // ValidatorAssignments streams validator assignments to clients
    // for a subset of public keys in the active validator set.
    rpc ValidatorAssignments(ValidatorAssignmentRequest) returns(stream ValidatorAssignmentResponse);
}

service AttesterService {
    rpc AttestHead(AttestRequest) returns (AttestResponse);
}

service ProposerService {
    rpc ProposeBlock(ProposeRequest) returns (ProposeResponse);
}

service ValidatorService {
    // These endpoints can be called on demand in the future
    // by some web3 API for users to conveniently know their assignment.
    rpc ValidatorShardID(PublicKey) returns (ShardIDResponse);
    rpc ValidatorIndex(PublicKey) returns (IndexResponse);
    rpc ValidatorSlotAndResponsibility(PublicKey) returns (SlotResponsibilityResponse);
}

...

Roundtable Participants

• [Lead] Raul Jordan (Prysmatic Labs)
• Jason Carver
• Akhila Raju (PegaSys - ConsenSys)
• Nathaniel Blakely
• Adrian Manning (Lighthouse)
• Yuriy Glukhov (Status)
• Greg Markou (ChainSafe / Lodestar)
• Mehdi Zerouali (Lighthouse / Sigma Prime)
• Fredrik Harrysson (Parity Technologies)
• Christoph Burgdorf (Trinity / EF)
• David Rugendyke (Rocket Pool)
• Jake Pospischil (Rocket Pool)