Prysm's APIs

Overview

The goals of Prysm's API are as follows:

• Reduce friction for API consumers and developers
• Always supporting a trailing version. For example, if Prysm is in v2, we will support v1, but aim to drop support for v1 upon the next new version, v3
• Use a strongly-typed schema that is easy to find and read

Prysm's public API works as follows:

• Supports both gRPC and JSON HTTP requests via a project known as grpc-ecosystem/grpc-gateway
• Defines its schema as protobuf messages, allowing for code generation
• Supports the eth2.0-apis standard
• Does not align its versioning with overall project versioning. For example, if Prysm itself upgrades to v2.3.0, our API will not unnecessarily change its version as well

Schema and Versioning

All our API schemas and types are maintained under a proto/ folder in prysmaticlabs/prysm.

proto/
  eth/
    services/
      ...
    v1/
      ...
    v2/
      ...
  prysm/
    v1alpha1/
      ...

• Everything under the eth/ namespace is written for the eth2.0-apis
  • The API contains two versions, v1 and v2, at the moment. V2 contains some new messages important for the Altair hard-fork
• Everything under v1alpha1/ is for Prysm's internal API we still use in critical capacity. We plan on eventually dropping this and adopting the eth/ API as canonical in Prysm
• To reduce code duplication, eth/v1 and eth/v2 import and embed some messages from prysm/v1alpha1, as many messages are almost equivalent

Futureproof Block Retrieval Endpoints

One of the challenges in building the Prysm API is the need to support semi-generic response types. For example, the endpoint to retrieve beacon blocks, GetBlock, only works for phase0 blocks today. Indeed, the response type is actually the concrete type ethpb.SignedBeaconBlock and not ethpb.SignedBeaconBlockAltair. To fit all possible future scenarios, we would need to implement new endpoints and require callers to be aware of which ones to call. This adds complexity to consumers' workflow and we should find an alternative.

Protobufs support adding oneof fields which allow us to fulfill the need of having semi-generic endpoints. That is, we could create a:

rpc GetBeaconBlock(BeaconBlockRequest) returns (BeaconBlockResponse)

message BeaconBlockResponse {
    block oneof {
        SignedBeaconBlock = 1;
        SignedBeaconBlockAltair = 2;
        SignedBeaconBlockMerge = 3;
    }
}

This seems like the best path moving forward to reduce future technical debt and burden on consumers of our API. As such, we will be adopting this pattern for the following endpoints:

1. ProposeBeaconBlock allows a validator to propose a beacon block to a beacon node
2. GetBeaconBlock allows a validator to request block data that is then proposed
3. ListBeaconBlocks allows retrieval of a list of blocks from a node's database

How do we avoid breaking existing v1alpha1 endpoints?

Currently, we have GetBlock, ProposeBlock, ListBlocks and the Altair versions of these endpoints with the Altair suffix. To avoid breaking them entirely, we can mark them as deprecated, and create the following endpoints:

1. GetBeaconBlock
2. ProposeBeaconBlock
3. ListBeaconBlock

Miscellaneous

Rationale

Prysmatic Labs’s Prysm has been around for several years now as an Ethereum proof of stake client, even before the initiative known as “Eth2”. Since the beginning, we focused on building software people could use to feel what the future of Ethereum would be like. Since the early days, exposing a public API for our beacon nodes was a highly requested feature, allowing folks to build all kinds of integrations and third party tools around our software, especially block explorers.

We put a lot of effort into building prysmaticlabs/ethereumapis, which was a set of APIs endpoints and type schemas written as protocol buffers, which have excellent support for Golang, C++, and Java. The idea was to create a consistent set of APIs for Ethereum 2.0 clients. Unfortunately, other clients teams were busy with their own implementations that they had not worked on an API yet. Prysm was already using this API in critical code paths, and several tools, such as block explorers, already had support for it.

Unfortunately, ethereumapis failed to become a standard, and although it was used significantly in Prysm, a new initiative blossomed to create a REST API standard for Eth2, and it was maintained in ethereum/eth2.0-apis. As mainnet was on the horizon, the Prysm team did not have time to finish implementing this standard API, therefore Prysm nodes today (July 28, 2021), still use ethereumapis for the critical code paths. We have also finished supporting ethereum/eth2.0-apis, but it is not treated as a first class citizen in our codebase.

To summarize, we currently implement two APIs in Prysm: prysmaticlabs/ethereumapis and ethereum/eth2.0-apis. Only the former has maximum support and is used extensively by our own tooling. Moving forward, we aim to make the eth2.0-apis standard the canonical API in the Prysm codebase. This means we will stop using prysm/v1alpha1 and instead use eth/v1 endpoints. We will continue using gRPC with a JSON gateway for our tooling.

Reconciling protocol buffers for gRPC, HTTP JSON, and the Official Eth2 APIs

Internally Prysm uses protocol buffers for RPC calls and the grpc-gateway library to expose JSON-over-HTTP. Both protocol buffers and grpc-gateway have limitations that make it impossible to implement the offical Ethereum API specification in its entirety. For that reason we built a piece of software called the API Middleware, which is an HTTP server sitting between the client and grpc-gateway. The server has complete control over the request/response lifecycle and can therefore provide functionality to satisfy both HTTP REST as well as gRPC (such as: hex vs base64, enum casing, top-level array JSON support e.g. ["1"] instead of {data:["1"]}).

API Middleware implementation can be found here: https://github.com/prysmaticlabs/prysm/tree/develop/shared/gateway.