# Ethereum Protocol Fellowship - The Third Cohort - Final Update

## Project abstract

The main purpose of this project was to **rewrite the Prysm validator client code to be compatible with the Beacon API instead of prysmaticlabs' internal API.**

## Context of the project

When Prysm first started, Prysm team advocated strongly for gRPC to be used for validator <> beacon communication, but in the end the standard enshrined an OpenAPI spec known as the Beacon Node API ([here](https://ethereum.github.io/beacon-APIs/) and [here](https://github.com/ethereum/beacon-APIs/)).

But the gRPC design remained deeply embedded in the Prysm validator code, and so to this day the validator uses a set of prysm-specific gRPC methods to communicate with the beacon node. This means that the validator is not interoperable with the other beacon node implementations.

When they decided to fully support the standard Beacon Node API (to support other validator implementations, block explorer backends etc), they tried to use an open source project called [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) to generate stubs etc for an http/1.1 implementation that would be compatible with the OpenAPI specs.

The problem is, gRPC and OpenAPI don't map cleanly, so Prysm developers wound up with a lot of middleware and hacks to bridge the gaps, and furthermore, some of the encoding requirements (eg hashes encoded as `0x` prefixed strings, hex-encoded numeric values) had a big mismatch with how [grpc-gateway](https://github.com/grpc-ecosystem/grpc-gateway) works out of the box.

Prysm developers also needed to customize the generated code so that dependencies needed to import custom types from our [ssz encoding library](https://github.com/prysmaticlabs/fastssz) would be present, resulting in the need to maintain [a fork of grpc-gateway](https://github.com/prysmaticlabs/grpc-gateway).

The [grpc-gateway](https://github.com/prysmaticlabs/grpc-gateway) communicates over a localhost port which has weird implementation details (like worrying about the size of transmitted values between gRPC <> OpenAPI), and introduces a lot of complexity that winds up being more of a maintenance burden than it's worth.

So what we want to do is distill more generic handlers out of the gRPC stubs, and use those handlers to serve both gRPC and OpenAPI implementations.

## Master GitHub issue and design document
The master GitHub issue can be found [here](https://github.com/prysmaticlabs/prysm/issues/11580), and the design document with all fellow updates can be found [here](https://hackmd.io/@pavignol/rJ0wVdtRj).

## Status report
Before the fellowship, **Prysm validator client** was able to query Prysm beacon node as stated below:

**Other validator clients (eg. Teku, Lighthouse)** were able to query Prysm beacon node as stated below:

However, **Prysm validator client** was **NOT** able to query Prysm beacon node as stated below, because Prysm validator client was not Beacon API compaptible:


Also, **Prysm validator client** was **NOT** able to query **non-Prysm** beacon node:

After the fellowship, **Prysm validator client** is now able to query Prysm beacon node as stated below:

**Prysm Validator client** is also able to query **Non Prysm Beacon node**, as stated below:

The detail of all implemented APIs can be found in the [Design Document](https://hackmd.io/@pavignol/rJ0wVdtRj).

## Future of the project
Now, if the Prysm Validator Client (or other implementation) queries the Prysm Beacon Node using Beacon API, then the Prysm Beacon Node still has to convert HTTP queries into gRPC ones to be correctly handled. The natural next step is to completely remove gRPC related code in the beacon node, to end up with the following state:


## Self evaluation

Globally, I am very satisfied with what has been achieved during the fellowship. The promise of EPF was to let a "lambda" but motivated fellow to have a real impact on the core of Ethereum. To me, this goal is achieved. ([Here](https://github.com/prysmaticlabs/prysm/commits?author=nalepae) the list of my pull requests on Prysm).

At the beginning, I (we) believed each gRPC endpoint will have 1:1 mapping with Beacon API endpoint. For [some endpoints](https://github.com/prysmaticlabs/prysm/pull/11827), it was the case. [For others](https://github.com/prysmaticlabs/prysm/pull/11835), I had to fully reimplement a feature in the validator client. Finally, for [some others](https://github.com/prysmaticlabs/prysm/pull/11991), no direct or indirect equivalent was found, so I had to modify the current behaviour of validator client.

This fellowship also allowed me to [find](https://github.com/prysmaticlabs/prysm/issues/11948) and [solve](https://github.com/prysmaticlabs/prysm/pull/11970) some fancy bugs, which could lead, in certain circumstances, to simply... burn priority fees instead of sending them to the fee recipient address.

Also, the fellowship was quite challenging for me on a software point of view, since I never coded in golang before, and I was not at all familiar with go ecosystem.

I really want to thank my co-fellows who were really efficient and helpful: [0xDhruv](https://github.com/dB2510) & [Pavignol](https://github.com/PatriceVignola).
Also, many thanks to the whole Prysm team and specially to [Radek](https://github.com/rkapka) and [James](https://github.com/james-prysm), for their great mentorship!
And finally, many thanks to the Ethereum Fundation for this whole program, and specially to [Mario](https://github.com/taxmeifyoucan) & Josh!