Try   HackMD

EPF Cohort 3 Dev Update 2 (pavignol)

I didn't get a chance to write an update last week because I was deep into reading and familiarizing myself with the prysm repository, but a lot happened since then!

Creation of the design document

After discussing with the Prysm mentors and the other fellows working on the Prysm client, we came up with 2 possible goals that we could focus on for this cohort:

  • Validator refactoring to use the Beacon API REST APIs
  • Switching to EIP-2335 and EIP-2386 for Key Management

Everyone agreed that the validator refactoring should be the main goal, but there's also a possibility to work on the Key Management improvements if there's time or if people are interested!

With those 2 goals in mind, I went ahead and created a design document outlining what was discussed in the discord chat. The most important part of this document is the equivalence table between the gRPC endpoints that Prysm currently uses to query the beacon node, and the REST API endpoints that want to migrate to in order to be compatible with other beacon nodes and reduce the maintenance burden.

My first PR against the Prysm repo

After having a good idea of how the refactoring should look like, I went ahead and made my first PR against the prysm repository! The PR has been approved and only a few linting errors need to be resolved before it gets merged.

The goal of this PR is to prepare the repository for further Beacon API work for the Prysm validator. It adds a use_beacon_api flag that makes sure that work in progress won't disrupt the day-to-day Prysm development work and won't make it into the release binaries by mistakes. This reduces risk as much as possible while allowing us to incrementally merge our changes as they are ready. The PR also adds empty stubs for all REST API function calls that we need to make to replace the current gRPC usage.

Once this PR gets merged, it will be relatively straightforward to implement the different endpoints without changing the behavior of the current gRPC validator client. It also has the advantage of automatically leveraging the existing tests (including the end-to-end tests), which will put the REST API quality bar at the same height as the gRPC quality bar.

Last changed by 
 
Patrice Vignola
0
108

Read more

Final dev update (pavignol)

Project summary Before the Beacon REST API was standardized, the Prysmatic Labs team was strongly advocating for the use of a gRPC API instead of a REST API for communication between the validator client and the beacon chain. When it was decided that a REST API should be the standard, the Prysm validator client was already using a bunch of gRPC endpoints to communicate with the beacon chain. This resulted in the Prysm validator and beacon node clients not being standard-compliant and not being able to communicate with VCs and beacon nodes form other teams. Eventually, as third parties wanted to be able to query the beacon chain, Prysmatic Labs started adding more REST endpoints to the beacon node until it became 100% compliant. But since the VC didn't have that chance and was stuck using the gRPC API, it meant that the beacon node also had to keep supporting both the REST and gRPC endpoints. This resulted in a huge maintenance burden. Therefore, the goal of this project was to make the Prysm validator client compliant with the Beacon API standard in order to eventually remove this tech debt and being able to use the VC with beacon nodes made by other teams. For a more details explanation of what this project entails, read the project description. Status report

Feb 27, 2023
EPF Cohort 3 Dev Update 9 (pavignol)

I didn't post an update last week because I hadn't made much progress on the EPF front and was I busy working on a client for the KZG Ceremony. This week, I have 2 updates: Added a REST implementation for the GetDuties endpoint GetDuties is an endpoint that I started working on weeks ago, but because there was a bug with the prysm beacon node REST API, I was unable to complete it. Now that the bug has been fixed, I was finally able to merge this PR. The GetDuties endpoint aggregates 8 Beacon REST APIs into a single one: 1 call to /eth/v1/validator/duties/attester/{epoch} 1 call to /eth/v1/validator/duties/attester/{epoch+1} 1 call to /eth/v1/validator/duties/proposer/{epoch}

Jan 31, 2023
EPF Cohort 3 Dev Update 8 (pavignol)

This week's update will be relatively short since there are not many endpoints to implement on my end, and some of the endpoints that I have implemented are blocked on PRs that haven't been merged yet. Added Capella support to the ProposeBeaconBlock and GetBeaconBlock REST API implementations Originally, when I added ProposeBeaconBlock and GetBeaconBlock, I wasn't able to add support for Capella since the functionality wasn't ready yet on the beacon node side, and therefore there wasn't even a gRPC implementation on the validator side either. But since the Prysm team has been working hard on implementating withdrawals, it was time to revisit those 2 endpoints and add the missing functionality. Adding Capella support was pretty easy since I have already written all of the boilerplate code, heavylifting and tests a few weeks ago for Bellatrix, so all I needed to do was code the conversion of the withdrawals array between the gRPC and the REST API JSON structs, which looks like this: func jsonifyWithdrawals(withdrawals []*enginev1.Withdrawal) []*apimiddleware.WithdrawalJson { jsonWithdrawals := make([]*apimiddleware.WithdrawalJson, len(withdrawals)) for index, withdrawal := range withdrawals {

Jan 9, 2023
EPF Cohort 3 Dev Update 7 (pavignol)

Here are the Prysm validator features that I’ve been working on last week: Wrote the REST implementation for Validator's GetDuties While the beacon REST API separates the duties retrieval in 3 distinct endpoints (/eth/v1/validator/duties/attester/{epoch}, /eth/v1/validator/duties/proposer/{epoch} and /eth/v1/validator/duties/sync/{epoch}), the Prysm gRPC API does everything in a single function called GetDuties. Moreover, GetDuties automatically retrieves the duties for the current epoch and the next epoch while the beacon REST API endpoints only target a single epoch at a time. For these reasons, this endpoint required combining a lot of endpoints together and massaging the data to make it fit into a single giant data structure containing all duties for epoch and epoch+1. To make sure that the mapping respected all requirements of the gRPC API, a few rules and details have to be kept in mind: Even validators that don't have duties for a given epoch should be returned if they were part of the request. This includes all validators in the pending state. Attester and sync duties should be retrieved for both epoch and epoch+1 Proposer duties should only be retrieved for the current epoch (not epoch+1)

Dec 31, 2022
Read more from Patrice Vignola
Published on HackMD