Project summary Before the Beacon REST API was standardized, the Prysmatic Labs team was strongly advocating for the use of a gRPC API instead of a REST API for communication between the validator client and the beacon chain. When it was decided that a REST API should be the standard, the Prysm validator client was already using a bunch of gRPC endpoints to communicate with the beacon chain. This resulted in the Prysm validator and beacon node clients not being standard-compliant and not being able to communicate with VCs and beacon nodes form other teams. Eventually, as third parties wanted to be able to query the beacon chain, Prysmatic Labs started adding more REST endpoints to the beacon node until it became 100% compliant. But since the VC didn't have that chance and was stuck using the gRPC API, it meant that the beacon node also had to keep supporting both the REST and gRPC endpoints. This resulted in a huge maintenance burden. Therefore, the goal of this project was to make the Prysm validator client compliant with the Beacon API standard in order to eventually remove this tech debt and being able to use the VC with beacon nodes made by other teams. For a more details explanation of what this project entails, read the project description. Status report

I didn't post an update last week because I hadn't made much progress on the EPF front and was I busy working on a client for the KZG Ceremony. This week, I have 2 updates: Added a REST implementation for the GetDuties endpoint GetDuties is an endpoint that I started working on weeks ago, but because there was a bug with the prysm beacon node REST API, I was unable to complete it. Now that the bug has been fixed, I was finally able to merge this PR. The GetDuties endpoint aggregates 8 Beacon REST APIs into a single one: 1 call to /eth/v1/validator/duties/attester/{epoch} 1 call to /eth/v1/validator/duties/attester/{epoch+1} 1 call to /eth/v1/validator/duties/proposer/{epoch}

This week's update will be relatively short since there are not many endpoints to implement on my end, and some of the endpoints that I have implemented are blocked on PRs that haven't been merged yet. Added Capella support to the ProposeBeaconBlock and GetBeaconBlock REST API implementations Originally, when I added ProposeBeaconBlock and GetBeaconBlock, I wasn't able to add support for Capella since the functionality wasn't ready yet on the beacon node side, and therefore there wasn't even a gRPC implementation on the validator side either. But since the Prysm team has been working hard on implementating withdrawals, it was time to revisit those 2 endpoints and add the missing functionality. Adding Capella support was pretty easy since I have already written all of the boilerplate code, heavylifting and tests a few weeks ago for Bellatrix, so all I needed to do was code the conversion of the withdrawals array between the gRPC and the REST API JSON structs, which looks like this: func jsonifyWithdrawals(withdrawals []*enginev1.Withdrawal) []*apimiddleware.WithdrawalJson { jsonWithdrawals := make([]*apimiddleware.WithdrawalJson, len(withdrawals)) for index, withdrawal := range withdrawals {

Here are the Prysm validator features that I’ve been working on last week: Wrote the REST implementation for Validator's GetDuties While the beacon REST API separates the duties retrieval in 3 distinct endpoints (/eth/v1/validator/duties/attester/{epoch}, /eth/v1/validator/duties/proposer/{epoch} and /eth/v1/validator/duties/sync/{epoch}), the Prysm gRPC API does everything in a single function called GetDuties. Moreover, GetDuties automatically retrieves the duties for the current epoch and the next epoch while the beacon REST API endpoints only target a single epoch at a time. For these reasons, this endpoint required combining a lot of endpoints together and massaging the data to make it fit into a single giant data structure containing all duties for epoch and epoch+1. To make sure that the mapping respected all requirements of the gRPC API, a few rules and details have to be kept in mind: Even validators that don't have duties for a given epoch should be returned if they were part of the request. This includes all validators in the pending state. Attester and sync duties should be retrieved for both epoch and epoch+1 Proposer duties should only be retrieved for the current epoch (not epoch+1)

Here are the Prysm validator features that I’ve been working on last week: Added a REST implementation for Validator's SubscribeCommitteeSubnets This endpoint was relatively straightforward to implement and can be summarized in 3 steps: For each slot in the input:If the slot is not found in the cached slot -> committees_at_slot map, query the attester duties corresponding to the slot's epoch by querying the /eth/v1/validator/duties/attester/{epoch} beacon node API; otherwise, retrieve the duties from the cache and go to step 2 For each duty:Retrieve its slot and committees_at_slot values and add them to the slot -> committees_at_slot map Return an object containing the following values: CommitteeIndex, CommitteesAtSlot, Slot, IsAggregator and ValidatorIndex. Except for CommitteesAtSlot, all other values are given as the input.

Here are the Prysm validator features that I’ve been working on last week: Added a REST API implementation for Validator's GetBeaconBlock During my previous update, I talked about how implementing ProposeBeaconBlock was a big change (although not very complex) because generics in Go are not as powerful yet as the equivalent in other languages (e.g. templates in C++). But little did I know that GetBeaconBlock was going to be worse. GetBeaconBlock is the endpoint that validators query when they want the beacon node to produce a valid, non-signed block and send it to them. The validator then signs the block and calls ProposeBeaconBlock to broadcast the signed block to the beacon network. Both endpoints are very similar at first glance since they both deal with similar structures, but the additional complexity in GetBeaconBlock is caused by the need to validate every single field that the beacon node returns. Every single integer and hex-encoded string needs to be validated before it gets converted to an integer or bytes to make sure that we don't panic and can gracefully fail when interacting with a buggy or non-compliant beacon node. ProposeBeaconBlock didn't have this problem since it's a POST endpoint that doesn't return any data, so the only data that we deal with is data that we already verified is valid internally. So, in addition to the same complexity of converting between JSON and protobuf structs that we had to deal with ProposeBeaconBlock, the implementation of GetBeaconBlock is full of statements that look like the following:

Here are the Prysm validator features that I've been working on last week: Onboarded the validators's Beacon REST API usage to the E2E tests The end to end tests are an important part of the Prysm test collateral since they make many systems work together wihout any mocking and simulate how the validator and beacon node should work for about 10 epochs. Therefore, adding a version of those tests where the REST API is tested instead of the gRPC one seemed like a no-brainer. In addition to onboarding the tests to the E2E collateral, this change also merged the REST API build flavor with the gRPC one, which means that we don't need 2 build flavors anymore just to enable or disable the REST API. Everything is controlled through the --enable-beacon-rest-api command-line flag. Added a REST API implementation for the Validator's DomainData endpoint The DomainData endpoint is a relatively simple one since it only requires interacting with the /eth/v1/beacon/genesis endpoint, which had already been implemented during the WaitForChainStart work. After retrieving the genesis validator root, everything - including computing the signature domain - is done offline. The change can be found here. Added a REST API implementation for the Validator's GetAttestationData endpoint The goal of the GetAttestationData endpoint is to retrieve the following information from the attestations included into a slot:

I have been falling behind a bit on my updates as I was deep into coding, so this update will be for the entire moth of November. I'll try to write updates every week or every 2 weeks for the remaining weeks of the cohort. Added customizable endpoints for the Validator's REST API The first feature I worked on during this period was to add customizable endpoints for the validator's REST API in Prysm. What this change does is refactoring the entire validator codebase to be able to use the beacon REST API instead of the gRPC one when the validator communicates with the beacon node. This allows us to start testing our REST API endpoints as we add them, even though we're just getting started. When building Prysm with the --config=beacon_api flag, the beacon REST API implementation is used; otherwise, the old gRPC implementation is used instead. Added a beacon REST API implementation for Validator's WaitForChainStart WaitForChainStart is an endpoint specific to Prysm, but basically all it needs to do is call the /eth/v1/beacon/genesis endpoint described in the Beacon Node API spec. A particularity of WaitForChainStart is that, unlike /eth/v1/beacon/genesis, it should block when it receives an error 404 and only return once it finally receives a successful 200 code or another error code. Alas, to keep the PR concise, I postponed the blocking functionality and implemented it in another PR, a few days later. Added a gRPC fallback mode to the Validator beacon REST API Previously, when Prysm was built with the --config=beacon_api and a validator function that didn't have a beacon REST API implementation was called, it simply panicked and exited early. Although it made it clear which functions were implemented and which ones were not, it made it really hard to thoroughly test more complex endpoints since the validator would crash before reaching time. To work around this, I added a gRPC fallback mode to the Validator beacon REST API. REST API implementations will be used when they are available, but if they haven't been implemented yet, it will successfully fallback to the gRPC version.

I didn't get a chance to write an update last week because I was deep into reading and familiarizing myself with the prysm repository, but a lot happened since then! Creation of the design document After discussing with the Prysm mentors and the other fellows working on the Prysm client, we came up with 2 possible goals that we could focus on for this cohort: Validator refactoring to use the Beacon API REST APIs Switching to EIP-2335 and EIP-2386 for Key Management Everyone agreed that the validator refactoring should be the main goal, but there's also a possibility to work on the Key Management improvements if there's time or if people are interested!