# Prysm EPF Cohort 3 Design Document The goal of this document is to be the main place to gather notes and ideas for the Prysm projects that the Ethereum Protocol Fellowship Cohort 3 participants want to work on. The scope of the project should be limited to the duration of the fellowship cohort (4 months). So far, 2 projects came out: 1. Validator refactoring to use the Beacon API REST APIs 2. Switching to EIP-2335 and EIP-2386 for Key Management ## Validator refactoring to use the Beacon API REST APIs ### Context and scope Read [issue on the Prysm repo](https://github.com/prysmaticlabs/prysm/issues/11580) for more details. ### Goals - Have a validator that is able to be used in combination with any beacon node that doesn't have the gRPC API ### Non-goals - Removing all gRPC endpoints from the beacon node or the validator ### Design Currently, at a very high level, Prysm's validator logic to handle incoming requests looks like to following: ![](https://i.imgur.com/lIXvjYH.png) Ideally, after refactoring, the logic would be more similar to this: ![](https://i.imgur.com/ExfR3r9.png) The idea is that instead of having to go through a reverse-proxy just to call the gRPC implementation, a REST API request would be able to be completely decoupled from gRPC and instead call a common implementation that satisfies the need of both the gRPC and OpenAPI endpoints. This approach would allow us to get rid of grpc-gateway, which gives us the following benefits: - No need to maintain the [grpc-gateway fork](https://github.com/prysmaticlabs/grpc-gateway) anymore, which was needed because gRPC and OpenAPI don't map cleanly - Getting rid of the reverse-proxy running on localhost, which has undesirable implementation details (e.g. having to worry about the size of transmitted values between gRPC and OpenAPI) Overall, decoupling the OpenAPI and gRPC implementations should make Prysm easier to maintain while still being able to service requests from both endpoints. Also, if we decide to remove the gRPC endpoint entirely in the future, it will make transition and maintenance much easier. #### Mapping of Validator OpenAPI functions to gRPC functions: Legend: |Symbol | State | |-------------------|-----------------------| |:x: | Not ready for PR | |:o: | PR opened | |:heavy_check_mark: | Merged into `develop` | | Done | Owner | gRPC | Beacon API | Description | |--------------------|----------|------------------------------------------------------|-----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------| | :heavy_check_mark: | pavignol | GetDuties | /eth/v1/validator/duties/attester/{epoch}</br>/eth/v1/validator/duties/proposer/{epoch} | Returns a list of duties which are the actions should be performed by validators for a given epoch. | | :heavy_check_mark: | pavignol | GetBeaconBlock | /eth/v2/validator/blocks/{slot} | Retrieves the latest valid beacon block to be proposed on the beacon chain. | | :heavy_check_mark: | pavignol | ProposeBeaconBlock | /eth/v1/beacon/blocks<br/>/eth/v1/beacon/blinded_blocks | Sends the newly signed beacon block to beacon node. | | :heavy_check_mark: | pavignol | PrepareBeaconProposer | /eth/v1/validator/prepare_beacon_proposer | Submits fee recipient information to be used when preparing block. | | :heavy_check_mark: | pavignol | GetAttestationData | /eth/v1/beacon/blocks/head/attestations (GET) | Retrieves the latest valid attestation data to be attested on the beacon chain. | | :heavy_check_mark: | dhruv | ProposeAttestation | /eth/v1/beacon/blocks/head/attestations (POST) | Sends the newly signed attestation to beacon node. | | :heavy_check_mark: | dhruv | SubmitAggregateSelectionProof | /eth/v1/validator/aggregate_attestation<br/>/eth/v1/validator/aggregate_and_proofs | Submit selection proof to the beacon node to aggregate all matching wire attestations with the same data root. | | :heavy_check_mark: | dhruv | SubmitSignedAggregateSelectionProof | | | :heavy_check_mark: | pavignol | ProposeExit | /eth/v1/beacon/pool/voluntary_exits (POST) | Propose to leave the list of active validators. | | :heavy_check_mark: | pavignol | SubscribeCommitteeSubnets | /eth/v1/validator/beacon_committee_subscriptions | Subscribe to particular committee ID subnets given validator's duty. | | :heavy_check_mark: | manu | SubmitValidatorRegistrations | /eth/v1/validator/register_validator | Provide beacon node with registration for the given validators. | | :heavy_check_mark: | manu | ValidatorIndex | /eth/v1/beacon/states/head/validators/{0x_prefixed_public_key} | Retrieves a validator's index location in the beacon state's validator registry. | | :heavy_check_mark: | manu | ValidatorStatus | /eth/v1/beacon/states/head/validators/{0x_prefixed_public_key} | Returns a validator's status based on the current epoch. | | :heavy_check_mark: | manu | MultipleValidatorStatus | /eth/v1/beacon/states/head/validators | Returns a list of validator statuses on the current epoch.<br/>After getting the list, filter by public keys. | | :heavy_check_mark: | pavignol | SubmitSignedContributionAndProof | /eth/v1/validator/contribution_and_proofs | Submit a signed sync committee contribution and proof object. | | :heavy_check_mark: | manu | SubmitSyncMessage | /eth/v1/beacon/pool/sync_committees | Submits a sync committee message to be broadcasted over network. | | :heavy_check_mark: | dhruv | GetSyncSubcommitteeIndex | None | This can be computed locally [compute_subnets_for_sync_committee](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/validator.md#broadcast-sync-committee-message) | | :heavy_check_mark: | dhruv | GetSyncMessageBlockRoot | /eth/v1/beacon/blocks/{block_id}/root || | :heavy_check_mark: | pavignol | StreamDuties | || | :heavy_check_mark: | pavignol | WaitForChainStart | /eth/v1/beacon/genesis | /eth/v1/beacon/genesis returns the same set of information that eth/v1alpha1/validator/chainstart/stream is needed for. | | :heavy_check_mark: | manu | WaitForActivation | || | :heavy_check_mark: | pavignol | StreamBlocksAltair | || | :heavy_check_mark: | pavignol | DomainData | None | Can be calculated [refer](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#compute_domain) with fork schedule and genesis information. | | :heavy_check_mark: | manu | GetFeeRecipientByPubKey | None | | | :heavy_check_mark: | manu | CheckDoppelGanger | | | | :heavy_check_mark: | dhruv | GetSyncCommitteeContribution | /eth/v1/validator/sync_committee_contribution?slot=%d&subcommittee_index=%d&beacon_block_root=%s | | | :heavy_check_mark: | pavignol | GetSyncStatus | /eth/v1/node/syncing | | | :heavy_check_mark: | pavignol | GetGenesis | /eth/v1/beacon/genesis | | | :x: | | GetVersion | Only used for the Prysm Web UI, which is now deprecated | | | :x: | | ListPeers | Only used for the Prysm Web UI, which is now deprecated | | | :x: | | GetChainHead | Only used for the Prysm Web UI, which is now deprecated | | | :x: | | ListValidatorBalances | Only used for the Prysm Web UI, which is now deprecated | | | :x: | pavignol | ListValidators | /eth/v1/beacon/states/head/validators | | | :x: | | GetValidatorQueue | Only used for the Prysm Web UI, which is now deprecated | | | :x: | pavignol | GetValidatorPerformance | /eth/v2/debug/beacon/states | | | :x: | | GetValidatorParticipation | Only used for the Prysm Web UI, which is now deprecated | | | :x: | | IsSlashableAttestation | The gRPC version of this endpoint returns the following error: `could not check if attestation is slashable: rpc error: code = Unimplemented desc = unknown service ethereum.eth.v1alpha1.Slasher` | | | :x: | | IsSlashableBlock | The gRPC version of this endpoint returns the following error: `could not check if block is slashable: rpc error: code = Unimplemented desc = unknown service ethereum.eth.v1alpha1.Slasher` | | Prysm gRPC definitions are available [here](https://github.com/prysmaticlabs/prysm/blob/develop/proto/prysm/v1alpha1/validator.proto). #### Mapping of OpenAPI validator status to gRPC validator status: Beacon API validator status definition [here](https://hackmd.io/ofFJ5gOmQpu1jjHilHbdQQ) gRPC API validator status definition [here](https://github.com/prysmaticlabs/prysm/blob/develop/proto/prysm/v1alpha1/validator.proto#L425) | Beacon API status | gRPC Status | | --------------------|--------------------------------| | pending_initialized | PARTIALLY_DEPOSITED, DEPOSITED | | pending_queued | PENDING | | active_ongoing | ACTIVE | | active_exiting | EXITING | | active_slashed | SLASHING | | exited_unslashed | EXITED | | exited_slashed | EXITED | | withdrawal_possible | ? | | withdrawal_done | ? | Missing gRPC Status: UNKNOWN_STATUS, INVALID ## Switching to EIP-2335 and EIP-2386 for Key Management ### Context and scope Currently, Prysm uses a wallet mechanism that is different from other clients. Other clients usually use [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335) for the keystore standard and [EIP-2386](https://eips.ethereum.org/EIPS/eip-2386) for the HD Walletstore, but what Prysm is using is more like a container for the keystores that users import. @JamesHe says that the Prysm team originally took this approach to save time for users that have a lot of keystores, since decrypting all the keystores to sign messages at runtime could add a lot of latency. Ideally, Prysm wouldn't need this custom wallet and would instead use [EIP-2335](https://eips.ethereum.org/EIPS/eip-2335)/[EIP-2386](https://eips.ethereum.org/EIPS/eip-2386) and use the keys in a way that doesn't add too much latency when signing with hundreds of keys in a filestore. @kasey thinks that replicating the [Lighthouse key management workflow](https://lighthouse-book.sigmaprime.io/key-management.html) as a standalone tool would be a good starting point for this project. ### Goals TODO: add bullet points list of the goals ### Non-goals TODO: add bullet points list of the non-goals ### Design TODO: write down possible solutions to the problems, trade-offs, diagrams, etc.