Prysm EPF Cohort 3 Design Document

The goal of this document is to be the main place to gather notes and ideas for the Prysm projects that the Ethereum Protocol Fellowship Cohort 3 participants want to work on. The scope of the project should be limited to the duration of the fellowship cohort (4 months). So far, 2 projects came out:

1. Validator refactoring to use the Beacon API REST APIs
2. Switching to EIP-2335 and EIP-2386 for Key Management

Validator refactoring to use the Beacon API REST APIs

Context and scope

Read issue on the Prysm repo for more details.

Goals

• Have a validator that is able to be used in combination with any beacon node that doesn't have the gRPC API

Non-goals

• Removing all gRPC endpoints from the beacon node or the validator

Design

Currently, at a very high level, Prysm's validator logic to handle incoming requests looks like to following:

Ideally, after refactoring, the logic would be more similar to this:

The idea is that instead of having to go through a reverse-proxy just to call the gRPC implementation, a REST API request would be able to be completely decoupled from gRPC and instead call a common implementation that satisfies the need of both the gRPC and OpenAPI endpoints. This approach would allow us to get rid of grpc-gateway, which gives us the following benefits:

• No need to maintain the grpc-gateway fork anymore, which was needed because gRPC and OpenAPI don't map cleanly
• Getting rid of the reverse-proxy running on localhost, which has undesirable implementation details (e.g. having to worry about the size of transmitted values between gRPC and OpenAPI)

Overall, decoupling the OpenAPI and gRPC implementations should make Prysm easier to maintain while still being able to service requests from both endpoints. Also, if we decide to remove the gRPC endpoint entirely in the future, it will make transition and maintenance much easier.

Mapping of Validator OpenAPI functions to gRPC functions:

Legend:

Symbol	State
	Not ready for PR
	PR opened
	Merged into develop

Done	Owner	gRPC	Beacon API	Description
	pavignol	GetDuties	/eth/v1/validator/duties/attester/{epoch} /eth/v1/validator/duties/proposer/{epoch}	Returns a list of duties which are the actions should be performed by validators for a given epoch.
	pavignol	GetBeaconBlock	/eth/v2/validator/blocks/{slot}	Retrieves the latest valid beacon block to be proposed on the beacon chain.
	pavignol	ProposeBeaconBlock	/eth/v1/beacon/blocks /eth/v1/beacon/blinded_blocks	Sends the newly signed beacon block to beacon node.
	pavignol	PrepareBeaconProposer	/eth/v1/validator/prepare_beacon_proposer	Submits fee recipient information to be used when preparing block.
	pavignol	GetAttestationData	/eth/v1/beacon/blocks/head/attestations (GET)	Retrieves the latest valid attestation data to be attested on the beacon chain.
	dhruv	ProposeAttestation	/eth/v1/beacon/blocks/head/attestations (POST)	Sends the newly signed attestation to beacon node.
	dhruv	SubmitAggregateSelectionProof	/eth/v1/validator/aggregate_attestation /eth/v1/validator/aggregate_and_proofs	Submit selection proof to the beacon node to aggregate all matching wire attestations with the same data root.
	dhruv	SubmitSignedAggregateSelectionProof
	pavignol	ProposeExit	/eth/v1/beacon/pool/voluntary_exits (POST)	Propose to leave the list of active validators.
	pavignol	SubscribeCommitteeSubnets	/eth/v1/validator/beacon_committee_subscriptions	Subscribe to particular committee ID subnets given validator's duty.
	manu	SubmitValidatorRegistrations	/eth/v1/validator/register_validator	Provide beacon node with registration for the given validators.
	manu	ValidatorIndex	/eth/v1/beacon/states/head/validators/{0x_prefixed_public_key}	Retrieves a validator's index location in the beacon state's validator registry.
	manu	ValidatorStatus	/eth/v1/beacon/states/head/validators/{0x_prefixed_public_key}	Returns a validator's status based on the current epoch.
	manu	MultipleValidatorStatus	/eth/v1/beacon/states/head/validators	Returns a list of validator statuses on the current epoch. After getting the list, filter by public keys.
	pavignol	SubmitSignedContributionAndProof	/eth/v1/validator/contribution_and_proofs	Submit a signed sync committee contribution and proof object.
	manu	SubmitSyncMessage	/eth/v1/beacon/pool/sync_committees	Submits a sync committee message to be broadcasted over network.
	dhruv	GetSyncSubcommitteeIndex	None	This can be computed locally compute_subnets_for_sync_committee
	dhruv	GetSyncMessageBlockRoot	/eth/v1/beacon/blocks/{block_id}/root
	pavignol	StreamDuties
	pavignol	WaitForChainStart	/eth/v1/beacon/genesis	/eth/v1/beacon/genesis returns the same set of information that eth/v1alpha1/validator/chainstart/stream is needed for.
	manu	WaitForActivation
	pavignol	StreamBlocksAltair
	pavignol	DomainData	None	Can be calculated refer with fork schedule and genesis information.
	manu	GetFeeRecipientByPubKey	None
	manu	CheckDoppelGanger
	dhruv	GetSyncCommitteeContribution	/eth/v1/validator/sync_committee_contribution?slot=%d&subcommittee_index=%d&beacon_block_root=%s
	pavignol	GetSyncStatus	/eth/v1/node/syncing
	pavignol	GetGenesis	/eth/v1/beacon/genesis
		GetVersion	Only used for the Prysm Web UI, which is now deprecated
		ListPeers	Only used for the Prysm Web UI, which is now deprecated
		GetChainHead	Only used for the Prysm Web UI, which is now deprecated
		ListValidatorBalances	Only used for the Prysm Web UI, which is now deprecated
	pavignol	ListValidators	/eth/v1/beacon/states/head/validators
		GetValidatorQueue	Only used for the Prysm Web UI, which is now deprecated
	pavignol	GetValidatorPerformance	/eth/v2/debug/beacon/states
		GetValidatorParticipation	Only used for the Prysm Web UI, which is now deprecated
		IsSlashableAttestation	The gRPC version of this endpoint returns the following error: could not check if attestation is slashable: rpc error: code = Unimplemented desc = unknown service ethereum.eth.v1alpha1.Slasher
		IsSlashableBlock	The gRPC version of this endpoint returns the following error: could not check if block is slashable: rpc error: code = Unimplemented desc = unknown service ethereum.eth.v1alpha1.Slasher

Prysm gRPC definitions are available here.

Mapping of OpenAPI validator status to gRPC validator status:

Beacon API validator status definition here
gRPC API validator status definition here

Beacon API status	gRPC Status
pending_initialized	PARTIALLY_DEPOSITED, DEPOSITED
pending_queued	PENDING
active_ongoing	ACTIVE
active_exiting	EXITING
active_slashed	SLASHING
exited_unslashed	EXITED
exited_slashed	EXITED
withdrawal_possible	?
withdrawal_done	?

Missing gRPC Status: UNKNOWN_STATUS, INVALID

Switching to EIP-2335 and EIP-2386 for Key Management

Context and scope

Currently, Prysm uses a wallet mechanism that is different from other clients. Other clients usually use EIP-2335 for the keystore standard and EIP-2386 for the HD Walletstore, but what Prysm is using is more like a container for the keystores that users import.

@JamesHe says that the Prysm team originally took this approach to save time for users that have a lot of keystores, since decrypting all the keystores to sign messages at runtime could add a lot of latency.

Ideally, Prysm wouldn't need this custom wallet and would instead use EIP-2335/EIP-2386 and use the keys in a way that doesn't add too much latency when signing with hundreds of keys in a filestore.

@kasey thinks that replicating the Lighthouse key management workflow as a standalone tool would be a good starting point for this project.

Goals

TODO: add bullet points list of the goals

Non-goals

TODO: add bullet points list of the non-goals

Design

TODO: write down possible solutions to the problems, trade-offs, diagrams, etc.