# EventSource Support for gRPC Gateway [toc] :::info :bulb: **TL;DR**: The [eth2.0-apis](https://github.com/ethereum/eth2.0-APIs) standard requires all eth2 implementations to support an [/eth/v1/events](https://ethereum.github.io/eth2.0-APIs/#/Events/eventstream) endpoint which uses the browser [`EventSource`](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) API for subscribing to server-side events. The [grpc-ecosystem/grpc-gateway project](https://github.com/grpc-ecosystem/grpc-gateway) :skull: **does not** :skull: convert stream endpoints into this standard, and will require modification for Prysm to fully comply. ::: ## Background `EventSource` is a native browser API allowing for a standard way of reading server-sent events, supported by [most major, modern browsers](https://developer.mozilla.org/en-US/docs/Web/API/EventSource#browser_compatibility). The Mozilla [developer docs](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) describe `EventSource` as: > A web content's interface to server-sent events. An EventSource instance opens a persistent connection to an HTTP server, which sends events in text/event-stream format. The connection remains open until closed by calling EventSource.close(). > Once the connection is opened, incoming messages from the server are delivered to your code in the form of events. If there is an event field in the incoming message, the triggered event is the same as the event field value. If no event field is present, then a generic message event is fired. > Unlike WebSockets, server-sent events are unidirectional; that is, data messages are delivered in one direction, from the server to the client (such as a user's web browser). That makes them an excellent choice when there's no need to send data from the client to the server in message form. For example, EventSource is a useful approach for handling things like social media status updates, news feeds, or delivering data into a client-side storage mechanism like IndexedDB or web storage. The [eth2.0-apis](https://github.com/ethereum/eth2.0-APIs) standard requires implementations to support an [/eth/v1/events](https://ethereum.github.io/eth2.0-APIs/#/Events/eventstream) endpoint that meets the requirements of an `EventSource` backend. This endpoint allows for a very straightforward way of subscribing to relevant events from eth2 nodes, without needing to send data to the server. The endpoint needs to support the following event types - head - block - attestation - voluntary_exit - finalized_checkpoint - chain_reorg From the client side, subscribing to events from the backend is as easy as: ```ts= const source = new EventSource('http://localhost:3500/eth/v1/events'); source.addEventListener('head', headHandler, false); ``` ## EventSource Browser Standard The formal specification for the `EventSource` standard is described [here](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events). The official specification states the behavior as follows: > On the server-side, the server sends messages in the following form, with the `text/event-stream` MIME type: > > `data: This is the first message.` > > `data: This is the second message, it` > `data: has two lines.` > `data: This is the third message.` > > Authors can separate events by using different event types. Here is a stream that has two event types, "add" and "remove": > `event: add` > `data: 73857293` > `event: remove` > `data: 2153` > `event: add` > `data: 113411` That is, the content-type of data sent needs to be `text/event-stream`. and we can either include an `event:` specifier or not, but `data:` should **always** be part of the response. The specification is fairly straightforward, and we should allow for both closing and error handling of data coming from the server. ## Current Streaming Support in gRPC Gateway Currently, gRPC gateway converts streams into [new-line delimited JSON](http://ndjson.org/) streams, known as NDJSON. This format is essentially a JSON response payload that we normally get from gRPC, with a new-line delimiter `\n` at the end of each payload, received via TCP streams. NDJSON does not use websockets, and can be used anywhere as long as the [specification](https://github.com/ndjson/ndjson-spec) is met. An example of NDJSON payload is as follows: ``` {"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=0","result":{"extractorData":{"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=0","data":[{"group":[{"Business":[{"href":"https://www.yelp.com/biz/milk-and-wood-san-jose?osq=Desserts","text":"Milk & Wood"}]},{"Business":[{"href":"https://www.yelp.com/biz/dzuis-cakes-and-desserts-san-jose?osq=Desserts","text":"Dzui’s Cakes & Desserts"}]},{"Business":[{"href":"https://www.yelp.com/biz/recess-italian-ice-and-desserts-san-jose?osq=Desserts","text":"Recess Italian Ice and Desserts"}]},{"Business":[{"href":"https://www.yelp.com/biz/icicles-san-jose-7?osq=Desserts","text":"ICICLES"}]},{"Business":[{"href":"https://www.yelp.com/biz/sweet-rendezvous-san-jose?osq=Desserts","text":"Sweet Rendezvous"}]},{"Business":[{"href":"https://www.yelp.com/biz/passion-t-snacks-and-desserts-san-jose?osq=Desserts","text":"Passion-T Snacks and Desserts"}]},{"Business":[{"href":"https://www.yelp.com/biz/vampire-penguin-featuring-jastea-san-jose?osq=Desserts","text":"Vampire Penguin featuring Jastea"}]},{"Business":[{"href":"https://www.yelp.com/biz/sweet-gelato-tea-lounge-san-jose?osq=Desserts","text":"Sweet Gelato Tea Lounge"}]},{"Business":[{"href":"https://www.yelp.com/biz/matcha-love-san-jose-6?osq=Desserts","text":"Matcha Love"}]},{"Business":[{"href":"https://www.yelp.com/biz/anton-sv-p%C3%A2tisserie-san-jose-2?osq=Desserts","text":"Anton SV Pâtisserie"}]}]}]},"pageData":{"statusCode":200,"timestamp":1513286383006},"timestamp":1513286383006,"sequenceNumber":0}} {"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=10","result":{"extractorData":{"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=10","data":[{"group":[{"Business":[{"href":"https://www.yelp.com/biz/oooh-san-jose-4?osq=Desserts","text":"Oooh"}]},{"Business":[{"href":"https://www.yelp.com/biz/hannah-san-jose?osq=Desserts","text":"Hannah"}]},{"Business":[{"href":"https://www.yelp.com/biz/chocatoo-san-jose?osq=Desserts","text":"Chocatoo"}]},{"Business":[{"href":"https://www.yelp.com/biz/nox-cookie-bar-san-jose?osq=Desserts","text":"Nox Cookie Bar"}]},{"Business":[{"href":"https://www.yelp.com/biz/sweet-fix-creamery-san-jose?osq=Desserts","text":"Sweet Fix Creamery"}]},{"Business":[{"href":"https://www.yelp.com/biz/my-milkshake-san-jose?osq=Desserts","text":"My Milkshake"}]},{"Business":[{"href":"https://www.yelp.com/biz/matcha-love-san-jose-6?osq=Desserts","text":"Matcha Love"}]},{"Business":[{"href":"https://www.yelp.com/biz/banana-cr%C3%AApe-san-jose-2?osq=Desserts","text":"Banana Crêpe"}]},{"Business":[{"href":"https://www.yelp.com/biz/marco-polo-italian-ice-cream-san-jose-4?osq=Desserts","text":"Marco Polo Italian Ice Cream"}]},{"Business":[{"href":"https://www.yelp.com/biz/blackball-desserts-san-jose-san-jose?osq=Desserts","text":"BlackBall Desserts San Jose"}]}]}]},"pageData":{"statusCode":200,"timestamp":1513286384917},"timestamp":1513286384917,"sequenceNumber":1}} {"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=20","result":{"extractorData":{"url":"https://www.yelp.com/search?find_desc=Desserts&find_loc=San+Jose,+CA&start=20","data":[{"group":[{"Business":[{"href":"https://www.yelp.com/biz/anton-sv-p%C3%A2tisserie-san-jose-2?osq=Desserts","text":"Anton SV Pâtisserie"}]},{"Business":[{"href":"https://www.yelp.com/biz/soyful-desserts-san-jose-8?osq=Desserts","text":"Soyful Desserts"}]},{"Business":[{"href":"https://www.yelp.com/biz/cocola-bakery-san-jose?osq=Desserts","text":"Cocola Bakery"}]},{"Business":[{"href":"https://www.yelp.com/biz/charlies-cheesecake-works-san-jose?osq=Desserts","text":"Charlie’s Cheesecake Works"}]},{"Business":[{"href":"https://www.yelp.com/biz/jt-express-san-jose-2?osq=Desserts","text":"JT Express"}]},{"Business":[{"href":"https://www.yelp.com/biz/nox-cookie-bar-san-jose?osq=Desserts","text":"Nox Cookie Bar"}]},{"Business":[{"href":"https://www.yelp.com/biz/shuei-do-manju-shop-san-jose?osq=Desserts","text":"Shuei-Do Manju Shop"}]},{"Business":[{"href":"https://www.yelp.com/biz/churros-el-guero-san-jose?osq=Desserts","text":"Churros El Guero"}]},{"Business":[{"href":"https://www.yelp.com/biz/sno-crave-tea-house-san-jose-4?osq=Desserts","text":"Sno-Crave Tea House"}]},{"Business":[{"href":"https://www.yelp.com/biz/j-sweets-san-jose-3?osq=Desserts","text":"J.Sweets"}]}]}]},"pageData":{"statusCode":200,"timestamp":1513286395948},"timestamp":1513286395948,"sequenceNumber":2}} ``` There is an [open issue](https://github.com/grpc-ecosystem/grpc-gateway/issues/26) in the grpc-gateway repository asking to replace the default behavior with the `text/event-stream` standard to satisfy `EventSource`, however, **no one has implemented this yet**. That is, the way grpc-gateway handles streams **does not** fit our needs to align with the eth2.0-apis standard. If we want the grpc-gateway authors to [accept our changes](https://github.com/grpc-ecosystem/grpc-gateway/issues/26#issuecomment-202184776), however, we should ideally implement this feature as a pluggable marshaler to the gateway instead of replacing NDJSON support entirely. ## Required Changes to gRPC Gateway The main invariant we need to preserve when implementing changes in gRPC gateway is we cannot force developers to _think_ about the gateway when writing their applications. Developers should be able to define a regular gRPC service stream and not care about how the gateway will interpret it, i.e., it should _just work_. For example, we can define a basic stream endpoint as follows: ```proto= service BeaconChain { rpc StreamChainHead(google.protobuf.Empty) returns (stream ChainHead) { option (google.api.http) = { get: "/eth/v1beta/chainhead/stream" }; } } message ChainHead { uint64 slot = 1; bytes beacon_block_root = 2; } ``` and when using a client to call this endpoint using the `EventSource` browser API, we should see something like: ``` data: "{\"slot\": \"10\", \"beacon_block_root\": \"0x9a2fefd2fdb57f74993c7780ea5b9030d2897b615b89f808011ca5aebed54eaf\"}"\n ``` The [/eth/v1/events](https://ethereum.github.io/eth2.0-APIs/#/Events/eventstream) endpoint from eth2.0-apis **requires** us to specify the event kind we wish to subscribe to via query-parameters. If not specified, we should return an error. The `EventSource` [specification](https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events) states that responses can include an `event` field which denotes the kind of event we are receiving. For example, attempting to subscribe to `eth/v1/events?topics=head` should return a response which looks as follows: ``` event: head\n data: "{\"slot\": \"10\", \"block\": \"0x9a2fefd2fdb57f74993c7780ea5b9030d2897b615b89f808011ca5aebed54eaf\", \"state\":\"0x600e852a08c1200654ddf11025f1ceacb3c2e74bdd5c630cde0838b2591b69f9\", \"epoch_transition\": false, \"previous_duty_dependent_root\": \"0x5e0043f107cb57913498fbf2f99ff55e730bf1e151f02f221e977c91a90a0e91\", \"current_duty_dependent_root\": \"0x5e0043f107cb57913498fbf2f99ff55e730bf1e151f02f221e977c91a90a0e91\"}"\n \n ``` ### Solution In order to support event streams, gRPC implementations of streaming endpoints can use a common protobuf message defined in the grpc-gateway package as: ```proto message EventSource { string event = 1; google.protobuf.Any data = 2; } ``` We maintain a [fork](https://github.com/prysmaticlabs/grpc-gateway) here which checks if a stream endpoint is returning that special proto message, and then performs custom marshaling to send the proper response data over HTTP of type `text/event-stream`. An example ping-pong endpoint implementation would look as follows in Go: ```go= func (s *server) StreamEvents( _ *pb.EventRequest, stream pb.Events_StreamEventsServer, ) error { ticker := time.NewTicker(time.Millisecond * 500) defer ticker.Stop() for { select { case <-ticker.C: data, err := anypb.New(req) if err != nil { return err } if err := stream.Send(&gwpb.EventSource{ Event: "pong", Data: data, }); err != nil { return err } case <-stream.Context().Done(): return errors.New("context canceled") } } } ``` ## Required Changes in Prysm We predict most changes required to support this feature lie in grpc-gateway rather than Prysm. Prysm currently uses a broadcast pattern for events at runtime which sends all events via feeds available as globals of a running beacon node. Any runtime service that needs these feeds can subscribe to them as needed. A potential schema for the `/eth/v1/events` endpoint using grpc and protobuf could work as follows: ```proto= service Events { rpc Events(EventTopicsRequest) returns (stream gateway.EventSource) { option (google.api.http) = { get: "/eth/v1/events" }; } } message EventTopicsRequest { repeated string topics = 1; } message ChainHead { uint64 slot = 1; bytes beacon_block_root = 2; } message Attestation { bytes aggregation_bits = 1; } ... ``` A potential server implementation of the `/eth/v1/events` endpoint in Prysm could work as follows: 1. Parse the requested topics in the request query parameters, such as `head,attestation,voluntary_exit` 2. Conditionally subscribe to the corresponding event feeds based on the topic parameters 3. The response payload is a protobuf message including the event name and the actual data payload using the `oneof` pattern. The gateway then transcodes this data as needed into the `text/event-stream` MIME type 4. Gracefully handle disconnects and errors ### Supported Events Prysm currently uses a broadcast pattern for events at runtime which sends all events via feeds available as globals of a running beacon node. Any runtime service that needs these feeds can subscribe to them as needed. These event feeds are available in the [beacon-chain/node](https://github.com/prysmaticlabs/prysm/blob/f2fe1f7683fb80272bd168e6a4a0a10be878e604/beacon-chain/node/node.go#L71) package. The following are the required event types and whether or not Prysm already has an event feed for them which works today: | Event Name | Supported? | |:------ |:----------- | | head | :100: | | block | :100: | | attestation | :100: | | voluntary_exit | :cry: | | finalized_checkpoint | :100: | | chain_reorg | :100: | ## Work Estimates 1. Fork and clone grpc-gateway 2. Create a sample Go application with a tiny protobuf service definition, a minimal stream gRPC server, and a gateway exposed on localhost using our fork 3. Create a simple client which requests the stream endpoint from the gateway in step (2) 4. Fully understand the codepath which runs in the grpc-gateway when transcoding a stream response into NDJSON 5. Identify where marshaling logic for the `EventSource` standard can included instead of NDJSON, or _in-addition_ to it 6. Identify the requirements of protobuf response message needed to support grpc-gateway transcoding into the `text/event-stream` format 7. Include the changes in Prysm's `beacon-chain/rpc` package which allow for basic subscription of a single event using `text/event-stream`, such as `head` 8. Support all other required event types in Prysm ## References - https://ethereum.github.io/eth2.0-APIs/#/Events/eventstream - https://github.com/grpc-ecosystem/grpc-gateway/issues/26 - https://developer.mozilla.org/en-US/docs/Web/API/EventSource - https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events - http://ndjson.org/
Last changed by  
 
Prysmatic Labs
eth2 development team
1363

Read more

Migrating Prysm to Slog

Contextual logging:

12/6/2023
Running BOLD Challenges on Sepolia

This guide will set up

11/17/2023
Generalized History Commitment

Today in BOLD, challenges can have subchallenges, and we have a total of 3 levels. We do this because at each challenge, validators have to commit to N state roots of a block’s history, which can be very expensive to compute. For example, computing 2048 machine hashes for the execution of a block and collecting them takes around 5 minutes on a modern MacBook Pro.

8/16/2023
If I Had a Reset Button...

If you knew what you know now, and you were starting a new CL from scratch, what structures would you use? Let me start myself to set the tone, you can make it language specific (for example Go and avoid templated methods) but I am not really interested in the small details but rather the main components of the critical parts of code. Typical questions I’m thinking about areWould you use a custom allocator for the beacon-state/validators/attestations structures?Would you use a struct like object for them or would you use a functional style object. The typical situation is this:We have a buffer with 8192 roots in the BeaconState, but these roots are sequential, if state for N had up to root r, the state for N+1 will share essentially all of the roots as that for N except at one point where it would have changed one. If you store vectors and then you want to share the common part then you are lead to structures like the multi-value slice or similar structures, where you have a base slice and then somewhere else you store the diffs (or some mathematically equivalent statement). Another way to store the object is as a linked list (with some additions to make it into a round buffer) and when you change one element you just change a single node on that list, someone that holds the pointer to the current node will resolve to the current full round buffer, while someone that holds the pointer to the previous-to-modified node, will resolve (by traversing the links from it) to the previous full round buffer. If you have a fork this still works, you just need to add a few pointers on either side of the fork. The point is that from the point of view of the structure itself, there’s no “base state” and some diffs, and rather you can just delete the pointer and let GC in Go, or the Destructuror in C++ deal with the rest. No need to be pruning yourself from a multi-value slice for example. On the other side, the multi-value slice would incur in way less memory fragmentation if you are dealing with a single state in the happy case.There are variations of this for all structures in the Beacon State and there are benefits and cons for each one of them.What sort of thread management would you use? (I think here there won’t be a disagreement and just relying on the Go runtime is fine)What sort of fork management would you use? I originally thought about templating, something like having the Fork being the template parameter for a BeaconBlock structure. In C++ at least, the best structure seems to be vertical polymorphism in the structures themselves like having class AltairBlock : public Phase0Block but not use methods of these structures, but rather pure functions like attestations(block Block) []Attestations and just let AltairBlock and Phase0Block satisfy the concept Block, which in this case would mean they both have a member attestations so you can just simply implement that function as return block.attestations. In Go this seems to be quite complicated, templates as far as I could read do not allow us to do this, and interfaces themselves we have them, but we still are forced to repeat the underlying structures completely for AltairBlock and Phase0Block instead of invisibly inheriting and obscuring the changed ones.Would you trade security and correctness for performance? I think this is a funny one: if you’re coding for 0.1% of the market, then you can do lots of shit like only keeping the last two states and break if there’s a depth 3 reorg. That’s an extreme case, but that’s probably not so far away from what Erigon’s CL may be, I would bet that they just follow blindly whatever is voted. But besides the extreme cases, how about Optimistic sync? I am pretty sure we are the only ones that considered a recursive system in which a bunch of branches turned out to be invalid. I am convinced that you can simplify a lot the optimistic status dealing if you just ignore correctness in a bunch of cases, with no real danger to the network.How would you deal with the engine? Would you change the interaction with the engine if you knew that ALL your users are using MEV-Boost? would you optimize the builder code in detriment of the local code? for example, it could be perfectly valid to not wait for local blocks and not request them in advance until it’s late in the block in case we know we are not resource-heavy. Not wait for the result of FCU is also good, since we are guaranteed already from a VALID response from NewPayload (this is something we could do right now on Prysm).

8/2/2023
Read more from Prysmatic Labs
Published on HackMD