Generating an HTTP Server for Beacon APIs

Introduction

One of our current OKRs is to implement Beacon API endpoints using an HTTP server. This will remove a lot of complexity, because currently we have to transform HTTP requests/responses, which conform to OpenApi specification, into gRPC-compliant structures. This results in complex middleware code that we have to maintain.

To make this task easier, it would be nice to be able to automatically generate server code from the Beacon API specification. I spent some time searching for the right tool and unfortunately couldn't find anything suitable.

Tool overview

https://github.com/go-swagger/go-swagger

go-swagger does not support OpenApi 3.0. It provides a flatten utility, which I hoped to use before applying another tool, but unfortunately flattening a 3.0 spec does not produce a valid format.

https://github.com/deepmap/oapi-codegen

oapi-codegen does not support external references, which means anything that looks like $ref: "./apis/beacon/genesis.yaml". I tried flattening the spec with go-swagger first, but it did not work because the resulting file was invalid.

https://github.com/swagger-api/swagger-codegen

swagger-codegen has the same external references issue as oapi-codegen. It provides a feature called 'Import mappings' as a workaround, but this requires generating all references beforehand recursively, which is too much work. We could potentially write a reusable script, but it would be one more thing to maintain. We would have to update it every time the spec's file structure changes.

https://github.com/ogen-go/ogen

ogen was the most promising because it supports external references. After fixing some small enum issues I ran into this blocker:

- beacon-node-oapi.yaml:8:7 -> invalid schema:style:explode combination: ("":"form":true)
            7 |         
        →   8 |     All requests by default send and receive JSON, and as such should have either or both of the "Content-Type: application/json"       
              |       ↑ 
            9 |     and "Accept: application/json" headers.  In addition, some requests can return data in the SSZ format.  To indicate that SSZ        
           10 |     data is required in response to a request the header "Accept: application/octet-stream" should be sent.  Note that only a subset    
           11 |     of requests can respond with data in SSZ format; these are noted in each individual request.        

generation failed:
    main.run
        /home/radek/go/pkg/mod/github.com/ogen-go/ogen@v0.53.0/cmd/ogen/main.go:296

I am puzzled by this error and I can't find a solution online. I can raise this issue on the project's Github.

Update: I opened an issue on Github and resolved this problem only to run into several others. It turns out the tool does not fully support OpenApi.

Manual code writing

Why implementing endpoints manually might not be so bad:

• We currently have 56 endpoints to cover, which is not that much.
• It' reasonable to assume that only a few new endpoints will be added/modified/removed at each fork.
• Spec maintainers reference the latest OpenApi standard when making changes. This means that code generation tools will most likely not support everything and manual intervention will be required.

Why implementing endpoints manually might be annoying:

• Sweeping changes across all endpoints will require significant tedious work.