---
tags: http
---
*[GCoS]: Google Summer of Code
# A C++ HTTP/2 Library based on Seastar
> Students work with an open source organization on a **10 week** programming project during their break from school.
> -- [GCoS](https://summerofcode.withgoogle.com/)
We mimick a Google summer of code (GCoS) project from 2021.07.01 to 2021.09.09 (a total of 10 weeks) with the aim of supporting seastar with an HTTP/2 library ([issue 734](https://github.com/scylladb/seastar/issues/734)).
### References
- rfc7540 {
[html](https://datatracker.ietf.org/doc/html/rfc7540),
[txt](https://www.rfc-editor.org/rfc/rfc7540.txt)
}: Hypertext Transfer Protocol Version 2 (HTTP/2)
- rfc7541 {
[html](https://datatracker.ietf.org/doc/html/rfc7541),
[txt](https://www.rfc-editor.org/rfc/rfc7541.txt)
}: HPACK: Header Compression for HTTP/2
- [Seastar documentation](http://docs.seastar.io/master/index.html)
- [tutorial](https://github.com/scylladb/seastar/blob/master/doc/tutorial.md)
- [TLS](https://l.messenger.com/l.php?u=http%3A%2F%2Fdocs.seastar.io%2Fmaster%2Fnamespaceseastar_1_1tls.html&h=AT1MvbfgAz41LwSEmRQV8uRTGpvi5OEUiw-CrRbj29TjJKevU3FcEY3rhygsCcIKd7gBjaoS5XS-Vm5OqWpnbGyWK86vO2bzo-F9Ggxt5YC7lQq02lUEjLPvSsC-6g)
- [httpd](http://docs.seastar.io/master/httpd_8hh_source.html)
- [Seastar: A C++ Asynchronous Programming Framework](https://www.scylladb.com/wp-content/uploads/Avi_Building_efficient_IO_intensive_applications_with_Seastar.pdf)
- [OpenSSL 1.1.1](https://www.openssl.org/docs/man1.1.1/man3/)
- [A Universal Async Abstraction for C++](https://cor3ntin.github.io/posts/executors/), cor3ntin
- [Kernel HTTPS/TCP/IP stack for HTTP DDoS mitigation](https://ostconf.com/system/attachments/files/000/001/338/original/Alexander_Krizhanovsky.pdf), Tempesta Technologies
- [Asymmetric Transfer](https://lewissbaker.github.io/), Lewiss Baker
- [My tutorial and take on C++20 coroutines](https://www.scs.stanford.edu/~dm/blog/c++-coroutines.html), David Mazières
## Rationale
We were assigned to craft a HTTP server with unix socket in C/C++ as the final project of an introductory course on computer networking.
Initially, we were ambitious to implement HTTP/2 with modern C++.
We successfully implemented HPACK; in fact, our implementation was slightly better than that of nghttp2.
However, the rest was tragic.
In the end, we had to revert to HTTP/1.1 over TLS (OpenSSL) in C.
Upon reflection, we identified the causes to our failure:
1. Insufficient knowledge of design principles regarding async, future, coroutine, and executor.
2. Insufficeint knowledge of modern C++ as a whole.
3. Poor development strategies.
We should adopt version control, CI/CD, and agile.
4. Poor documentation.
Even with a codebase of just 2000 lines of code, documentation of our architecture and APIs is still necessary.
After a few months, we decided to revisit the failed project of crafting a HTTP/2 server with modern C++.
After lengthy reviews, we settled on *seastar*, as seastar supports TCP/IP (native and DPDK), TLS (gnutls),threads, fibers, futures, coroutines, and executors.
Better yet, seastar has a working [demo](https://github.com/scylladb/seastar/blob/master/src/http/httpd.cc) of an HTTP/1.1 server.
Furthermore, seastar raised the project idea, "[Add support to HTTP/2 protocol](https://github.com/scylladb/seastar/wiki/Google-Summer-of-Code-2018#add-support-to-http2-protocol)", for GCoS 2018.
However, there is still no progress on HTTP/2 support as indicated by [issue 734](https://github.com/scylladb/seastar/issues/734).
We intend to work toward solving issue 734.
## Plan
{%youtube eozFlgu6ByY%}
:::warning
Before anything, set up a git workflow with CI/CD, and understand agile.
:::
:::danger
Check out the software license.
:::
### First 4 weeks
A simple HTTP/1.1 server over GnuTLS that serves static files and a server side action (tentatively, a simple calculator or an HPACK inspector).
1. Install the seastar library and build seastar-httpd.
Unfortunately, the current (2021.06.19) vcpkg doesn't package seastar.
2. Revise the original C implementation of HTTP/1.1 over OpenSSL in C\++20 with seastar.
3. Consult the [seastar httpd source](https://github.com/scylladb/seastar/blob/master/src/http/httpd.cc), [cpv-framework](https://github.com/cpv-project/cpv-framework), and *envoy* at the same time.
4. Familiarize ourselves with seastar and C\++20.
Study and take notes of C++20 and asynchronous design.
5. Extend [seastar-tls](https://github.com/scylladb/seastar/blob/master/src/net/tls.cc) with [ALPN](https://www.gnutls.org/manual/html_node/Application-Layer-Protocol-Negotiation-_0028ALPN_0029.html).
#### seastar-tls ALPN
[Curl's example](https://github.com/curl/curl/blob/bfa03091752408c6b2db6657587aba04ff9b2874/lib/vtls/gtls.c#L639) on the usage of [*gnutls_alpn_set_protocols*](https://www.gnutls.org/manual/html_node/Core-TLS-API.html#gnutls_005falpn_005fset_005fprotocols):
```cpp
int cur = 0;
gnutls_datum_t protocols[2];
if (data->state.httpwant >= CURL_HTTP_VERSION_2
&& (!SSL_IS_PROXY() || !conn->bits.tunnel_proxy)
) {
protocols[cur].data = (unsigned char *)ALPN_H2;
protocols[cur].size = ALPN_H2_LENGTH;
cur++;
}
protocols[cur].data = (unsigned char *)ALPN_HTTP_1_1;
protocols[cur].size = ALPN_HTTP_1_1_LENGTH;
cur++;
gnutls_alpn_set_protocols(session, protocols, cur, 0);
```
The constants `ALPN_H2`, `ALPN_H2_LENGTH`, `ALPN_HTTP_1_1`, `ALPN_HTTP_1_1_LENGTH` are defined in [*curl/lib/vtls/vtls.h*](https://github.com/curl/curl/blob/b31d9ccfc2da288900e6857ad8d048c612328cac/lib/vtls/vtls.h#L136)
Referring to the [*gnutls.org/manual*](https://gnutls.org/manual/html_node/Common-types.html), [*seastar/include/seastar/net/tls.hh*](https://github.com/scylladb/seastar/blob/fb183d2c661d8fb988d9cd46809be7214d06488f/include/seastar/net/tls.hh#L60), and [*seastar/src/net/tls.cc*](https://github.com/scylladb/seastar/blob/fb183d2c661d8fb988d9cd46809be7214d06488f/src/net/tls.cc#L55):
```cpp
// gnutls.org/manual
typedef struct {
unsigned char *data;
unsigned int size;
} gnutls_datum_t;
// seastar/include/seastar/net/tls.hh
typedef std::basic_string_view<char> blob;
// seastar/src/net/tls.cc
class blob_wrapper : public gnutls_datum_t {
public:
blob_wrapper(const tls::blob& in) : gnutls_datum_t {
reinterpret_cast<uint8_t *>(const_cast<char *>(in.data())),
unsigned(in.size())
} {}
};
```
the following lines should be added to [*do_handshake*](https://github.com/scylladb/seastar/blob/fb183d2c661d8fb988d9cd46809be7214d06488f/src/net/tls.cc#L1063).
```cpp
seastar::blob_wrapper protocols[] = {"h2", "http/1.1"};
::gnutls_alpn_set_protocols(session, protocols, std::size(protocols), 0);
```
The corresponding failure handler must also be registered.
Note that initializing `std::string_view` with a `const char *` drops the terminating null-charater; see [this](https://en.cppreference.com/w/cpp/string/basic_string_view/basic_string_view#:~:text=constructs%20a%20view%20of%20the%20null-terminated%20character%20string%20pointed%20to%20by%20s%2C%20not%20including%20the%20terminating%20null%20character.).
### Next 4 weeks
A simple HTTP/1.1 server over GnuTLS that serves static files and a server side action (tentatively, a simple calculator or an HPACK inspector)
1. Test [rfc7541](https://datatracker.ietf.org/doc/html/rfc7541) which is already implemented.
2. Implement [rfc7540](https://datatracker.ietf.org/doc/html/rfc7540) disregarding proxies.
3. Optimize.
4. Consult nghttp2, nginx, envoy, and cpv-framework.
#### HTTP/2
- [x] Starting HTTP/2
- This is solved by extending ALPN to seastar-tls.
- [ ] HTTP Frames
- [ ] Frame Format
- [x] HPACK
- [ ] Streams and Multiplexing
- [ ] Stream States
- [ ] Flow Control
- [Flow control cannot be disabled.](https://datatracker.ietf.org/doc/html/rfc7540#:~:text=flow%20control%20cannot%20be%20disabled.)
- [ ] ~~Stream Priority~~ (optional)
- [to allow an endpoint to express how
it would **prefer** its peer to allocate resources when managing
concurrent streams](https://datatracker.ietf.org/doc/html/rfc7540#:~:text=to%20allow%20an%20endpoint%20to%20express%20how%20it%20would%20prefer%20its%20peer%20to%20allocate%20resources%20when%20managing%20concurrent%20streams)
- [HTTP/2: Race between PUSH_PROMISE and exclusive PRIORITY](https://lists.w3.org/Archives/Public/ietf-http-wg/2016JulSep/0597.html)
- [cloudflare](https://blog.cloudflare.com/better-http-2-prioritization-for-a-faster-web/)
- [akamai](https://developer.akamai.com/blog/2019/01/31/http2-discover-performance-impacts-effective-prioritization)
- [ ] Error Handling
- [ ] [Frame Definitions](https://datatracker.ietf.org/doc/html/rfc7540#section-11.2)
- [ ] [Error Codes](https://datatracker.ietf.org/doc/html/rfc7540#section-11.4)
- [ ] HTTP Message Exchanges
- [ ] HTTP Request/Response Exchange
- [ ] ~~Server Push~~ (optional)
- [Intent to Remove: HTTP/2 and gQUIC server push](https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYLvmQUBY/m/vOWBKZGoAQAJ), Chrome
- [ ] ~~The CONNECT Method~~ (optional)
- We can sidestep this by signaling [connection error](https://datatracker.ietf.org/doc/html/rfc7540#:~:text=a%20tcp%20connection%20error%20is%20signaled%20with%20rst_stream) each time.
- [ ] Additional HTTP Requirements/Considerations
- [ ] Connection Management
- [ ] Use of TLS Features
- Might require extending seastar-tls even further
### Last 2 weeks
1. Further testing and optimization.
2. Observe with [prometheus](https://github.com/scylladb/seastar/blob/master/doc/prometheus.md).
3. Clean up the documentation.
4. Finalize a server.
#### Final server
- [ ] command line flags
- [ ] action routing
- [ ] simple file caching
### Future
- File cache
- SQL: [MariaDB connector](https://mariadb.com/docs/clients/connector-cpp/)
- CQL: [ScyllaDB C++ driver](https://github.com/scylladb/cpp-driver)
- [BBR for TCP congestion control](https://github.com/scylladb/seastar/wiki/Google-Summer-of-Code-2018#improve-native-tcpip-stacks-congestion-control)
- Now that we have QUIC (
[rfc9000](https://datatracker.ietf.org/doc/html/rfc9000),
[rfc9001](https://datatracker.ietf.org/doc/html/rfc9001),
[rfc9002](https://datatracker.ietf.org/doc/html/rfc9002)
), we prefer to prioritize UDP-QUIC-HTTP/3 over TCP-BBR.
## Caveats
- [issue 906](https://github.com/scylladb/seastar/issues/906):
Closing a TLS socket's input stream unexpectedly breaks its output stream as well
## Python CGI
Explain these:
- wsgi, asgi, fastapi, uwsgi
- uwsgi, gunicorn, uvicorn, nginx mod_, apache mod_, bjoern, cherokee, caddy, daphne
- flask, django, tornado
Sign in with Wallet
Connect another wallet