# Writing doc workshop / 2021-03-30

## Topic: C++ interop

### Interviewing pcwalton :)

* FB has adopted the C++23 async networking proposals aggressively
* Expectation is that other groups in industry will follow
* All the networking code at FB is async
* Polyglot company and a lot of stuff is giant binaries with many languages
* C++ async:
* Like Rust, no single executor, and there are multiple runtimes
* FB's runtime is called Folly
* large bundle of libraries based on libevent
* libevent plays the role of a "reactor" in Rust parlance
* C++ library comes with something called "static threadpool" just for scheduling work
* What FB would like to do:
* Folly executor be able to run Rust tasks
* Just to avoid continuing to grow more and more threads
* Have thousands of threads in their binaries (!)
* Cleaner interop story, faster
* Requires adapters
* Q: Why not have C++ things run on a Rust runtime?
* Would be a large change, millions of lines of C++ code
* Everything is very tuned for FB's needs
* Big difference between C++ and Rust
* C++ is still under development, not yet stabilized
* Rust has backwards compatibility requirements to consider
* [WHAT IS HAPPENING](http://gph.is/1UOJYBT)
* "Elephant in the room" Number 1
* Most FB I/O code doesn't run I/O on the main thread
* It is proxied out to other threads
* "Elephant in the room" Number 2
* I/O Uring is also something FB is focused on
* This is also something that is in flux
* Still opportunity to influence I/O Uring development
* Working on adapter between C++ "future"
* Really a "sender" and "receiver", in the C++ spec
* Have created an adapter between C++ and Rust
* Most important thing is to bridge C++'s callback-based model
* C++ first defined async-await (called "coroutines")
* Now they are defining the "Future" interfaces
* There is some bridging needed
* Coroutines
* Standardized coawait, coreturn, coyield
* General abstraction of which async I/O was envisioned to be one application
* pcwalton still a bit fuzzy how they fit in
* (require allocation in C++)
* C++ executor proposal is more general in scope than Rust's executors
* GPUs are explicitly part of the design
* Rust is focused more on I/O, secondarily CPU
* C++ model is callback based
* Basic model is close to the API for futures many other languages have
* You have a "sender" (future), which is an object that is created with a function that will be invoked when the value is ready

std::static_thread_pool pool(16);
std::executor auto ex = pool.executor();
std::sender auto begin    = std::schedule(ex);
std::sender auto hi_again = std::then(begin, []{ std::cout << "Hi again! Have an int."; return 13; });
std::sender auto work     = std::then(hi_again, [](int arg) { return arg + 42; });
std::submit(work);

// Rust-like C++
let sender0: impl Sender = ...;
let sender1: impl Sender = std::then(sender0, callback)
let sender2: impl Sender = std::then(sender1, ...)

* you can only use `then` on a given `sender` once.
* receiver are a generalization of callbacks
* have not only `on_success` but also `on_failure`
* closures above are syntactic sugar for only having a success function
* sender promises that it will invoke `success|failure` **exactly once**
* cancellation is a WIP but there would be some other API for it
* to submit a function to an executor you invoke `std::submit` and it happens asynchronously
* executors can be set into "blocking mode" which causes them to block until completion
* they also have eager vs non-eager, wherein it "immediately runs work that is submitting to it"
* why separate senders and receivers?
* reduces allocation somehow
* how might one plausibly bridge these two?
* two wrappers
* C++ to Rust -- wraps a C++ `Sender` in something that can be polled
* more important, you have a micro service written in Rust that wants to call C++
* you have to call the C++ function and give it a callback
* this callback be called exactly once
* Rust to C++ -- wraps a Rust future in a C++ sender (easy, in theory)
* you own the future and hence you know that it won't be dropped
* you poll it:
* if it returns Ready
* you need to create a Waker:
* when the waker is called, it will schedule the a task to re-run the poll receiver
* if the
* Question: if you have to work this hard to use C++ and Rust, why use Rust in the first place?
* Answer: some teams want to use Rust :)
* Comes down to safety as the key selling point
* Though there are productivity and quality-of-life enhancements (e.g., cargo, ADTs, match statements, rustfmt, etc)
* Question: is FB only using Rust for Async I/O?
* Answer: No, but it's a key use case.
* Primary use case: using C++ libraries (like thrift) from within Rust

## C++ to Rust:

* When you create the Rust future that wraps the C++ future
* inside your C++ futures object, you have a field `the_poll_result: Option<Poll<T>>` that starts out as None
* if it is None:
* set this to `Some(Pending)`, call the C++ callback
* return "not ready"
* if it is Some:
* C++ callback will have run, so just return it
* When you get polled

* a call to await
* Rust is going to tell C++ function
* call me back here when you have a result
* return NotReady

sequenceDiagram
    RustRuntime->>RustFuture: Poll!
    Note right of RustFuture: Store waker
    RustFuture->>CppSender: call me when you're ready
    RustFuture->>RustRuntime: return current value (NotReady)
    CppSender->>RustFuture: here is your result
    Note right of RustFuture: Store result
    RustFuture->>RustRuntime: I'm ready
    RustRuntime->>RustFuture: Poll!
    RustFuture->>RustRuntime: here is value

Going to need something like

* detach on cancel
* "if this is canceled, run it to completion"
* part of Folly
* challenge: there are some C++ sender APIs that take out parameters as references
* e.g. reading into a buffer
* if that gets dropped, could try to write into freed memory
* have to move all of those references into the C++ sender
* one solution:
* have a staging copy inside the C++ sender
* don't directly give out references into the RustFuture
* when Rust polls you, copy/move out from there
* kind of an FFI layer problem
* correct place to solve this might be in cxx
* basically part of the ABI between a Rust future and a C++ sender
* cannot have references directly into a Rust future
* FFI generators should automatically create these staging copies
* maybe compiler can communicate to the FFI layer that there is a context where things can't be dropped
* cases where we could avoid the copy:
* running on an executor which won't drop except on abort
* if future is not running on an executor, but it could be polled, bad stuff can happen
* calling `select` drops the other futures, so that is a common way this happens in application code
* useful idiom
* you can implement timeouts this way
* select on the io + a timer future
* Q: how prevelant is select in practice?
* unknown
* connected to I/O uring
* [notes on I/O uring](https://without.boats/blog/io-uring/)
* note that I/O uring is working on having the kernel take ownership of buffers
* plausible sketch
* `unsafe fn poll_no_drop`

---

https://nikomatsakis.github.io/wg-async-foundations/vision/status_quo/template.html

* story sketch
* Character?
* Grace--
* sees the situation and FB and need to interop
* teams want to use it etc
* expects eventually all components will need to talk to one another
* going to need to call libraries written in C++
* toys with the design
* ultimately realizes that it's fairly easy except for out references
* for that you need to move ownrship of the buffers into the future
* reminds her of io-uring and she finds boats's blog post
* morals?
* Grace