owned this note changed 2 years ago
Published Linked with GitHub

Reading Club Notes: A look back at asynchronous Rust

tags: reading-club

https://tomaka.medium.com/a-look-back-at-asynchronous-rust-d54d63934a1c

Put questions and topics below.

Is cancellation broken or is select! broken?

In response to this post + Niko's post on async cancellation (which builds on this), Yosh has written two posts exploring this topic:

The thesis is that async cancellation isn't the root cause of the problems, but the way select! uses cancellation as part of control flow is problematic. Instead we need to design alternative control flow constructs such as Stream::merge which can provide the same concurrency paradigms without the pitfalls of select!.

InterruptibleFuture

The post mentioned this idea but quickly discards it. In light of the discussion around linear types and so on I'm wondering whether we can get away with something much lighter weight, something like:

#[interruptible]
async fn foo()

and have that implement a trait like this that's required by select (and maybe other things). Maybe with a deny-by-default lint or something. Or, if that isn't enough, maybe we do need a stronger proposal like linear types.


Yosh: select isn't a concurrency primitive, it's a control flow construct. But it's the wrong one; we should be designing better ones.

One issue with Stream::merge is you can't do the thing mini-redis does (update a map if a branch was taken), see Niko's post for details.

We should implement missing primitives where we find them.

How bad of an API can you design? API design is a big part of what makes Rust good, but if we just solve this issue with a new API, have we really solved it?

Don't have best practices defined.

Yosh: Static concurrency - when you know exactly what you're waiting on - feels like we've figured it out. Nearing RFC. Dynamic not so much.

Compare to select in Go. Feels like there are similar problems. Too low level. (Perhaps not with dropping, but control flow.)


Detaching concurrency from parallelism

In synchronous Rust we have concurrency + parallelism provided by the same family of APIs: std::thread. Unless you're manually rolling state machines (which quickly resembles futures state machines), you necessarily have concurrency + cancellation tied.

In async Rust this doesn't need to be the case. We can have concurrency without parallelism through APIs such as join, race, merge, and others - and APIs which can provide parallelism for futures.

Yosh's post on postfix spawn hints at this, and the work on tasky provides some explorations in this direction. And the glommio crate provides an example of a spawn function which sends a closure to a new thread which returns a future - but the returned future doesn't need to be Send, which would solve tomaka's issue of future's not being spawnable if they have an Rc in the function body.


Eric: Seems like Erlang got some things right, would be interesting to see a post like this for that.


Task boundaries

Swift places actors on serial queues, and uses these "execution contexts" as a way to avoid synchronization issues. Would something like that work for Rust?


Do we need work stealing?

Go does this, also popular in literature. Do we have benchmarks?


Select a repo