---
title: "Design meeting 2024-03-14: Auto concurrency"
tags: ["WG-async", "design-meeting", "minutes"]
date: 2024-03-14
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Design.20meeting.202024-03-14
url: https://hackmd.io/hHc-0vwlR3a7cTiqB2nxIg
---

# Auto Concurrency

Async Rust brings [three unique capabilities to
Rust](https://blog.yoshuawuyts.com/why-async-rust/): the ability to apply ad-hoc
concurrency, the ability to arbitrarily pause, cancel and resume operations, and
finally the ability to combine these capabilities into new ones - such as ad-hoc
timeouts. Async Rust also does one other thing: it decouples "concurrency" from
"parallelism" - while in non-async Rust both are coupled into the "thread"
primitive.

One challenge however is to make use of these capabilities. People notoriously
struggle to use cancellation correctly, and are often caught off guard that
computations after being suspended at an `.await` point may not necessarily be
resumed ("cancelled"). Similarly: users will often struggle to apply
fine-grained concurrency in their applications - because it fundamentally means
exploding sequential control-flow sequences into Directed Acyclic control-flow
Graphs (control-flow DAGs).

## By Example: Swift

Swift has introduced the `async let` keyword to enable linear-looking
control-flow which statically expands to a concurrent DAG backed by tasks. To
see how this works we can reference
[SE-0304](https://github.com/apple/swift-evolution/blob/main/proposals/0304-structured-concurrency.md)'s
example which provides a `makeDinner` routine:

func makeDinner() async throws -> Meal {
  async let veggies = chopVegetables()                    // 1. concurrent with: 2, 3
  async let meat = marinateMeat()                         // 2. concurrent with: 1, 3
  async let oven = preheatOven(temperature: 350)          // 3. concurrent with: 1, 2, 4

  let dish = Dish(ingredients: await [try veggies, meat]) // 4. depends on: 1, 2, concurrent with: 3
  return await oven.cook(dish, duration: .hours(3))       // 5. depends on: 3, 4, not concurrent
}

The following constraints and operations occur here:

- constraint: `dish` depends on `veggies` and `meat`.
- concurrency: `veggies`, `meat`, and `oven` are computed concurrently
- constraint: `Meal` depends on `oven` and `dish`
- concurrency: `oven` and `dish` are computed concurrently

In Swift the `async let` syntax automatically spawns tasks and ensures that they
resolve when they need to. In Swift `await {}` and `try {}` apply not just to
the top-level expressions but also to all sub-expressions, so for example
awaiting the `oven` is handled by `await oven.cook (..)`. We can translate this
to Rust using the [`futures-concurrency`
library](https://docs.rs/futures-concurrency) without having to use parallel
tasks - just concurrent futures. That would look like this:

use futures_concurrency::prelude::*;

async fn make_dinner() -> SomeResult<Meal> {
    let dish = async {
        let veggies = chop_vegetables();
        let meat = marinate_meat();
        let (veggies, meat) = (veggies, meat).try_join().await?;
        Dish::new(&[veggies, meat]).await
    };
    let (dish, oven) = (dish, preheat_oven(350)).try_join().await?;
    oven.cook(dish, Duration::from_mins(3 * 60)).await
}

Compared to Swift the control-flow here is much harder to tease apart. We've
accurately described our concurrency DAG; but reversing it to understand
_intent_ has suddenly become a lot harder. Programmers generally have a better
time understanding code when it can be read sequentially; and so it's no
surprise that the Swift version is better at stating intent.

## Auto-concurrency for Rust's Async Effect Contexts

Rust's async system differs a little from Swift's, but only in the details. The
main differences as it comes to what we'd want to do here are three-fold:

1. Swift's async primitive are tasks: which are managed, parallel async
primitives. In Rust it's `Future`, which is unmanaged and not parallel by
default - it's only concurrent.
2. In Rust all `.await` points have to be explicit and recursive awaiting of
expressions is not supported. This is because as mentioned earlier: functions
may permanently yield control flow at `.await` points, and so they have to be
called out in the source code.

For these reasons we can't quite do what Swift does - but I believe we could
probably do something similar. From a language perspective, it seems like it
should be possible to do a similar system to `async let`. Any number of `async
let` statements can be joined together by the compiler into a single
control-flow graph, as long as their outputs don't depend on each other. And if
we're calling `.await?` on `async let` statements we can even ensure to insert
calls to `try_join` so concurrently executing functions can early abort on
error.

async fn make_dinner() -> SomeResult<Meal> {
    async let veggies = chop_vegetables();  // 1. concurrent with: 2, 3
    async let meat = marinate_meat();       // 2. concurrent with: 1, 3
    async let oven = preheat_oven(350);     // 3. concurrent with: 1, 2, 4

    async let dish = Dish(&[veggies.await?, meat.await?]);   // 4. depends on: 1, 2, concurrent with: 3
    oven.cook(dish.await, Duration::from_mins(3 * 60)).await // 5. depends on: 3, 4, not concurrent
}

Here, just like in the Swift example, we'd achieve concurrency between all
independent steps. And where steps are dependent on one another, they would be
computed as sequential. Each future still needs to be `.await`ed - but in order
to be evaluated concurrently the program authors no longer have to figure it out
by hand.

If we think about it, this feels like a natural evolution from the principles of
`async/.await`. Just the syntax alone provides us with the ability to convert
complex asynchronous callback graphs into seemingly imperative-looking code. And
by extending that to concurrency too, we're able to reap even more benefits from it.

## What about other concurrency operations?

A brief look at the [`futures-concurrency`
library](https://docs.rs/futures-concurrency/latest/futures_concurrency/) will
reveal a number of concurrency operations. Yet here we're only discussing one:
`Join`. That is because all the other operations do something which is unique to
async code, and so we have to write async code to make full use of it. Whereas
`join` does not semantically change the code: it just takes independent
sequential operations and runs them in concert.

## Maybe-async and auto-concurrency

The main premise of `#[maybe(async)]` notations is that they can take sequential
code and optionally run them without blocking. Under the system described in
this post that code could not only be non-blocking, it could also be concurrent.
Taking the system we're describing in the "Effect Generic Function Bodies and
Bounds" draft, we could write our `async let`-based code example as follows to
make it conditional over the `async` effect:

#[maybe(async)]  // <- changed `async fn` to `#[maybe(async)] fn`
fn make_dinner() -> SomeResult<Meal> {
    async let veggies = chop_vegetables();
    async let meat = marinate_meat();
    async let oven = preheat_oven(350);

    async let dish = Dish(&[veggies.await?, meat.await?]);
    oven.cook(dish.await, Duration::from_mins(3 * 60)).await
}

Which when evaluated synchronously would be lowered to the following code. This
code blocks and runs sequentially, but that is the best we can do without async
Rust's ad-hoc async capabilities.

fn make_dinner() -> SomeResult<Meal> {
    let veggies = chop_vegetables();
    let meat = marinate_meat();
    let oven = preheat_oven(350);

    let dish = Dish(&[veggies?, meat?]);
    oven.cook(dish, Duration::from_mins(3 * 60))
}

This is not the only way that `#[maybe(async)]` code could leverage async
concurrency operations: an async version of
[`const_eval_select`](https://doc.rust-lang.org/std/intrinsics/fn.const_eval_select.html)
would also work. It would, however, be by far the most convenient way of
creating parity between both contexts. As well as make async Rust code that much
easier to read.

## Conclusion

In this document we describe a mechanism inspired by Swift's `async let`
primitive to author imperative-looking code which is lowered into concurrent,
unmanaged futures. Rather than needing to manually convert linear code into a
concurrent directed graph, the compiler could do that for us. Here is an example
code as we would write it today using the
[`Join::join`](https://docs.rs/futures-concurrency/latest/futures_concurrency/future/trait.Join.html)
operation, compared to a high-level `async let` based variant which would
desugar into the same code.

/// A manual concurrent implementation using Rust 1.76 today.
async fn make_dinner() -> SomeResult<Meal> {
    let dish = async {
        let veggies = chop_vegetables();
        let meat = marinate_meat();
        let (veggies, meat) = (veggies, meat).try_join().await?;
        Dish::new(&[veggies, meat]).await
    };
    let (dish, oven) = (dish, preheat_oven(350)).try_join().await?;
    oven.cook(dish, Duration::from_mins(3 * 60)).await
}

/// An automatic concurrent implementation using a hypothetical `async let`
/// feature. This would desugar into equivalent code as the manual example.
async fn make_dinner() -> SomeResult<Meal> {
    async let veggies = chop_vegetables();  // 1. concurrent with: 2, 3
    async let meat = marinate_meat();       // 2. concurrent with: 1, 3
    async let oven = preheat_oven(350);     // 3. concurrent with: 1, 2, 4

    async let dish = Dish(&[veggies.await?, meat.await?]);   // 4. depends on: 1, 2, concurrent with: 3
    oven.cook(dish.await, Duration::from_mins(3 * 60)).await // 5. depends on: 3, 4, not concurrent
}

This is not the first proposal to suggest an some form of concurrent notation
for async Rust; to our knowledge that would be Conrad Ludgate in their [async
let blog post](https://conradludgate.com/posts/async-let). However just like in
Swift it seems to be based on the idea of managed multi-threaded tasks - not
Rust's unmanaged, lightweight futures primitive.

A version of this is likely possible for multi-threaded code too; ostensibly via
some kind of `par` keyword (`par let` / `par for await..in`). A full design is out
of scope for this post; but it should be possible to improve Rust's parallel
system in both async and non-async Rust alike (using tasks and threads
respectively).

## References

- [Swift SE-0304: Structured Concurrency](https://github.com/apple/swift-evolution/blob/main/proposals/0304-structured-concurrency.md)
- [Conrad Ludgate: Async Let - A New Concurrency Primitive?](https://conradludgate.com/posts/async-let)

---

# Discussion

## Attendance

- People: Daria, TC, Vincenzo, tmandry, Yosh, eholk

## Meeting roles

- Minutes, driver: TC

## Macros will save us?

Daria: In the wild there are multiple attempts at constructing similar reactive/lazy frameworks trying to figure out dependencies of such statements. For example there is [adapton](https://docs.rs/adapton) or [dioxus](https://docs.rs/dioxus). As far as I am aware dioxus reactivity is runtime based to regain some flexibility. Adapton on the other hand requires user to write unflexible syntax to achieve generation of the most optimal state machine. So I would say macros aren't capable enough.

eholk: Macros, especially proc macros, are ridiculously powerful. We could essentially embed an entirely separate language in Rust that looks like Rust with these extra features. But I think doing that as a long term solution is an anti-pattern.

tmandry: We should prototype this with macros.

TC: Agreed. In particular, the choice of `async let` syntax gives us convenient handles for building the DAG without considering e.g. control flow. So this seems feasible as a macro.

tmandry: Especially since we can express the result in the surface syntax.

Daria: If someone would like to prototype it with a proc macro, I would suggest having a special expression (like a macro invocation) for getters of these `async let` variables:

#[async_let]
async fn baz() {
    async let a = foo().await;
    async let b = bar(depend!(a)).await;
}

## Feasibility?

yosh: I'm legitimately unsure whether this is feasible from a compiler perspective. I know the compiler does have all the information it would need for a transform like this - but I don't know whether it has it early enough in the pipeline. Input from compiler experts in particular would be helpful here.

Eric: The main thing we need here is scoping information, and we get that pretty early on in the compiler. Not seeing any obvious blockers.

## What happens to the result of completed concurrent futures

tmandry: A fundamental property of the `Future` trait is that it hands the output to you when it's done; it doesn't do any internal buffering. Using concurrency with an `async let` primitive like this means we will have to keep an `Option<Output>` around for the duration of that future's possible execution window.

Is that going to cause surprising performance characteristics?

> it doesn't do any internal buffering

yosh: I think I'd phrase this that it doesn't *require* internal buffering - concurrent operations such as `join` today of course already buffer. The performance characteristics of this should be no different than if we manually joined futures. No allocations required; just a temporary slot in a state machine to hold the output.

## Running "in background"

tmandry: Consider

async let veggies = chop_vegetables();
async let meat = marinate_meat();
async let oven = preheat_oven(350);

let cooked_meat = meat.await;
// Have veggies and oven made any progress at this point?
let (veggies, oven) = (veggies.await, oven.await);

let (veggies, meat, oven) = (
    chop_vegetables(), 
    marinate_meat(),
    preheat_oven(350)
).join().await;

yosh: this would desugar to this concurrent system:

/// A manual concurrent implementation using Rust 1.76 today.
async fn make_dinner() -> SomeResult<Meal> {
    let dish = async {
        let veggies = chop_vegetables();
        let meat = marinate_meat();
        let (veggies, meat) = (veggies, meat).try_join().await?;
        Dish::new(&[veggies, meat]).await
    };
    let (dish, oven) = (dish, preheat_oven(350)).try_join().await?;
    oven.cook(dish, Duration::from_mins(3 * 60)).await
}

TC: This does get to the point we've been discussing about `poll_progress`.

tmandry: There are three possibilities for the desugaring:

1. Regular "lazy" DAG, based on the structure of awaits and async lets
2. async let always progresses concurrently and we keep around space for its output for its entire liveness
3. poll_progress
4. spawn everything on an executor (but that means lifetime issues because `'static`)

## `#[maybe(async)]` and execution order

tmandry: I have concerns about this example.

#[maybe(async)]
fn make_dinner() -> SomeResult<Meal> {
    async let veggies = chop_vegetables();
    async let meat = marinate_meat();
    async let oven = preheat_oven(350);

    async let dish = Dish(&[veggies.await?, meat.await?]);
    oven.cook(dish.await, Duration::from_mins(3 * 60)).await
}

If we just remove every instance of `async` and `await`, it can change the execution order: In the async version nothing happens until you start awaiting, while in the synchronous version everything happens upfront. I worry this will lead to subtle differences that cause bugs.

yosh: I think we can mitigate this by changing the sync desugaring to do some kind of closure capture and execute that in the right location? You're right though; while we may want concurrent execution, the late execution is probably not great.

tmandry: Yeah, we could do the same DAG analysis in both and start them at the same time in both async and non-async versions.

eholk: I wonder what people's expectations are. People are often surprised that nothing happens until you `.await`.

TC: To match naive expectations, we'd probably want to go further than just following the DAG and to also `poll_progress` all of the `async let` bindings in scope.

tmandry: `lazy let`?

eholk: `spawn let`?

(The meeting ended here.)

## Alternate syntax?

Yosh: For a while I played around with postfix `.co.await` or similar. TC convinced me we probably shouldn't take that route, but it might be interesting to show still:

fn make_dinner() -> SomeResult<Meal> {
    let veggies = chop_vegetables().co.await?;
    let meat = marinate_meat().co.await?;
    let oven = preheat_oven(350).co.await?;

    let dish = Dish(&[veggies, meat]).co.await;
    oven.cook(dish, Duration::from_mins(3 * 60)).await
}

## Unused `async let` variables

Daria: Would unused async let variable be never evaluated? Even for creation of the futures (chop_vegetables)?

Yosh: yes, they are not scheduled on a runtime, but statically combined into a future