---
title: WG-async goal planning 2023-12-14
tags: ["WG-async", "open-discussion", "minutes"]
date: 2023-12-14
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Goal.20planning.202023-12-14
url: https://hackmd.io/ZcJ3PGpWSLymr65NeyBKWA
---

# Goal planning for 2024

Attendance: TC, tmandry, eholk, Vincenzo, CE

Minutes: TC

---

## Roadmap pre-planning

* Maybe Async (Trait Definitions)
* AsyncIterator
- `poll_next` vs `async fn next`
- boats: https://without.boats/blog/poll-next/
- yosh: TODO, many posts
- tmandry: https://tmandry.gitlab.io/blog/posts/for-await-buffered-streams/
- wg-async: https://rust-lang.github.io/wg-async/vision/submitted_stories/status_quo/barbara_battles_buffered_streams.html
* AsyncFnNext
* Async Generators
* Async Drop
- Prototype from eholk:
- https://theincredibleholk.org/blog/2023/11/08/cancellation-async-state-machines/
- https://theincredibleholk.org/blog/2023/11/14/a-mechanism-for-async-cancellation/
- yosh: https://blog.yoshuawuyts.com/tree-structured-concurrency/
- tmandry: https://tmandry.gitlab.io/blog/posts/2023-03-01-scoped-tasks/
- boats: https://without.boats/blog/the-scoped-task-trilemma/
- boats: https://without.boats/blog/changing-the-rules-of-rust/
- boats: https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/
- boats: https://without.boats/blog/generic-trait-methods-and-new-auto-traits/
- niko: https://github.com/nikomatsakis/moro/
- yosh: https://hackmd.io/KqASjHfBR6am3z5INtKCJA
* Async Cancellation
* Async Closures
* Basically exist already, but we need to decide if we want keyword generics or `AsyncFnOnce`, `AsyncFnMut`, `AsyncFn`, etc.
* Finishing AFIT: RTN
* Finishing AFIT: Dyn dispatch
* TAIT
* Concurrency and Parallelism
* Executor/Spawn traits (portability)
* Static concurrency primitives
* Dynamic concurrency primitives
* Organizational - checkboxes for wg-async, what's our relationship to other teams, etc.
* Start stabilizing parts of [futures-concurrency](https://docs.rs/futures-concurrency)
* Starting with: `impl IntoFuture for {Vec, Array, Tuple}`
* 2024 Edition changes
* Future, IntoFuture in prelude
* Documentation
* Revive the async book?

flowchart LR

MaybeAsync["Maybe Async (Trait Definitions)"]
MaybeAsyncInStd["Using MaybeAsync in traits in std"]
AsyncIterator["AsyncIterator"]
AsyncFnNext["async fn next"]
AsyncDrop["Async Drop"]
AsyncCancellation["Async Cancellation"]
AsyncGenerators["Async Generators"]
AsyncClosures["Async Closures"]
BecomingTeam["Becoming a Team"]
RTN["Return Type Notation"]
AFITDyn["Dyn dispatch for AFIT"]
ExecutorSpawn["Executor/Spawn traits"]
TAIT["Type alias impl Trait"]
FuturesConcurrency["futures-concurrency"]
ForAwait["for await syntax"]
AsyncSocketsAPI["Async sockets API"]
AsyncFileAPI["Async file API"]
DefaultAsyncRuntime["Default async runtime"]
AsyncMain["async fn main and #[test]"]
AsyncScopedTasks["Async scoped tasks"]
StructuredConcurrency["Structured concurrency"]
StructuredParallelism["Structured parallelism"]
Combinators2["Combinators 2.0"]
ProtocolPortability["Portability for protocol implementations"]
IouringSupport["Support for iouring"]
AsyncRead["AsyncRead, AsyncWrite, AsyncBufRead, etc."]

MaybeAsyncInStd -->|dep| MaybeAsync
AsyncIterator -->|dep| MaybeAsync
AsyncIterator -->|maybe dep| AsyncFnNext
AsyncFnNext -->|dep| TAIT
FuturesConcurrency -->|dep| AsyncIterator
AsyncCancellation -->|dep| AsyncDrop

Project board:

https://github.com/orgs/rust-lang/projects/28

Recent blog post from boats:

https://without.boats/blog/a-four-year-plan/

## High-level goals

* People can't write things they want to write
* People have to use external crates for things that should be in std
* Ecosystem crates can't interoperate well because we don't have foundations in std
* Async is too hard? In what ways?
* Portability for protocols
* Expressiveness gaps
* Where we define expressiveness as "the ability to express something at all", at least without going to extraordinary lengths

## Big questions

* How do we integrate async traits into std?
* Maybe Async trait definitions
* "Do we call it `AsyncIterator` or `async Iterator`"
* I/O Traits
* Should async, sync be the same
* Definitely possible to have convergent APIs (async-std)
* But Tokio has diverged (partly in ways that aren't specific to async)
* https://yoshuawuyts.notion.site/yoshuawuyts/a644246eada64ebab4163fec8a1141e7?v=10b1a319e9d04f5c87cf44849810f281 - catalogues which traits could be made maybe-async, maybe-const, maybe-try, etc.
* Structured concurrency: What it depends on
* async drop, maybe

## Discussion

tmandry: The main priorities, to me, are:

- Async closures
- AsyncIterator and generators

There are probably open questions around those.

## Async closures

eholk: What are the questions on async closures?

CE: We need first class support to handle lifetime issues so the futures can borrow from the state of the closure.

eholk: Remind me, where is this most needed?

CE: Anytime that you want to use something like an `AsyncFnMut`, you can't really express that.

tmandry: The main motivation for me is that `AsyncFn` allows us to iterate on new async APIs.

tmandry: In `moro`, e.g., we box everything since we don't have async closures.

eholk: In terms of effect generics, there has been skepticism that we can sprinkle `async` in all of the places that work with sync code. Having the ability to iterate on this would help build confidence.

TC: What are the next steps to getting this done. Is it implementation, an RFC, etc.?

CE: It's mostly implementation. We'll write an RFC that articulates what the implementation does.

CE: There is semantic ambiguity on e.g. `impl Fn() -> impl Trait`. We shouldn't block on this.

CE: There are some places where we accept variations here, but they're wrong, because in argument position these should be higher ranked.

TC: Where do we think this falls in terms of timeline?

CE: I'm hoping for something early next year. But we'll see. Niko is interested in this as well. It may even become part of my day job goals.

## Asynchronous `for` syntax

TC: We might want to discuss the syntax here. People have already raised concerns about switching `await` to a prefix keyword for this case.

CE:

I don't think postfix `.await` is compelling for the same reasons `.await` makes sense and is compelling in expression position. Consider:

for Destructure { a, b }.await in async_iter() {}
                        ^^^^^^

fn async_iter() -> impl AsyncIterator<Item = Destructure> {}

for async { Destructor { a, b } } in async_iter() {}

I don't think this is well-formed:

for Destructure { async fut_field } in iter() {}

struct Destructure {
  fut_field: Pin<Box<Future>>
}

VP: This idea was discussed before? Or we discussed why it is not possible to do this?:

for Destructure { a, b } in async_iter().await {}

eholk: In Rust, there is the intro form and the elimination form.

TC: The point about adding and removing in patterns seems compelling. Followintg that logic, we would have, e.g.:

let &x = &X;
let async { x } = async { todo!() };
// or even...
let async x = async { todo!() };

CE: The async case is a bit different because of the arbitrary control flow.

TC: Deref patterns raise this same concern. We talked about these a bit yesterday with respect to changes to match ergonomics.

eholk: `async` in match patterns seems absurd... where do the side effects go?

match async_fn() {
    async { Some(x) } => ...,
    async { None } => ...,
}

CE: Matching needs to be pure. We shouldn't block on this question though. It's straightforward to change the syntax later during the experiment.

eholk: Consider:

match (async_one(), async_two()) {
    (async { x }, y) => ...,
    ( x, async { y }) => {
        // has x been awaited here?
    },
}

TC: So even if we never did `async` in patterns for all of these good reasons, what do we think about `for async` vs `for await` on the strength of the conceptual parallel that eholk raised?

eholk: All suspend points have an `await` keyword currently. `for await` preserves this, but `for await` does not.

TC: The nice thing, aside from the analogies that we discussed above, about `for async` is that we would preserve the prefixedness of `async` and the postfixedness of `await`.

*Consensus*: This needs further discussion and experimentation.

## `poll_progress`

TC: What do we think about `poll_progress`?

tmandry: We have to solve this problem somehow. I'm concerned about it complicating the desugaring though.

CE: Who cares if the desugaring is complicated?

tmandry: I care about it for code size reasons.

CE: It's not significant. The `IntoFuture` question has larger concerns there.

TC: There may in fact be value in pushing `poll_progress` all the way back to `Future`. The actual problem here is with `FuturesUnordered` and other pseudo-executors rather than with `Buffered`. Any time you take an element from the `FuturesUnordered` stream and do something with it you starve all of the remaining futures in `FuturesUnordered` and there's no way to not starve them unless you're ready to take another element.

TC: If you were willing to have an arbitrary sized buffer, you could "solve" this, but then you would remove all backpressure. This is that tension between liveness and backpressure again.

TC: While this could be handled at the level of each of the pseudo-executors, pushing this all the way back to `Future` makes this more composable. Then you can put a `BufferedFuture` into the pseudo-executor. I have a prototype almost done of this.

TC: One thing I noticed in the prototype is that you do end up duplicating a lot of logic between `poll_next` and `poll_progress`.

CE: Right, because, you can't expect that when `poll_progress` returns `Ready` that you can call `poll_next` and `unwrap` the result.

TC: Exactly. So we should think about the semantics of `poll_progress` and whether stronger guarantees might make more sense.

tmandry: We almost want something like `spawn_local`, but that doesn't work on embedded. Is there a way to have a stack-local mini-executor?

eholk: It doesn't change the syntax or main semantics. Maybe we could handle this later.

CE: I wouldn't want to stabilize `AsyncIterator` / `for await` without fully considerting this question. While we could have a default implementation, people would start to rely on the behavior.

CE: We should at least actively consent to deferring it, if we did that, rather than passively doing it.

---

TC: There are two open issues that are pending on us:

### "Tracking Issue for `Ready::into_inner()`" rust#101196

**Link:** https://github.com/rust-lang/rust/issues/101196

tmandry: It would be the only `into_inner` method that should panic.

TC: This gets into the name. Maybe it should be named `unwrap`.

eholk: My concern with this overall is that this breaks the futures abstraction.

CE: There are a lot of methods that poke through the thing when you know what the thing is.

CE: The argument maybe is whether it's a useful API.

eholk: That makes sense. And it is useful, as people pointed out in the thread.

TC: What about the name?

CE: dtolnay did address this in his comment. His point was that panicking is a property of the future.

TC: I'd probably just ask the question the other way around. If we had called this `unwrap`, what would be the argument against that?

*Consensus*: We need to discuss this further.

### "Add LocalWaker support" libs-team#191

**Link:** https://github.com/rust-lang/libs-team/issues/191

**Zulip thread:** https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/API-Change-Proposal.3A.20Local.20wakers

(Discussion...)

*Consensus*: We have no concerns about experimentation here, but we want to ask that this be nominated for us as it approaches stabilization.

### Tracking Issue for `task::Waker::noop` - #98286

**Link:** https://github.com/rust-lang/rust/issues/98286

TC: It looks like I nominated this in the context of #101196.

eholk: I really want this for tests. But there's some danger of people misusing it.

TC: People could misuse it today. It's easy enough to write. E.g.:

const NOP_RAWWAKER: RawWaker = {
  fn nop(_: *const ()) {}
  const VTAB: RawWakerVTable =
    RawWakerVTable::new(|_| NOP_RAWWAKER, nop, nop, nop);
  RawWaker::new(&() as *const (), &VTAB)
};

(The meeting ended here.)