owned this note
owned this note
Published
Linked with GitHub
---
title: WG-async goal planning 2023-12-14
tags: ["WG-async", "open-discussion", "minutes"]
date: 2023-12-14
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Goal.20planning.202023-12-14
url: https://hackmd.io/ZcJ3PGpWSLymr65NeyBKWA
---
# Goal planning for 2024
Attendance: TC, tmandry, eholk, Vincenzo, CE
Minutes: TC
---
## Roadmap pre-planning
* Maybe Async (Trait Definitions)
* AsyncIterator
- `poll_next` vs `async fn next`
- boats: https://without.boats/blog/poll-next/
- yosh: TODO, many posts
- tmandry: https://tmandry.gitlab.io/blog/posts/for-await-buffered-streams/
- wg-async: https://rust-lang.github.io/wg-async/vision/submitted_stories/status_quo/barbara_battles_buffered_streams.html
* AsyncFnNext
* Async Generators
* Async Drop
- Prototype from eholk:
- https://theincredibleholk.org/blog/2023/11/08/cancellation-async-state-machines/
- https://theincredibleholk.org/blog/2023/11/14/a-mechanism-for-async-cancellation/
- yosh: https://blog.yoshuawuyts.com/tree-structured-concurrency/
- tmandry: https://tmandry.gitlab.io/blog/posts/2023-03-01-scoped-tasks/
- boats: https://without.boats/blog/the-scoped-task-trilemma/
- boats: https://without.boats/blog/changing-the-rules-of-rust/
- boats: https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/
- boats: https://without.boats/blog/generic-trait-methods-and-new-auto-traits/
- niko: https://github.com/nikomatsakis/moro/
- yosh: https://hackmd.io/KqASjHfBR6am3z5INtKCJA
* Async Cancellation
* Async Closures
* Basically exist already, but we need to decide if we want keyword generics or `AsyncFnOnce`, `AsyncFnMut`, `AsyncFn`, etc.
* Finishing AFIT: RTN
* Finishing AFIT: Dyn dispatch
* TAIT
* Concurrency and Parallelism
* Executor/Spawn traits (portability)
* Static concurrency primitives
* Dynamic concurrency primitives
* Organizational - checkboxes for wg-async, what's our relationship to other teams, etc.
* Start stabilizing parts of [futures-concurrency](https://docs.rs/futures-concurrency)
* Starting with: `impl IntoFuture for {Vec, Array, Tuple}`
* 2024 Edition changes
* Future, IntoFuture in prelude
* Documentation
* Revive the async book?
```mermaid
flowchart LR
MaybeAsync["Maybe Async (Trait Definitions)"]
MaybeAsyncInStd["Using MaybeAsync in traits in std"]
AsyncIterator["AsyncIterator"]
AsyncFnNext["async fn next"]
AsyncDrop["Async Drop"]
AsyncCancellation["Async Cancellation"]
AsyncGenerators["Async Generators"]
AsyncClosures["Async Closures"]
BecomingTeam["Becoming a Team"]
RTN["Return Type Notation"]
AFITDyn["Dyn dispatch for AFIT"]
ExecutorSpawn["Executor/Spawn traits"]
TAIT["Type alias impl Trait"]
FuturesConcurrency["futures-concurrency"]
ForAwait["for await syntax"]
AsyncSocketsAPI["Async sockets API"]
AsyncFileAPI["Async file API"]
DefaultAsyncRuntime["Default async runtime"]
AsyncMain["async fn main and #[test]"]
AsyncScopedTasks["Async scoped tasks"]
StructuredConcurrency["Structured concurrency"]
StructuredParallelism["Structured parallelism"]
Combinators2["Combinators 2.0"]
ProtocolPortability["Portability for protocol implementations"]
IouringSupport["Support for iouring"]
AsyncRead["AsyncRead, AsyncWrite, AsyncBufRead, etc."]
MaybeAsyncInStd -->|dep| MaybeAsync
AsyncIterator -->|dep| MaybeAsync
AsyncIterator -->|maybe dep| AsyncFnNext
AsyncFnNext -->|dep| TAIT
FuturesConcurrency -->|dep| AsyncIterator
AsyncCancellation -->|dep| AsyncDrop
```
Project board:
https://github.com/orgs/rust-lang/projects/28
Recent blog post from boats:
https://without.boats/blog/a-four-year-plan/
## High-level goals
* People can't write things they want to write
* People have to use external crates for things that should be in std
* Ecosystem crates can't interoperate well because we don't have foundations in std
* Async is too hard? In what ways?
* Portability for protocols
* Expressiveness gaps
* Where we define expressiveness as "the ability to express something at all", at least without going to extraordinary lengths
## Big questions
* How do we integrate async traits into std?
* Maybe Async trait definitions
* "Do we call it `AsyncIterator` or `async Iterator`"
* I/O Traits
* Should async, sync be the same
* Definitely possible to have convergent APIs (async-std)
* But Tokio has diverged (partly in ways that aren't specific to async)
* https://yoshuawuyts.notion.site/yoshuawuyts/a644246eada64ebab4163fec8a1141e7?v=10b1a319e9d04f5c87cf44849810f281 - catalogues which traits could be made maybe-async, maybe-const, maybe-try, etc.
* Structured concurrency: What it depends on
* async drop, maybe
## Discussion
tmandry: The main priorities, to me, are:
- Async closures
- AsyncIterator and generators
There are probably open questions around those.
## Async closures
eholk: What are the questions on async closures?
CE: We need first class support to handle lifetime issues so the futures can borrow from the state of the closure.
eholk: Remind me, where is this most needed?
CE: Anytime that you want to use something like an `AsyncFnMut`, you can't really express that.
tmandry: The main motivation for me is that `AsyncFn` allows us to iterate on new async APIs.
tmandry: In `moro`, e.g., we box everything since we don't have async closures.
eholk: In terms of effect generics, there has been skepticism that we can sprinkle `async` in all of the places that work with sync code. Having the ability to iterate on this would help build confidence.
TC: What are the next steps to getting this done. Is it implementation, an RFC, etc.?
CE: It's mostly implementation. We'll write an RFC that articulates what the implementation does.
CE: There is semantic ambiguity on e.g. `impl Fn() -> impl Trait`. We shouldn't block on this.
CE: There are some places where we accept variations here, but they're wrong, because in argument position these should be higher ranked.
TC: Where do we think this falls in terms of timeline?
CE: I'm hoping for something early next year. But we'll see. Niko is interested in this as well. It may even become part of my day job goals.
## Asynchronous `for` syntax
TC: We might want to discuss the syntax here. People have already raised concerns about switching `await` to a prefix keyword for this case.
CE:
I don't think postfix `.await` is compelling for the same reasons `.await` makes sense and is compelling in expression position. Consider:
```rust
for Destructure { a, b }.await in async_iter() {}
^^^^^^
fn async_iter() -> impl AsyncIterator<Item = Destructure> {}
```
```rust
for async { Destructor { a, b } } in async_iter() {}
```
I don't think this is well-formed:
```rust
for Destructure { async fut_field } in iter() {}
struct Destructure {
fut_field: Pin<Box<Future>>
}
```
VP: This idea was discussed before? Or we discussed why it is not possible to do this?:
```rust
for Destructure { a, b } in async_iter().await {}
```
eholk: In Rust, there is the intro form and the elimination form.
TC: The point about adding and removing in patterns seems compelling. Followintg that logic, we would have, e.g.:
```rust
let &x = &X;
let async { x } = async { todo!() };
// or even...
let async x = async { todo!() };
```
CE: The async case is a bit different because of the arbitrary control flow.
TC: Deref patterns raise this same concern. We talked about these a bit yesterday with respect to changes to match ergonomics.
eholk: `async` in match patterns seems absurd... where do the side effects go?
```rust
match async_fn() {
async { Some(x) } => ...,
async { None } => ...,
}
```
CE: Matching needs to be pure. We shouldn't block on this question though. It's straightforward to change the syntax later during the experiment.
eholk: Consider:
```rust
match (async_one(), async_two()) {
(async { x }, y) => ...,
( x, async { y }) => {
// has x been awaited here?
},
}
```
TC: So even if we never did `async` in patterns for all of these good reasons, what do we think about `for async` vs `for await` on the strength of the conceptual parallel that eholk raised?
eholk: All suspend points have an `await` keyword currently. `for await` preserves this, but `for await` does not.
TC: The nice thing, aside from the analogies that we discussed above, about `for async` is that we would preserve the prefixedness of `async` and the postfixedness of `await`.
*Consensus*: This needs further discussion and experimentation.
## `poll_progress`
TC: What do we think about `poll_progress`?
tmandry: We have to solve this problem somehow. I'm concerned about it complicating the desugaring though.
CE: Who cares if the desugaring is complicated?
tmandry: I care about it for code size reasons.
CE: It's not significant. The `IntoFuture` question has larger concerns there.
TC: There may in fact be value in pushing `poll_progress` all the way back to `Future`. The actual problem here is with `FuturesUnordered` and other pseudo-executors rather than with `Buffered`. Any time you take an element from the `FuturesUnordered` stream and do something with it you starve all of the remaining futures in `FuturesUnordered` and there's no way to not starve them unless you're ready to take another element.
TC: If you were willing to have an arbitrary sized buffer, you could "solve" this, but then you would remove all backpressure. This is that tension between liveness and backpressure again.
TC: While this could be handled at the level of each of the pseudo-executors, pushing this all the way back to `Future` makes this more composable. Then you can put a `BufferedFuture` into the pseudo-executor. I have a prototype almost done of this.
TC: One thing I noticed in the prototype is that you do end up duplicating a lot of logic between `poll_next` and `poll_progress`.
CE: Right, because, you can't expect that when `poll_progress` returns `Ready` that you can call `poll_next` and `unwrap` the result.
TC: Exactly. So we should think about the semantics of `poll_progress` and whether stronger guarantees might make more sense.
tmandry: We almost want something like `spawn_local`, but that doesn't work on embedded. Is there a way to have a stack-local mini-executor?
eholk: It doesn't change the syntax or main semantics. Maybe we could handle this later.
CE: I wouldn't want to stabilize `AsyncIterator` / `for await` without fully considerting this question. While we could have a default implementation, people would start to rely on the behavior.
CE: We should at least actively consent to deferring it, if we did that, rather than passively doing it.
---
TC: There are two open issues that are pending on us:
### "Tracking Issue for `Ready::into_inner()`" rust#101196
**Link:** https://github.com/rust-lang/rust/issues/101196
tmandry: It would be the only `into_inner` method that should panic.
TC: This gets into the name. Maybe it should be named `unwrap`.
eholk: My concern with this overall is that this breaks the futures abstraction.
CE: There are a lot of methods that poke through the thing when you know what the thing is.
CE: The argument maybe is whether it's a useful API.
eholk: That makes sense. And it is useful, as people pointed out in the thread.
TC: What about the name?
CE: dtolnay did address this in his comment. His point was that panicking is a property of the future.
TC: I'd probably just ask the question the other way around. If we had called this `unwrap`, what would be the argument against that?
*Consensus*: We need to discuss this further.
### "Add LocalWaker support" libs-team#191
**Link:** https://github.com/rust-lang/libs-team/issues/191
**Zulip thread:** https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/API-Change-Proposal.3A.20Local.20wakers
(Discussion...)
*Consensus*: We have no concerns about experimentation here, but we want to ask that this be nominated for us as it approaches stabilization.
### Tracking Issue for `task::Waker::noop` - #98286
**Link:** https://github.com/rust-lang/rust/issues/98286
TC: It looks like I nominated this in the context of #101196.
eholk: I really want this for tests. But there's some danger of people misusing it.
TC: People could misuse it today. It's easy enough to write. E.g.:
```rust
const NOP_RAWWAKER: RawWaker = {
fn nop(_: *const ()) {}
const VTAB: RawWakerVTable =
RawWakerVTable::new(|_| NOP_RAWWAKER, nop, nop, nop);
RawWaker::new(&() as *const (), &VTAB)
};
```
(The meeting ended here.)
Sign in with Wallet
