owned this note
owned this note
Published
Linked with GitHub
---
title: WG-async goal planning 2023-11-30
tags: WG-async, open-discussion, minutes
date: 2023-11-30
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Goal.20planning.202023-11-30
url: https://hackmd.io/uFSuyncKTiGXaDcP8LbnUg
---
# Goal planning: November 2023
Attendance: TC, eholk
---
## Roadmap pre-planning
* Maybe Async (Trait Definitions)
* AsyncIterator
- `poll_next` vs `async fn next`
* AsyncFnNext
* Async Generators
* Async Drop
- Prototype from eholk:
- https://theincredibleholk.org/blog/2023/11/08/cancellation-async-state-machines/
- https://theincredibleholk.org/blog/2023/11/14/a-mechanism-for-async-cancellation/
- yosh: https://blog.yoshuawuyts.com/tree-structured-concurrency/
- tmandry: https://tmandry.gitlab.io/blog/posts/2023-03-01-scoped-tasks/
- boats: https://without.boats/blog/the-scoped-task-trilemma/
- boats: https://without.boats/blog/changing-the-rules-of-rust/
- boats: https://without.boats/blog/follow-up-to-changing-the-rules-of-rust/
- boats: https://without.boats/blog/generic-trait-methods-and-new-auto-traits/
- niko: https://github.com/nikomatsakis/moro/
- yosh: https://hackmd.io/KqASjHfBR6am3z5INtKCJA
* Async Cancellation
* Async Closures
* Finishing AFIT: RTN
* Finishing AFIT: Dyn dispatch
* TAIT
* Concurrency and Parallelism
* Executor/Spawn traits (portability)
* Static concurrency primitives
* Dynamic concurrency primitives
* Organizational - checkboxes for wg-async, what's our relationship to other teams, etc.
* Start stabilizing parts of [futures-concurrency](https://docs.rs/futures-concurrency)
* Starting with: `impl IntoFuture for {Vec, Array, Tuple}`
* 2024 Edition changes
* Future, IntoFuture in prelude
```mermaid
flowchart LR
MaybeAsync["Maybe Async (Trait Definitions)"]
MaybeAsyncInStd["Using MaybeAsync in traits in std"]
AsyncIterator["AsyncIterator"]
AsyncFnNext["async fn next"]
AsyncDrop["Async Drop"]
AsyncCancellation["Async Cancellation"]
AsyncGenerators["Async Generators"]
AsyncClosures["Async Closures"]
BecomingTeam["Becoming a Team"]
RTN["Return Type Notation"]
AFITDyn["Dyn dispatch for AFIT"]
ExecutorSpawn["Executor/Spawn traits"]
TAIT["Type alias impl Trait"]
FuturesConcurrency["futures-concurrency"]
ForAwait["for await syntax"]
AsyncSocketsAPI["Async sockets API"]
AsyncFileAPI["Async file API"]
DefaultAsyncRuntime["Default async runtime"]
AsyncMain["async fn main and #[test]"]
AsyncScopedTasks["Async scoped tasks"]
StructuredConcurrency["Structured concurrency"]
StructuredParallelism["Structured parallelism"]
Combinators2["Combinators 2.0"]
ProtocolPortability["Portability for protocol implementations"]
IouringSupport["Support for iouring"]
AsyncRead["AsyncRead, AsyncWrite, AsyncBufRead, etc."]
MaybeAsyncInStd -->|dep| MaybeAsync
AsyncIterator -->|dep| MaybeAsync
AsyncIterator -->|maybe dep| AsyncFnNext
AsyncFnNext -->|dep| TAIT
FuturesConcurrency -->|dep| AsyncIterator
AsyncCancellation -->|dep| AsyncDrop
```
Project board:
https://github.com/orgs/rust-lang/projects/28
Recent blog post from boats:
https://without.boats/blog/a-four-year-plan/
## High-level goals
* People can't write things they want to write
* People have to use external crates for things that should be in std
* Ecosystem crates can't interoperate well because we don't have foundations in std
* Async is too hard? In what ways?
* Portability for protocols
* Expressiveness gaps
* Where we define expressiveness as "the ability to express something at all", at least without going to extraordinary lengths
## Big questions
* How do we integrate async traits into std?
* Maybe Async trait definitions
* "Do we call it `AsyncIterator` or `async Iterator`"
* I/O Traits
* Should async, sync be the same
* Definitely possible to have convergent APIs (async-std)
* But Tokio has diverged (partly in ways that aren't specific to async)
* https://yoshuawuyts.notion.site/yoshuawuyts/a644246eada64ebab4163fec8a1141e7?v=10b1a319e9d04f5c87cf44849810f281 - catalogues which traits could be made maybe-async, maybe-const, maybe-try, etc.
* Structured concurrency: What it depends on
* async drop, maybe
## Random thoughts
All Generators that are not pinned are Iterators.
All Iterators are Generators.
```rust
impl<T> Generator for T
where
T: Iterator + Unpin // Iterator is always Unpin already
{ ... }
impl<T> Iterator for G
where
T: Generator + Unpin
{ ... }
```
Can we have both of these blanket impls at once?
If we only have one path into this cycle, maybe this is OK?
Would not be allowed if there were both a hand-written impl of Iterator and Generator. But these would be more specific, so perhaps we'd just ignore the other two.
We need a way to be polymorphic over the Self type. Is there an angle here with trait aliases?
```rust
#![feature(trait_alias)]
use core::pin::Pin;
trait GeneralIterator {
type This;
type Item;
fn next(x: Self::This) -> Self::Item;
}
trait Generator<'a> = GeneralIterator<This = Pin<&'a mut Self>>;
trait Iterator<'a> = GeneralIterator<This = &'a mut Self>;
```
```rust
#![feature(trait_alias)]
use core::pin::Pin;
trait GeneralIterator {
type This;
type Item;
fn next(x: Self::This) -> Self::Item;
}
trait Generator = GeneralIterator<This = for<'a> Pin<&'a mut Self>>;
trait Iterator = GeneralIterator<This = for<'a> &'a mut Self>;
```
This is HKT, right?
So can we express this with GATs?
What about internal iteration for generators?
Does the coroutine transform buy us a solution there? Rather than passing in a closure, we pass in a coroutine, then resume = yield.
We'd probably need a dual coroutine transform to make a for loop into a coroutine to pass into an iterator.
We don't want to let RTN sit around forever.
An alternate to RTN, inferred associated types...
```rust
trait Trait {
type Foo;
fn method() -> Self::Foo {
()
}
async fn foo() -> Self::Foo {}
// Where RTN doesn't quite work.
fn foo2() -> (impl Future<..>, impl Future<..>) { todo!() }
// Instead:
fn foo2() -> (Self::Ret1, Self::ret2) { todo!() }
}
```
`dyn*` is somewhat stalled. CE was working on making it work for async and `dyn Trait`. It was put on hold pending AFIT. So it's at a good spot to pick up again and start going.
Do we do the most general version that Niko proposed, or start with only the pointer-sized ones?
If our `dyn*` could store two or three or four pointers worth of data, that means a lot more futures could be stored inline rather than having to be allocated. That may be convincing.
Sign in with Wallet
