---
title: "Meetup 2024: Omnibus async bill and reconciliation act"
date: 2024-09-12
url: https://hackmd.io/fQ_LVdGaSyyuL7B_nuOfug
---

# Meetup 2024: Omnibus async bill and reconciliation act

**see [draft RFC](https://hackmd.io/Ykr320pHRoyfPmvCKkJkdw) which was extracted to its own hackmd**

---

## What kind of document

options

* "informal consensus", start blogging + authoring RFC(s), but merge async closures in the meantime
* design note with checkboxes, merge async closures in parallel
* design note with checkboxes *before* merging async closures
* rfc, merge async closures in parallel
* rfc *before* merging async closures

merge = rfcbot merge the tracking issue for async closure syntax

josh proposal:

* author design note, get checkboxes on it from T-lang, merge it into the lang-team repository
* fcp merge the tracking issue #128129 on that basis
* begin work towards one or more real RFCs

TC/Niko proposal:

- Fuzzy consensus via this discussion.
- Resolve #128129.
- File RFC on big picture.

pnkfelix proposal?:

- File and accept RFC on big picture.
- Resolve #128129.

Here's the issue for resolving the syntax:

https://github.com/rust-lang/rust/issues/128129

## Outline

* We see a need for `async {Trait,fn}` and `const {Trait,fn}`.
* Trait: Sync trait and corresponding async trait; same for const.
* We want to see a consistent Rust picture for async in the future, which `async Fn` syntax is a consistent part of. In order to have that consistent picture, we want `async` to be a **transformer** that can apply to:
* Traits: producing e.g. `async Iterator`
* Types: with some syntax that desugars to `impl Future<Output = T>`
* Functions: our existing `async fn`
* Blocks: our existing `async {}` as well as a version of those blocks with an explicit return type `T`
* We see this transformer model as also naturally extending to `const` (as is already being experimented with using `~const`), and potentially to `try` (as a family of transformers varying by type, for instance `Result<_, E>` or `Option<_>` or `ControlFlow<B, _>`), `gen`, and `async gen`. Each of these (with some quirks and caveats)
* Therefore we propose
* `async Trait` will be a general form to select the "async version of a trait" (or "async bound on the trait")
* just as `const Trait` will be for "const version" (or "const bound on the trait")
* there will be a way to use keyword async for type, (potentially/hypothetically: `async<T>` or `async -> T`), that expands to `impl Future<Output = T>`
*

Note: "we are precommitting to this entire vision" vs "we are painting a vision that we generally favor"

More typing

* We want to see a consistent Rust picture for async in the future, which `async Fn` syntax is a consistent part of. In order to have that consistent picture, we want the `async` keyword to be usable
* Traits: producing e.g. `async Iterator`
* Types: with some syntax that desugars to `impl Future<Output = T>`
* Functions: our existing `async fn`
* Blocks: our existing `async {}` as well as a version of those blocks with an explicit return type `T`
* `async` and `const` trait modifiers
* The async closures RFC proposed `async Fn`, meaning an "async version of the `Fn` trait" (one in which the `call` method is `async`).
* The work on const has revealed the need for trait references like `const Default`, meaning a "const version of the `Default` trait". (There may be some quirks or caveats to unifying this with const).
*
* What we are firmly committed to now
* There will be an `async Trait` modifier
* There will be an `const Trait` modifier
* There will be a syntax for an "async version" of a type (e.g., either `async<T>` or `async -> T`).
* ...and that it applies in expression syntax.

Motivation:
- We can't stabilize `for` loops in const context until we can make `Iterator::next()` const.

Future work

* probably in the future `try {Trait,fn,T}`
* `do`

// There is "one trait".
//
// In the sense that for M types, there may be at most M impls.
trait Tr {
  type T: Sized;
}

// There are "N traits".
//
// In the sense that for M types, there may be at most M*N impls.
trait Tr<T: Sized> {}

async -> u8 {
  
}

try yeets E -> u8 {
  
}

gen yields u8 {
  
}

|| -> u8 {
  
}

coro yields u8 -> u16 {
  
}

## Transformers

Resolved

* We want to settle questions like `AsyncFn` vs `async Fn` vs `async fn`
* We feel that which of those we want to do depends on a variety of future language changes that we want to be consistent with, to build what feels like a complete and consistent and orthogonal solution.
* We want to provide ambitious comprehensive changes to the language, rather than rounding off all the sharp edges and doing path-dependent hill-climbing or annealing
* Nobody loves `AsyncFn` (as the *only* syntax) unless we're *not* making *any* of the other exciting changes
* "Fn is not that special": Did not like the idea of supporting `async Fn` but not `async OtherTraits`
* `async Fn` is a name for a first-class concept, not a syntax for something else: if the user writes `impl async Fn` it should be presented to the user (in rustdoc, error messages, etc) as `async Fn` (not e.g. `AsyncFn`)
* "Don't break the Rust book": Did not want a solution for async closures that requires extensive changes to other common patterns that exist today
* e.g. using `impl async fn()`, which implies using `impl fn()` (resp. `impl fn mut`, `impl fn once`) for consistency, which:
* is a change from `impl Fn` (resp. `impl FnMut`, `impl FnOnce`) which are very common
* introduces `mut` and `once` modifiers that aren't planned to be generalized and do not have particular precedent
* would be uncomfortably close to `fn` types (and perhaps require finding a new notation for them)
* We expect to support additional trait modifiers like `const Trait`, which will be "a variant of trait with const versions of (at least some) its methods" (much like `async Fn`)

What is a transformer?

A transformer TX =
* a type transformation `TX<T>` that takes the return type and modifies
* not quite a type alias because it can expand to `impl SomeTrait<T>` or `impl SomeTrait<AssocType=T>`
* a kind of block that you can apply to a block that produces a `T` to produce a `TX<T>`
* an operation to extract the `T` from a `TX<T>`
* this can do random stuff but must, if it returns, produce a `T`

Example transformer `async` =
* `async<T> = impl Future<Output = T>`
* `async { ... }` (or `async<T> { ... }` to set the return type)
* extract(E) = `E.await`

Example transformer `gen(T)` =
* `gen(T)<R> = impl Gen<Item = T, Return = R>` where expected `R = ()`
* `gen { ... }`
* extract(E) = `yield_from!(E)`

Example transformer `coro(x: T)` =
* `coro(x:T)<T> = impl Gen<Out = T>`
* `coro(x:T) { ... }` (or `async<T> { ... }` to set the return type)
* extract(E) = `E.await`

Example transformer `try(E)` =
* `try(E)<T> = Result<T, E>` // Note: we need to handle Option and ControlFlow and others, too
* `try { ... }`
* extract(E) = `E?`

Example transformer `const` = (let's ignore it for the moment)
* `try(E)<T> = Result<T, E>`
* `try { ... }`
* extract(E) = `E` -- `E` must not do things not compatible with const
* `if $in_const { const { E} } else { E }`

Example transformer `runtime` AKA `IO` =
* `runtime<T> = T`
* `runtime { .. }` -- allows non-const-safe actions
* extract(E) = (actually do the side effects, not allow in a non-runtime context)
* in practice we flip the default, so you write `const` instead of `?runtime`

Example transformer `lazy` = ?
* `lazy<T> = impl FnOnce<(), Output = T>`
* `move || { ... }`
* extract(E) = `E()`

Syntax idea based on transformer...

For every transformer TX, you can write `TX fn f(args) -> R { body }` or, equivalently

fn f(args) -> TX<R> {
    TX {
        body
    }
}

(some specific examples involving `async`)

async fn do_something(x: async<u32>) -> i32 {
    x.await as i32
}

fn do_something(x: async<u32>) -> async<i32> {
    async {
        // gloss over: `x` drops
        x.await as i32
    }
}

fn do_something(x: impl Future<Output = u32>) -> impl Future<Output = i32> {
    async {
        x.await as i32
    }
}

fn do_something(x: async<u32>) = async<i32> {
    x.await as i32
}

closure expressions

* `TX || ..` would implement the `TX Fn` (etc) traits
* `TX || x ~~~> || TX { x }` doesn't really hold because of details
* but it "feels like" it does
* there's an operation we do that is

async fn do_something(x: async<u32>) -> i32 {
    x.await as i32
}

fn do_something(x: async<u32>) -> async<i32> {
    async {
        x.await as i32
    }
}

fn do_something(x: impl Future<Output = u32>) -> impl Future<Output = i32> {
    async {
        x.await as i32
    }
}

General technical ideas/areas

* Function/trait transformers like "maybe `async`", "maybe `const`"
* Type shortcuts `async<T>`, `try<T>`, maybe `try(E)<T>` (TBD), `do<T>`
* Allow defining `try` as an alias, in order to default the error type. Have a non-defaulting version where you can specify the error type.
* `use eyre::try`

//Haskell-style
`try E T`

// in anyhow
type try<T> = Result<T, Error>;

// in user of anyhow
use anyhow::try;

try fn could_fail() -> u32 {
    22
}

// in std
mod option {
    type try<T> = Option<T>;
}
mod io {
    type try<T> = Result<T, Error>;
}
mod control_flow {
    type try<B, C> = ControlFlow<B, C>;
    
    // try(B) fn foo() -> u32
    // fn foo() -> try<B, u32>
}
pub enum ControlFlow<B, C = ()> {
    Continue(C),
    Break(B),
}



use std::option::try;

type try<T, E = anyhow::Error> = Result<T, E>;

try fn could_find_nothing() -> u32 {
    let x = find_something()?;
    22 + x
}

fn something() = try<u32, io::Error> {
};


fn something() = async<try<u32, io::Error>> {
};


try(io::Error) fn something() -> u32 {
    read()?
}

fn could_find_nothing() -> try<u32> {
    // ....
    try {
        let x = find_something()?;
        22 + x
    }
}

fn could_find_nothing() = try<u32> {
    let x = find_something()?;
    22 + x
}

fn find_something() -> Option<u32> {
    Some(e)
}

// Would we be able to support error types?
// Here `try<E>` feels bad because normally it means an ok type

try(E) try(AllocationError) fn could_fail<T>(func: impl try(E) Fn() -> T) -> T {
    22
}

## TC's big picture thought

Regarding `AsyncGen`....

Had a thought about how this might lend some support to an `impl async -> u8` syntax (though I don't claim it's dispositive).

Eric Holk and I have been talking, and we've at various points considered trying to RFC/stabilize `async gen { .. }` blocks while hiding the details of the trait, as with `async Fn() -> T`. Not sure we need to, but let's say that we did. How do users write `impl AsyncGen<..>`?

We could have them write that, of course, but people will complain this is inconsistent with the decision we made (hopefully) on `async Fn()`. We don't want to do `async Iterator`, because that essentially yields the entire case in favor of an unpinned `async fn next()` design, in terms of framing this as "the asynchronous version of the unpinned iterator". We could come up with a synchronous pinned `Gen` trait and then say `async Gen`, but that's still kind of weird that we keep the `async` keyword/effect but convert the `gen` keyword/effect to a carried trait.

So, the end of this train of thought lands in something like:

fn f() -> impl async gen yields u8 {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen { for x await in g() { yield x; } }
}

There's also parity with async gen closures to consider. E.g., we can make the above higher-ranked by just adding `||` and `Fn()`:

fn f() -> impl async gen Fn() yields u8 {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen || { for x await in g() { yield x; } }
}

fn f() -> impl async gen || yields u8 {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen || { for x await in g() { yield x; } }
}

fn f() -> impl async gen fn() yields u8 {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen || { for x await in g() { yield x; } }
}

Otherwise, it's kind of odd that the transformation is a bigger step, i.e. from this instead:

fn f() -> impl AsyncGen<Item = u8> {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen { for x await in g() { yield x; } }
}

...and that when making a closure of it, the effects end up getting dropped back to keywords at that point.

Of course, I could make the argument the other way, that this train of thought suggests why we should do `AsyncFn` instead. I suppose, really, what this train of thought suggests is how our choices here might end up binarized. That either we double down on encoding all of these effects as carried traits, i.e. `AsyncGen`, `AsyncFn`, etc., or that we double down on using the keywords in bounds.

Possibly in favor of the keywords in bounds approach is that this could let us hide for awhile the kind of combinatorial blow-up of these effects when flattened into traits while we work out if there's something we can do here.

## Gen closures vs coroutines closures

async { 
  ...
}

async |..| {
  
}

~=

|..| async {
  ...
}

gen { ... }

gen || {
  ..
}

~=

|..| gen {
  ..
}


coro |x: u32| {
    x = yield;
}


coro |x: u32| {
    x = yield;
}

// TC: Crazy idea
// pnkfelix: I actually don't hate this.
coro |..| <- x: u8 -> u8 {
  //                  ^^ Output
  //         ^^ Resume
  // ^^^ Closure args
  x = yield ();
}


|a| coro |x| {
    x = yield;
}

// NIKO
coro(y: u32) {
    y = yield;
}

coro(y: u32) || {
    y = yield;
}

~=

|| coro(y: u32) {
    y = yield;
}

pub trait Coroutine<R = ()> {
    type Yield;
    type Return;

    // Required method
    fn resume(
        self: Pin<&mut Self>,
        arg: R,
    ) -> CoroutineState<Self::Yield, Self::Return>;
}

// Possible, but probably not worth it.
pub trait Coroutine<R = ()> {
    type Yield;
    type Return;

    // Required method
    fn start(self: Pin<&mut Self>) -> ..;
    fn resume(
        self: Pin<&mut Self>,
        arg: R,
    ) -> CoroutineState<Self::Yield, Self::Return>;
}

## Net-net on coroutines

TC: I agree that:

- It's possible and probably desirable to preserve the `async` -> async closures, `gen` -> gen closures pattern for general coroutines.
- That is, that the `|..|` args are closure arguments.
- And to then have a separate syntax for the `Resume` argument.

...and in that world, can `gen` be the same as `coro`? Maybe.

## Other things maybe we hate or not?

TC: There's I think a general question here of to what degree we want the type syntax to match the expression syntax.

fn f() -> impl async gen || yields u8 {
    async gen fn g() yields u8 { loop { yield ready(42).await; } }
    async gen || { for x await in g() { yield x; } }
}

TC: Another example of this is the `&raw ..` syntax and the desire to make that a type system (rather than `*const`.)

## Niko's digression into iterator chains

trait do Iterator {
    type Item;

    do fn next(&mut self) -> Option<Self::Item>;

    do fn map<U>(
        self,
        f: impl do FnMut(Self::Item) -> U,
    ) -> impl do Iterator<Item = U> {
        Map {
            self, f
        }
    }

    do fn collect<C>(self) -> C
    where
        C: do FromIterator<Item = Self::Item>
    {
        
    }
}

struct do Map<I, F>
where
    I: do Iterator,
    F: do FnMut(I::Item)
{
    i: I,
    f: F,
}

struct Map<tx X, I, F>
where
    I: Iterator<X>,
    F: FnMut<X>(I::Item),
{
    i: I,
    f: F,
}

trait Iterator<tx X> {
    type Item;
    
    fn next(&mut self) -> X<Option<Self::Item>>;
    fn map(&mut self) -> ..;
}

impl do Iterator for do Map<F> {
    do fn next(&mut self) -> Option<Self::Item>;
}

#[maybe(async, try)]
async try fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load().await?;
    bytes
        .split("\n")
        .map(async try |s| s.parse_as_data.await?)
        .collect/*::<Vec<Data>>*/().await?
}

// ??
fn load_data_from_disk<effect E: async + try>(p: &Path) -> Vec<Data> {
    let bytes = p.load().await?;
    bytes
        .split("\n")
        .map(async try |s| s.parse_as_data.await?)
        .collect/*::<Vec<Data>>*/().await?
}

do(async) fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load().await.do;
    bytes
        .split("\n")
        .map(do |s| s.parse_as_data.await.do)
        .collect/*::<Vec<Data>>*/().await.do
}

do fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load().do;
    bytes
        .split("\n")
        .map(do |s| s.parse_as_data.do)
        .collect/*::<Vec<Data>>*/().do
}

try fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load()?;
    bytes
        .split("\n")
        .map(try |s| s.parse_as_data?)
        .collect/*::<Vec<Data>>*/()?
}

async fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load().await;
    bytes
        .split("\n")
        .map(async |s| s.load_data().await)
        .collect/*::<Vec<Data>>*/().await
}

async try fn load_data_from_disk(p: &Path) -> Vec<Data> {
    let bytes = p.load().await?;
    bytes
        .split("\n")
        .map(async try |s| s.load_data().await?)
        .collect/*::<Vec<Data>>*/().await?
}