---
title: AsyncIterator design considerations
tags: ["WG-async", "analysis"]
date: 2023-11-28
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Blog.20post.20about.20poll_next
url: https://hackmd.io/xiduBZwVQ0y_z0XSJZNHfA
---

# AsyncIterator design considerations

## Effect on other poll traits

Tomás Vallotton:

Like `Stream`, there are other poll traits that also benefit from staying as poll traits as opposed to async traits. For example, types that support the traits `AsyncRead` and `AsyncWrite` can be split. The implementation of split requires minimal unsafe code and little synchronization (which can be omitted in single threaded executors). Split is easy to implement because there are no long lived mutable borrows when you use poll methods, but this is not the case for async traits.

If AsyncRead and AsyncWrite were async traits instead, we would need a new unsafe trait (likely named Split), and this trait would need to implement unsafe versions of read and write that take a shared reference instead of a mutable reference.

If we wish to support split for async trait versions of `Read` and `Write`, it would may have to look like this:

/// Safety: The implementor of the trait must ensure that it is safe
/// to read and write concurrently to this type.
pub unsafe trait Split: AsyncRead + AsyncWrite {
    /// Safety: this method cannot be called concurrently twice.
    /// It is only safe to call it concurrently with `write`
    unsafe fn read(&self, buf: &mut [u8]) -> io::Result<usize>;
    /// Safety: this method cannot be called concurrently twice.
    /// It is only safe to call it concurrently with `read`
    unsafe fn write(&self, buf: &[u8]) -> io::Result<usize>;
}

In comparison, currently `split` looks like a function that only requires `AsyncRead + AsyncWrite + Sized` and requires no special support or marker traits.

To see another implementation of split in the ecosystem you can look at [monoio's](https://docs.rs/monoio/latest/monoio/io/trait.Splitable.html). Although I must say that at a first glance, the `AsyncRead` and `AsynWrite` implementations for `ReadHalf` and `WriteHalf` doesn't look sound.

## Effect on `async gen` implementation

@_**Michael (compiler-errors) Goulet|426609** [said](https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Blog.20post.20about.20poll_next/near/404709504):

> I would like to offer a datapoint that I gathered when [proving out an impl for `async gen` blocks](https://github.com/rust-lang/rust/pull/118420) while I was bored last night.
>
> Specifically, the built-in implementation of `async gen` blocks is far more receptive to an `fn poll_next()`-based API due to the way that the generator desugaring operates. There is a very compelling parallel between the APIs of the existing methods (`Future::poll`/`Iterator::next`/`Coroutine::resume`) lowered by the ["coroutine transform"](https://github.com/rust-lang/rust/blob/5facb422f8a5a61df515572fe79b02433639d565/compiler/rustc_mir_transform/src/coroutine.rs#L1) pass, and `AsyncIterator::poll_next` that, from a technical (compiler impl) perspective, I think is worth considering.
>
> To support a `async fn AsyncIterator::next()`-based API, we'd at least need to introduce an additional layer of indirection to turn the coroutine-resume function into a future implementation that can be returned by `AsyncIterator::next`. I'm not saying that's impossible, though I'd say that it would at least be a bit annoying, especially since we've supported work (e.g. #104321) to make coroutines implement these traits as natively as possible.

## Effect on inlining and indirect calls with `dyn AsyncIterator`

eholk:

`async fn next` makes it absolutely impossible to inline the calls to `poll`, at least if you have a `dyn AsyncIterator`.

Okay, so we write:

// iter needs to be pinned in boats' version
async fn sum(iter: &mut dyn AsyncIterator<Item = usize>) -> usize {
    let mut x = 0;
    for await i in iter {
        x += i;
    }
    x
}

actually, let's get simpler:

async fn first(iter: &mut dyn AsyncIterator<Item = usize>) -> Option<usize> {
    iter.next().await
}

If we assume the next boxes the future, the body would be equivalent to:

let f: Box<dyn AsyncIter> = iter.next();
f.await

and `f.await` is basically a call to `poll`, but since `f` is a trait object, it is an indirect call

so either we devirtualized iter, in which case it's a concrete type and we have a direct call for `next()` and a direct call for `poll()`

or we can't devirtualize it, in which case we have an indirect call for `next()` and an indirect call for `poll()`

but let's do the same function with `poll_next`

In `poll_next`, `AsyncIterator` doesn't include a `next()` method, so we have to add one as an extension method

so this would be something like

impl AsyncFnNext for T where T: AsyncIterator {
    fn next(&mut self) -> impl Future<Output = Option<T::Item>> {
        // use poll_fn to return a future that makes a dynamic call to poll_next
        ...
    }
}

But the key thing here is we _don't_ have a `dyn AsyncFnNext` when we call `next()`, we have a `impl AsyncFnNext for dyn AsyncIterator`, which means the hidden future returned by `next()` is a concrete type and we can make direct calls

but we're still making repeated indirect calls to `poll_next` in the loop that `await` desugars to after we inline the calls to `poll`

So, I think the `poll_next` version saves us one indirect call (the call to `.next()`), but in both cases we make indirect calls in the await loop