---
title: "Reading notes: Async Iteration III: The Async Iterator Trait"
tags: WG-async, reading-club, minutes
date: 2023-10-12
url: https://hackmd.io/N6TfbzyPSfmqaOSycUKfsg
---

# Reading notes: Async Iteration III: The Async Iterator Trait

Link to document: https://blog.yoshuawuyts.com/async-iterator-trait/

## Attendance

- People: Tyler Mandry, Vincenzo Palazzo, TC, Zach Mitchell, Eric Holk, Yosh Wuyts

## What are the difference between `AtomicFuture` and `UnwindSafe`?

yosh: A colleague at work read this post, and was asking what the differences between "cancel safety" and "unwind safety" are. And I don't think we arrived at a particularly good answer?

yosh: They both seem to relate to: "hey if control flow hits in the middle of this operation, does my type remain in a good state?" I'm not super comfortable with what `UnwindSafe` actually does, and I think conversations have been had about possibly deprecating it. So I wanted to raise this to the group.

TC: The fact that we have an `UnwindSafe` trait provides some precedent for `CancellationSafe`.

tmandry: One difference is that you can catch and stop an unwind, but with cancellation you can write code that will (hopefully) happen on cancellation (with `drop`) but that won't stop the cancellation itself.

eholk: In the future, we hope to have a way to stop a cancellation. The things I've been working on with my series of posts would open up with possibility. Cancellation is weird with respect to panic, because it's the other way around. Panic starts at the bottom, but cancellation starts at the top.

TC: That's related to the `async` transform; it converts the code inside out. Inversion of control.

## Why not both?

zmitchell: Is there a reason not to have both `poll_next` as `AsyncIterator` and then something like `AsyncIteratorExt` that contains the `next` method that most users will interact with (e.g. `Stream` and `StreamExt` today)? It seems like this would allow things that _need_ pinning to have it, while also having better ergonomics for people that don't care about pinning.

yosh: I really believe we should be treating "self-referential iterators" as orthogonal to "async iterator". If we're going to add support for self-references to async iterator, we should try and take the same approach for non-async iterators. And tbh probably also for other traits. Otherwise we end up with really weird asymmetric designs.

TC: Do we believe that for the self-referential cases that `async fn next(self: Pin<&mut Self>) { .. }` has all of the needed expressiveness? It seems the piece under discussion is arguing that it does.

yosh: It does.

yosh: One fundamental flaw of the existing `AsyncIterator` design is that it unifies things into one state that should be two states. We're overconstraining, essentially. The Self state of Iterator is not pinned. We only care about the Self state of `Future` being pinned. It's only when you want a self-referential iterator that the pin matters. And that's a problem shared with sync iterators. If we did it one way for async and another way for sync, that would be inconsistent.

TC: If we think the sync ones were done in error, and we plan to fix them somehow, that could justify doing the async ones in the way that we want to do the sync ones. (But I'm not arguing whether that's the case or whether that would be sufficient.)

yosh: I'd say the starting point would be to fix the sync ones first, rather than assuming that we could do a fix. It's more important to have parity between the sync and async space.

eholk: If we had had generators earlier, then `Iterator` may have taken pinned Self.

---

(Clarification of earlier discussion...)

TC: There are two separate questions here: 1) does `AsyncIterator` take a pinned `Self` type?, 2) does `async fn next(self: Pin<&mut Self>)` have all of the expressive power to represent self-referential iterators?

TC: People say, "we need `poll_next` because we might need `AsyncIterator` to take a pinned `Self` type." And yosh, you argue, "but we don't need `AsyncIterator` to take a pinned `Self` type." Maybe the better argument is that "`async fn next(self: Pin<&mut Self>)` has all of the needed expressivessness for a hypothetical`AsyncIterator` with a pinned `Self` type". That is, whether or not `AsyncIterator` should take a pinned `Self` type is a separate question from whether `async fn next` would somehow preclude a pinned `Self` type.

## Zero-cost depends on TAIT

TC: Without TAIT (or equivalently, "RTN Everywhere"), one advantage of `poll_next`-based traits and manual implementations is that the resulting `Future` can be named, so it can be stored unboxed in a `struct`, e.g. With RPITIT/AFIT-based traits and RPIT/`async fn` impls, the resulting `Future` cannot be named. In a world with TAIT, this isn't really a problem. But until/unless we do that, it's hard to argue in favor of returning more types that cannot be named from APIs, and particularly from stdlib APIs.

TC: (This is the reason that, in Tokio, [ReusableBoxFuture](https://docs.rs/tokio-util/latest/tokio_util/sync/struct.ReusableBoxFuture.html) exists.)

yosh: That's a good point. I put it under the same box as `dyn*` as something that we want to fix.

## Effect of `AtomicFuture`

tmandry: What would the practical impact of the `AtomicFuture` trait be on a user's workflow? Would the compiler lint on loops that create/destroy futures that are not `AtomicFuture`?

yosh: Oh that's a good question; I'm not sure. That seems like a cool option worth exploring. I'm basically hoping that APIs which depend on dropping/recreating futures will check for `AtomicFuture`. But also by providing [structured concurrency operations](https://docs.rs/futures-concurrency/latest/futures_concurrency/) we can largely supplant the need for some of the most popular APIs using this pattern today (e.g. replace `select!`).

yosh: I'm also hoping an `async` iteration syntax will make other patterns less common.

## Two state machines

From [The AsyncIterator Interface](https://without.boats/blog/async-iterator/):
> One major problem that arises from this has been framed as a problem of “cancellation safety.” Suppose you are selecting from two async iterators in a loop. If, when one completes, you drop the next future, any progress made in that state machine will be cancelled. When you go around the loop again and return to waiting for that next, instead you will start a new next future over from the beginning state. To implement this behavior correctly, you would be responsible for storing the next futures outside of the loop and replacing them whenever they finish. This is a major footgun that would be specifically problematic for AsyncIterator (though it can show up in other places already) because AsyncIterator is almost always polled repeatedly in a loop like this.

zmitchell: The cancellation-safety aspect is addressed in the post, but the additional complexity of managing a second state machine is not.

yosh: can you elaborate on the problem you're seeing there? I'm not sure where the "cancellation safety" problem ends, and where what you're describing begins.

zmitchell: Cancellation-safety is one aspect of logical correctness. In general there's likely to be other types of correctness issues that are made more difficult by needing to manage more state. This is absolutely hand-wavy, but more complexity is often the source of bugs.

yosh: I'd be interested in reading about examples of this which don't involve `select!` specifically. My position is that [select is not great](https://blog.yoshuawuyts.com/futures-concurrency-3/), and we should move to alternative APIs instead. At least for the `futures-concurrency` APIs, I don't believe there should be any negative interactions by switching to `async fn next`. And it may solve at least one potential soundness bug.

## Is there any expressiveness *not* restored by `poll_fn`?

TC: Having not thought deeply about this yet, I wonder whether there is any expressiveness *not* restored by `poll_fn`. If we could show that any `poll_next` impl could be mechanically-translated to use `poll_fn`, perhaps we could *prove* that no expressiveness is lost.

yosh: Yes. There is the orthogonal issue of the pinned Self type.

yosh: And we are relying heavily on inlining here. Of course, we could treat it as a compiler bug if it didn't.

TC: We could guarantee this inlining property, just as tail call elimination is an optimization that is guaranteed by some languages.

## Appendix

- vincent: It is missing the pin blog post link :) I would like to read it too

## LendingIterator/PinIterator

tmandry: If we make Iterator lending with a GAT can we have only two traits, one async and one sync?

## Iterator state and future state

eholk: One property of `poll_next` is that it keeps the iterator state and the future state machine as a single data structure.
`async fn next()` separates these two, and potentially introduces weird borrowing constraints, since the `next()` future will need to borrow the iterator.
(I don't know off hand of cases where this causes problems, but maybe someone else does?)
Should these concepts be separate or combined?

I think Rust has generally had a long-standing design principle of using separate language constructs for orthogonal features, so separating iterator state and future state (which is what you get with `async fn next()`) seems in line with this principle.

Separating the states might cause problems too, such as trying to stick the future returned by `next()` in a collection.