---
title: "Design meeting 2024-05-30: Open discussion"
tags: ["WG-async", "design-meeting", "minutes"]
date: 2024-05-30
discussion: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Design.20meeting.202024-05-30
url: https://hackmd.io/uK3aDMXnQNq8gUFerD5REA
---
# Discussion
## Attendance
- People: TC, eholk, Daria, Yosh
## Joining on tuples
Yosh: What stops us from doing, e.g. `(a, b).join()`?
Daria: You need allocation to do perfect waking (or my (future version of) the scope-lock crate.)
https://github.com/zetanumbers/scope-lock
eholk: There's a crate called unicycle that does tricks here also.
https://github.com/udoprog/unicycle
eholk: Something we should explore is task-local reference counted space.
Yosh: It would be interesting if we could ship this as a demonstration that this WG can ship things.
Daria: With async drop we'd have to run the destructors sequentially because otherwise we have to allocate.
Yosh: What if we keep another vec of the drop impls, like this?
```rust
#[repr(stable)]
struct Vec<F: Future> {}
impl IntoFuture for Vec<F> {
type IntoFuture = IntoFuture;
...
}
struct IntoFuture<F: IntoFuture> {
vec: Vec<F::IntoFuture>,
drop_impls: Vec<&'self impl Future<...>>,
}
```
eholk: It seems like this is a design choice, we can allocate and join the async drop futures, or we can run them sequentially in constant space. Is this an arbitrary choice, or is there a compelling reason to do one over the other?
Daria: Futures are interface types, like iterators, you have separate iterator types. One exception from this would be Range type which implements Iterator, which is considered a wrong turn today.
TC: Vectors can be arbitrarily large. Neither doing all of the drop operations sequentially nor concurrently seem optimal. One would probably want some concurrency, but if you have 100M items in a vector, you probably want e.g. a window of 1000 operations happening concurrently. This would seem to require more control than we've typically had with `Drop`.
Yosh: If you want control over concurrency you should use other mechanisms.
Daria: The user probably wants to specify whether they want sequential or concurrent awaiting?
Yosh: Let's leave Vec out of this, what's forward compatible with async destructors? How does this work for arrays or tuples?
eholk: There are a lot of policy questions here. You have to be opinionated in std, but there may not be a right answer. Maybe these choices are better left to the ecosystem.
Yosh: Let's not push hard choices to the users. We should adopt in the standard library reasonable choices for users.
TC: This gets to a philosophical question. There's some feeling that the standard library should only have in it what needs to be in the standard library. Where we've gone beyond that in the past, e.g. with channels, there are questions about whether doing those in std may have been a mistake in retrospect. So when there are many possible choices users could want, and it's not clear this needs to be in the standard library, this is where people raise questions.
Daria: Why can't we have a `Join` trait?
Yosh: `Vec` already has a `join` method.
Yosh: Another reason for why I think `impl IntoFuture` makes sense for container types is because it is a lossless concurrency operation which returns types wholesale. We have one input structure of futures, and return another output structure of outputs. It's also the reason why we can use it to express `async let`. It's sort of like a "default control flow" for lack of a different word?
## Experience report mapping a gen block to a coroutine
TC: Related to our discussions on progress and push versus pull iteration, I found it instructive mapping a non-trivial `gen` block (that pulls values from an iterator) to an equivalent coroutine (that has the values pushed into it).
TC: This made me think about the various axes of flow control that are needed.
TC: E.g., after `resume` is called, at the next `yield` point, we can be in one of these 16 possible states (RTR = "ready to receive", RTS = "ready to send"):
| | Caller RTR | Caller RTS |
|---------------|:----------:|:----------:|
| Coroutine RTR | Y/N | Y/N |
| Coroutine RTS | Y/N | Y/N |
> Run-length encoding is a compression mechanism which prints the count, then the value. So if you have 25 zeros, you can compress it by yielding `24` and then yielding `0`.
Here's the `gen` block version of run-length encoding (we included this in the RFC for gen blocks):
```rust
fn rl_encode<I: IntoIterator<Item = u8>>(
xs: I,
) -> impl Iterator<Item = u8> {
gen {
let mut xs = xs.into_iter();
let (Some(mut cur), mut n) = (xs.next(), 0) else { return };
for x in xs {
if x == cur && n < u8::MAX {
n += 1;
} else {
yield n; yield cur;
(cur, n) = (x, 0);
}
}
yield n; yield cur;
}.into_iter()
}
```
[Playground link][gen-rle]
And here's the coroutine version:
```rust
fn rl_encode()
-> impl Coroutine<Option<u8>, Yield = (Poll<u8>, bool), Return = ()> {
#[coroutine] |x: Option<u8>| {
let (Some(mut cur), mut n) = (x, 0) else { return };
while let Some(x) = yield (Pending, true) {
if x == cur && n < u8::MAX {
n += 1;
} else {
yield (Ready(n), false); yield (Ready(cur), false);
(cur, n) = (x, 0);
}
}
yield (Ready(n), false);
yield (Ready(cur), false);
}
}
```
[Playground link][coroutine-rle]
The `bool` is a flag to indicate whether `resume` should next be called with `Some(_)` or with `None`. The effect of that is to "control" the return value of the `yield`, such that we expect these to be correct:
```rust
let Some(_) = yield (Pending, true) else { unsafe { unreachable_unchecked() }};
let None = yield (Pending, false) else { unsafe { unreachable_unchecked() }};
```
(Discussion explaining and walking through the above.)
Daria: Code above looks like a function signatures.
(Discussion about how this applies to sans I/O use cases for coroutines and how that use case needs this same flow control mechanism.)
eholk/Yosh: This is interesting. We'll be thinking about this after the meeting.
## Async code should not have been cancellable
Daria: Cancel is just an effect like Result type and the question mark operator, but today it's engraved into `.await`... There is a lot to talk about in the future...
eholk: I'm not sure cancellation was added on purpose, it kind of fell out of the other semantics. It definitely makes our job harder, and it makes the user's job harder, but it also makes some things easier. It highlights missing features for the language.
Daria: Aspect of it that "makes some things easier" should have been a separate thing (like a question mark operator).
(The meeting ended here.)
---
## Playground links
[gen-rle]: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&code=%2F%2F+This+demonstrates+run-length+encoding+using+gen+blocks+and+how+we%0A%2F%2F+might+use+this+via+a+method+combinator+on+%60Iterator%60.%0A%2F%2F%0A%2F%2F+Author%3A+TC%0A%2F%2F+Date%3A+2024-05-23%0A%0A%2F%2F%40+edition%3A+2024%0A%23%21%5Bfeature%28gen_blocks%29%5D%0Afn+rl_encode%3CI%3A+IntoIterator%3CItem+%3D+u8%3E%3E%28%0A++++xs%3A+I%2C%0A%29+-%3E+impl+Iterator%3CItem+%3D+u8%3E+%7B%0A++++gen+%7B%0A++++++++let+mut+xs+%3D+xs.into_iter%28%29%3B%0A++++++++let+%28Some%28mut+cur%29%2C+mut+n%29+%3D+%28xs.next%28%29%2C+0%29+else+%7B+return+%7D%3B%0A++++++++for+x+in+xs+%7B%0A++++++++++++if+x+%3D%3D+cur+%26%26+n+%3C+u8%3A%3AMAX+%7B%0A++++++++++++++++n+%2B%3D+1%3B%0A++++++++++++%7D+else+%7B%0A++++++++++++++++yield+n%3B+yield+cur%3B%0A++++++++++++++++%28cur%2C+n%29+%3D+%28x%2C+0%29%3B%0A++++++++++++%7D%0A++++++++%7D%0A++++++++yield+n%3B+yield+cur%3B%0A++++%7D.into_iter%28%29%0A%7D%0A%0Atrait+IteratorExt%3A+Iterator+%2B+Sized+%7B%0A++++fn+then%3CF%2C+I%3E%28self%2C+f%3A+F%29+-%3E+impl+Iterator%3CItem+%3D+Self%3A%3AItem%3E%0A++++where%0A++++++++%2F%2F+For+the+same+reasons+that+we+need+async+closures%2C+we%27d%0A++++++++%2F%2F+actually+want+to+use+gen+closures+here+in+a+real%0A++++++++%2F%2F+implementation.%0A++++++++F%3A+FnOnce%28Self%29+-%3E+I%2C%0A++++++++I%3A+IntoIterator%3CItem+%3D+Self%3A%3AItem%3E%2C%0A++++%7B%0A++++++++f%28self%29.into_iter%28%29%0A++++%7D%0A%7D%0A%0Aimpl%3CI%3A+Iterator%3E+IteratorExt+for+I+%7B%7D%0A%0Afn+main%28%29+%7B%0A++++for+x+in+%5B1u8%3B+513%5D.into_iter%28%29.then%28rl_encode%29+%7B%0A++++++++%2F%2F+++%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%0A++++++++%2F%2F+++%5E+Produces+an+iterator+that+yields+the+run-length%0A++++++++%2F%2F+++++encoding+of+this+array.%0A++++++++println%21%28%22%7B%3A%3F%7D%22%2C+x%29%3B%0A++++%7D%0A%7D%0A
[coroutine-rle]: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&code=%2F%2F+This+demonstrates+run-length+encoding+using+a+coroutine+and+how+we%0A%2F%2F+might+use+this+via+a+method+combinator+on+%60Iterator%60.%0A%2F%2F%0A%2F%2F+The+interesting+bit+here+is+that+the+input+rate+and+the+output+rate%0A%2F%2F+are+not+the+aligned.++Sometimes+we+want+to+accept+many+values+as%0A%2F%2F+input+without+yielding+anything%2C+and+sometime+we+want+to+yield%0A%2F%2F+multiple+things+without+taking+any+further+input.%0A%2F%2F%0A%2F%2F+We+encode+one+half+of+this+flow+control+by+having+the+coroutine%0A%2F%2F+yield+a+flag+indicating+whether+it+wants+to+receive+further+input.%0A%2F%2F+If+it+doesn%27t%2C+then+the+caller+is+expected+to+resume+with+%60None%60%3B%0A%2F%2F+otherwise+with+%60Some%28_%29%60.%0A%2F%2F%0A%2F%2F+For+the+other+half+of+this+flow+control%2C+we+have+the+coroutine%0A%2F%2F+yield+either+%60Ready%28_%29%60+or+%60Pending%60.++If+it+yields+%60Pending%60%2C+that%0A%2F%2F+means+it+doesn%27t+yet+have+a+value+for+us%3B+if+it+yields+%60Ready%28_%29%60%2C%0A%2F%2F+that+means+it+has+a+value+for+us%2C+but+it+doesn%27t+mean+that+it%27s%0A%2F%2F+done.++The+coroutine+should+continue+to+be+resumed+until+it%0A%2F%2F+returns.%0A%2F%2F%0A%2F%2F+Author%3A+TC%0A%2F%2F+Date%3A+2024-05-22%0A%0A%2F%2F%40+edition%3A+2024%0A%23%21%5Bfeature%28coroutines%2C+coroutine_trait%29%5D%0A%23%21%5Bfeature%28gen_blocks%29%5D%0A%23%21%5Bfeature%28stmt_expr_attributes%29%5D%0A%0Ause+core%3A%3A%7B%0A++++ops%3A%3A%7BCoroutine%2C+CoroutineState%7D%2C%0A++++pin%3A%3A%7BPin%2C+pin%7D%2C%0A++++task%3A%3APoll%3A%3A%7Bself%2C+%2A%7D%2C%0A%7D%3B%0A%0Afn+rl_encode%28%29%0A-%3E+impl+Coroutine%3COption%3Cu8%3E%2C+Yield+%3D+%28Poll%3Cu8%3E%2C+bool%29%2C+Return+%3D+%28%29%3E+%7B%0A++++%23%5Bcoroutine%5D+%7Cx%3A+Option%3Cu8%3E%7C+%7B%0A++++++++let+%28Some%28mut+cur%29%2C+mut+n%29+%3D+%28x%2C+0%29+else+%7B+return+%7D%3B%0A++++++++while+let+Some%28x%29+%3D+yield+%28Pending%2C+true%29+%7B%0A++++++++++++if+x+%3D%3D+cur+%26%26+n+%3C+u8%3A%3AMAX+%7B%0A++++++++++++++++n+%2B%3D+1%3B%0A++++++++++++%7D+else+%7B%0A++++++++++++++++yield+%28Ready%28n%29%2C+false%29%3B+yield+%28Ready%28cur%29%2C+false%29%3B%0A++++++++++++++++%28cur%2C+n%29+%3D+%28x%2C+0%29%3B%0A++++++++++++%7D%0A++++++++%7D%0A++++++++yield+%28Ready%28n%29%2C+false%29%3B%0A++++++++yield+%28Ready%28cur%29%2C+false%29%3B%0A++++%7D%0A%7D%0A%0Atrait+IteratorExt%3A+Iterator+%2B+Sized+%7B%0A++++fn+then_co%3CC%3E%28self%2C+mut+co%3A+Pin%3C%26mut+C%3E%29%0A++++-%3E+impl+Iterator%3CItem+%3D+Self%3A%3AItem%3E%0A++++where%0A++++++++C%3A+Coroutine%3C%0A++++++++++++Option%3CSelf%3A%3AItem%3E%2C%0A++++++++++++Yield+%3D+%28Poll%3CSelf%3A%3AItem%3E%2C+bool%29%2C%0A++++++++++++Return+%3D+%28%29%2C%0A++++++++%3E%2C%0A++++%7B%0A++++++++gen+move+%7B%0A++++++++++++let+mut+xs+%3D+self%3B%0A++++++++++++let+mut+x+%3D+Some%28xs.next%28%29%29%3B%0A++++++++++++while+let+CoroutineState%3A%3AYielded%28%28v%2C+ready_for_next%29%29%0A++++++++++++++++%3D+co.as_mut%28%29.resume%28x.take%28%29.flatten%28%29%29%0A++++++++++++%7B%0A++++++++++++++++if+let+Ready%28v%29+%3D+v+%7B+yield+v%3B+%7D%0A++++++++++++++++if+ready_for_next+%7B+x+%3D+Some%28xs.next%28%29%29+%7D%3B%0A++++++++++++%7D%0A++++++++%7D%0A++++%7D%0A%7D%0A%0Aimpl%3CI%3A+Iterator%3E+IteratorExt+for+I+%7B%7D%0A%0Afn+main%28%29+%7B%0A++++let+co+%3D+pin%21%28rl_encode%28%29%29%3B%0A++++for+x+in+%5B1u8%3B+513%5D.into_iter%28%29.then_co%28co%29+%7B%0A++++++++%2F%2F+++%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%5E%0A++++++++%2F%2F+++%5E+Produces+an+iterator+that+yields+the+run-length%0A++++++++%2F%2F+++++encoding+of+this+array.%0A++++++++println%21%28%22%7B%3A%3F%7D%22%2C+x%29%3B%0A++++%7D%0A%7D%0A
Sign in with Wallet
Connect another wallet