owned this note   changed a year ago  
Published Linked with GitHub

Goal planning for 2024

Attendance: TC, tmandry, eholk, Vincenzo, CE

Minutes: TC

Roadmap pre-planning

flowchart LR

MaybeAsync["Maybe Async (Trait Definitions)"]
MaybeAsyncInStd["Using MaybeAsync in traits in std"]
AsyncIterator["AsyncIterator"]
AsyncFnNext["async fn next"]
AsyncDrop["Async Drop"]
AsyncCancellation["Async Cancellation"]
AsyncGenerators["Async Generators"]
AsyncClosures["Async Closures"]
BecomingTeam["Becoming a Team"]
RTN["Return Type Notation"]
AFITDyn["Dyn dispatch for AFIT"]
ExecutorSpawn["Executor/Spawn traits"]
TAIT["Type alias impl Trait"]
FuturesConcurrency["futures-concurrency"]
ForAwait["for await syntax"]
AsyncSocketsAPI["Async sockets API"]
AsyncFileAPI["Async file API"]
DefaultAsyncRuntime["Default async runtime"]
AsyncMain["async fn main and #[test]"]
AsyncScopedTasks["Async scoped tasks"]
StructuredConcurrency["Structured concurrency"]
StructuredParallelism["Structured parallelism"]
Combinators2["Combinators 2.0"]
ProtocolPortability["Portability for protocol implementations"]
IouringSupport["Support for iouring"]
AsyncRead["AsyncRead, AsyncWrite, AsyncBufRead, etc."]

MaybeAsyncInStd -->|dep| MaybeAsync
AsyncIterator -->|dep| MaybeAsync
AsyncIterator -->|maybe dep| AsyncFnNext
AsyncFnNext -->|dep| TAIT
FuturesConcurrency -->|dep| AsyncIterator
AsyncCancellation -->|dep| AsyncDrop

Project board:

https://github.com/orgs/rust-lang/projects/28

Recent blog post from boats:

https://without.boats/blog/a-four-year-plan/

High-level goals

  • People can't write things they want to write
  • People have to use external crates for things that should be in std
  • Ecosystem crates can't interoperate well because we don't have foundations in std
  • Async is too hard? In what ways?
  • Portability for protocols
  • Expressiveness gaps
    • Where we define expressiveness as "the ability to express something at all", at least without going to extraordinary lengths

Big questions

Discussion

tmandry: The main priorities, to me, are:

  • Async closures
  • AsyncIterator and generators

There are probably open questions around those.

Async closures

eholk: What are the questions on async closures?

CE: We need first class support to handle lifetime issues so the futures can borrow from the state of the closure.

eholk: Remind me, where is this most needed?

CE: Anytime that you want to use something like an AsyncFnMut, you can't really express that.

tmandry: The main motivation for me is that AsyncFn allows us to iterate on new async APIs.

tmandry: In moro, e.g., we box everything since we don't have async closures.

eholk: In terms of effect generics, there has been skepticism that we can sprinkle async in all of the places that work with sync code. Having the ability to iterate on this would help build confidence.

TC: What are the next steps to getting this done. Is it implementation, an RFC, etc.?

CE: It's mostly implementation. We'll write an RFC that articulates what the implementation does.

CE: There is semantic ambiguity on e.g. impl Fn() -> impl Trait. We shouldn't block on this.

CE: There are some places where we accept variations here, but they're wrong, because in argument position these should be higher ranked.

TC: Where do we think this falls in terms of timeline?

CE: I'm hoping for something early next year. But we'll see. Niko is interested in this as well. It may even become part of my day job goals.

Asynchronous for syntax

TC: We might want to discuss the syntax here. People have already raised concerns about switching await to a prefix keyword for this case.

CE:

I don't think postfix .await is compelling for the same reasons .await makes sense and is compelling in expression position. Consider:

for Destructure { a, b }.await in async_iter() {}
                        ^^^^^^

fn async_iter() -> impl AsyncIterator<Item = Destructure> {}

for async { Destructor { a, b } } in async_iter() {}

I don't think this is well-formed:

for Destructure { async fut_field } in iter() {}

struct Destructure {
  fut_field: Pin<Box<Future>>
}

VP: This idea was discussed before? Or we discussed why it is not possible to do this?:

for Destructure { a, b } in async_iter().await {}

eholk: In Rust, there is the intro form and the elimination form.

TC: The point about adding and removing in patterns seems compelling. Followintg that logic, we would have, e.g.:

let &x = &X;
let async { x } = async { todo!() };
// or even...
let async x = async { todo!() };

CE: The async case is a bit different because of the arbitrary control flow.

TC: Deref patterns raise this same concern. We talked about these a bit yesterday with respect to changes to match ergonomics.

eholk: async in match patterns seems absurd… where do the side effects go?

match async_fn() {
    async { Some(x) } => ...,
    async { None } => ...,
}

CE: Matching needs to be pure. We shouldn't block on this question though. It's straightforward to change the syntax later during the experiment.

eholk: Consider:

match (async_one(), async_two()) {
    (async { x }, y) => ...,
    ( x, async { y }) => {
        // has x been awaited here?
    },
}

TC: So even if we never did async in patterns for all of these good reasons, what do we think about for async vs for await on the strength of the conceptual parallel that eholk raised?

eholk: All suspend points have an await keyword currently. for await preserves this, but for await does not.

TC: The nice thing, aside from the analogies that we discussed above, about for async is that we would preserve the prefixedness of async and the postfixedness of await.

Consensus: This needs further discussion and experimentation.

poll_progress

TC: What do we think about poll_progress?

tmandry: We have to solve this problem somehow. I'm concerned about it complicating the desugaring though.

CE: Who cares if the desugaring is complicated?

tmandry: I care about it for code size reasons.

CE: It's not significant. The IntoFuture question has larger concerns there.

TC: There may in fact be value in pushing poll_progress all the way back to Future. The actual problem here is with FuturesUnordered and other pseudo-executors rather than with Buffered. Any time you take an element from the FuturesUnordered stream and do something with it you starve all of the remaining futures in FuturesUnordered and there's no way to not starve them unless you're ready to take another element.

TC: If you were willing to have an arbitrary sized buffer, you could "solve" this, but then you would remove all backpressure. This is that tension between liveness and backpressure again.

TC: While this could be handled at the level of each of the pseudo-executors, pushing this all the way back to Future makes this more composable. Then you can put a BufferedFuture into the pseudo-executor. I have a prototype almost done of this.

TC: One thing I noticed in the prototype is that you do end up duplicating a lot of logic between poll_next and poll_progress.

CE: Right, because, you can't expect that when poll_progress returns Ready that you can call poll_next and unwrap the result.

TC: Exactly. So we should think about the semantics of poll_progress and whether stronger guarantees might make more sense.

tmandry: We almost want something like spawn_local, but that doesn't work on embedded. Is there a way to have a stack-local mini-executor?

eholk: It doesn't change the syntax or main semantics. Maybe we could handle this later.

CE: I wouldn't want to stabilize AsyncIterator / for await without fully considerting this question. While we could have a default implementation, people would start to rely on the behavior.

CE: We should at least actively consent to deferring it, if we did that, rather than passively doing it.

TC: There are two open issues that are pending on us:

"Tracking Issue for Ready::into_inner()" rust#101196

Link: https://github.com/rust-lang/rust/issues/101196

tmandry: It would be the only into_inner method that should panic.

TC: This gets into the name. Maybe it should be named unwrap.

eholk: My concern with this overall is that this breaks the futures abstraction.

CE: There are a lot of methods that poke through the thing when you know what the thing is.

CE: The argument maybe is whether it's a useful API.

eholk: That makes sense. And it is useful, as people pointed out in the thread.

TC: What about the name?

CE: dtolnay did address this in his comment. His point was that panicking is a property of the future.

TC: I'd probably just ask the question the other way around. If we had called this unwrap, what would be the argument against that?

Consensus: We need to discuss this further.

"Add LocalWaker support" libs-team#191

Link: https://github.com/rust-lang/libs-team/issues/191

Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/API-Change-Proposal.3A.20Local.20wakers

(Discussion…)

Consensus: We have no concerns about experimentation here, but we want to ask that this be nominated for us as it approaches stabilization.

Tracking Issue for task::Waker::noop - #98286

Link: https://github.com/rust-lang/rust/issues/98286

TC: It looks like I nominated this in the context of #101196.

eholk: I really want this for tests. But there's some danger of people misusing it.

TC: People could misuse it today. It's easy enough to write. E.g.:

const NOP_RAWWAKER: RawWaker = {
  fn nop(_: *const ()) {}
  const VTAB: RawWakerVTable =
    RawWakerVTable::new(|_| NOP_RAWWAKER, nop, nop, nop);
  RawWaker::new(&() as *const (), &VTAB)
};

(The meeting ended here.)

Select a repo