# Harmonic synthesis of the asynchronous vision document ![Bohehmian Rapsody](https://media.giphy.com/media/v0FUFlLbad7IA/giphy.gif) ## What is this document This document describes a sketch of the work required to make "joyful async". Each section maps to what would eventually become one "shiny future" story in the final document. They begin with a description of what it will feel like when this is done ("we'll know we've achieved it when you can...") and then have a list of the work that is required to get there ("requires"). In some cases, the work items are other sections of the document. The groupings are meant to follow the following heuristic: * They contain a minimal set of work items such that completing all of them produces an experience that is "greater than the sum of its parts". So e.g. type alias impl trait is very useful, as are GATs, but *together* they allow you to model `async fn` in traits, albeit in an unergonomic way. ## Draft shiny future ### Template (useful to copy and paste) * We'll know we've achieved it when you can... * Requires: ### Unergonomic async fns in traits * We'll know we've achieved it when you can... * Write non-dyn-safe traits that can have fns that return futures * Requires: * Type alias impl Trait (`type Foo = impl Trait`) usable in particular locations * Generic Associated Types ### Async fn everywhere * We'll know we've achieved it when you can... * Write `async fn` anywhere you can write `fn` * Requires: * *Unergonomic async fns in traits* * Support for `dyn Future` where `Future` has async fn * Async fn sugar in traits * Async closure support * Boxable / recursive async fns ### Async iteration is awesome * We'll know we've achieved it when you can... * Use Async iterators as easily as sync iterators * Write async and sync iterators with equal ease * Requires: * *Async fn everywhere* * AsyncIterator * Common combinators on `AsyncIterator` * Generators (both sync and async * "Iterator map when given an async closure yields an async iterator" or other similar niceties ### Async read and write are a pleasure to use * We'll know we've achieved it when you can... * Easy to pass around interoperable readers and writers * Easy to impl AsyncRead and AsyncWrite traits * Wrapping an AsyncRead or AsyncWrite is easy * Easy to write adapters that wrap async read and async write * Requires: * *Async fn everywhere* * Trait for AsyncRead * Trait for AsyncWrite * TBD: * Could be that we crack the async fn in traits dyn problem * Could be that we convert async read / async write to streams and generators * Could be that everyone switches to IO U-Ring and nobody cares about async read and async write anyway :) ### Portability across runtimes is possible * We'll know we've achieved it when you can... * Grab a library from crates.io and use it and it works with your chosen runtime easily, as long as the author does a good job * Possible to author libraries that can be used with many runtimes, but requires careful use of traits * Create a new runtime and have all existing (portable) libraries work with no modifications * Requires: * *Async iteration is awesome* * *Async read and write are a pleasure to use* * Trait for spawning tasks * Trait for spawn-blocking * Trait for timers * Utilities like `select`, `join`, async mutex, and friends are available in some standard way ### Retargetable async I/O constructs * We'll know we've achieved it when you can... * Use Tcp Streams and other constructs without baking in a specific runtime or implementation * Use Tcp Streams without threading generics all throughout your code * Requires: * TBD ### Portability across runtimes by default * We'll know we've achieved it when you can... * The easiest, most natural code to write is portable across runtimes * Requires: * *Portability across runtimes is possible* * *Retargetable async I/O constructs* * Able to write `async fn main` and `#[test]` with async fn * Able to run tests from dependencies with your runtime ### Better compiler diagnostics targeting async patterns * We'll know we've achieved it when you can... * compile async rust code and understand all the errors you get * be confident you've avoided common async footguns * Requires: * Improvements to diagnostics (Expand) * Lint holding things over an await that should not be held over an await * Lint for blocking functions in an async context * Lint for `poll_fn` that will take too long * Lint when potentially canceling "futures not known to be cancel safe" * Lint for sync dropping when there's an async close * Other lints... ### Joyful, jaw-dropping async documentation * We'll know we've achieved it when you can... * Search for "async Rust" on google, click "I feel lucky" and get a book that... * takes you from basic Rust knowledge to productive in async Rust * identifies scenarios where async is a good fit (and where it is not) * gives practical advice for common design patterns * gives detailed coverage of how async works on demand, but doesn't force you to learn it * Requires: * *Portability* * [...book toc...](https://hackmd.io/sz3uSs2nRP-Wczvt_W3F3Q) * Work on identifying common design patterns ### Async RAII * We'll know we've achieved it when you can... * add an async drop and be confident that it will be invoked without effort * reliably detect cases where sync drop might be used instead * Requires: * *Unergonomic async fns in traits* * Ability to write an async disposal method * Lint for sync dropping when there's async drop ### Sync-async "parity" * We'll know we've achieved it when you can... * Convert most sync code into async code by adding `async` in various places and tweaking some libraries in a straightforward way * Requires: * *Async iteration is awesome* * *Async fn everywhere* * *Portability* * *Async RAII* * Combinators and utilities on async traits: * futures * async read * async write * etc * Standard API for async I/O: * open tcp stream * ...other operations... ### Debugger integration * We'll know we've achieved it when you can... * Use a debugger to step through Rust code, print expressions, in a way that is roughly analogous to C++ code * Requires: * Better debuginfo -- needs elaboration ### Debugging tooling * We'll know we've achieved it when you can... * Find out what tasks your program currently has * Detect common "colored flags" like overly long poll functions, long time from wake to poll executing * Detecting what tasks are blocked on * Requires: * *Debugger integration* * Runtime:Debugger interface * Runtime bug detectors ### Profiling tooling * We'll know we've achieved it when you can... * Profiles of specific tasks (heap, memory, etc) * Overall profiles * Requires: * Debugging tooling ### First-class learning experience * We'll know we've it when you can... * When async doesn't work as I expect (whether at compilation time, runtime, debugging)... * something identifies the problem * something explains the problem * something proposes solutions * after reading the explanation and the solutions, I understand what I did wrong * Requires: * *Portability across runtimes by default* * *Better compiler diagnostics targeting async patterns* * *Joyful, jaw-dropping async documentation* * *Debugging tooling* * Runtime warnings in debug mode * Cross-referencing between docs, lints, errors, and so forth ### Portability across Send * We'll know we've achieved it when you can... * write a library that can be used in a Send or not Send context as a "zero-cost abstraction" * that library uses Rc or Arc transparently, for example * Requires: * *Unergonomic async fns in traits* * associated traits, most likely ### If it compiles, it works * We'll know we've achieved it when you can... * "Bugs are generally logic bugs, not a result of surprising async mechanisms", * "Easy to create parallelism operating on borrowed data", * "When tasks are no longer needed, they can be reliably canceled", * "It is easy to visualize your program's task structure", * Requires: * Way to avoid tasks being dropped unexpectedly while they continue to execute * Mechanism for launching tasks within those scopes that can reference borrowed data * Hierarchical structure for tasks with cancelation propagation * cancel the parent: cancel the children * Integration into the visualization infrastructure and debug tools ### Zero copy works beautifully * We'll know we've achieved it when you can... * schedule memory read/writes with an `&`/`&mut` and have it get read/written in parallel * Requires: * *If it compiles, it works* * some way to have guaranteed execution * new traits, probably ### Mocking runtime environments and testing is easy * We'll know we've achieved it when you can... * take code that you wrote and write tests for it without a lot of pre-planning * Requires: * *Retargetable async I/O constructs* * Ability to fire timers programmatically (mock time) * `run_until_stalled` * Mock sockets and file objects? * Simulate I/O layers * Inject errors or long delays * Inject fake objects for real ones without modifying your code ### Long-running sequential loops are easy to find and remedy * We'll know we've achieved it when you can... * have tooling that helps you identify long-running loops * have mechanisms and design patterns you can use to address them * Requires: * *Profiling tooling* * *Footgun-free control flow* because you may have references you have to deal with * Integration of profiling output into lint tools * Combinators and hooks to make it easy to yield in long-running loops * Highly optimized `spawn_blocking` ## Out of scope These items would be nice, but we have *no idea* how to do them right now, or if they are even a good idea. Perhaps as we make progress, we'll have a clearer picture. ### Transitioning between sync and async code is easy * We'll know we've achieved it when you can... * have an async-sync-async sandwich that "generally works" and avoids deadlock * Requires: * TBD, we may never solve this ### Long-running sequential loops don't cause horrible performance * We'll know we've achieved it when you can... * have a long running loop and avoid stalls * Requires: * TBD, we may never solve this