TC0
    • Create new note
    • Create a note from template
      • Sharing URL Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Customize slides
      • Note Permission
      • Read
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Write
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Engagement control Commenting, Suggest edit, Emoji Reply
    • Invite by email
      Invitee

      This note has no invitees

    • Publish Note

      Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

      Your note will be visible on your profile and discoverable by anyone.
      Your note is now live.
      This note is visible on your profile and discoverable online.
      Everyone on the web can find and read all notes of this public team.
      See published notes
      Unpublish note
      Please check the box to agree to the Community Guidelines.
      View profile
    • Commenting
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
      • Everyone
    • Suggest edit
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
    • Emoji Reply
    • Enable
    • Versions and GitHub Sync
    • Note settings
    • Note Insights
    • Engagement control
    • Transfer ownership
    • Delete this note
    • Save as template
    • Insert from template
    • Import from
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
    • Export to
      • Dropbox
      • Google Drive
      • Gist
    • Download
      • Markdown
      • HTML
      • Raw HTML
Menu Note settings Versions and GitHub Sync Note Insights Sharing URL Create Help
Create Create new note Create a note from template
Menu
Options
Engagement control Transfer ownership Delete this note
Import from
Dropbox Google Drive Gist Clipboard
Export to
Dropbox Google Drive Gist
Download
Markdown HTML Raw HTML
Back
Sharing URL Link copied
/edit
View mode
  • Edit mode
  • View mode
  • Book mode
  • Slide mode
Edit mode View mode Book mode Slide mode
Customize slides
Note Permission
Read
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Write
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Engagement control Commenting, Suggest edit, Emoji Reply
  • Invite by email
    Invitee

    This note has no invitees

  • Publish Note

    Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

    Your note will be visible on your profile and discoverable by anyone.
    Your note is now live.
    This note is visible on your profile and discoverable online.
    Everyone on the web can find and read all notes of this public team.
    See published notes
    Unpublish note
    Please check the box to agree to the Community Guidelines.
    View profile
    Engagement control
    Commenting
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Suggest edit
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    Emoji Reply
    Enable
    Import from Dropbox Google Drive Gist Clipboard
       owned this note    owned this note      
    Published Linked with GitHub
    Subscribed
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    Subscribe
    --- title: "Type Alias impl Trait (TAIT) Stabilization (Mini Design Meeting 2023-06-29)" tags: mini-design-meeting date: 2023-06-29 url: https://hackmd.io/r1oqcjrzTAK5e_T1IOXeXg --- # Type Alias impl Trait (TAIT) Stabilization (Mini Design Meeting 2023-06-29) ## Appendix C: Why constraining through encapsulation is necessary The ability to constrain the hidden type after it has been encapsulated in other types is a central part of this proposal. In the sections below, we'll show why this is necessary both conceptually and practically. ### Concept: The details of a good abstraction can be (mostly) forgotten This document has spent a lot of words discussing what it means to constrain the hidden type, when that's allowed, and how that is different from not constraining it. We do this because we need to specify this precisely for our purposes in designing and implementing a language. However, in day to day use of this feature, we believe that users will not be thinking about these distinctions carefully and that they should not need to do so. This proposal makes a number of tradeoffs in pursuit of the goal that most of the code that users will want to write should just work without having to carefully consider the details of this mechanism. As we'll see in the following sections, constraining through encapsulation is necessary to achieve this. ### Symmetry with `dyn Trait` The `impl Trait` feature has long been thought of as a statically-dispatched version of `dyn Trait`. For example, [RFC 1951] suggests that: > [Extending the use of `impl Trait` to more places] provides a greater degree of analogy between static and dynamic dispatch when working with traits. Introducing trait objects is easier when they can be understood as a variant of impl Trait, rather than a completely different approach. There is a large body of existing Rust code that today uses boxing and `dyn Trait` to solve the kinds of problems discussed in the motivations above. We expect that much of this code will migrate to use TAIT to take advantage of static dispatch. One goal of this proposal is to minimize the amount of refactoring, churn, and careful thinking that is necessary to accomplish that. Consider, for example, how a simplified version of [Motivation 1](#Motivation-1-Closing-the-expressiveness-gap-on-unnameable-types) might be written today using `Box<dyn Trait>`: ```rust // This is a simplified version of the example in Motivation 1. type JobFut = Box<dyn Future<Output = ()>>; struct Job(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, Box::new(async { todo!() }))).await; } ``` To migrate this code to `impl Trait`, the user simply replaces the `Box<dyn Trait>` with `impl Trait` and removes the boxing: ```rust type JobFut = impl Future<Output = ()>; struct Job(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, async { todo!() })).await; } ``` Without the ability to constrain the hidden type through encapsulation, this code would need to be significantly reworked. But worse, the user would have to think carefully about why this seemingly simple substitution didn't work. We argue that in the common case, users should not have to think more carefully about the difference between using and constraining an `impl Trait` than they do about the difference between using and assigning to an abstract `dyn Trait`. ### Robustness to code evolution, principle of least surprise Without the ability to constrain through encapsulation, seemingly trivial changes to code would require the user to refactor code for reasons that may not be obvious. For example, consider that the user first wrote this simple version of the job queue where each `Job` is only a future: ```rust type Job = impl Future<Output = ()>; // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(async { todo!() }).await; } ``` This works and the user is happy. Then the user realizes that every job in the queue needs an identifier, so the user adds one using a tuple in the type alias: ```rust type Job = (u64, impl Future<Output = ()>); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send((0u64, async { todo!() })).await; } ``` This still works and the user is happy. But then, during code review, the reviewer suggests that it would be more idiomatic and better long-term to make the `Job` into a `struct`. The user first tries to convert the type alias to a `struct`: ```rust struct Job(u64, impl Future<Output = ()>); // ^---- Error: We don't support `impl Trait` everywhere yet. // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, async { todo!() })).await; } ``` That doesn't work. We don't support `impl Trait` everywhere yet. So the user extracts the `impl Trait` into a type alias, then uses that type alias in the `Job` `struct`: ```rust type JobFut = impl Future<Output = ()>; struct Job(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, async { todo!() })).await; } ``` If we were to not support constraining through encapsulation, this code wouldn't work either. We claim that this violates the principle of least surprise. Even worse, if we were to not permit this code, it may not be possible to emit an error message that would guide the user to the set of tricks that the user would have to employ instead. We detail these tricks below and the problems with them. ### Making the human into a compiler To solve problems like the one above without constraining through encapsulation, the user must think like a compiler and desugar the code until the type alias is returned by name from a function. Consider this variation on our job queue with some non-trivial input: ```rust type JobFut = impl Future<Output = (u8, u8)>; struct Job(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>, data: [u8; 16]) { // Without loss of generality, assume that the function // transforms its inputs in a non-trivial way. let data1: [u8; 8] = data[0..8].try_into().unwrap(); let data2: [u8; 8] = data[8..16].try_into().unwrap(); let mut feeder = data1.into_iter().zip(data2.into_iter()); // It uses the transformed inputs itself. let id = feeder.next().unwrap().0 as u64; // And the hidden type captures these transformed inputs. tx.send(Job(id, async move { feeder.next().unwrap() })).await; } ``` To make this idea work without constraining through encapsulation, the user must refactor the code until each type alias is returned by name from some function. Any state needed to construct the value for the hidden type must be passed down the stack, which will require repeating the names of those types or naming types that were previously entirely inferred because the values were created and used within one function body. Here's how we might need to "compile" the code above without the ability to constrain through encapsulation: ```rust type JobFut = impl Future<Output = (u8, u8)>; struct Job(u64, JobFut); // We have to manually desugar an `async fn` here // and name the type of the `Iterator`. fn make_future( // We need to pass down all of the transformed inputs from // `send_job` that need to be captured by the hidden type. mut feeder: std::iter::Zip< std::array::IntoIter<u8, 8>, std::array::IntoIter<u8, 8>, >, ) -> JobFut { // The body must be wrapped in an `async` block. async move { feeder.next().unwrap() } } // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>, data: [u8; 16]) { let data1: [u8; 8] = data[0..8].try_into().unwrap(); let data2: [u8; 8] = data[8..16].try_into().unwrap(); let mut feeder = data1.into_iter().zip(data2.into_iter()); let id = feeder.next().unwrap().0 as u64; tx.send(Job(id, make_future(feeder))).await; } ``` The first thing to note here is that we would not be able to use `async fn` when performing this manual compilation. Since we would need the type alias to appear explicitly by name in the return value, we would have to manually desugar `make_future` into a synchronous function with its body wrapped in an `async` block. We've been making efforts in other areas to reduce the need for users to desugar `async` functions, and this would seem to pull in the opposite direction from that. The second thing to note is that we needed to name a complicated `Iterator` type to make this work. In the original code, we had avoided naming this type. We believe that it's likely in this scenario that a user may try to use `impl Trait` again, in `make_future`, to avoid naming the `Iterator`, as follows: ```rust // Can we use `impl Iterator` rather than naming the complex type? fn make_future(mut feeder: impl Iterator<Item = (u8, u8)>) -> JobFut { async move { // ^--- Error: type parameter `impl Iterator<Item = (u8, u8)>` // is part of concrete type but not used in parameter list // for the `impl Trait` type alias feeder.next().unwrap() } } ``` Using `impl Trait` in argument position introduces a generic parameter that the type alias must capture. The compiler tells the user that this can work, but that the hidden type must capture a type parameter. Following the directions of the error message, we suspect the user may next try this: ```rust // We added a type parameter like the error message told us to do. type JobFut<I: Iterator<Item = (u8, u8)>> = impl Future<Output = (u8, u8)>; struct Job<I: Iterator<Item = (u8, u8)>>(u64, JobFut<I>); // We can't use `impl Trait` in argument position because we // need to name the type parameter so as to pass it through to // the type alias. fn make_future<I>(mut feeder: I) -> JobFut<I> where I: Iterator<Item = (u8, u8)> { async move { feeder.next().unwrap() } } // Now we need a named type parameter for the `Job` struct, but // this doesn't work because we're choosing the type, not the caller. async fn send_job<I>(tx: Sender<Job<I>>, data: [u8; 16]) where I: Iterator<Item = (u8, u8)> { let data1: [u8; 8] = data[0..8].try_into().unwrap(); let data2: [u8; 8] = data[8..16].try_into().unwrap(); let mut feeder = data1.into_iter().zip(data2.into_iter()); let id = feeder.next().unwrap().0 as u64; tx.send(Job(id, make_future(feeder))).await; // ^---- Error: mismatched types. } ``` At this point, hopefully, the user will realize that this isn't going to work. Even if it did, it would require polluting all encapsulating types and all code up the stack with a type parameter that isn't actually needed. But this can't work here because the callee is choosing the type of the `Iterator`, not the caller. So instead, the user would have to ignore the error message and think to use TAIT again to name the type of the argument that needs to be passed down the stack: ```rust type JobFut = impl Future<Output = (u8, u8)>; struct Job(u64, JobFut); // We use TAIT again to name the type that needs // to be passed down the stack. type JobIterArg = impl Iterator<Item = (u8, u8)>; // We have to manually desugar an `async fn` here // and name the type of the Iterator using TAIT. fn make_future(mut feeder: JobIterArg) -> JobFut { async move { feeder.next().unwrap() } } // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>, data: [u8; 16]) { let data1: [u8; 8] = data[0..8].try_into().unwrap(); let data2: [u8; 8] = data[8..16].try_into().unwrap(); let mut feeder = data1.into_iter().zip(data2.into_iter()); let id = feeder.next().unwrap().0 as u64; tx.send(Job(id, make_future(feeder))).await; } ``` To pass the data down the stack as would be required without constraining through encapsulation, the user needed to add an extra type alias `impl Trait` to avoid naming the complicated `Iterator` type. We argue that forcing the creation of more type aliases with hidden types just to satisfy the compiler does not improve code clarity. ### The tricks above lead to cycle errors The tricks above often result in rearranging code in ways that are more prone to fundamental cycle errors. Following our motivating example, let's assume that `Sender` requires its type parameter to implement `Send`. It's implemented in a separate crate as follows: ```rust pub struct Sender<T>(PhantomData<T>); impl<T: Send> Sender<T> { pub async fn send(&self, x: T) { todo!() } } ``` Consider our motivating example, but now with type parameters that are captured by the hidden type. The `Send`ness of `JobFut<T>` is covariant with the `Send`ness of `T` due to auto trait leakage: ```rust // JobFut<T> will be `Send` only if `T: Send`. type JobFut<T> = impl Future<Output = ()>; struct Job<T>(u64, JobFut<T>); // This function constrains the hidden type, so it's OK for us // to reveal the auto traits of that hidden type within this // function. async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) { tx.send(Job(0u64, async move { let _x = x; () })).await; } ``` Under the rules of this proposal, the compiler is allowed to accept this code. The function that constrains the hidden type also checks whether that hidden type implements `Send`. That is allowed. However, if we were to use the trick from the last section, we end up with code that *must* be rejected: ```rust // JobFut<T> will be `Send` only if `T: Send`. type JobFut<T> = impl Future<Output = ()>; struct Job<T>(u64, JobFut<T>); // This function constrains the hidden type. fn make_job<T>(x: T) -> JobFut<T> { async { let _x = x; () } } // This function does not constrains the hidden type, but since // it's within a scope that is allowed to constrain it, under // the rules of this proposal it may not check the leaked auto // traits of the hidden type. async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) { tx.send(Job(0u64, make_job(x))).await; } ``` That code will result in a cycle error. This cycle error is fundamental in the sense that it's inherent and necessary under the rules of the signature restriction. As with other cycle errors, we have to work around this by moving the hidden type and its constraining use into a more narrow scope: ```rust mod jobfut { // JobFut<T> will be `Send` only if `T: Send`. pub(crate) type JobFut<T> = impl Future<Output = ()>; pub(crate) struct Job<T>(pub(crate) u64, pub(crate) JobFut<T>); // This function constrains the hidden type. pub(crate) fn make_job<T>(x: T) -> JobFut<T> { async { let _x = x; () } } } use jobfut::{Job, make_job}; // This function is allowed to check for the leaked auto traits // of the hidden type since it is not in a scope that's allowed // to constrain the hidden type. async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) { tx.send(Job(0u64, make_job(x))).await; } ``` We argue that this is a lot to ask from users when the obvious code could "just work". ### ATPIT would require a different trick If we were to not support constraining through encapsulation, `impl Trait` in associate type position would require a different trick. Consider this implementation of `IntoIterator` in which the `Item` associated type uses `impl Trait`: ```rust struct S; impl IntoIterator for S { type Item = impl Future<Output = ()>; type IntoIter = std::iter::Once<Self::Item>; fn into_iter(self) -> Self::IntoIter { std::iter::once(async {}) } } ``` Without constraining through encapsulation, the user would need to know to "alpha-expand" the method's return type until the "correct" associated type appeared explicitly by name in the return type: ```rust struct S; impl IntoIterator for S { type Item = impl Future<Output = ()>; type IntoIter = std::iter::Once<Self::Item>; fn into_iter(self) -> std::iter::Once<Self::Item> { std::iter::once(async {}) } } ``` We argue that this may not be obvious and that it would be less idiomatic. Because the return type does not match the standard presentation of the trait by using `IntoIter`, it might look like a refinement at first glance even though it is not (pedantically: it's not any more of a refinement than associated types are inherently). ### ATPIT needs constraining through encapsulation For ATPIT, it's not always possible for all of the associated type aliases with hidden types to appear explicitly by name in the signature of the method that constrains them. Consider this implementation of `IntoFuture` where both the `Output` and `IntoFuture` associated types use `Impl Trait`: ```rust struct S; impl IntoFuture for S { type Output = impl Future<Output = ()>; type IntoFuture = impl Future<Output = Self::Output>; fn into_future(self) -> Self::IntoFuture { async { async { () } } } } ``` It's not possible to rewrite this example in such a way that both `Self::Output` and `Self::IntoFuture` appear in the signature of `into_future`. Consequently, ATPIT requires some form of constraining through encapsulation so as to not restrict the fundamental expressiveness of the language. ### The grepping concern Constraining through encapsulation means that to find where a hidden type is constrained using grep may require more care. We would need to "walk" through other types that encapsulate the hidden type. Using our job queue example: ```rust type JobFut = impl Future<Output = ()>; // Grep for `JobFut`. struct Job(u64, JobFut); // ...and grep for `Job`. // We would find this function by grepping for `Job`. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, async { todo!() })).await; } ``` To use grep to find where the hidden type might be constrained, we would need to search both for appearances of `JobFut` and `Job`. We argue that this is fine for the reasons that follow. #### Using grep is still possible To find all uses of the hidden type with grep, the user would follow this algorithm: 1. Add the name of the innermost type alias containing the hidden type to the search list. 2. Search for all tokens in the search list. 3. Whenever a data structure is found, add it to the search list and go to step 2. Though it may look overwrought when written out explicitly, this process is common and required more often than not to search code thoroughly with grep (and is one reason that people use IDEs). (As we'll demonstrate [below](#Constraining-uses-can-be-found-easily-without-grep), even without IDE support, using grep is *never* needed to find the constraining uses for a hidden type.) #### It's not any worse than `Box<dyn Trait>` Any time that an abstract type is encapsulated in other types, we may need to look through uses of those other types to find the concrete type used for the abstract type. For example, here's the code that someone might write today using `Box<dyn Trait>`: ```rust type JobFut = Box<dyn Future<Output = ()>>; struct Job(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, Box::new(async { todo!() }))).await; } ``` To find the concrete types that may underlie the abstract type represented by `JobFut`, we must consider all uses of any types that may encapsulate it, such as `Job` in this example. We argue that this has all of the same problems in terms of grep as the `impl Trait` version. #### The restriction would force an unlikely cost to always be paid To use grep, the user may have to unwind some encapsulation. But if we were to forbid constraining through encapsulation, then users would *always* have to unwind this encapsulation just to write code that the compiler would accept (as described in the "tricks" above). With the permissive approach, the cost will rarely be paid as there are [better](#Constraining-uses-can-be-found-easily-without-grep) ways to find all constraining uses of any hidden type. With the restrictive approach, the cost would *always* have to be paid. Furthermore, with the permissive approach, the maximum cost is that the user must iteratively build a search list when choosing to use grep. With the restrictive approach, the cost would be potentially-tricky and elaborate code refactoring. #### On the width of constraining scopes One of the reasons that people have worried about being able to use grep to find where a hidden type might be constrained is that the rule allowing child modules to constrain the hidden type might in theory result in wide constraining scopes. We believe that in practice the scopes for constraining `impl Trait` types will be narrow because of the effect of cycle errors. The presence of the cycle errors required by this proposal will encourage best practices and coding guidelines to favor moving `impl Trait` hidden types to the narrowest possible scope. When we teach people this feature, we will encourage them to adopt the narrowest possible scope because that will minimize the chances that these users will encounter cycle errors. When teaching users how to handle cycle errors, we will say things such as: > Modules in Rust are cheap. If you encounter a cycle error, move the hidden type to a more narrow scope, creating a new child module if needed. Keeping the `impl Trait` in the narrowest possible scope is a good best practice so that you don't later stumble over these cycle errors when making changes to your code. To whatever degree that education, best practices, and coding guidelines were not enough, we believe that this could be adequately addressed by linting. Clippy today lints about functions that it scores as being too long or complex. It would certainly be possible for Clippy to lint about a hidden type that is introduced in "too wide" of a scope. #### Constraining uses can be found easily without grep Grep is never needed to find all constraining uses of a hidden type. The better option, aside from IDE support, is to simply provoke a compiler error. For example, to find all constraining uses of `JobFut` in the code above, we add a bogus constraint that will conflict with any other constraining use: ```rust // Let's find all constraining uses of `JobFut` // by adding a bogus constraint. fn nop() -> JobFut { async {} } ``` This will provoke a compiler error that will point us to exactly what we want: ``` error: concrete type differs from previous defining opaque type use --> src/lib.rs:16:13 | 16 | tx.send(Job(0u64, async { todo!() })).await; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `[async block@src/lib.rs:12:22: 12:30]`, got `[async block@src/lib.rs:16:23: 16:40]` | ``` Provoking an error to extract information from the compiler is a common technique used by Rust programmers, so this is not out of the ordinary. (For example, users often write `let _: () = ...` to query for type information.) #### Conflicts are the main reason to look for constraining uses We believe that the main practical reason that a user would want to find the constraining uses for a hidden type is because the user has written a conflicting constraint. Whenever that happens, neither grep nor any special action on the part of the user is required to find the constraining uses. The compiler will point the user to exactly the right places. ### Regarding `impl Trait` everywhere This proposal only seeks to stabilize `impl Trait` in type aliases and in associated type position. However, we argued above in the main body of this proposal that constraining through encapsulation is required to avoid surprising behavior if we were to support `impl Trait` inside of `struct`s or `enum`s. In this section, we provide more discussion of this claim. Consider a version of our job queue that uses `impl Trait` everywhere: ```rust struct Job(u64, impl Future<Output = ()>); // Not part of this proposal. // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<Job>) { tx.send(Job(0u64, async { todo!() })).await; } // This function takes a job out of a channel and processes it. async fn process_job(rx: Receiver<Job>) { let job = rx.recv().await.unwrap(); job.await; } ``` If we were to forbid constraining through encapsulation, presumably the above code would still work because the innermost named type that contains the hidden type appears explicitly by name in the signature. However, now imagine that we perform this seemingly trivial modification because we want to use the hidden type in two different `struct`s: ```rust type JobFut = impl Future<Output = ()>; struct InnerJob(u64, JobFut); struct OuterJob(u64, JobFut); // This function creates a future and sends it out a channel. async fn send_job(tx: Sender<InnerJob>) { tx.send(Job(0u64, async { todo!() })).await; } // This function takes a job out of a channel and processes it. async fn process_job(rx: Receiver<InnerJob>) { let job = rx.recv().await.unwrap(); job.await; } // ...elided code that uses `OuterJob`. ``` Without constraining through encapsulation, the above code now would not work. The user would need to refactor `send_job` using the tricks above. We argue that the need to do this would be surprising and the reasons this would be required would be unintuitive. Currently in Rust it's always legal to replace a type in a `struct` or `enum` with an equivalent type alias. The authors believe it would be difficult to explain to our colleagues why this was no longer the case. Note that `process_job` would *not* need to be refactored using one of the tricks. We argue that without carefully considering the mechanics of `impl Trait`, it would not be obvious why `send_job` needs to be refactored but `process_job` does not. We argue that forcing the user to be so aware that `send_job` is *more special* than `process_job` decreases the value of the abstraction. #### Constraining the `Self` type If we were to forbid constraining through encapsulation while supporting `impl Trait` everywhere, then depending on the exact rules, it may become difficult to constrain a hidden type within a `struct` or `enum` from a method on that type. Consider if we were to add methods to our job queue: ```rust struct Job(u64, impl Future<Output = ()>); // Not part of this proposal. impl Job { // This method constrains the hidden type. fn prepare_job(&mut self) { self.0 = 0u64; self.1 = async { todo!() }; } // ...other methods... } ``` If we were to allow this code because `self` appears even though the containing type does not appear explicitly by name in the signature, then that would re-introduce a certain complexity when trying to grep for where the hidden type is constrained, as discussed [above](#The-grepping-concern). However, if this were not allowed, it would be one more place where special tricks would be required. Note too that this issue comes up even without `impl Trait` everywhere. Under the current proposal, we could equivalently write the following code by moving the hidden type into a type alias: ```rust type JobFut = impl Future<Output = ()>; struct Job(u64, JobFut); impl Job { // This method constrains the hidden type. fn prepare_job(&mut self) { self.0 = 0u64; self.1 = async { todo!() }; } // ...other methods... } ``` We argue that it would be surprising if this variant did not work but that (in a world with `impl Trait` everywhere) putting the `impl Trait` directly into the `Job` struct did. #### Potential future note: enum-variant types There's an early proposal for Rust to support [enum-variant types]. Such a proposal would allow for the variants in an `enum` to be used directly as types. When combined with `impl Trait` everywhere, this code would become legal: ```rust enum Job { // Not part of this proposal. Pending(impl Future<Output = ()>), Backlog(impl Future<Output = ()>), } // Definitely not part of this proposal. async fn send_job(tx: Sender<Job::Pending>) { tx.send(Job::Pending(async { todo!() })).await; } ``` This would add another wrinkle to the rules if we were to forbid constraining through encapsulation. Given that the innermost named type that contains the hidden type would now be the *variant*, allowing only the name of the `enum` to appear in the signature would itself become a form of constraining through encapsulation. Such a future is very speculative and we do not mean to suggest that much weight should be placed on this. But we do claim this may be suggestive that constraining through encapsulation is robust to the ways that Rust might likely evolve in the future, and that a more restrictive rule might start to accumulate annoying edge cases as the language evolves. [enum-variant types]: https://internals.rust-lang.org/t/prerfc-enum-variant-types/18984 ### Regarding optional `#[defines(..)]` It has been proposed that we could in the alternative allow `#[defines(..)]` to be used in cases where the name of the innermost type alias containing the hidden type is not present in the signature of the function. Such a design would ameliorate some but not all of the problems described above. In particular, we would still be: - Rejecting otherwise valid code that could in principle be made to "just work". - Creating divergence between `impl Trait` and `dyn Trait`. - Forcing users to pay the cost upfront to unwind their abstractions. - Forcing users to think carefully about the mechanism of `impl Trait` and the difference between using an `impl Trait` type and constraining it in a way that they don't have to do when using `dyn Trait`. - Forced to teach users about *both* the signature restriction and `#[defines(..)]`. Additionally, depending on how the rules of this new syntax were defined, the user may still have to engage in certain kinds of careful refactoring (such as breaking out hidden types into multiple type aliases) just to satisfy the compiler. Because under this alternative `#[defines(..)]` would be optional and the hidden types would still be anonymous, we would be paying these costs without getting any of the benefits that would accrue to a fundamentally different design strategy. From a process perspective, any `#[defines(..)]`-like mechanism would require careful design to address the exact handling of nits such as the behavior of type and lifetime parameters within the annotation. The authors believe this would require a new RFC. There is always the possibility that building consensus on such an RFC could take a long time or might never happen at all. During this time, if we were to forbid constraining through encapsulation, users would be writing and refactoring code according to the tricks and workarounds described in this document. We argue that this would cause unnecessary effort by users and unnecessary churn within the ecosystem. ### Concept: The minimum required restriction Neither [RFC 2071] nor [RFC 2515] envisioned any restrictions at all on which functions within the scope in which the hidden type was introduced could constrain it. These RFCs implicitly allowed all of the code whose legality is preserved by constraining through encapsulation. Due to concern over implementation considerations in the compiler and IDEs, as discussed in [Appendix A](#Appendix-A-Signature-restriction-details) and [Appendix B](#Appendix-B-The-IDE-concern), we have adopted the signature restriction. The details of this restriction, including constraining through encapsulation, have been carefully designed to fully address these implementation considerations while still supporting the use cases described in the [motivations](#Motivation-1-Closing-the-expressiveness-gap-on-unnameable-types) above and allowing most of the useful code that people would want to write using this feature to "just work". The signature restriction as proposed in this document represents the minimum restriction required to get these implementation benefits. We argue that this approach is the most congruent with the intent of the accepted RFCs. We argue that further restrictions that would make more otherwise correct code illegal would serve a fundamentally stylistic purpose, and that such a purpose can be better addressed by education, best practices, coding guidelines, and linting. [RFC 2071]: https://github.com/rust-lang/rfcs/blob/master/text/2071-impl-trait-existential-types.md ### Tradeoffs This proposal makes a set of design choices to achieve these goals: - We want people to be able to replace the large body of existing `Box<dyn Trait>` code with `impl Trait` whenever they were using the former as a workaround. - We want that replacement to be easy and to result in minimal churn to existing code. - We want these and other common expected uses of this feature to "just work" as much as possible. - We want people to be able to rely on the abstraction and mostly forget about the mechanics. - We want people to be able to mostly apply their intuitions about `dyn Trait` to `impl Trait` and vice versa. The ability to constrain the hidden type through encapsulation works together with the other design choices in this proposal (such as the other details of the signature restriction) to achieve these goals. These goals lead us to design choices that incur certain costs. The most notable of these costs are the presence of cycle errors in certain (hopefully not-too-common) scenarios that must be worked around. If we were to abandon these goals -- in particular, if we wanted to force users to be very aware of the mechanism and make it very different from `dyn Trait` -- then essentially all of the choices in this proposal -- and perhaps even those within [RFC 2071] and [RFC 2515] -- should be reconsidered. Forbidding constraining through encapsulation would incur the many costs described above without getting any of the benefits that might accrue by reconsidering all of the design choices with a different set of goals. ## Appendix D: The signature restriction on nested inner functions [Appendix D]: #Appendix-D-The-signature-restriction-on-nested-inner-functions This proposal requires that: - Nested functions may not constrain a hidden type from an outer scope unless the outer function also includes the hidden type in its signature. For example, in the proposal, this code is invalid: ```rust // Example A. type Foo = impl Sized; fn foo() { fn bar() -> Foo { () } } ``` However, it has been suggested that this restriction feels surprising and inconsistent with the rule that allows child modules to constrain the hidden type. In the remainder of this appendix, we'll discuss the tradeoffs involved in deciding whether this restriction should be kept or discarded. ### Sneaky inner impls in Rust Consider this code that stable Rust allows today: ```rust // Example B. trait Trait {} struct S; fn foo() { impl Trait for S {} } ``` The `impl` within function body of `foo` affects the broader scope. We could say that it does this "sneakily" because it does so without mentioning anything about what it plans to affect in its signature. Likewise, without the nested inner function restriction, we could say the same about `foo` in Example A above. ### Why is this a problem? As long as either Example A or Example B are legal in Rust, tools such as rust-analyzer must parse all function bodies to be sure that they have correctly inferred any type. The authors of these tools would prefer that this were not necessary as that could enable better performance while achieving correct behavior. ### How might this be addressed? There is a [draft proposal][restrict-sneaky] that would make these sneaky inner impls illegal in a future edition of Rust. If that proposal were adopted, then it would be logical for the nested inner function restriction to apply to TAIT. ### What has @matklad said about this? Regarding these sneaky inner impls, why they are an issue, and what to do about them, @matklad has [said](https://github.com/rust-lang/rfcs/pull/3373#issuecomment-1406888432): > I guess my overall thrust here is that, if we do this, its important to do 100% of it. We don't want to forbid this because it's an unreadable mess: no need to forbid something which isn't used anywhere outside of dtolnay's famous quiz. We want to forbid this, because we want to make the implementation faster. In some future edition, we want to just [throw] every function onto a threadpool and typecheck the world on all cores, without any synchronization. If there's at least one case where we need to invalidate typechcking results of `foo` when we learned something new when checking `bar`, this optimization simply doesn't work. The authors have confirmed with @matklad that Example A above is the same problem as the other sneaky impls and that his statement applies to this case as well. ### What decision do we have to make? If we were sure that we were going to adopt the [draft proposal][restrict-sneaky] to restrict sneaky impls in the next edition, then it might make sense to adopt the nested inner function restriction with the TAIT stabilization to avoid adding a new case that must be addressed over an edition. On the other hand, even if we were planning to adopt that proposal, we could take the view that it might still be more consistent to simply allow this in this edition and restrict it when the other cases are restricted. This may be even more true if we're not sure that we will in fact adopt the proposal, or if we're not sure that we will adopt it for the 2024 edition. [restrict-sneaky]: https://github.com/rust-lang/rfcs/pull/3373 ## Appendix E: Nested inner modules This proposal allows the hidden type to be constrained within a child scope, such as a child module, subject to the signature restriction for nested inner functions as discussed in [Appendix D]. However, it has been suggested that this affordance feels inconsistent with the restriction on nested inner functions. In the alternate, it has been suggested that this affordance is too powerful that might result in too wide of a scope in which the hidden type may be constrained. In the sections that follow in this appendix, we'll discuss these concerns. ### Restricting child modules would be inconsistent If we were to forbid constraining the hidden type in child modules, that would be inconsistent with the proposed rule on nested inner functions. The rule on those is that while this code is illegal, due to the signature restriction: ```rust type Foo = impl Sized; fn foo() { fn bar() -> Foo { () } } ``` This code is legal: ```rust type Foo = impl Sized; fn foo() -> Foo { fn bar() -> Foo { () } bar() } ``` That is, we allow nested inner functions to constrain the hidden type *only if* they and all outer functions pass the signature restriction. If we were to forbid constraining the hidden type in child modules, that would be a more severe restriction as there would be no corresponding way to make it work. ### Restricting child modules has no tooling benefits As discussed in [Appendix D] (the signature restriction on nested inner functions), [Appendix A] (the signature restriction), and [Appendix B] (the IDE concern), the signature restriction was introduced to facilitate a simpler and more performant implementation and to ease other tooling concerns. Forbidding the hidden type from being constrained in child modules has no analogous tooling benefits. ### Child modules and cycle errors People may hit cycle errors when using type alias `impl Trait`. When they do, we will suggest that they move the hidden type to a more narrow scope. If there are multiple interacting types using `impl Trait`, then a restriction on constraining the hidden type in a child module would impose additional constraints on this refactoring. We are still analyzing whether it would always be possible to find a legal refactoring under this restriction, but even if it is, we would still worry about forcing an undesired factoring onto the user and making the [human into a compiler](#Making-the-human-into-a-compiler). ### Constraining scopes will be narrow in practice Regarding the concern that allowing the hidden type to be constrained in a child scope may result in allowing the hidden type to be constrained in "too wide" of a scope, please see [this section](#On-the-width-of-constraining-scopes) for a complete discussion of that point. ### Constraining uses can be found easily Regarding the concern that allowing the hidden type to be constrained in a child scope may make it too difficult to find any constraining uses, note that it's always easy to find constraining uses, as discussed in [this section](#Constraining-uses-can-be-found-easily-without-grep). ## Questions ### Constraining the hidden type and cycle errors tmandry: Under the rules that prevent constraining through encapsulation, I think this should compile: ```rust // JobFut<T> will be `Send` only if `T: Send`. type JobFut<T> = impl Future<Output = ()>; struct Job<T>(u64, JobFut<T>); // This function constrains the hidden type. fn make_job<T>(x: T) -> JobFut<T> { async { let _x = x; () } } // This function does not constrains the hidden type, but since // it's within a scope that is allowed to constrain it, under // the rules of this proposal it may not check the leaked auto // traits of the hidden type. async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) { tx.send(Job(0u64, make_job(x))).await; } ``` TC: But we throw as many cycle errors as possible to maximize the options that we have to evolve going forward. tmandry: So we can't go from not allowing constraining through encapsulation to allowing it if we let this compile, otherwise we'd end up with cycle errors. ### Committing to `#[defines]` TC: I think this means committing to the (named) existential type syntax. TAIT creates anonymous existential types. tmandry: ```rust enum Bar { V1(impl Debug), V2(impl Debug), } struct Foo(Bar); #[defines(Bar)] fn bar1() -> Foo { Foo(Bar::V1(22u32)) } #[defines(Bar)] fn bar2() -> Bar { Bar::V2("hello") } ``` I'm not convinced that's so bad. TC: That means you're defining some but not all tmandry: Removing the encapsulation is nice, that's what scares me a bit about this proposal. TC: Symmetry with dyn Trait is what motivates me here. ```rust enum Bar { V1(Box<dyn Debug>), V2(Box<dyn Debug>), } struct Foo(Bar); fn bar1() -> Foo { Foo(Bar::V1(22u32)) } fn bar2() -> Bar { Bar::V2("hello") } ``` ### Symmetry with dyn Trait tmandry: Not sure about this goal TC: impl Trait is easier than dyn Trait because there can only be one. tmandry: The fact that you have a hidden type that the compiler knows about *and that you can observe* (via auto trait leakage) makes it.. annoying.. that you can't easily figure out what it is TC: I'd disagree with the idea that we want people thinking about the concrete type. Really want people to see `dyn Trait` and `impl Trait` and bridge them together, from a teaching perspective. Yeah auto trait leakage is a caveat. Some people even want that for dyn, though it's hard. ```rust type Foo = impl Future<Output = ()> + Send; type Foo = Box<dyn Future<Output = ()> + Send>; ``` ```rust type Foo<'t, T> = impl Future<Output = impl Iterator<Item = &'t T>>>; ``` tmandry: I agree with wanting symmetry, but I worry that we'll get to a point where they're somewhat symmetric but not quite the same and this only serves to confuse users more than help them. TC: I think of the main use case as `impl Future` vs `Box<dyn Future>`. tmandry: ..that points to a difference I'm concerned about, `Box` vs none. `dyn*` would be nice but that's a tangent. ### Nested modules TC: I think the stronger argument for this is that it's consistent with the signature rule. We aren't constraining all nested functions, just recursively applying the signature rule (inner functions of a function whose signature mentions the TAIT are allowed to constrain). tmandry: I could imagine a future where we have generic modules (but that wouldn't be inconsistent) tmandry: I'm not fully convinced by that argument, but I'm also not strongly opposed to allowing nested modules. TC: I think it'll make stylistic sense to push these into narrow scopes. I agree that local reasoning is great, but a language restriction isn't really the right place to enforce it. We could have clippy come up with a lint here. ### Semantics vs visibility TC: This was true in the original RFCs, we've only added restrictions here. tmandry: Weakened by the fact that we're adding restrictions TC: But we're adding them for tooling reasons, not language design. ### General concerns about TAIT TC: My goal in writing this doc was to see if there was a way to deliver this feature in a way that gives users a consistent mental model. Should be able to swap out a concrete type with `impl Trait` and it just works. ## Conclusions

    Import from clipboard

    Paste your markdown or webpage here...

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lose their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template has been removed or transferred.
    Upgrade
    All
    • All
    • Team
    No template.

    Create a template

    Upgrade

    Delete template

    Do you really want to delete this template?
    Turn this template into a regular note and keep its content, versions, and comments.

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password

    or

    By clicking below, you agree to our terms of service.

    Sign in via Facebook Sign in via Twitter Sign in via GitHub Sign in via Dropbox Sign in with Wallet
    Wallet ( )
    Connect another wallet

    New to HackMD? Sign up

    Help

    • English
    • 中文
    • Français
    • Deutsch
    • 日本語
    • Español
    • Català
    • Ελληνικά
    • Português
    • italiano
    • Türkçe
    • Русский
    • Nederlands
    • hrvatski jezik
    • język polski
    • Українська
    • हिन्दी
    • svenska
    • Esperanto
    • dansk

    Documents

    Help & Tutorial

    How to use Book mode

    Slide Example

    API Docs

    Edit in VSCode

    Install browser extension

    Contacts

    Feedback

    Discord

    Send us email

    Resources

    Releases

    Pricing

    Blog

    Policy

    Terms

    Privacy

    Cheatsheet

    Syntax Example Reference
    # Header Header 基本排版
    - Unordered List
    • Unordered List
    1. Ordered List
    1. Ordered List
    - [ ] Todo List
    • Todo List
    > Blockquote
    Blockquote
    **Bold font** Bold font
    *Italics font* Italics font
    ~~Strikethrough~~ Strikethrough
    19^th^ 19th
    H~2~O H2O
    ++Inserted text++ Inserted text
    ==Marked text== Marked text
    [link text](https:// "title") Link
    ![image alt](https:// "title") Image
    `Code` Code 在筆記中貼入程式碼
    ```javascript
    var i = 0;
    ```
    var i = 0;
    :smile: :smile: Emoji list
    {%youtube youtube_id %} Externals
    $L^aT_eX$ LaTeX
    :::info
    This is a alert area.
    :::

    This is a alert area.

    Versions and GitHub Sync
    Get Full History Access

    • Edit version name
    • Delete

    revision author avatar     named on  

    More Less

    Note content is identical to the latest version.
    Compare
      Choose a version
      No search result
      Version not found
    Sign in to link this note to GitHub
    Learn more
    This note is not linked with GitHub
     

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub
        • Please sign in to GitHub and install the HackMD app on your GitHub repo.
        • HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.
        Learn more  Sign in to GitHub

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions
        Include title and tags
        Available push count

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch
        Available push count

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully