changed 2 years ago  
Linked with GitHub

We accepted RFC 2515 in 2019. That RFC proposed that impl Trait syntax be allowed in type aliases and in associated types. This is a long-awaited feature. It helps API designers to not leak unwanted implementation details and it closes certain fundamental expressiveness gaps in Rust. We would like to stabilize this feature quickly if possible.

This meeting will be a success if everyone walks away with a clear understanding of the motivations for this feature and the key problems it addresses, the details of the proposed plan for stabilization, and how we can move forward.

In this document, TAIT stands for "type alias impl Trait", and ATPIT stands for "associated type position impl Trait.

Motivation 1: Closing the expressiveness gap on unnameable types

In Rust, there are types that cannot be named directly such as the type for each closure and future. These types can only be described by the traits that they implement. Type alias impl Trait allows type aliases to contain these (and other) types by using type inference. This is similar to impl Trait in return position (RPIT), but unlocks new use cases by allowing hidden types to appear in more places.

Example: Sending futures down a channel

Consider this example which might be part of a job queueing system:

use std::future::Future;
use tokio::sync::mpsc;

type Iter = impl Iterator<Item = u8>;

type Fut = impl Future<Output = impl FnOnce() -> Iter>;

struct Job {
    id: u64,
    fut: Fut,
}

async fn send_job(tx: mpsc::Sender<Job>) {
    let iter = std::iter::once(42u8);
    let k = move || iter;
    let fut = async move { k };
    let job = Job { id: 0, fut };
    let _ = tx.send(job).await;
}

In this example, we can see that:

  • We can place a Future within a struct without boxing.
  • We can use this "hidden type" in argument position, which allows us to send this type down a channel.
  • impl Trait can appear multiple times within one type alias, which allows our Future to resolve to an unboxed closure.

Without TAIT, we would need to use boxing and trait objects to achieve a similar result.

Example: Replacing Tokio's ReusableBoxFuture

In async Rust, when implementing Future::poll for a type, we often want to update some inner future. For example:

struct RecvManager {
    inner: ??? // This is a `Future`, but without TAIT we can't name it.
}

async fn make_inner(mut rx: Receiver<T>) -> (Option<T>, Receiver<T>) {
    let v = rx.recv().await;
    (v, rx)
}

impl Future for RecvManager {
    type Output = ();

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let (v, rx) = ready!(self.inner.poll(cx));
        self.inner = make_inner(rx);
        // ...
    }
}

If we just naively box the inner future, then we would have to allocate a new Box every time the outer future is polled. Clearly that's undesirable.

To solve this problem, both internally and for its users, Tokio implements a ReusableBoxFuture. The implementation does a lot of unsafe magic. With TAIT, the future returned by make_inner can be named, so all this clever trickery can simply be avoided.

Quick aside: Why it's important to close expressiveness gaps

The problem with an expressiveness gap in a language is that users often only hit it after having made a substantial commitment to an architecture or code factoring. The engineer has built an entire wall, and only upon trying to lay the last brick does it become apparent that, though the design is logically sound, it cannot be expressed in the language, so the whole wall has to be torn down and built some other way.

What may be worse is the long-term effects of these expressiveness gaps on language experts. Because we've deeply internalized these expressiveness gaps, we stop thinking of architectures that would be better for the problem because we subconsciously know we'll run into the gap. The expressiveness gap can start to stunt our thinking.

Motivation 2: Closing the expressiveness gap on captured lifetimes

RPIT captures the lifetimes of any generic type parameters that are in scope. This can cause the returned hidden type to have undesirably tight bounds. For example, this code does not work, even though the returned hidden type clearly does not capture any references:

fn foo<T>(_t: T) -> impl Sized {
    () // Returned hidden type captures lifetimes in T.
}

fn bar<'x>(x: &'x ()) -> impl Sized + 'static {
    foo(x) // Type captures 'x.
}

error: lifetime may not live long enough
  |
  | fn bar<'x>(x: &'x ()) -> impl Sized + 'static {
  |        -- lifetime `'x` defined here
  |     foo(x) // Type captures 'x.
  |     ^^^^^^ returning this value requires that `'x` must outlive `'static`

There are no good ways to work around this in stable Rust today. As we'll see below, TAIT gives us a way to express the correct bounds for this hidden type.

Motivation 3: Cleaner APIs with impl Trait

Currently, using RPIT in a function exposed in an API is discouraged. This is because for callers of a function behind an API, it's painful to receive a return value whose type cannot be named. Such a type cannot be stored unboxed in any data structure. This reduces the usefulness of RPIT.

TAIT fixes this problem. Because the type alias or other type (e.g. a struct or enum) containing the hidden type or types can be exposed via the API, callers to the API can name the types that the API returns. These types can be placed in data structures and used anywhere any other type may be used.

Motivation 4: Doing what we said we would do

Back when RFC 1951 was being debated, these very same problems and motivations were raised. The feeling was that these were important and should be addressed. Members of the lang-team and others raised serious concerns about the design in that RFC because of these issues. The RFC resolved these concerns and justified the expressiveness restrictions that it imposed by leaning heavily on the assumption that we would later stabilize a fully explicit syntax. It's now many years later. It's time that we take a step in the direction of fulfilling that assumption.

How it works: Desugaring

A hypothetical existential type syntax

To help with understanding impl Trait, let's suppose that Rust supported this explicit syntax (not part of this proposal):

existential type H: Default; // Not part of this proposal.

This would introduce a type parameter H and add a trait bound such that H must implement the trait Default. As with other type parameters in Rust, H may be used in places where we would expect to find a type, but the type is opaque. We can only assume that it implements the traits in the bound (with the exception of leaked auto traits, described below). We can say, e.g., H::default().

This type parameter is existential in the sense that no monomorphization is performed as with type parameters in argument position. H must represent exactly one concrete hidden type.

We can extend this syntax to support type and lifetime parameters:

existential type H<'t, T>: Iterator<Item = &'t T>; // Demonstration only.

This means that, for each type T and lifetime 't, there is exactly one concrete hidden type H<'t, T>.

impl Trait in type aliases

The TAIT proposal differs from the explicit syntax above in that the hidden type (H above) is anonymous and can not necessarily be named explicitly. Let's desugar the first example to show how this works:

type Iter = impl Iterator<Item = u8>;
type Fut = impl Future<Output = impl FnOnce() -> Iter>;

// Desugars to:

existential type _H0: Iterator<Item = u8>;
existential type _H1: FnOnce() -> _H0;
existential type _H2: Future<Output = _H1>;

type Iter = _H0;
type Fut = _H2;

We can see that's there's actually nothing special about the type alias. The type alias is just a normal type alias. Each use of the impl Trait syntax simply causes a new anonymous type parameter to be introduced.

Quick aside: impl Trait everywhere

We can see from the above desugaring why it's normal and natural to want impl Trait everywhere. There's no conceptual distinction between a use of impl Trait in a type alias and in a struct or enum. I.e.:

type S = (impl Default, impl Default);
// Is conceptually similar to:
struct S(impl Default, impl Default); // Not part of this proposal.

In both cases, S is just a normal type that has "holes punched in it" that are later filled in with concrete types.

This is not part of the proposed stabilization.

How it works: Constraining the hidden type

As described above, each hidden type represents exactly one concrete type. The concrete hidden type is chosen by type inference within the scope of whatever item contains the impl Trait. Typically this is going to be a module, but it could also be a function or any other kind of item that may contain other items.

Within that scope the hidden type may be used in two different ways:

  1. It may be used only for the traits that it implements; or
  2. It may be used in a way that constrains it to a particular hidden type.

Outside of that scope, it may only be used for the traits that it implements.

Inside of that scope, the hidden type may be constrained more than once. But if it is, all of those uses must constrain it to the exact same concrete type. All items that constrain the hidden type must fully constrain that type. An item cannot partially constrain a type and rely on other items to fill in the gaps.

For a function to constrain the hidden type, the hidden type must appear in the signature of the function – in the type of the function's return value, in the type of one or more of its arguments, or in a type within a bound.

Note that we talk here about constraining the hidden type, not about constraining "the TAIT" or the type alias. It's important to remember the hidden types are anonymous, that there can be more than one of them in a single type alias, and that these hidden types can be constrained through encapsulating tuples, structs, enums, etc.

Let's look at some examples of how these hidden types may be constrained.

Example: Return position

A hidden type that appears in return position may be constrained by the function returning a value of some concrete type:

type Foo = impl Default; // Let's call the hidden type `_H`.

fn foo() -> Foo { () } // Constrains _H to ().

Note that if we simply pass through the hidden type, it has not been constrained:

type Foo = impl Default; // Let's call the hidden type `_H`.

fn foo(x: Foo) -> Foo { x } // Does not constrain _H.

(To preserve our ability to make backward compatible changes in the future, this and other non-constraining functions within the scope in which the hidden type was introduced nonetheless are treated by the implementation as if they could constrain the hidden type, with an error being thrown later if needed, as we detail in an appendix.)

Allowing the hidden type to be constrained when it appears in return position is necessary to address the expressiveness gap related to more precisely capturing lifetimes as described in Motivation 2 and is necessary to support the use of impl Trait in API designs as described in Motivation 3.

Example: Argument position

A hidden type that appears in argument position may be constrained by the function using the hidden type in a way that coerces that hidden type to some concrete type. For example:

type Foo = impl Copy; // Let's call the hidden type `_H`.

fn foo(x: Foo) {
    let _: () = x; // Constrains _H to ().
    let _ = x as (); // Also constrains _H to ().
    std::convert::identity::<()>(x) // Also constrains _H to ().
}

Allowing the hidden type to be constrained when it appears in argument position is necessary to address the expressiveness gap related to output arguments described in Motivation 1. This pattern of accepting, as an argument, a channel to which output will be sent is ubiquitous in asynchronous Rust code. For example:

// This is a simplified version of the example in Motivation 1.
type JobFut = impl Future<Output = ()>;
struct Job(u64, JobFut);
// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

Example: Trait bound

A hidden type that appears in a bound may be constrained by its use within the function or by the function's return type. For example:

type Foo = impl Sized; // Let's call the hidden type `_H`.

// Constrains _H to ().
fn foo<I: Iterator<Item = Foo>>(i: I) -> impl Iterator<Item = ()> { i }

Allowing the hidden type to be constrained when it appears in trait bounds is necessary so as to not make traits "second-class citizens" of this feature and of the language as a whole. Consider the example in Motivation 1 that involves a function that accepts a channel parameterized by the hidden type as an argument. If the function is changed to accept a trait representing all types of channels that implement a particular interface, then the hidden type will appear only in the trait bound. For example:

// This is a variation on the example in Motivation 1.
trait Channel<T> {
    async fn send(&self, value: T);
}

type JobFut = impl Future<Output = ()>;
struct Job(u64, JobFut);

// The hidden type is contained within the `Job` struct which
// appears in the bounds.
async fn send_job<C: Channel<Job>>(tx: C) {
    async fn make_job() { todo!() }
    tx.send(Job(0u64, make_job())).await;
}

Example: Statics and constants

A hidden type may be constrained by using it in the type of a static or constant:

type Foo = impl Default; // Let's call the hidden type `_H`.

const C: Foo = (); // Constrains _H to ().
static S: Foo = (); // Also constrains _H to ().

We can see here that it's OK for _H to be constrained multiple times, as each use constrains it to the same concrete type.

Allowing the hidden type to be constrained when it appears in the type of statics and constants is necessary as a matter of consistency and so as to support the use of this feature in APIs as described in Motivation 3.

Example: Constraining through encapsulation

Remember, we're constraining the hidden type, not the type alias, so it's totally OK to constrain the hidden type through some other type that encapsulates it. For example:

type Foo = impl Default; // Let's call the hidden type `_H`.

struct Bar(Foo);

fn foo() -> Bar { // Constrains _H to ().
    Bar(())
}

We say here that the hidden type appears in the signature of foo() because Bar contains within it the existential type parameter _H (which is anonymous and cannot be written explicitly).

It's important that constraining through encapsulation works, as it would be a very severe usability and expressiveness limitation if it did not, as it would considerably restrict normal abstraction; see Motivation 1 for an example of this. This behavior is also required for a future "impl Trait everywhere" to work as we would expect.

For many more details on the motivation behind constraining through encapsulation and why it's an integral part of this proposal, please see Appendix C.

Example: Constraining hidden types separately

All hidden types that are introduced within the scope of an item must be constrained within that scope. However, a function, const, or static that constrains one of the hidden types does not need to constrain all of them, even if those hidden types are all contained within the same outer type. For example:

// Let's call the hidden types `Result<_H0, _H1>`.
type Foo = Result<impl Default, impl Default>;

const FOO_OK: Foo = Ok(()); // Constrains _H0 to ().
const FOO_ERR: Foo = Err(()); // Constrains _H1 to ().

This is fine as each item fully constrains each hidden type.

Example: Cannot partially constrain hidden type

An item that constrains the hidden type must fully constrain that type. It cannot partially constrain it and rely on other items to fill in the gaps. For example, this does not work:

type Foo = impl Sized;

fn foo_ok() -> Foo { Ok(()) } // Error.
fn foo_err() -> Foo { Err(()) } // Error.

These items individually try to constrain the hidden type without fully constraining its type. This results in an error that type annotations are needed.

Note carefully the difference between this and the earlier example. We can separately constrain two hidden types contained in one concrete type. But we cannot constrain a hidden type without constraining all of the contained hidden types.

How it works: Leaked auto traits

Return position impl Trait (RPIT) hidden types leak auto traits that are not specified in the bounds. While callers cannot see the concrete hidden type behind an opaque type returned by the function, they can see whether it implements these auto traits. For example:

fn foo() -> impl Sized { () } // We only promised `Sized`

fn is_send<T: Send>(_t: T) {}

fn test() {
    is_send(foo()); // But we can see that it's also `Send`.
}

This was a conscious design decision. On the one hand the behavior is convenient and useful, but on the other it can require the compiler and associated tooling to do more work and it can present SemVer hazards. These tradeoffs were considered and accepted during the design and stabilization of RPIT.

Type alias impl Trait exactly matches the RPIT behavior with respect to leaked auto traits.

How it works: Associated type position

Everything said here about impl Trait in type aliases is also true about impl Trait in associated type position, and associated type position impl Trait (ATPIT) is part of this stabilization proposal. For example:

struct S;
impl Iterator for S {
    type Item = impl Future<Output = ()>; // Let's call the hidden type `_H`.

    fn next(&mut self) -> Option<Self::Item> {
        Some(async {}) // Constrains _H to an unnameable Future.
    }
}

For ATPIT, in addition to the other restrictions discussed in this document, only methods and associated constants on the same impl can constrain the hidden type; functions and other items nested within the methods of the impl cannot constrain it. This is not a restriction introduced by this proposal; it's simply consequence of our existing restriction against using generic parameters from outer scopes in inner items.

How it works: Capturing in-scope type parameters

Hidden types in RPIT implicitly capture the lifetimes within all generic type parameters in scope when the hidden type is introduced with impl Trait. Hidden types in TAIT work in exactly this same way. For example:

mod m {
    //  _H implicitly captures the lifetimes of any references in T.
    pub type Foo<T> = impl Sized; // Let's call the hidden type `_H`.
    pub fn foo<T: Sized>(t: T) -> Foo<T> { t } // Constrains _H to T.
}

fn bar<'x>(x: &'x ()) -> impl Sized + Captures<'x> {
    m::foo(x) // The return type from foo captures 'x.
}

Similarly for ATPIT:

struct S<T>(Option<T>);
impl<T: Clone> Iterator for S<T> {
    // _H implicitly captures the lifetimes of any references in T.
    type Item = impl Clone; // Let's call the hidden type `_H`.

    fn next(&mut self) -> Option<Self::Item> {
        Some(self.0.clone()) // Constrains _H to T.
    }
}

fn bar<'x>(mut x: S<&'x ()>) -> impl Clone + Captures<'x> {
    x.next().unwrap()
}

Note, however, that the lifetimes of generic type parameters that are only in scope of where the hidden type is used are not captured. Therefore this code works:

mod m {
    // No type parameters are in scope, so the hidden type
    // does not capture any.
    pub type Foo = impl Sized; // Let's call the hidden type `_H`.

    pub fn foo<T>(_t: T) -> Foo { () } // Constrains _H to ().
}

fn bar<'x>(x: &'x ()) -> impl Sized + 'static {
    m::foo(x) // Hidden type does not capture 'x.
}

This allows for expressing correct bounds on hidden types in a way that is not possible in stable Rust today.

How it works: Generic parameters must be used generically

When the hidden type captures a type parameter, that type parameter must be used generically. It's an error to fill it in with a concrete type. For example, this code is invalid:

type Foo<T> = impl Sized;

fn foo() -> Foo<()> { () }

error[E0792]: expected generic type parameter, found `()`
  |
  | type Foo<T> = impl Sized;
  |          - this generic parameter must be used with a generic type parameter
  |
  | fn foo() -> Foo<()> { () }
  |                       ^^

Intuitively, what this restriction says is that when a hidden type has captured a generic type parameter, we must constrain the hidden type for all possible values of that type parameter. Without this restriction, things get a bit wonky. We could write:

mod m {
    pub type Foo<T> = impl Sized;
    const _: Foo<()> = ();
}
const _: m::Foo<u8> = ??; // Not sound.

In this example, Foo<T> has no bound that prevents expressing Foo<u8>, but there's no concrete type that we can use for its hidden type. Conversely, note that all of these are fine and are what you would want instead:

type Foo = impl Sized;
const fn foo<T>() -> Foo { () }
type Bar<T> = impl Sized;
const fn bar<T>() -> Bar<T> { () }
type Baz<T> = impl Sized;
const fn baz<T>(t: T) -> Baz<T> { t }

const _: Foo = foo::<()>();
const _: Bar<()> = bar::<()>();
const _: Baz<()> = baz(());

How it works: The signature restriction

When a function constrains a hidden type to a particular concrete type, the hidden type must appear somewhere in the signature of the function – in the type of the function's return value, in the type of one of its arguments, or in a type within a bound.

Note that, as in many of the examples above, a hidden type may be nested arbitrarily deeply within other types, and those outer types appearing in the signature satisfy this requirement. For example:

type Foo = impl Sized; // Let's call the hidden type `_H`.
struct Bar(Foo);

fn foo() -> Bar { // Bar contains _H, so this is OK.
    Bar(()) // Constrains _H to ().
}

When a function contains a nested inner function, the inner function may not constrain hidden types introduced outside of the outer function unless the hidden type appears in the signature of the outer function. For example, this is invalid:

type Foo = impl Sized;

fn foo() {
    fn bar() -> Foo { () }
}

Stabilization proposal summary

In summary, we propose for stabilization:

  • Type aliases and associated types may contain multiple instances of impl Trait.
  • Each impl Trait introduces a new hidden type.
  • The hidden type may be constrained only within the scope of the item (e.g. module) in which it was introduced, and within any sub-scopes thereof, except that:
    • Functions and methods must have the hidden type that they intend to constrain within their signature – within the type of their return value, within the type of one or more of their arguments, or within a type in a bound.
    • Nested functions may not constrain a hidden type from an outer scope unless the outer function also includes the hidden type in its signature.
  • The hidden type may be constrained by functions, methods, constants, and statics.
  • The hidden type leaks its auto traits; these can be observed through the opaque type.
  • The hidden type implicitly captures the lifetimes within all generic parameters in scope.
  • Generic type parameters captured by the hidden type must be used generically.

Appendix A: Signature restriction details

Editor's note: As this appendix focuses on the implementation, it uses terminology that makes sense from the point of view of the compiler. Most notably, "registering a hidden type" is what the compiler does when a user "constrains a hidden type".

Implementation considerations

The signature restriction that's part of this proposal simplifies the implementation and improves its performance.

Because of this restriction, outside of a function, we can check the opaque types for which that function may register hidden types by just looking at its signature. Using this information, we can avoid computing typeck for that function when we need to resolve hidden types that the function cannot possibly register.

Inside of a function, because we know the opaque types for which we may register hidden types before we start typeck, we can create a single inference variable for the hidden type ahead of time. That allows us to perform inference across all use sites of the opaque type within that function. This is what RPIT already does across the return statements and the trailing return expression.

The net result is that we can use better caching and generally be more performant because we don't have to carry information about the current function into all cache keys for the various things that we try to prove during typeck.

Cycle errors

If a function defined within the scope in which an impl Trait hidden type is introduced does not register that hidden type, but does ask whether its corresponding opaque type implements an auto trait (e.g. Send), we need to compute the concrete hidden type to check that bound. That involves further type checking. If we have to assume that this function could itself register the hidden type, then this will produce a cycle error.

For code affected by this, the workaround is either to avoid checking for the implementation of an auto trait on the opaque type or to move the hidden type into a more narrow scope.

Planning for backward compatibility

With the signature restriction in this proposal, we could eliminate some cycle errors. Because we can determine before we start typeck the opaque types for which a function may register hidden types, within the function we could reveal the leaked auto traits for those hidden types that the function could not possibly register. We are proposing not to do this for now.

Under this proposal, for the purpose of computing cycle errors, we conservatively assume that all functions in the scope might register the hidden type and we throw any errors needed according to the rules of the signature restriction. This produces the maximum possible number of cycle errors. We do this to preserve the maximum scope for making future changes to this feature in a backward compatible way. The rule is that we need to know before running typeck which opaque types a function may register hidden types for implicitly in the future.

Here's how the implementation works:

  • We scrape the hidden types in the signature of each function in the same module or in a child module of where the hidden type was introduced.
  • For ATPIT, we scrape mentions of the associated type from the signatures of the methods and associated functions. This is done iteratively by looking for associated types, then normalizing each to its concrete type and repeating the process.
  • If a function within this scope tries to constrain a hidden type that did not appear in the signature, we throw an error.
  • If a function within this scope that does not constrain the hidden type uses the opaque type in such a way that we would need to leak its auto traits, we throw an error.

If we were to decide at any point to commit to never removing the signature restriction, we could accept more correct code without cycle errors.

Appendix B: The IDE concern

To maximize responsiveness, IDEs try to minimize the amount of work that they need to do to infer types and provide completions.

Without the signature restriction proposed here, because impl Trait opaque types leak auto traits, IDEs would have to check more function bodies than they do today to provide correct completions and type inference annotations. The problem the IDEs face is similar to the problem faced by the compiler itself when performing incremental compilation.

Because of the proposed signature restriction, this is not an issue. The IDEs do not have to type check a function body to determine whether a hidden type may be constrained by that function.

@matklad, the author of rust-analyzer, has confirmed that with the signature restriction, TAIT is equivalent to RPIT in terms of the complications it presents for the IDE.

Even if we were to remove the signature restriction, the following considerations mitigate the IDE concern:

  • Even today, because of leaked auto traits and RPIT, rust-analyzer presents incorrect completions and type inference hints. TAIT cannot break anything that is not already broken.
  • Stable Rust today allows items to be nested in ways that, even without RPIT or TAIT, require IDEs and other tooling to check all function bodies to correctly infer all types. There is a proposal (as-yet not accepted) that would add restrictions to Rust in a future edition to remove this requirement.
  • RFC 1522, which decided to leak auto traits, explicitly acknowledged and accepted the additional burdens that this decision put on all tooling.

Appendix C: Why constraining through encapsulation is necessary

The ability to constrain the hidden type after it has been encapsulated in other types is a central part of this proposal. In the sections below, we'll show why this is necessary both conceptually and practically.

Concept: The details of a good abstraction can be (mostly) forgotten

This document has spent a lot of words discussing what it means to constrain the hidden type, when that's allowed, and how that is different from not constraining it. We do this because we need to specify this precisely for our purposes in designing and implementing a language.

However, in day to day use of this feature, we believe that users will not be thinking about these distinctions carefully and that they should not need to do so. This proposal makes a number of tradeoffs in pursuit of the goal that most of the code that users will want to write should just work without having to carefully consider the details of this mechanism. As we'll see in the following sections, constraining through encapsulation is necessary to achieve this.

Symmetry with dyn Trait

The impl Trait feature has long been thought of as a statically-dispatched version of dyn Trait. For example, RFC 1951 suggests that:

[Extending the use of impl Trait to more places] provides a greater degree of analogy between static and dynamic dispatch when working with traits. Introducing trait objects is easier when they can be understood as a variant of impl Trait, rather than a completely different approach.

There is a large body of existing Rust code that today uses boxing and dyn Trait to solve the kinds of problems discussed in the motivations above. We expect that much of this code will migrate to use TAIT to take advantage of static dispatch. One goal of this proposal is to minimize the amount of refactoring, churn, and careful thinking that is necessary to accomplish that.

Consider, for example, how a simplified version of Motivation 1 might be written today using Box<dyn Trait>:

// This is a simplified version of the example in Motivation 1.

type JobFut = Box<dyn Future<Output = ()>>;
struct Job(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, Box::new(async { todo!() }))).await;
}

To migrate this code to impl Trait, the user simply replaces the Box<dyn Trait> with impl Trait and removes the boxing:

type JobFut = impl Future<Output = ()>;
struct Job(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

Without the ability to constrain the hidden type through encapsulation, this code would need to be significantly reworked. But worse, the user would have to think carefully about why this seemingly simple substitution didn't work.

We argue that in the common case, users should not have to think more carefully about the difference between using and constraining an impl Trait than they do about the difference between using and assigning to an abstract dyn Trait.

Robustness to code evolution, principle of least surprise

Without the ability to constrain through encapsulation, seemingly trivial changes to code would require the user to refactor code for reasons that may not be obvious.

For example, consider that the user first wrote this simple version of the job queue where each Job is only a future:

type Job = impl Future<Output = ()>;

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(async { todo!() }).await;
}

This works and the user is happy. Then the user realizes that every job in the queue needs an identifier, so the user adds one using a tuple in the type alias:

type Job = (u64, impl Future<Output = ()>);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send((0u64, async { todo!() })).await;
}

This still works and the user is happy. But then, during code review, the reviewer suggests that it would be more idiomatic and better long-term to make the Job into a struct. The user first tries to convert the type alias to a struct:

struct Job(u64, impl Future<Output = ()>);
// ^---- Error: We don't support `impl Trait` everywhere yet.

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

That doesn't work. We don't support impl Trait everywhere yet. So the user extracts the impl Trait into a type alias, then uses that type alias in the Job struct:

type JobFut = impl Future<Output = ()>;
struct Job(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

If we were to not support constraining through encapsulation, this code wouldn't work either. We claim that this violates the principle of least surprise.

Even worse, if we were to not permit this code, it may not be possible to emit an error message that would guide the user to the set of tricks that the user would have to employ instead. We detail these tricks below and the problems with them.

Making the human into a compiler

To solve problems like the one above without constraining through encapsulation, the user must think like a compiler and desugar the code until the type alias is returned by name from a function. Consider this variation on our job queue with some non-trivial input:

type JobFut = impl Future<Output = (u8, u8)>;
struct Job(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>, data: [u8; 16]) {
    // Without loss of generality, assume that the function
    // transforms its inputs in a non-trivial way.
    let data1: [u8; 8] = data[0..8].try_into().unwrap();
    let data2: [u8; 8] = data[8..16].try_into().unwrap();
    let mut feeder = data1.into_iter().zip(data2.into_iter());
    // It uses the transformed inputs itself.
    let id = feeder.next().unwrap().0 as u64;
    // And the hidden type captures these transformed inputs.
    tx.send(Job(id, async move { feeder.next().unwrap() })).await;
}

To make this idea work without constraining through encapsulation, the user must refactor the code until each type alias is returned by name from some function. Any state needed to construct the value for the hidden type must be passed down the stack, which will require repeating the names of those types or naming types that were previously entirely inferred because the values were created and used within one function body. Here's how we might need to "compile" the code above without the ability to constrain through encapsulation:

type JobFut = impl Future<Output = (u8, u8)>;
struct Job(u64, JobFut);

// We have to manually desugar an `async fn` here
// and name the type of the `Iterator`.
fn make_future(
    // We need to pass down all of the transformed inputs from
    // `send_job` that need to be captured by the hidden type.
    mut feeder: std::iter::Zip<
        std::array::IntoIter<u8, 8>,
        std::array::IntoIter<u8, 8>,
    >,
) -> JobFut {
    // The body must be wrapped in an `async` block.
    async move {
        feeder.next().unwrap()
    }
}

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>, data: [u8; 16]) {
    let data1: [u8; 8] = data[0..8].try_into().unwrap();
    let data2: [u8; 8] = data[8..16].try_into().unwrap();
    let mut feeder = data1.into_iter().zip(data2.into_iter());
    let id = feeder.next().unwrap().0 as u64;
    tx.send(Job(id, make_future(feeder))).await;
}

The first thing to note here is that we would not be able to use async fn when performing this manual compilation. Since we would need the type alias to appear explicitly by name in the return value, we would have to manually desugar make_future into a synchronous function with its body wrapped in an async block. We've been making efforts in other areas to reduce the need for users to desugar async functions, and this would seem to pull in the opposite direction from that.

The second thing to note is that we needed to name a complicated Iterator type to make this work. In the original code, we had avoided naming this type. We believe that it's likely in this scenario that a user may try to use impl Trait again, in make_future, to avoid naming the Iterator, as follows:

// Can we use `impl Iterator` rather than naming the complex type?
fn make_future(mut feeder: impl Iterator<Item = (u8, u8)>) -> JobFut {
    async move {
    // ^--- Error: type parameter `impl Iterator<Item = (u8, u8)>`
    // is part of concrete type but not used in parameter list
    // for the `impl Trait` type alias
        feeder.next().unwrap()
    }
}

Using impl Trait in argument position introduces a generic parameter that the type alias must capture. The compiler tells the user that this can work, but that the hidden type must capture a type parameter. Following the directions of the error message, we suspect the user may next try this:

// We added a type parameter like the error message told us to do.
type JobFut<I: Iterator<Item = (u8, u8)>> = impl Future<Output = (u8, u8)>;
struct Job<I: Iterator<Item = (u8, u8)>>(u64, JobFut<I>);

// We can't use `impl Trait` in argument position because we
// need to name the type parameter so as to pass it through to
// the type alias.
fn make_future<I>(mut feeder: I) -> JobFut<I>
where
    I: Iterator<Item = (u8, u8)>
{
    async move {
        feeder.next().unwrap()
    }
}

// Now we need a named type parameter for the `Job` struct, but
// this doesn't work because we're choosing the type, not the caller.
async fn send_job<I>(tx: Sender<Job<I>>, data: [u8; 16])
where
    I: Iterator<Item = (u8, u8)>
{
    let data1: [u8; 8] = data[0..8].try_into().unwrap();
    let data2: [u8; 8] = data[8..16].try_into().unwrap();
    let mut feeder = data1.into_iter().zip(data2.into_iter());
    let id = feeder.next().unwrap().0 as u64;
    tx.send(Job(id, make_future(feeder))).await;
    // ^---- Error: mismatched types.
}

At this point, hopefully, the user will realize that this isn't going to work. Even if it did, it would require polluting all encapsulating types and all code up the stack with a type parameter that isn't actually needed. But this can't work here because the callee is choosing the type of the Iterator, not the caller. So instead, the user would have to ignore the error message and think to use TAIT again to name the type of the argument that needs to be passed down the stack:

type JobFut = impl Future<Output = (u8, u8)>;
struct Job(u64, JobFut);

// We use TAIT again to name the type that needs
// to be passed down the stack.
type JobIterArg = impl Iterator<Item = (u8, u8)>;

// We have to manually desugar an `async fn` here
// and name the type of the Iterator using TAIT.
fn make_future(mut feeder: JobIterArg) -> JobFut {
    async move {
        feeder.next().unwrap()
    }
}

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>, data: [u8; 16]) {
    let data1: [u8; 8] = data[0..8].try_into().unwrap();
    let data2: [u8; 8] = data[8..16].try_into().unwrap();
    let mut feeder = data1.into_iter().zip(data2.into_iter());
    let id = feeder.next().unwrap().0 as u64;
    tx.send(Job(id, make_future(feeder))).await;
}

To pass the data down the stack as would be required without constraining through encapsulation, the user needed to add an extra type alias impl Trait to avoid naming the complicated Iterator type. We argue that forcing the creation of more type aliases with hidden types just to satisfy the compiler does not improve code clarity.

The tricks above lead to cycle errors

The tricks above often result in rearranging code in ways that are more prone to fundamental cycle errors. Following our motivating example, let's assume that Sender requires its type parameter to implement Send. It's implemented in a separate crate as follows:

pub struct Sender<T>(PhantomData<T>);
impl<T: Send> Sender<T> {
    pub async fn send(&self, x: T) { todo!() }
}

Consider our motivating example, but now with type parameters that are captured by the hidden type. The Sendness of JobFut<T> is covariant with the Sendness of T due to auto trait leakage:

// JobFut<T> will be `Send` only if `T: Send`.
type JobFut<T> = impl Future<Output = ()>;
struct Job<T>(u64, JobFut<T>);

// This function constrains the hidden type, so it's OK for us
// to reveal the auto traits of that hidden type within this
// function.
async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) {
    tx.send(Job(0u64, async move { let _x = x; () })).await;
}

Under the rules of this proposal, the compiler is allowed to accept this code. The function that constrains the hidden type also checks whether that hidden type implements Send. That is allowed.

However, if we were to use the trick from the last section, we end up with code that must be rejected:

// JobFut<T> will be `Send` only if `T: Send`.
type JobFut<T> = impl Future<Output = ()>;
struct Job<T>(u64, JobFut<T>);

// This function constrains the hidden type.
fn make_job<T>(x: T) -> JobFut<T> {
    async {
        let _x = x;
        ()
    }
}

// This function does not constrains the hidden type, but since
// it's within a scope that is allowed to constrain it, under
// the rules of this proposal it may not check the leaked auto
// traits of the hidden type.
async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) {
    tx.send(Job(0u64, make_job(x))).await;
}

That code will result in a cycle error. This cycle error is fundamental in the sense that it's inherent and necessary under the rules of the signature restriction.

As with other cycle errors, we have to work around this by moving the hidden type and its constraining use into a more narrow scope:

mod jobfut {
    // JobFut<T> will be `Send` only if `T: Send`.
    pub(crate) type JobFut<T> = impl Future<Output = ()>;
    pub(crate) struct Job<T>(pub(crate) u64, pub(crate) JobFut<T>);

    // This function constrains the hidden type.
    pub(crate) fn make_job<T>(x: T) -> JobFut<T> {
        async {
            let _x = x;
            ()
        }
    }
}

use jobfut::{Job, make_job};

// This function is allowed to check for the leaked auto traits
// of the hidden type since it is not in a scope that's allowed
// to constrain the hidden type.
async fn send_job<T: Send>(tx: Sender<Job<T>>, x: T) {
    tx.send(Job(0u64, make_job(x))).await;
}

We argue that this is a lot to ask from users when the obvious code could "just work".

ATPIT would require a different trick

If we were to not support constraining through encapsulation, impl Trait in associate type position would require a different trick. Consider this implementation of IntoIterator in which the Item associated type uses impl Trait:

struct S;
impl IntoIterator for S {
    type Item = impl Future<Output = ()>;
    type IntoIter = std::iter::Once<Self::Item>;

    fn into_iter(self) -> Self::IntoIter {
        std::iter::once(async {})
    }
}

Without constraining through encapsulation, the user would need to know to "alpha-expand" the method's return type until the "correct" associated type appeared explicitly by name in the return type:

struct S;
impl IntoIterator for S {
    type Item = impl Future<Output = ()>;
    type IntoIter = std::iter::Once<Self::Item>;

    fn into_iter(self) -> std::iter::Once<Self::Item> {
        std::iter::once(async {})
    }
}

We argue that this may not be obvious and that it would be less idiomatic. Because the return type does not match the standard presentation of the trait by using IntoIter, it might look like a refinement at first glance even though it is not (pedantically: it's not any more of a refinement than associated types are inherently).

ATPIT needs constraining through encapsulation

For ATPIT, it's not always possible for all of the associated type aliases with hidden types to appear explicitly by name in the signature of the method that constrains them. Consider this implementation of IntoFuture where both the Output and IntoFuture associated types use Impl Trait:

struct S;
impl IntoFuture for S {
    type Output = impl Future<Output = ()>;
    type IntoFuture = impl Future<Output = Self::Output>;

    fn into_future(self) -> Self::IntoFuture {
        async { async { () } }
    }
}

It's not possible to rewrite this example in such a way that both Self::Output and Self::IntoFuture appear in the signature of into_future. Consequently, ATPIT requires some form of constraining through encapsulation so as to not restrict the fundamental expressiveness of the language.

The grepping concern

Constraining through encapsulation means that to find where a hidden type is constrained using grep may require more care. We would need to "walk" through other types that encapsulate the hidden type. Using our job queue example:

type JobFut = impl Future<Output = ()>; // Grep for `JobFut`.
struct Job(u64, JobFut); // ...and grep for `Job`.

// We would find this function by grepping for `Job`.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

To use grep to find where the hidden type might be constrained, we would need to search both for appearances of JobFut and Job.

We argue that this is fine for the reasons that follow.

Using grep is still possible

To find all uses of the hidden type with grep, the user would follow this algorithm:

  1. Add the name of the innermost type alias containing the hidden type to the search list.
  2. Search for all tokens in the search list.
  3. Whenever a data structure is found, add it to the search list and go to step 2.

Though it may look overwrought when written out explicitly, this process is common and required more often than not to search code thoroughly with grep (and is one reason that people use IDEs).

(As we'll demonstrate below, even without IDE support, using grep is never needed to find the constraining uses for a hidden type.)

It's not any worse than Box<dyn Trait>

Any time that an abstract type is encapsulated in other types, we may need to look through uses of those other types to find the concrete type used for the abstract type. For example, here's the code that someone might write today using Box<dyn Trait>:

type JobFut = Box<dyn Future<Output = ()>>;
struct Job(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, Box::new(async { todo!() }))).await;
}

To find the concrete types that may underlie the abstract type represented by JobFut, we must consider all uses of any types that may encapsulate it, such as Job in this example.

We argue that this has all of the same problems in terms of grep as the impl Trait version.

The restriction would force an unlikely cost to always be paid

To use grep, the user may have to unwind some encapsulation. But if we were to forbid constraining through encapsulation, then users would always have to unwind this encapsulation just to write code that the compiler would accept (as described in the "tricks" above).

With the permissive approach, the cost will rarely be paid as there are better ways to find all constraining uses of any hidden type. With the restrictive approach, the cost would always have to be paid.

Furthermore, with the permissive approach, the maximum cost is that the user must iteratively build a search list when choosing to use grep. With the restrictive approach, the cost would be potentially-tricky and elaborate code refactoring.

On the width of constraining scopes

One of the reasons that people have worried about being able to use grep to find where a hidden type might be constrained is that the rule allowing child modules to constrain the hidden type might in theory result in wide constraining scopes.

We believe that in practice the scopes for constraining impl Trait types will be narrow because of the effect of cycle errors. The presence of the cycle errors required by this proposal will encourage best practices and coding guidelines to favor moving impl Trait hidden types to the narrowest possible scope.

When we teach people this feature, we will encourage them to adopt the narrowest possible scope because that will minimize the chances that these users will encounter cycle errors. When teaching users how to handle cycle errors, we will say things such as:

Modules in Rust are cheap. If you encounter a cycle error, move the hidden type to a more narrow scope, creating a new child module if needed. Keeping the impl Trait in the narrowest possible scope is a good best practice so that you don't later stumble over these cycle errors when making changes to your code.

To whatever degree that education, best practices, and coding guidelines were not enough, we believe that this could be adequately addressed by linting. Clippy today lints about functions that it scores as being too long or complex. It would certainly be possible for Clippy to lint about a hidden type that is introduced in "too wide" of a scope.

Constraining uses can be found easily without grep

Grep is never needed to find all constraining uses of a hidden type. The better option, aside from IDE support, is to simply provoke a compiler error. For example, to find all constraining uses of JobFut in the code above, we add a bogus constraint that will conflict with any other constraining use:

// Let's find all constraining uses of `JobFut`
// by adding a bogus constraint.
fn nop() -> JobFut { async {} }

This will provoke a compiler error that will point us to exactly what we want:

error: concrete type differs from previous defining opaque type use
  --> src/lib.rs:16:13
   |
16 |     tx.send(Job(0u64, async { todo!() })).await;
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `[async block@src/lib.rs:12:22: 12:30]`, got `[async block@src/lib.rs:16:23: 16:40]`
   |

Provoking an error to extract information from the compiler is a common technique used by Rust programmers, so this is not out of the ordinary. (For example, users often write let _: () = ... to query for type information.)

Conflicts are the main reason to look for constraining uses

We believe that the main practical reason that a user would want to find the constraining uses for a hidden type is because the user has written a conflicting constraint.

Whenever that happens, neither grep nor any special action on the part of the user is required to find the constraining uses. The compiler will point the user to exactly the right places.

Regarding impl Trait everywhere

This proposal only seeks to stabilize impl Trait in type aliases and in associated type position. However, we argued above in the main body of this proposal that constraining through encapsulation is required to avoid surprising behavior if we were to support impl Trait inside of structs or enums. In this section, we provide more discussion of this claim. Consider a version of our job queue that uses impl Trait everywhere:

struct Job(u64, impl Future<Output = ()>); // Not part of this proposal.

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<Job>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

// This function takes a job out of a channel and processes it.
async fn process_job(rx: Receiver<Job>) {
    let job = rx.recv().await.unwrap();
    job.await;
}

If we were to forbid constraining through encapsulation, presumably the above code would still work because the innermost named type that contains the hidden type appears explicitly by name in the signature. However, now imagine that we perform this seemingly trivial modification because we want to use the hidden type in two different structs:

type JobFut = impl Future<Output = ()>;
struct InnerJob(u64, JobFut);
struct OuterJob(u64, JobFut);

// This function creates a future and sends it out a channel.
async fn send_job(tx: Sender<InnerJob>) {
    tx.send(Job(0u64, async { todo!() })).await;
}

// This function takes a job out of a channel and processes it.
async fn process_job(rx: Receiver<InnerJob>) {
    let job = rx.recv().await.unwrap();
    job.await;
}

// ...elided code that uses `OuterJob`.

Without constraining through encapsulation, the above code now would not work. The user would need to refactor send_job using the tricks above. We argue that the need to do this would be surprising and the reasons this would be required would be unintuitive.

Currently in Rust it's always legal to replace a type in a struct or enum with an equivalent type alias. The authors believe it would be difficult to explain to our colleagues why this was no longer the case.

Note that process_job would not need to be refactored using one of the tricks. We argue that without carefully considering the mechanics of impl Trait, it would not be obvious why send_job needs to be refactored but process_job does not. We argue that forcing the user to be so aware that send_job is more special than process_job decreases the value of the abstraction.

Constraining the Self type

If we were to forbid constraining through encapsulation while supporting impl Trait everywhere, then depending on the exact rules, it may become difficult to constrain a hidden type within a struct or enum from a method on that type. Consider if we were to add methods to our job queue:

struct Job(u64, impl Future<Output = ()>); // Not part of this proposal.

impl Job {
    // This method constrains the hidden type.
    fn prepare_job(&mut self) {
        self.0 = 0u64;
        self.1 = async { todo!() };
    }
    // ...other methods...
}

If we were to allow this code because self appears even though the containing type does not appear explicitly by name in the signature, then that would re-introduce a certain complexity when trying to grep for where the hidden type is constrained, as discussed above. However, if this were not allowed, it would be one more place where special tricks would be required.

Note too that this issue comes up even without impl Trait everywhere. Under the current proposal, we could equivalently write the following code by moving the hidden type into a type alias:

type JobFut = impl Future<Output = ()>;
struct Job(u64, JobFut);

impl Job {
    // This method constrains the hidden type.
    fn prepare_job(&mut self) {
        self.0 = 0u64;
        self.1 = async { todo!() };
    }
    // ...other methods...
}

We argue that it would be surprising if this variant did not work but that (in a world with impl Trait everywhere) putting the impl Trait directly into the Job struct did.

Potential future note: enum-variant types

There's an early proposal for Rust to support enum-variant types. Such a proposal would allow for the variants in an enum to be used directly as types. When combined with impl Trait everywhere, this code would become legal:

enum Job { // Not part of this proposal.
    Pending(impl Future<Output = ()>),
    Backlog(impl Future<Output = ()>),
}

// Definitely not part of this proposal.
async fn send_job(tx: Sender<Job::Pending>) {
    tx.send(Job::Pending(async { todo!() })).await;
}

This would add another wrinkle to the rules if we were to forbid constraining through encapsulation. Given that the innermost named type that contains the hidden type would now be the variant, allowing only the name of the enum to appear in the signature would itself become a form of constraining through encapsulation.

Such a future is very speculative and we do not mean to suggest that much weight should be placed on this. But we do claim this may be suggestive that constraining through encapsulation is robust to the ways that Rust might likely evolve in the future, and that a more restrictive rule might start to accumulate annoying edge cases as the language evolves.

Regarding optional #[defines(..)]

It has been proposed that we could in the alternative allow #[defines(..)] to be used in cases where the name of the innermost type alias containing the hidden type is not present in the signature of the function.

Such a design would ameliorate some but not all of the problems described above. In particular, we would still be:

  • Rejecting otherwise valid code that could in principle be made to "just work".
  • Creating divergence between impl Trait and dyn Trait.
  • Forcing users to pay the cost upfront to unwind their abstractions.
  • Forcing users to think carefully about the mechanism of impl Trait and the difference between using an impl Trait type and constraining it in a way that they don't have to do when using dyn Trait.
  • Forced to teach users about both the signature restriction and #[defines(..)].

Additionally, depending on how the rules of this new syntax were defined, the user may still have to engage in certain kinds of careful refactoring (such as breaking out hidden types into multiple type aliases) just to satisfy the compiler.

Because under this alternative #[defines(..)] would be optional and the hidden types would still be anonymous, we would be paying these costs without getting any of the benefits that would accrue to a fundamentally different design strategy.

From a process perspective, any #[defines(..)]-like mechanism would require careful design to address the exact handling of nits such as the behavior of type and lifetime parameters within the annotation. The authors believe this would require a new RFC. There is always the possibility that building consensus on such an RFC could take a long time or might never happen at all. During this time, if we were to forbid constraining through encapsulation, users would be writing and refactoring code according to the tricks and workarounds described in this document. We argue that this would cause unnecessary effort by users and unnecessary churn within the ecosystem.

Concept: The minimum required restriction

Neither RFC 2071 nor RFC 2515 envisioned any restrictions at all on which functions within the scope in which the hidden type was introduced could constrain it. These RFCs implicitly allowed all of the code whose legality is preserved by constraining through encapsulation.

Due to concern over implementation considerations in the compiler and IDEs, as discussed in Appendix A and Appendix B, we have adopted the signature restriction. The details of this restriction, including constraining through encapsulation, have been carefully designed to fully address these implementation considerations while still supporting the use cases described in the motivations above and allowing most of the useful code that people would want to write using this feature to "just work".

The signature restriction as proposed in this document represents the minimum restriction required to get these implementation benefits. We argue that this approach is the most congruent with the intent of the accepted RFCs.

We argue that further restrictions that would make more otherwise correct code illegal would serve a fundamentally stylistic purpose, and that such a purpose can be better addressed by education, best practices, coding guidelines, and linting.

Tradeoffs

This proposal makes a set of design choices to achieve these goals:

  • We want people to be able to replace the large body of existing Box<dyn Trait> code with impl Trait whenever they were using the former as a workaround.
  • We want that replacement to be easy and to result in minimal churn to existing code.
  • We want these and other common expected uses of this feature to "just work" as much as possible.
  • We want people to be able to rely on the abstraction and mostly forget about the mechanics.
  • We want people to be able to mostly apply their intuitions about dyn Trait to impl Trait and vice versa.

The ability to constrain the hidden type through encapsulation works together with the other design choices in this proposal (such as the other details of the signature restriction) to achieve these goals.

These goals lead us to design choices that incur certain costs. The most notable of these costs are the presence of cycle errors in certain (hopefully not-too-common) scenarios that must be worked around. If we were to abandon these goals – in particular, if we wanted to force users to be very aware of the mechanism and make it very different from dyn Trait – then essentially all of the choices in this proposal – and perhaps even those within RFC 2071 and RFC 2515 – should be reconsidered. Forbidding constraining through encapsulation would incur the many costs described above without getting any of the benefits that might accrue by reconsidering all of the design choices with a different set of goals.

Appendix D: The signature restriction on nested inner functions

This proposal requires that:

  • Nested functions may not constrain a hidden type from an outer scope unless the outer function also includes the hidden type in its signature.

For example, in the proposal, this code is invalid:

// Example A.
type Foo = impl Sized;

fn foo() {
    fn bar() -> Foo { () }
}

However, it has been suggested that this restriction feels surprising and inconsistent with the rule that allows child modules to constrain the hidden type.

In the remainder of this appendix, we'll discuss the tradeoffs involved in deciding whether this restriction should be kept or discarded.

Sneaky inner impls in Rust

Consider this code that stable Rust allows today:

// Example B.
trait Trait {}
struct S;
fn foo() {
    impl Trait for S {}
}

The impl within function body of foo affects the broader scope. We could say that it does this "sneakily" because it does so without mentioning anything about what it plans to affect in its signature.

Likewise, without the nested inner function restriction, we could say the same about foo in Example A above.

Why is this a problem?

As long as either Example A or Example B are legal in Rust, tools such as rust-analyzer must parse all function bodies to be sure that they have correctly inferred any type. The authors of these tools would prefer that this were not necessary as that could enable better performance while achieving correct behavior.

How might this be addressed?

There is a draft proposal that would make these sneaky inner impls illegal in a future edition of Rust. If that proposal were adopted, then it would be logical for the nested inner function restriction to apply to TAIT.

What has @matklad said about this?

Regarding these sneaky inner impls, why they are an issue, and what to do about them, @matklad has said:

I guess my overall thrust here is that, if we do this, its important to do 100% of it. We don't want to forbid this because it's an unreadable mess: no need to forbid something which isn't used anywhere outside of dtolnay's famous quiz. We want to forbid this, because we want to make the implementation faster. In some future edition, we want to just [throw] every function onto a threadpool and typecheck the world on all cores, without any synchronization. If there's at least one case where we need to invalidate typechcking results of foo when we learned something new when checking bar, this optimization simply doesn't work.

The authors have confirmed with @matklad that Example A above is the same problem as the other sneaky impls and that his statement applies to this case as well.

What decision do we have to make?

If we were sure that we were going to adopt the draft proposal to restrict sneaky impls in the next edition, then it might make sense to adopt the nested inner function restriction with the TAIT stabilization to avoid adding a new case that must be addressed over an edition. On the other hand, even if we were planning to adopt that proposal, we could take the view that it might still be more consistent to simply allow this in this edition and restrict it when the other cases are restricted. This may be even more true if we're not sure that we will in fact adopt the proposal, or if we're not sure that we will adopt it for the 2024 edition.

Appendix E: Nested inner modules

This proposal allows the hidden type to be constrained within a child scope, such as a child module, subject to the signature restriction for nested inner functions as discussed in Appendix D.

However, it has been suggested that this affordance feels inconsistent with the restriction on nested inner functions. In the alternate, it has been suggested that this affordance is too powerful that might result in too wide of a scope in which the hidden type may be constrained.

In the sections that follow in this appendix, we'll discuss these concerns.

Restricting child modules would be inconsistent

If we were to forbid constraining the hidden type in child modules, that would be inconsistent with the proposed rule on nested inner functions. The rule on those is that while this code is illegal, due to the signature restriction:

type Foo = impl Sized;

fn foo() {
    fn bar() -> Foo { () }
}

This code is legal:

type Foo = impl Sized;

fn foo() -> Foo {
    fn bar() -> Foo { () }
    bar()
}

That is, we allow nested inner functions to constrain the hidden type only if they and all outer functions pass the signature restriction. If we were to forbid constraining the hidden type in child modules, that would be a more severe restriction as there would be no corresponding way to make it work.

Restricting child modules has no tooling benefits

As discussed in Appendix D (the signature restriction on nested inner functions), Appendix A (the signature restriction), and Appendix B (the IDE concern), the signature restriction was introduced to facilitate a simpler and more performant implementation and to ease other tooling concerns. Forbidding the hidden type from being constrained in child modules has no analogous tooling benefits.

Child modules and cycle errors

People may hit cycle errors when using type alias impl Trait. When they do, we will suggest that they move the hidden type to a more narrow scope. If there are multiple interacting types using impl Trait, then a restriction on constraining the hidden type in a child module would impose additional constraints on this refactoring. We are still analyzing whether it would always be possible to find a legal refactoring under this restriction, but even if it is, we would still worry about forcing an undesired factoring onto the user and making the human into a compiler.

Constraining scopes will be narrow in practice

Regarding the concern that allowing the hidden type to be constrained in a child scope may result in allowing the hidden type to be constrained in "too wide" of a scope, please see this section for a complete discussion of that point.

Constraining uses can be found easily

Regarding the concern that allowing the hidden type to be constrained in a child scope may make it too difficult to find any constraining uses, note that it's always easy to find constraining uses, as discussed in this section.

Appendix F: Process problems with alternate proposals

During the recent push for stabilization of type alias impl Trait, various other proposals have been made for how this feature might work. Most notably, various flavors of a #[defines(..)] syntax have been suggested. In this appendix, we'll discuss the problems with these proposals as a matter of good process.

The signature restriction in this proposal accepts a strict subset of the code that is allowed under the accepted RFCs and can be later extended to the exact RFC semantics (if desired) without breaking backward compatibility. That is what makes it a straightforward partial stabilization.

Any alternate proposal that adds new syntax, such as #[defines(..)], would either allow code not allowed by the RFCs (such as if placed outside of the module in which the hidden type is introduced) or would disallow code that is allowed under the RFCs (if required to be placed anywhere else). This means that such a proposal would be adding new syntax and semantics rather than simply stabilizing a subset of the RFC.

Further, if we were to use that new syntax and those new semantics to reduce the number of cycle errors, then that would make it impossible to later allow the RFC semantics while preserving backward compatibility. That is, it would be a rejection of the RFCs.

Certainly sometimes what is stabilized does look different than what is in the RFC. But that tends to happen when a feature has evolved organically over a long period of time in unstable and a strong consensus has been built during that time that the RFC was imperfect, that the evolved semantics are clearly better, and that the evolution does not involve questions so deep, nuanced, and ponderous as to require those questions and their answers to be written out carefully.

Obviously that's not the case here. What has lived in unstable for years is an implementation that closely models the RFC semantics. Any #[defines(..)] proposal raises new and complicated questions for both language design and implementation. The compiler does not currently have any concept of types or even paths in attributes. On the language design side, such a proposal opens a wide design space including about the handling of paths, generic type parameters, and lifetimes, where the new syntax could be placed, what kinds of arguments the new syntax could accept, whether the syntax is optional, if the syntax is present whether the hidden type only may be constrained or whether it must be, the handling of hidden types contained within other hidden types that must be constrained together, and many other issues. Alternate designs would have to be considered, such as specifying the constraining items at the point that the hidden type is introduced or using a special where bound instead. And of course, all of this would need to be compared against the original goals and motivations for the feature as captured in the accepted RFCs.

For these and other reasons, the lang team decided that the details of any proposal similar to #[defines(..)] would need to be carefully considered in a new RFC.

The authors feel that the existing RFCs deserve considerable deference. These RFCs were the work of other smart people trying to thread this needle carefully and do what is best for Rust. The RFCs survived multiple consensus-building processes and were accepted. Over the long time that the implementation of these RFCs has soaked in unstable, no major problems have been found through experience and no major drawbacks have been uncovered that were not known and considered carefully at the time that these RFCs were accepted. The one major consideration that has been taken more seriously over time relates to tooling concerns, and those concerns have been addressed in this proposal via the signature restriction. As we've discussed above, this is the minimum required restriction.

As a practical matter, it's clear that many people strongly prefer the RFC semantics. Obviously we would expect that should be the case as those RFCs were in fact accepted. Those people are not being loud right now because there is no concrete proposal on the table to reject those semantics. However, this quiet consensus in favor of the existing RFCs would be a real problem for building an opposing consensus to reject those semantics. The view of the authors is that to block type alias impl Trait on trying to build this opposing consensus would be to block it indefinitely.

If type alias impl Trait were a new proposal without accepted RFCs and a long history in unstable, then it would be proper to consider all alternatives on an equal footing. However, that's not the situation. It would be wildly inefficient and rather perverse if accepted RFCs and the semantics of a feature during its time in unstable were given no deference. The RFCs deserve the benefit of the doubt.

Consequently, the authors propose that the relevant question is not whether the proposed partial stabilization strictly dominates all conceivable alternate solutions. Instead, we should ask:

  • Do we still feel the RFCs were solving real problems?
  • Have we found any reason to believe that the solution proposed in the RFCs does not actually solve those problems?
  • Have we since found any serious blocking issues that were unknown at the time of the RFCs, such as unsoundness?
  • For a partial stabilization, are we stabilizing a subset of the specified behavior that is reasonable, still addresses most or all of the problems the RFCs set out to address, and leaves room to stabilize more of the RFC-specified behavior in the future?

The view of the authors is that the type alias impl Trait RFCs were written to solve real problems that are still serious problems today in the Rust ecosystem. The solution defined in the RFCs is a reasonable solution to those problems. No issues have been found that were not considered and discussed at the time the RFCs were accepted. The tooling concerns, which we take more seriously today, have been addressed through the signature restriction. This restriction has been carefully designed. It leaves room for potential future stabilizations. And it addresses substantially all of the practical problems that the RFCs set out to address.

It is on this basis that we propose that we should proceed with the partial stabilization, as described in this proposal, of type alias impl Trait.

Select a repo