We are stabilizing #![feature(impl_trait_in_assoc_type)]
, commonly called either "associated type position impl Trait" (ATPIT) or "impl Trait in associated type" (ITIAT).
Among other things, this feature allows async
blocks and async fn
to be used in more places (instead of writing implementations of Future
by hand), filling gaps in the story of async Rust.
Stabilizing ATPIT helps the many Rust users who have well known use cases, including, e.g., the Tower library (in particular, e.g., its Service
trait) and its extensive ecosystem.
The theme of this stabilization report is simplicity. Through much work, we've found that we can stabilize a subset of RFC 2515 that solves the most demanded use cases while enabling a simple and robust implementation in the compiler, supporting efficient implementations in other tooling, and answering all previously-open language design questions in a principled way.
This is a partial stabilization of RFC 2515 and #63063.
We now allow impl Trait
to appear in type position in associated type definitions within trait implementations.
For the first time, we can now use async
blocks to implement IntoFuture
without boxing. E.g.:
Just as with RPIT, impl Trait
may appear syntactically multiple times within the type, and any item that does not register a hidden type for the opaque witnesses it as an opaque type. No items outside of the impl block may register hidden types for impl Trait
opaques defined within. The rules for which items within the impl block may do so are a simple extension of the RPIT rules and are described below.
Throughout this document, "registering a hidden type for an opaque type" and "defining the hidden type of an opaque type" can be treated as synonyms. More loosely, in context, "defining an opaque type" may also be taken to mean the same thing.
These phrases mean that an item is picking a concrete type or type constructor to underlie the opaque type.
Any item that is not allowed and required to register a hidden type for the opaque witnesses that opaque type as completely opaque. Such an item may only use the type via the interfaces defined by the traits that the opaque is declared to implement and those of the auto traits leaked through from the hidden type (exactly as with RPIT).
ATPIT is the extension of RPIT to associated type definitions and trait implementations. Everything that is true about RPIT opaque types is true about ATPIT opaque types modulo that:
impl Trait
opaque type appears in the signature of an item, that item may and must register a hidden type for that opaque.impl Trait
opaque type in the same impl block is syntactically reachable from an associated type in the signature of an item, that item may and must register a hidden type for that opaque.The rule to decide whether an item may and must register a hidden type for an opaque is strictly syntactic and strictly local to the impl block. It's an extension from considering just the signature, as with RPIT, to also considering the definitions of associated types in the same impl.
Intuitively, we collect all impl Trait
types within the same trait impl that are syntactically reachable from the signature of an item, considering only that item's signature and the associated type definitions within the impl block. We don't recurse into ADTs (e.g. the fields of a struct).
More precisely, the rule is as follows:
To determine the set of impl Trait
opaque types for which an item may and must register a hidden type, we first collect from the signature (including from any where
clauses) all types without normalization.
impl Trait<..>
and dyn Trait<..>
types, we:
impl Trait
into the set of opaque types for which this item may and must register a hidden type.Self
) match, collecting the unnormalized types, and repeating step 1.Note that RPIT already performs steps 1 and 1a. The stabilization of ATPIT adds only step 1b.
Following the rules for syntactic reachability, this works as we would expect:
If an impl Trait
opaque type is syntactically reachable from the signature of an item according to the syntactic reachability rule, then a hidden type may and must be registered by that item for the opaque.
Items not satisfying this predicate may not register a hidden type for the opaque.
Only the syntactic items that are direct children of the impl block may register hidden types for an impl Trait
opaque type in an associated type. Nested syntactic items within those items may not do so. As with RPIT, closures and async
blocks, which are in some sense items but are not syntactic ones and which share the generics and where
clauses of their parent, may register hidden types.
The design described in this document for ATPIT adheres to the following principles.
We believe that impl Trait
syntax in return position and in associated type position is for convenience and should have minimal overhead to use.
The design proposed for stabilization here solves common and important use cases well, conveniently, and with minimal overhead, similar to RPIT, by e.g. leaning on the syntactic reachability rule. Other use cases may prefer to wait for full type alias impl Trait
.
We believe that it should be easy to determine whether an item may and must register a hidden type of an impl Trait
opaque type.
In certain edge cases, whether or not an item may register a hidden type for an opaque can affect method selection and type inference. It should therefore be straightforward for the user to look syntactically at the impl block only to determine whether or not an item may register a hidden type for any opaque. This is what the syntactic reachability rule achieves.
We believe that the behavior of impl Trait
should be very crisp; if an item may register the hidden type for an opaque, then within that item the type should act exactly like an inference variable (existential quantification). If it cannot, then within that item the type should act like a generic type (universal quantification).
If items were allowed to register hidden types without being required to do so, then it is believed to be either difficult or impossible to maintain this kind of crispness in all circumstances. Consequently, this design adopts the "may define == must define" rule to preserve this crisp behavior.
We long ago stabilized async fn
and async { .. }
blocks as these make writing async Rust more pleasant than having to implement Future
everywhere by hand.
However, there's been a lingering problem with this story. The type of the futures returned by async fn
and async { .. }
blocks cannot be named, and we often need to name types to do useful things in Rust. This means that we can't use async
everywhere that we might want to use it, and it means that if our dependencies do use it, that can create problems for us that we can't fix ourselves.
It's for this reason that using RPIT in public APIs has long been considered an anti-pattern. Using the recently-stabilized RPITIT and AFIT in public APIs carries these same pitfalls while adding a new one: the inability for callers to set bounds on these types.
It is these problems, in the context of interfaces defined as traits, that ATPIT addresses.
async
in more placesToday, if we want to implement IntoFuture
for a type so that it can be awaited using .await
, we have no choice but to implement Future::poll
for some wrapper type by hand so that it can be named in the associated type of IntoFuture
.
With ATPIT, for the first time, we can use async { .. }
blocks to implement IntoFuture
. E.g.:
If someone were writing a Service
-like trait today, now that AFIT and RPITIT are stable, that person may think to write it as follows so that async
blocks or async fn
could be used in the impl:
However, as compared with using associated types, that would create four problems for users of the trait:
Future
can't be named so as e.g. to store it in a struct.Future
can't be named so as to set bounds on it, e.g. to require a Send
bound.Future
captures a lifetime we may not want to capture.In the future, there may be other and better solutions to some of these problems. But today, without ATPIT, trait authors face a dilemma. They must either accept all of these drawbacks or, alternatively, must accept that implementors of the trait will not be able to use async
blocks and will have to write manual implementations of Future
.
ATPIT offers us a way out of this dilemma. Trait authors can allow implementors to use async
blocks (and conceivably, in the future, async fn
) to implement the trait while preserving the object safety of the trait, allow users to name the type so as to store it and set bounds, and express precise capturing of type and lifetime generic parameters.
Opaque types in Rust today, including stable RPIT, allow themselves to be used opaquely in a body before a hidden type is registered for the opaque. There is a proposal to reject this (tracked in #117866). This is orthogonal to ATPIT except to the degree that ATPIT would provide new ways for people to write this sort of code. We're exploring this restriction in parallel with this proposed stabilization.
The stabilization of ATPIT would not have been possible without, in particular, the ongoing and outstanding work of @oli-obk, who has been quietly pushing forward on all aspects of type alias impl Trait for years. Thanks are also due, in no small part, to @compiler-errors for pushing forward both on this work directly and on critical foundations which have made this work possible. Similarly, we can't say enough positive things about the work that @lcnr has been and is doing on the new trait solver; that work has shaped this proposal and is also what gives us confidence about the ability to support and extend this feature into the long term.
Separately, the author of this stabilization report thanks @oli-obk, @compiler-errors, @tmandry, and @nikomatsakis for their personal support on this work.
Thanks are due to the types team generally for helping develop the "Mini-TAIT" proposal that preceded this work. And thanks are of course due to the authors of the RFC 1522, RFC 1951, RFC 2071, and RFC 2515, @Kimundi, @aturon, @cramertj, and @varkor, and to all those who contributed usefully to those designs and discussions.
Thanks to @nikomatsakis for setting out the design principles articulated in this document, and to @tmandry, @oli-obk, and @compiler-errors for reviewing drafts. All errors and omissions remain those of the author alone.
[This appendix will not appear in a separate document linked from the PR.]
TODO: Add ToC.
Yes. The idea here is to take what is good about RPIT, including its convenience, and to extend that to more places while fixing some of its flaws.
In particular, it's widely considered an anti-pattern to return an RPIT opaque type across a public interface as this creates problems for downstream callers since the type cannot be named. Since associated types can be named, this problem does not occur, and returning ATPIT types across a public interface is a correct and idiomatic thing to do.
The syntactic reachability rule means that we do not consider projections within ADTs, even if those projections lead back to an associated type defined within the impl block. Consequently, under the syntactic reachability rule, this does not work, even though it would under a semantic reachability rule:
In practice, under the syntactic reachability rule, to make this work, we must simply name the associated type Self::Item
within the signature of the method, such as by including it in a trivial where
clause.
The "may define == must define" rule requires that all items that are allowed to register a hidden type for an opaque under the syntactic reachability rule must register a hidden type for that opaque.
For example, the following does not work because the must_not_define
item attempts to register a hidden type for an impl Trait
opaque type not syntactically reachable from its signature.
The following does not work because the attempts_passthrough
item does not register a hidden type for the impl Trait
opaque type that is syntactically reachable from its signature, and it must do so:
In practice, to make that last example work, we must register a hidden type for the opaque while passing through the value:
The "sibling only" rule states that only the items that are direct children of the impl block may register hidden types for an impl Trait
opaque type in an associated type; nested items within those items may not do so.
For example, this does not work:
The first error is due to the sibling only rule. The second error is due to the fact that, under the "may define == must define" rule, the outer item must register a hidden type for the opaque but does not do so.
Tower users and library authors will now be able to implement the Tower Service
trait using async
blocks rather than having to write manual implementations of Future
.
Tower wants to support both statically-dispatched and dynamically-dispatched services. While Tower would probably prefer to allow async
blocks to be used to implement the Service
trait, even setting aside backward compatibility concerns, it cannot switch to using AFIT/RPITIT without giving up object safety.
Here's an example of how a simplified version of the trait may be used with dynamic dispatch today:
We may someday want to allow writing such a trait as follows, using AFIT:
However, today, using AFIT or RPITIT in the trait definition would prevent the trait from being object safe, meaning that downstream users could not use type erasure and dynamic dispatch.
ATPIT allows the existing object safe Tower Service
trait to be used while allowing it to be implemented using async
blocks.
Send
bound)We would like the Service
trait to be able to be implemented both by:
We would like for callers to be able to add bounds that rely on this distinction. This is known as the Send
bound problem. Consider:
If we then want to write this caller that relies on being able to send a generic service and its returned futures safely between threads, we need to be able to name the type of the returned future in the bounds of the caller. E.g.:
This isn't possible today in Rust using AFIT or RPITIT. It may be possible in the future with a solution such as Return Type Notation (RTN) or trait transformers (as embodied in the trait-variant
crate), but these would break compatibility.
ATPIT offers a solution today, allowing the Service
trait to be implemented using async
blocks while also allowing callers to set bounds.
Yes.
TODO.
TODO.
Yes.
Yes.
All generics are. See Lifetime Capture Rules 2024.
No.
No. We have fully separated TAIT from ATPIT in both design and implementation.
TODO.
TODO.
Inference changes.
The new trait solver is being designed around the restrictions.
defines
syntax for ATPIT?TODO.
TODO.
TODO.
Defining in inner items would be weird anyway since the outer item must still define.
RPIT can be used on inner items to get most of the same benefit.
TODO:
Only the items that are direct children of the impl block may register hidden types for an impl Trait
opaque type in an associated type. Nested items within those items may not do so.
This follows naturally from the "may define == must define" rule, the syntactic reachability rule, and an existing rule in Rust that inner items cannot use generic parameters from outer items.
For example, this does not work because though the impl Trait
is syntactically reachable from the signature of the inner item, the inner item is not allowed to use the outer associated type in its signature:
If, conversely, we did not use the associated type in the signature of the inner item, then it would fail the syntactic reachability rule.
If instead we were to not use Self
in the signature of the inner item and project to the associated type by naming the type, and we were to allow this, this would still fail due to the "may define == must define" rule. E.g.:
The problem here is that the outer item is required to register a hidden type for the opaque but, in this example, it instead just passes through the value, which we do not allow.
There is a correct and supported way to return opaques from inner items. We must simply use RPIT on these inner items. In this way, the outer item registers the hidden type for the opaque (by using a separate opaque type returned by the inner item), satisfying the "may define == must define" rule. Consequently this code is accepted:
TODO.
No.
No.
Due to the syntactic reachability rule, we can locally determine whether any item may register a hidden type for an impl Trait
opaque type by looking only at the signature of the item and the associated type definitions in the impl.
Due to the "may define == must define" rule, we know that any item that may register a hidden type for an impl Trait
opaque type must do so, and so must contain a "defining use."
Separately from that, it's always possible to provoke the compiler to tell us exactly where all defining uses for a particular opaque type are located simply by defining a conflicting one and running rustc
.
For example, if we want to find all defining uses in this code:
We can simply insert one line, run rustc
, and it will point us to all defining uses:
No. For the same reasons that humans can locally determine whether an item may register a hidden type for an opaque, tooling such as rust-analyzer can also.
Due to the syntactic reachability rule, tools can locally determine whether any item may register a hidden type for an impl Trait
opaque type by looking only at the signature of the item and the associated type definitions in the impl.
Due to the "may define == must define" rule, tolls can know that any item that may register a hidden type for an impl Trait
opaque type must do so, and so must contain a "defining use."
TODO.
There has been discussion that rather than requiring that users write, e.g.:
We could additionally accept:
Using the trait definition, we could infer that impl Future<Output = u8>
must be the definition for the Self::Item
associated type.
Similarly, this could allow us to use async fn
syntax in the trait definition, so that instead of writing, e.g.:
We could instead write:
This is a possibility that we leave to future work.
If we want to write:
We can't quite write it that way because the associated type with the impl Trait
does not appear in the signature of new
in the trait impl. What we can write instead is:
This takes advantage of the fact that we can always write where bounds that are trivially satisfied.
In the future, we may want to allow attributes on individual where clauses so that we can exclude these bounds from the documentation by writing:
TODO: Describe that this isn't the main use case for ATPIT. This isn't the intended use case, so we're OK with it being weird. We're just demonstrating that the feature is powerful enough to do this.
Sometimes, e.g. for embedded Rust, we might want to initialize static items with unnameable types such as futures. This has not previously been possible to do in a reasonable way in stable Rust. With ATPIT, we can now do this. E.g.:
Sometimes we may want to provide from our API an opaque type that callers can use only by handing it back to our API or for its declared trait bounds. However, within our own crate, we want to be able to rely on the concrete value of that type, including after it has been passed back to us from the outside.
The typical solution to this problem in Rust is using the newtype
pattern; i.e. wrapping the type in a struct and writing all necessary trait forwarding impls.
With ATPIT, there is now another option. We can use the "hide/reveal pattern". E.g.:
For example:
For example:
type_of
?You cannot do this. This requires TAIT.
const fn
?You cannot do this. This requires either TAIT or const fn
in traits.
Cannot write:
Could write:
I've heard that AFIT/RPITIT can be desugared into ATPIT. How?
If we have a trait and trait impl using AFIT:
We can first desugar that into RPITIT:
Then we can desugar to GATs and ATPIT:
Send
bounds problem for my traits?Desugar to GATs and use ATPIT in the trait impls. Since the types are named, callers can set bounds.
The scenario:
We can write:
The scenario:
Solution:
If the author of an upstream crate provides a trait that uses AFIT or RPITIT instead of naming the returned opaque type with an associated type, and we find that we do need to name that type, we can "wrap" the upstream trait in a new trait that does include the needed associated type, then provide a blanket impl that uses ATPIT. E.g.:
Yes. The implementation has been baking for years, and the increased simplicity of this design has made the implementation simpler and more robust.
On top of ATPIT, we could write a proc macro that RTN-ifies a trait definition by converting all RPITITs and AFITs to named GATs. It'd be similar to trait-variant
.
On top of ATPIT, we could write a #[capture(..)]
proc macro that would be applied to functions and list the generic arguments to be captured. It would transform it into a trait, a trait method, and a stub function.