Type Alias and Associated Type Impl Trait

Tracking issues

• "Tracking issue for RFC 2515, "Permit impl Trait in type aliases"" rust#63063
• "TAIT defining scope options" rust#107645

Definitions

Acronyms

• APIT – argument position impl trait (fn foo(x: impl Trait))
• RPIT – return position impl trait (fn foo() -> impl Trait)
• RPITIT – return position impl trait in trait (trait Foo { fn bar() -> impl Baz)
• TAIT – type alias referencing impl Trait, i.e., type Foo = impl Bar at module level
• AssocIT – associated type value referencing impl Trait, i.e., impl Foo for Bar { type Baz = impl Qux; }

Other terms

• Each use of impl Trait desugars to an opaque type that has a hidden type which must meet the declared bounds; note that a TAIT (or RPIT, etc) may have multiple opaque types (e.g., type Foo = (impl Bar, impl Baz)).
• The defining scope of an opaque type corresponds to the lexical scope in which constraints on the hidden may appear. Outside of this scope, the opaque type is always used opaquely. For purposes of this discussion, the defining scope is always the enclosing module or item.
• Within the defining scope, an item is called constraining if it puts constraints on the value of the TAIT. i.e., for the item to type check, the hidden type of the TAIT must have a particular value. This could occur because of a let (e.g., let x: TAIT = 22_u32), a return (e.g., return 22_u32 in a function whose return type is TAIT), or in other ways.
• We haven't yet fully defined where we wish to allow constrained items. Therefore, some subset of constraining uses are called defining uses, and that basically means "places where it is allowed to constrain". All defining uses must fully infer the hidden type of the TAIT and must infer the same type for the TAIT.
• Any constraining item within the defining scope that is not a defining use is a hard error. This means we can later opt to allow such a use; or to allow it with an annotation of some kind; or to make other such changes.

Desired patterns

"Single method traits" that return futures, iterators, etc

The defining characteristic here is that you have a simple trait that returns a future, iterator, or something like it.

Example: Tower service trait

impl Service for Something {
    type Future = impl Future<Output = ...>;
    
    fn service(&mut self) -> Self::Future {
        async move {
            ...
        }
    }
}

Example: IntoIterator trait

impl IntoIterator for &MyType {
    type IntoIter = impl Iterator<Item = Self::Item>;
    type Item = ...;
    
    fn into_iter(self) -> Self::IntoIter {
        ...
    }
    
}

Embassy: const/static initializers

Source

In Embassy, the user writes this:

#[embassy_executor::task]
async fn run() {
    // stuff
}

which gets expanded to the following, which references a TAIT Fut in the type of a constant:

async fn __run1_task() {
    // stuff
}

fn run1() -> ::embassy_executor::SpawnToken<impl Sized> {
    type Fut = impl ::core::future::Future + 'static;
    static POOL: ::embassy_executor::raw::TaskPool<Fut, 1usize> = ::embassy_executor::raw::TaskPool::new();

    // defining use here. Signature for `_spawn_async_fn` is:
    // impl<F: Future + 'static, const N: usize> TaskPool<F, N> {
    //     fn _spawn_async_fn<FutFn>(&'static self, future: FutFn) -> SpawnToken<impl Sized>
    //          where FutFn: FnOnce() -> F;
    // }
    // so it constrains `Fut` to be the future for `__run1_task()`.
    unsafe { POOL._spawn_async_fn(move || __run1_task()) }
}

The "jplatte" example

source

Based on the real-world request and response traits from Ruma, which the project is planning to switch from converting from one-step HTTP deserialization to two steps:

• Deserialize the body – converting http::Request<SomeRawByteType> (or Response) to http::Request<RequestTypeBodyFields>
• Convert the result of the first step to a Rust type containing both the body fields and extra fields extracted from headers / path parameters for requests

This is done for two reasons:

• It is planned that the Matrix protocol that Ruma implements will support CBOR as an extra encoding for the body contents in the future
• While this introduces an extra associated type, it actually makes a trait method non-generic (while introducing another one, but with smaller scope), so is hoped to improve compile times of downstream crates as they have to instantiate less generic code themselves

See this PR (which was merged, but not into the main branch; latest rebase here).

trait Request {
    type Body: Serialize;
    // More common case: Associated type is an output of a trait method
    fn into_http_request(self) -> http::Request<Self::Body>;
}

trait Response {
    type Body: DeserializeOwned;
    // Less common case: Associated type is an input of a trait method
    fn try_from_http_response(http::Response<Self::Body>) -> anyhow::Result<Self>;
}

/// Request type for `GET /foo/:bar`
#[derive(Request)]
#[request(method = GET, path = "/foo/{bar}")]
pub struct MyHttpRequest {
    // path fields
    #[request(path)]
    bar: String,

    // body fields
    #[serde(skip_serializing_if = "Option::is_none")]
    field_a: Option<i32>,
    field_b: SomethingElse,
}

// generated by Request derive
// note: not public
#[derive(Serialize)]
struct MyHttpRequestBody {
    #[serde(skip_serializing_if = "Option::is_none")]
    field_a: Option<i32>,
    field_b: SomethingElse,
}

// also generated by Request derive
impl Request for MyHttpRequest {
    type Body = impl Serialize;

    fn into_http_request(self) -> http::Request<Self::Body> {
        let Self { bar, field_a, field_b } = self;
        let uri = format!("/foo/{bar}", bar=bar);
        let body = MyHttpRequestBody { field_a, field_b };
        http::Request::builder().method(Method::GET).uri(uri).body(body).unwrap()
    }
}

/// Response type for `GET /foo/:bar`
#[derive(Response)]
pub struct MyHttpResponse {
    #[response(header = http::header::EXPIRES)]
    expires: Option<Timestamp>,

    #[serde(default)]
    field_x: u32,
    field_y: SomeData,
}

// generated by Response derive
// note: not public
#[derive(Deserialize)]
struct MyHttpResponseBody {
    #[serde(default)]
    field_x: u32,
    field_y: SomeData,
}

// also generated by Response derive
impl Response for MyHttpResponse {
    type Body = impl DeserializeOwned;

    fn try_from_http_response(res: http::Response<Self::Body>) -> anyhow::Result<Self> {
        let expires = res.headers()
            .get(http::header::EXPIRES)
            .map(Timestamp::try_from)
            .transpose()?;
        let MyHttpResponseBody { field_x, field_y } = req.into_body();
        Ok(Self { expires, field_a, field_b })
    }
}

Magic &'static mut macro

Converts from T to &'static mut T without alloc. (Sort of equivalent to Box::leak, except you can run it only once). source

macro_rules! singleton {
    ($val:expr) => {{
        type T = impl Sized;
        static STATIC_CELL: static_cell::StaticCell<T> = static_cell::StaticCell::new();
        // the 1-item tuple is a hack to ensure we return a `&mut T` and
        // not a `&mut impl Sized`, which would cause type inference to behave weirdly.
        let (x,) = STATIC_CELL.init(($val,));
        x
    }};
}

let x = singleton!(42u32); // gives a &'static mut u32

Major proposals

Stabilize TAIT + AssocIT, but only in return type or value of a const/static

A defining use for an opaque type O appearing in a TAIT or an AssocIT is…

• a function that contains O in its return type;
• a constant or static that contains O in its declared type;
• an impl where O appears in the value of an associated type and which contains either
  • a method that contains O in its return type; or
  • a constant that contains O in its type.

Rationale: You can define a TAIT in exactly those places where you could write impl Trait and have it desugar to a TAIT.

Advantages: Easy to explain what a defining use is.

Disadvantages: Supports all examples except for the "jplatte" example. This example doesn't work because the opaque type only appears as a method.

Commitments: Not impossible to add #[defines] later, but harder; the story is muddled. When do you need #[defines] exactly and why?

Question: do we accept this?

Stabilize AssocIT, allow constraints anywhere in impl

This proposal keeps TAITs unstable and only stabilizes AssocIT only. A defining use would be any impl that is a constraining use. This accepts all the impl examples. It does not accept embassy, though embassy can be rewritten to use this on stable (it is sort of awkward).

Rationale: Modules are different categorically from other items (including most impls). For most items, they are small, and when you use impl Trait there, it's natural that the "scope" that can constrain it is any place within that item. In contrast, modules often contain distinct "groupings" of items that belong together but for which it is not convenient to declare a distinct module. Let's worry about that case later, and focus just on the impl case to start.

Advantages: All examples work, albeit with some annoyance to manage true TAITs.

Disadvantages: TAITs remain unstable; we lose the analogy of an impl body to a module.

Commitments: We cannot add #[defines] later on impls, though we could lint for it and recommend its use if experience found that it is common to have confusion due to impls like the jplatte example.

Require #[defines] on both TAIT and AssocIT

A defining use for an opaque type O appearing in a TAIT or an AssocIT is…

• any item within the defining scope that is designated with #[defines(X)] (for a TAIT) or #[defines(Self::X)] (for an AssocIT)

Advantages: Explicit

Disadvantages: Verbose, especially for cases like the impl; not clear that this adds value.

Also, if we use #[defines] for AssocIT, we have to add ability to deal with types in attributes, decide if we want to normalize, etc. If we just limit #[defines] to TAITs, we only think about paths. Regardless we don't have a ton of precedent here.

Commitments: We can add more natural forms later, of course, but then we have more options.

Appendix and corner cases to consider

Constraint from an embedded item

This works today, should it?

impl SomeTrait for MyType {
    type SomeType = impl core::fmt::Debug;

    fn process(self) {
        const X: <MyType as SomeTrait>::SomeType = 22_u32;
    }
}

Constraint due to matching trait/impl

This doesn't work, should it? (Niko thinks no)

#![feature(type_alias_impl_trait)]

use std::ops::Add;

struct MyType;

impl Add for MyType {
    type Output = impl std::fmt::Debug;
    fn add(self, other: MyType) -> u32 {
        22
    }
}

fn main() { }

Alternatives and future possibilities

Here are some other points on the design space that might be useful as we consider issues with the current proposal.

Associated type inference

Some use cases can be addressed by simply not requiring the user to write the associated type value…

impl<T: Debug> IntoIterator for DebugSampler {
    type Item = String;
    // type IntoIter not specified
    
    fn into_iter(self) -> Self::IntoIter {
        self.inner
            .into_iter()
            .chunks(self.chunk_size)
            .enumerate()
            .map(|(i, c)| format!(...))
    }
}

Explicit type inference

The above proposal could be made more explicit with something like

type IntoIter = _;

Notably, inference would not include hiding details of the inner type. This might be desired in cases where a user actually wants to reveal those details, but can't or doesn't want to write the type. Hiding could still be accomplished with impl Trait, and these syntax proposals can even be combined:

type IntoIter = _ as impl Iterator<Item = Self::Item>;

If a user wants hiding but not inference, they are free to do that as well:

type Foo = u32 as impl Debug;

Then we would still have the question of what is considered a defining use, but it could be scoped to explicit uses of _ and not implied by any use of impl Trait. That way if there are nonobvious type inference behaviors that result from use of _ you only "pay for what you use".

Inspired by this blog series: https://david.kolo.ski/blog/a-new-impl-trait-2/.

No type inference

The above series suggests going further and removing type inference-based mechanisms altogether, replacing them with some explicit way of labelling the definition site of a type with a placeholder name:

fn generate<T>(value: T) -> type 'A {
    'A: move || value
}

This is like taking #[defines] past the function and going all the way down to the expression level.

Questions

Mark: clarification on meaning

Any constraining item within the defining scope that is not a defining use is a hard error. This means we can later opt to allow such a use; or to allow it with an annotation of some kind; or to make other such changes.

To make sure I follow this, is it true that any such use could be made into a defining use? Or are we saying that in some places within the defining scope, you can get "stuck" into a compile error you can't really do anything productive about?

type Foo = impl Debug;

fn foo() {
    // imagine this were allowed:
    let x: Foo = 22_u32;
    
    // imagine this were not allowed:
    let x: Foo = { let y: Foo = 22_u32; y };
}

pnkfelix: simple example, method with an explicit return type that you inline into a call?

type Internal = impl Debug;

fn produce() -> Internal {
    22
}

fn consume() {
    let x = produce();
}

// becomes

type Internal = impl Debug;

fn consume() {
    let x: Internal = 22; // ERROR
}

simulacrum: yeah, I could see this being annoying and confusing. Feels easy to get in a situation where you are stuck.

pnkfelix: could imagine that you have to add the #[defines] attribute to accept that case.

simulacrum: I feel like I don't have a good enough grasp on the limitations to say if that helps or not, but I can feel that most users will not read the rules and understand them, so they'll self-discover them based on usage.

pnkfelix: compiler could give you a nice error message here, right? Where would #[defines] go, anyway?

oscherer: should be on the function to get true benefit

nikomatsakis: part of the confusing is that the defining use / constraining use terminolgoy is there, it's meant to be a temporary thing to "hold space" for future decisions.

summary of the concern:

Whatever the rational, limitations, etc are: I would like to be able to refactor my code in ways. Particularly since as you refactor you'll get complex errors. You might get the error quite late after you've fixed a bunch of other things.

oscherer: error would occur early.

pnkfelix: would we be able to provide good feedback for defines attribute?

oscherer: moment you have a mismatch with an opaque type, we could suggest adding the attribute

clarifying the first proposal

allowed:

type Foo = impl Debug;

impl Iterator for Something {
    type Item = Foo;
    
    fn next(&mut self) -> Option<Foo> { }
}

type Foo = impl Debug;

impl Iterator for Something {
    type Item = Foo;
    
    fn next(&mut self) -> Option<Self::Item> { }
}

impl Iterator for Something {
    type Item = impl Debug;
    fn next(&mut self) -> Option<Self::Item> { }
}

not allowed:

type Foo = impl Debug;

impl Iterator for Something {
    type Item = Foo;
    
    fn next(&mut self) -> Option<u32> { 
    }
}

Oli: We haven't talked about privacy yet at all

We can end up having a function nameable, but not its return type, even though
Adts must be public if used in a public API.

mod foo {
    type Foo = impl Debug;
    pub fn foo() -> Foo {}
}
let x: ??? = foo();

the _ as impl Trait proposal

nikomatsakis: I don't hate the _ as impl Trait family of proposals, actually, but I'm also loath to revisit the design in such a big way. Is there a forwards compatible path to that if we ever want to do something like that?

(scottmcm: Going even further and having things like Add default to inferring their Output is tempting too – re-specifying it for those single-method traits is mostly just noise.)

other kinds of explicitness

nikomatsakis: The #[defines] attribute makes the "defining uses" explicit, but people still can't specify the "hidden type", which I guess is where the as impl Trait syntax comes into play. Are there other forms of implicitness not being specified here?

Just for fun

{
    foo::<{ type Bar = impl Sized; let x: Bar = 3; x}>(); // Defining scope?
}

Oli: use a where bound instead of a #[defines] attribute

• pro: already can handle associated types
• pro: exactly to how we want to handle it in the impl (put it as a new predicate into the param env)
• con: need to ignore it for things like matching impl method signatures against trait method signatures (as the trait won't have it)

type Foo = impl Debug;

fn foo() defines Foo {
    let x: Foo = 42;
}

fn foo() {
    let x: def Foo = 42;
}

• can also use the attribute and desugar to unstable syntax

  • i.e. k#defines ?

If we have syntax, inverting the arrows?

scottmcm: Rather than adding syntax to everything, we could have it special on type. That'd make the grammar change more scoped.

type Foo = impl Debug defined_by bar;

fn bar() {
    let x: Foo = 42;
}

That has the nice behaviour that if you're a human wondering what the type actually is, you can also look at the place that the type alias tells you to look. (In addition to being easier for R-A too.)

(Oh, or even as part of the impl Trait syntax, interesting.)

niko: How do we specify that it's constrained by an impl?

scottmcm: Maybe UFCS? That does mean it's not just a path, though :(

Rough consensus towards end of meeting

• Favorable towards "defines" but syntax is unclear