owned this note
owned this note
Published
Linked with GitHub
---
title: "Design meeting 2024-06-05: Precise capturing: `impl use<..> Trait` vs `use<..> impl Trait`"
tags: ["T-lang", "design-meeting", "minutes"]
date: 2024-06-05
discussion: https://rust-lang.zulipchat.com/#narrow/stream/410673-t-lang.2Fmeetings/topic/Design.20meeting.202024-06-05
url: https://hackmd.io/uvTw8ss4STSaSWs2PA1bQw
---
# Introduction
We've decided to add syntax for the precise capturing of generic parameters in RPIT-like `impl Trait` opaque types. This solves the problem of overcapturing and will allow us to fully stabilize the Lifetime Capture Rules 2024 for RPIT in Rust 2024.
To make this work, we need to stabilize this syntax ahead of or along with the edition. To best support migration efforts, it would be better to stabilize this sooner rather than later.
To do that, we need to finalize the exact syntax. Based on our earlier discussions leading to the acceptance of RFC 3617, the key choice here is between:
1. `impl use<..> Trait`
2. `use<..> impl Trait`
The goal of this meeting is to pick which one of these we want.
To lay my own cards on the table, as the author, having now written out the arguments every which way, I find myself deeply sympathetic to the good arguments for both choices. As discussed below, in favor of `impl use<..> Trait` is that it follows the natural ordering when applying generics as arguments to a type. In favor of `use<..> impl Trait` is an appealing kind of syntactic *principledness*, in that it keeps `use<..>` away from the bounds (which it is not a part of) in the same way that preferring `super let pat = expr` to `let super pat = expr` keeps `super` away from the pattern (which it is not a part of), and is why the former was preferred by the draft RFC for `super let`.
Mostly I think we're going to be happy either way. We'll just have to pick one.
Niko has been using this feature in nightly and recently reported his experience:
> I thought I didn't care but within a few minutes of using it I found that `use<> impl` just felt _much_ simpler to me. The best explanation I can give is that it feels less "stacked" -- i.e., I can look at the `use` and process it as a kind of "prefix" on the `impl Trait` type, rather than being an inner part that I have to think carefully about. That combined with the different scoping relative to `for` feels like a very strong argument to me.
The remainder of the document is a detailed comparative analysis of these options drawn from the relevant alternatives section from RFC 3617.
# Alternatives
## Syntax
### `use<..> impl Trait`
Putting the `use<..>` specifier *before* the `impl` keyword is potentially appealing as `use<..>` applies to the entire `impl Trait` opaque type rather than to just one of the bounds, and this ordering might better suggest that.
Let's discuss some arguments for this, some arguments against it, and then discuss the fundamental tension here.
#### The case for `use<..>` before `impl`
We've been referring to the syntax for RPIT-like opaque types as `impl Trait`, as is commonly done. But this is a bit imprecise. The syntax is really `impl $bounds`. We might say, e.g.:
```rust
fn foo() -> impl 'static + Unpin + for<'a> FnMut(&'a ()) {
|_| ()
}
```
Each *bound*, separated by `+`, may be a *lifetime* or a *trait bound*. Each trait bound may include a higher ranked `for<..>` *binder*. The lifetimes introduced in such a binder are in scope only for the bound in which that binder appears.
This could create confusion with `use<..>` after `impl`. If we say, e.g.:
```rust
fn foo<'a>(
_: &'a (),
) -> impl use<'a> for<'b> FnMut(&'b ()) + for<'c> Trait<'c> {
// ^^^^^^^ ^^^^^^^ ^^^^^^^
// | | ^ Applies to one bound.
// | ^ Applies to one bound.
// ^ Applies to the whole type.
|_| ()
}
```
...then it may feel like `use<..>` should apply to only the first bound, just as the `for<..>` binder right next to it does. Putting `use<..>` *before* `impl` might avoid this issue. E.g.:
```rust
fn foo<'a>(
_: &'a (),
) -> use<'a> impl for<'b> FnMut(&'b ()) + for<'c> Trait<'c> {
|_| ()
}
```
This would make it clear that `use<..>` applies to the entire type. This seems the strongest argument for putting `use<..>` before `impl`, and it's a *good* one.
#### The case for and against `use<..>` before `impl`
There are some other known arguments for this ordering that may or may not resonate with the reader; we'll present these, along with the standard arguments that might be made in response, as an imagined conversation between Alice and Bob:
> **Bob**: We call the base feature here "`impl Trait`". Anything that we put between the `impl` and the `Trait` could make this less recognizable to people.
>
> **Alice**: Maybe, but users don't literally write the words `impl Trait`; they write `impl` and then a set of bounds. They could even write `impl 'static + Fn()`, e.g. The fact that there can be multiple traits and that a lifetime or a `for<..>` binder could come between the `impl` and the first trait doesn't seem to be a problem here, so maybe adding `use<..>` won't be either.
>
> **Bob**: But what about the orthography? In English, we might say "using 'x, we implement the trait". We'd probably try to avoid saying "we implement, using 'x, the trait". Putting `use<..>` first better lines up with this.
>
> **Alice**: Is that true? Would we always prefer the first version? To my ears, "using 'x, we implement the trait" sounds a bit like something Yoda would say. I'd probably say the second version, if I had to choose. Really, of course, I'd mostly try to say instead that "we implement the trait using 'x", but there are probably good reasons to not use that ordering here in Rust.
>
> **Bob**: The RFC talks about maybe later extending the `use<..>` syntax to closure-like blocks, e.g. `use<> |x| x`. If it makes sense to put the `use<..>` first here, shouldn't we put it first in `use<..> impl Trait`?
>
> **Alice**: That's interesting to think about. In the case of closure-like blocks, we'd probably want to put the `use<..>` in the same position as `move` as it could be extended to serve a similar purpose. For closures, that would mean putting it before the arguments, e.g. `use<> |x| x`, just as we do with `move`. But this would also imply that `use<..>` should appear *after* certain keywords, e.g. for `async` blocks we currently write `async move {}`, so maybe here we would write `async use<> {}`.
>
> **Alice**: There is a key difference to keep in mind here. Closure-like blocks are *expressions* but `impl Trait` is syntax for a *type*. We often have different conventions between type position and expression position in Rust. Maybe (or maybe not) this is a place where that distinction could matter.
#### The case against `use<..>` before `impl`
The `use<..>` specifier syntax *applies* the listed generic *parameters* as generic *arguments* to the opaque type. It's analogous, e.g., with the generic arguments here:
```rust
impl Trait for () {
type Opaque<'t, T> = Concrete<'t, T>
// ^^^^^^^^ ^^^^^
// ^ Type ^ Generic arguments
where Self: 'static;
// ^^^^^^^^^^^^^
// ^ Bounds
}
```
Just as the above *applies* `<'t, T>` to `Concrete`, `use<..>` applies its arguments to the opaque type.
In the above example and throughout Rust, we observe the following order: *type*, *generic arguments* (applied to the type), *bounds*. In `impl Trait` syntax, the `impl` keyword is the stand-in for the opaque type itself. The `use<..>` specifier lists the generic arguments to be applied to that type. Then the bounds follow. Putting `use<..>` after `impl` is consistent with this rule, but the other way would be inconsistent.
This observation, that we're applying generic *arguments* to the opaque type and that the `impl` keyword is the stand-in for that type, is also a strong argument in favor of `impl<..> Trait` syntax. It's conceivable that we'll later, with more experience and consistently with [Stroustrup's Rule][], decide that we want to be more concise and adopt the `impl<..> Trait` syntax after all. One of the advantages of placing `use<..>` after `impl` is that there would be less visual and conceptual churn in later making that change.
Finally, there's one other practical advantage to placing `impl` before `use<..>`. If we were to do it the other way and place `use<..>` before `impl`, we would need to make a backward incompatible change to the `ty` macro matcher fragment specifier. This would require us to migrate this specifier according to our policy in [RFC 3531][]. This is something we could do, but it is a cost on us and on our users, even if only a modest one.
[RFC 3531]: https://github.com/rust-lang/rfcs/blob/master/text/3531-macro-fragment-policy.md
[Stroustrup's Rule]: https://www.thefeedbackloop.xyz/stroustrups-rule-and-layering-over-time/
#### The fundamental tension on `impl use<..>` vs. `use<..> impl`
Throughout this RFC, we've given two intuitions for the semantics of `use<..>`:
- **Intuition #1**: `use<..>` *applies* generic arguments to the opaque type.
- **Intuition #2**: `use<..>` brings generic parameters *into scope* for the hidden type.
These are *both* true and are both valid *intuitions*, but there's some tension between these for making this syntax choice.
It's often helpful to think of `impl Trait` in terms of generic associated types (GATs), and let's make that analogy here. Consider:
```rust
impl Trait for () {
type Opaque<'t, T> = Concrete<'t, T>;
// ^^^^^^ ^^^^^ ^^^^^^^^ ^^^^^
// | | | ^ Generic arguments applied
// | | ^ Concrete type
// | ^ Generic parameters introduced into scope
// ^ Alias type (similar to an opaque type)
fn foo<T>(&self) -> Self::Opaque<'_, T> { todo!() }
// ^^^^^^^^^^^^ ^^^^^
// ^ Alias type ^ Generic arguments applied
}
```
The question is, are the generics in `use<..>` more like the generic *parameters* or more like the generic *arguments* above?
If these generics are more like the generic *arguments* above (*Intuition #1*), then `impl<..> Trait` and `impl use<..> Trait` make a lot of sense as we're *applying* these arguments to the type. In Rust, when we're applying generic arguments to a type, the generic arguments appear *after* the type, and `impl` is the stand-in for the type here.
However, if these generics are more like the generic *parameters* above (*Intuition #2*), then `use<..> impl Trait` makes more sense. In Rust, when we're putting generic parameters into scope, they appear before the type.
Since both intuitions are valid, but each argues for a different syntax choice, picking one is tough. The authors are sympathetic to both choices.
---
# Discussion
## Attendance
- People: TC, scottmcm, tmandry, pnkfelix, nikomatsakis, eholk, Josh, Xiang
## Meeting roles
- Minutes, driver: TC
## Previous discussions
The 2024-04-24 design meeting:
https://hackmd.io/PohGwog9SLK-XCg0ZWTpuQ
The 2024-04-03 planning meeting with the first bikeshed:
https://hackmd.io/c_-Fm49QRASsH7m-0VmtZw
## Remind me where this is actually usable?
scottmcm: this mentions the ty matcher, but can this really be used in every `ty` position? What would it mean to have `fn foo<...>(x: use<...> impl Trait)`, for example? Can I do `impl Foo<...> for Bar<...> { type MyType = use<...> impl Trait; }`?
(I'm wondering if there's a more meaningful distinction here than just `ty_2021` vs `ty_2024`, like we have `pat_param` vs `pat`. Is there a `ty_return` vs normal `ty`, say?)
eholk: I don't immediately see why we couldn't, but I'd need to look more closely at the grammar.
NM: There are two questions. On is what we would parse. The other is what we accept semantically. We'd want to reject this in some places certainly.
eholk: As with other places, the macro matcher would probably accept it generally, but if it expands to a place where it can't be used, that would be an error. We have some precedent with how `_` is matched by `expr`.
scottmcm: There's also the precedent here of `pat_param` vs `pat`. If this can never show up in an argument and only in a return type... it makes me wonder if there's a possibility here to leave `ty` alone and add `ty_return`.
tmandry: Actually, I'm wondering why this would never be useful in argument position.
eholk: I like this idea of writing a `ty_return` or whatnot. There are ergonomics considerations here. There are places we could end up with really precise matchers that aren't generally useful. But on the other hand, it'd be good to have confidence that expanding something matched would be legal in the intended place of use.
pnkfelix: These could also be used in type alias position. That might raise questions for that name.
pnkfelix: I'm confused by how this would be used in argument position. This seems deeply tied to opaque types.
NM: That's correct. I can maybe imagine future words in which it could have a meaning, but I'm not sure we'd go there. What this says in the abstract is that, "as long as this type is valid, these other types must also be valid, as far as the borrows go." But for argument generics, the types have to be valid throughout the function anyway.
```rust
fn foo<T, U>(arg: impl Debug) {
// becomes `fn foo<T, U, V>() where V: Debug`
}
fn foo<T, U>(arg: use<T> impl Debug) {
// ...also becomes `fn foo<T, U, V>() where V: Debug` ...?
}
```
NM: Proposal:
* Parser accepts `use` everywhere; semantically it is rejected
* Matcher in 2024:
* ty = may start with `use`
* ty_2021 = may not
* Matcher in 2021
* (not possible) = may start with `use`
* ty = ty_2021 = may not
* Questions
* Name: ty_2021 = typaram -- similar to patparam?
* Edition rewrite: is it worth rewriting `ty` to `ty_2021`
## `impl use<..> + $bounds`
Josh: When we considered syntax options before, did we consider:
`impl use<'a, B> + Trait<C> + Other + ...` ?
That could make the scope clearer, since it'd no longer look like `use` applied only to the first term. "I return something that uses these lifetimes/parameters and implements this trait and ..."
TC: We did not previously discuss this.
Josh: Then, if I'm understanding correctly how this is used, I'd like to propose this as being clearer. Effectively, rather than `use<'a, B>` being a thing attached specifically to the `impl` at the beginning of the construct (whether before or after it), it's one *term* in the `impl X + Y + Z` construct. (Also, ideally that term could be placed anywhere in the list.)
NM: Similar to "the `Captures` hack".
tmandry: This sort of works in the opposite direction from the other bounds. It's a weakening rather than a strengthening. I don't like that about it. The 2024 capture rules were trying to align the language with the correct intuition here, and this would seem to push against it.
NM:
* `fn foo<T, U>() -> impl PartialEq` // could capture everything (default: `use<T, U>`)
* `fn foo<T, U>() -> impl use<> + PartialEq` // captures nothing
* `fn foo<T, U>() -> impl use<> + PartialEq<T>` // captures `T`
* `fn foo<T, U>() -> impl use<U> + PartialEq<T>` // captures `T`, `U`
* `fn foo<T, U>() -> impl use<T, U> + PartialEq<T>` // captures `T`, `U`
NM: `use` here acts lke any other bound -- adding `use` means fewer types meet those bounds, but you can do more with it (it "outlasts" other things)
tmandry: For me, the bounds seem to restrict what you can do with a type...
NM: The bounds restricts the set of types but expands what you can do with it.
NM: Most bounds are about what you can call on it (e.g. the methods of a trait). There are bounds like `'static`.
NM: Where I went wrong here was saying that `use<T> + use<U>` would be `use<T, U>`. That's not true.
NM: If you have "at most one use", it works well.
JT: Yes, because use is a "subtraction from all" in some sense.
*Intuition:* `use<B...>` is a trait that is implemented for all types that reference types in `B...`.
TC: While I agree with what NM is saying, perhaps we can more precisely frame the intuition that tmandry has. As a list of bounds gets longer, it offers more to the caller. But as the list of arguments to `use<..>` gets longer, it offers less to the caller. So the `use<..>` construct is *contravariant* in its length as compared with the bounds.
NM: Yes, +1; it's *contravariant*. That's a great framing.
NM: My preference:
* Restrict to at most one `use` in the list right now and require that it includes all the identifiers from other bounds
* If we ever wanted more than one:
* I would interpret it as "disjoint from the complement of the identifiers in the list"
* and then say that the "legal names to be included" are the union of all identifiers that appear
* but I'd want to work from needs and examples
NM: There are places where I want to do something similar to this, such as with `dyn`. So I think it's not bad that it scales. It may be comparable to the prefix `use<..>` in that way.
scottmcm: It'd be interesting to be able to use these on GATs.
NM: Yes, it's interesting to think about these in a where clause.
Josh: E.g., `where T: Debug + use<U>`.
NM: Yes, we'd make this part of the bounds list, it's just not a bound that could be used everywhere right now.
NM: GATs and opaque types are very related.
NM: We'd start with this where we need it right now, then we'd think about expanding it to e.g. where clauses on GATs.
```rust
fn foo<'a, T: 'a + Foo>(t: T) -> dyn Foo + 'a {
// Niko has always found this indirect and non-obvious...
Box::new(t)
}
fn foo<T>(t: T) -> dyn Foo + use<T> {
// ...I'd rather write this, though it could be `use<T> dyn Foo` just as well
Box::new(t)
}
```
NM: This also lets us put it at the end, which is nice. I like it at either the beginning or the end, just not where we had it at the beginning.
Josh: Yes, this isn't the most important thing to know about the return type, it's an added detail. The most important thing to know is usually the primary trait implemented. "This is a Future", not "Hey, before I tell you anything else, know that this uses `'a`".
NM:
```rust
fn foo<A, B, C>(
a: A,
b: B,
) -> impl Iterator + use<A, B> {
}
```
scottmcm: I've long wondered how we'd ever be able to do something like:
```rust
trait Iterator = StreamingIterator<Item<'a>: use<>>;
```
as a way to say it doesn't depend on the `'a`.
pnkfelix: Is there a current way to specify any kind of bounds on a closure expression directly, beyond the "return type" itself, e.g.:
```rust
fn foo<T>() -> impl Fn() {
// ~~~~~~~~~ this is the aforementioned "return type"
/* want to impose bounds on underlying type of the
closure expression below (presumably *not* by
annotating the return type above) */
|| { ... }
}
```
NM: It's sort of not surprising that we don't, as closures as expressions.
tmandry: we can still just do this...
```rust
fn foo() -> impl Fn() + use<> {
use<> || { .. }
}
```
tmandry: If we can use this for GATs, that's a strong argument.
scottmcm: If this can help us clean up `dyn`, that would be really nice. +1 to that.
scottmcm: to Niko's point earlier, I agree that I'd love to rethink how we do bounds on trait objects. See "Consider re-tuning the lifetime elision rules for trait objects" <https://github.com/rust-lang/rust/issues/91302>. In particular, I think `impl dyn Foo { … }` is too often a mistake, because people want `impl dyn Foo + '_ { … }`.
NM: What are the disadvantages of putting it in the bounds list rather than putting it in front?
tmandry: The main one is the contravariance, using TC's framing there.
NM: My hunch is that this will not cause undue confusion. If you don't think too hard about it, it's OK.
TC: To pnkfelix's question earlier, note that this does work:
```rust
#![feature(precise_capturing)]
#![allow(incomplete_features)]
//@ edition: 2024
fn outlives<'o, T: 'o>(_: T) {}
fn foo<'a>(_: &'a ()) -> impl use<> Fn() {
// ^^^^^
// We would get an error below without this.
|| ()
}
fn main() {
let x = ();
outlives::<'static>(foo(&x));
}
```
NM: It seems OK for things to be a bit complicated as long as the compiler will point people in the right direction.
others: Agreed.
Josh: We could have rustfmt move `use<..>` to the end.
scottmcm: The thing that comes to mind for me is `Fn()` where it may be clearer to not have anything after it. Otherwise, I agree that at the end is probably fine most of the time.
```rust
fn foo() -> impl Fn(A) -> B + use<…>
// ^^^^^^^^^^ unclear to human
```
Exact proposal:
* bound lists can contain `use<Parameter..>`.
* semantically, we enforce that:
* `use<Parameter..>` appears at most once in the bound list.
* it only appears in "opaque type" impl trait (i.e., RPIT, ATPIT, etc).
* error if the bounds reference other parameters that don't appear in the `use` list (with machine-applicable suggestion to add those parameters to the use list).
* Question mark: RPITIT -- either support it now in trait definitions or later (once we support in GATs)
* The `use` bound defines the set of parameters to appear on the hidden type
* if not listed, default is all parameters in scope
* expected additions in future:
* use to control capture in GATs, dyns
* similar syntax for closure expressions (can't be same, not a bounds list)
Pros:
* Can appear at end of the list (and rustfmt can move it)
* Can be used for GATs (in the future) to avoid capturing parameters from the trait
* Can be used for dyns (in the future) as part of a larger change
* Clearly covers hidden type in the same way as other bounds
* No macro changes required
Cons:
* Contravariant in the length of identifiers inside the `use` which is a bit odd
* but having at most one in the list (and one that covers all parameters that appear elsewhere) makes this less relevant for now; we can decide if/how to permit multiple and what it should mean
*Consensus*: We're agreed on the exact proposal above. We're interested in expanding this in the future to other places, such as the where clauses of GATs and to `dyn`.
(This section of the meeting ended here.)
---
## Role of the `use` keyword
tmandry: The framing of bringing parameters into scope vs applying arguments is helpful. I see it as bringing parameters into scope, and part of this is the fact that we are reusing the `use` keyword. The whole purpose of `use` directives is to bring an outside name into scope, and if we used it for closure captures I would frame its role in the same way.
---
# Short planning meeting
Availability for planning:
Dates:
- 5th: Today!
- 12th: tmandry out all week.
- 19th: Open.
- 26th: Open.
Availability:
* Josh: No conflicts.
* Niko: June 12 -- "maybe" but otherwise available.
* Tyler: Out June 12.
* Scott: should be available all.
Topics:
* Project goals revue
* x2 ? ergonomics initiative topics
* https://rust-lang.github.io/rust-project-goals/2024h2/ergonomics-initiative.html
* especially auto-clone
* maybe together with profiles + AFIDT
* Return type notation (RTN)
* a note: implementable trait aliases would be nice
* Match ergonomics
* AsyncIterator and async trait implementation strategy (`async fn next()` vs `fn poll_next()` vs ...) (note: libs-api inclined to punt to lang here)
* +1 to libs-api punting --niko
* eholk would be logical person to prepare doc on this I think --niko
* AFIDT
* Does supertrait item shadowing https://github.com/rust-lang/rfcs/pull/3624 need a design meeting, or can we check boxes async?
We need tmandry for match ergonomics. Let's schedule a special meeting:
- Friday 1630 UTC / 1230 EDT / 0930 PDT
Sign in with Wallet
