owned this note
owned this note
Published
Linked with GitHub
---
title: Lifetime Capture Rules 2024
tags: design-meeting
date: 2023-07-26
url: https://hackmd.io/sFaSIMJOQcuwCdnUvCxtuQ
authors: tmandry, TC (in close collaboration and in no particular order)
---
:::info
Note for later readers:
The consensus reached through this discussion resulted in [RFC 3498](https://github.com/rust-lang/rfcs/pull/3498). The RFC encompases and supersedes this document. Except for reasons of historical interest, it would be better to read the RFC.
:::
# Introduction
Rust's rules around capturing lifetimes in opaque and hidden types are inconsistent, unergonomic, and not helpful to users. In common scenarios, doing the correct thing requires a "trick" that most people don't know about. Of the people who do know about this trick, most of them do not fully understand it.
As we look forward to the 2024 edition and move toward stabilizing features such as type alias `impl Trait` (TAIT), associated type position `impl Trait` (ATPIT), return position `impl Trait` in trait (RPITIT), and `async fn` in trait (AFIT), we must decide on a clear vision of how lifetimes should be captured in Rust.
We want the upcoming features in the stabilization pipeline to capture lifetimes in a way that's consistent with each other and with the way we want Rust to work and develop going forward.
This meeting is about finding and building consensus on that shared vision of the lifetime capture rules. This meeting will be a success if we can agree in principle to how we want lifetimes to be captured in Rust going forward and whether it's more important for new features to be aligned with that direction than to be aligned with the behavior in Rust 2021. Such an agreement in principle will unblock important work on these features and will lay the groundwork for an RFC to be written formalizing these capture rules.
# Background
## Capturing lifetimes
In return position `impl Trait` and `async fn`, an **opaque type** is a type that can only be used for its specified trait bounds (and for the "leaked" auto trait bounds of its hidden type). A **hidden type** is the actual concrete type of the values hidden behind the opaque type.
A hidden type is only allowed to name lifetime parameters when those lifetime parameters have been *"captured"* by the corresponding opaque type. For example:
```rust
// Returns: Future<Output = &'a ()> + Captures<&'a ()>
async fn foo<'a>(x: &'a ()) -> &'a () { x }
```
In the above, we would say that the lifetime parameter `'a` has been captured in the return type.
When a caller receives an opaque type and wishes to prove that it outlives some lifetime, **the caller must prove, unless the opaque has a `+ 'other` bound stating the opaque outlives that lifetime (after transitively taking into consideration other known lifetime bounds), that all of the captured lifetime components of the opaque type outlive that lifetime**. The captured lifetime components are the set of lifetimes contained within captured type parameters and the lifetimes represented by captured lifetime parameters.
:::info
Instead of saying the above, people have previously said that the opaque type outlives the *intersection* of the captured lifetimes.
Conversely, a bound like `impl Trait + 'a + 'b` was said to require that the opaque type outlive the *union* of those captured lifetimes.
While this framing has some appeal in terms of type theory and subtyping relationships, there's some debate about its usefulness in describing Rust semantics. We mention it here only because earlier discussions and references may have used this terminology.
:::
## Capturing lifetimes in type parameters
In return position `impl Trait` and `async fn`, lifetimes inside of type parameters may also be captured. For example:
```rust
// Returns: Future<Output = T> + Captures<T>
async fn foo<T>(x: T) -> T { x }
fn bar<'a>(x: &'a ()) {
let y = foo(x);
// ^^^^^^^^^^^
// ^ Captures 'a.
}
```
In the above, we would say that `foo` captures the type parameter `T` or that it "captures all lifetimes contained in the type parameter `T`". Consequently, the call to `foo` captures the lifetime `'a` in the return type.
### Behavior of `async fn`
As we saw in the examples above, `async` functions automatically capture all type and lifetime parameters in scope.
This is different than the rule for return position `impl Trait` (RPIT) which requires that lifetime parameters (but not type parameters) be captured by writing them in the bound. As we'll see below, RPIT requires users to use a `Captures` "trick" to get the correct behavior.
The inconsistency is visible to users when desugaring from `async fn` to RPIT. As that's something users commonly do, users have to be aware of this complexity in Rust today.
For example, given this `async fn`:
```rust
async fn foo<'a, T>(x: &'a (), y: T) -> (&'a (), T) {
(x, y)
}
```
To correctly desugar this to RPIT, we must write:
```rust
use std::future::Future;
trait Captures<U> {}
impl<T: ?Sized, U> Captures<U> for T {}
fn foo<'a, T>(x: &'a (), y: T)
-> impl Future<Output = (&'a (), T)> + Captures<&'a ()> {
// ^^^^^^^^^^^^^^^^
// ^ Capture of lifetime.
async move { (x, y) }
}
```
(As we'll discuss below, other seemingly simpler desugarings are incorrect.)
If async had happened first, we could imagine that the lifetime capture rules for RPIT may have gone the other way.
:::info
Lifetimes in scope from an outer impl are also captured automatically by an `async fn`. For example:
```rust
struct Foo<'a>(&'a ());
impl<'a> Foo<'a> {
async fn foo(x: &'a ()) {}
// ^^^^^^^^^^^^^^
// ^ The lifetime 'a is automatically
// captured in the opaque return type.
}
```
Note that the lifetime is captured whether or not the lifetime appears in the return type and whether or not the lifetime is actually used in the hidden type at all.
:::
## Working with the lifetime capture rules in RPIT
For the borrow checker to function with an opaque type it must know what lifetimes it captures (and consequently what lifetimes may be used by the hidden type), so it's important that this information can be deduced from the signature, either by writing it out or by an automatic rule.
As we saw in the previous example, for RPIT (but not `async fn`), the rule is that opaque types automatically capture lifetimes within the type parameters but only capture lifetime parameters when those lifetime parameters are mentioned in their bounds.
When someone wants to capture a lifetime parameter not already in the bounds, that person must use one of the "tricks" we'll describe next.
### The outlives trick
Consider this example:
```rust
fn foo<'a>(x: &'a ()) -> impl Sized { x }
```
This does not compile today because the `'a` lifetime is not mentioned in the bounds of the opaque type. We can "fix" this by writing:
```rust
fn foo<'a>(x: &'a ()) -> impl Sized + 'a { x }
```
This called the "outlives trick". But this "solution" is a lie. To see this, think about what `impl Sized + 'a` means: we return an opaque type that implements `Sized` and outlives `'a`.
But *outliving* `'a` was never actually a promise we wanted to make to our API user. In fact, it was quite the opposite. We wanted to say that our opaque type *captures* `'a`. In other words, *`'a` outlives our opaque type* (if lifetimes could be said to outlive types in Rust).
Counterintuitively, adding an outlives relationship in the wrong direction satisfied our needs. This works for two reasons:
1. Naming the lifetime anywhere in the bounds allows us to capture it, according to the RPIT capture rules.
1. The relationship is still true because the lifetime we return is exactly `'a`. The direction doesn't matter if the lifetimes are equal.
Nevertheless, it is also problematic for two reasons:
1. Requiring users to write a bound that means the exact opposite of what they want makes it difficult to build a consistent mental model of Rust lifetime bounds, an area that is already ripe for confusion.
2. This solution does not compose well: It does not work when multiple lifetimes of different variance need to be captured, including lifetimes that may be contained within automatically captured type parameters.
### The `Captures` trick
There is another, more "honest" way to accomplish this. It's the only option when there are multiple lifetimes in play (including via automatically captured type parameters), especially when those lifetimes may be invariant. It's called the `Captures` trick. Consider again our example:
```rust
fn foo<'a>(x: &'a ()) -> impl Sized { x }
```
We could instead solve the problem like this:
```rust
trait Captures<U> {}
impl<T: ?Sized, U> Captures<U> for T {}
fn foo<'a>(x: &'a ()) -> impl Sized + Captures<&'a ()> { x }
```
Every type trivially implements `Captures<T>` for any type `T`, so we're not adding any new meaningful *trait* bounds. But we are affecting the *lifetime* bounds of the opaque type. And because we have now named the lifetime in the bounds, the lifetime parameter can be captured by the hidden type.
We can extend this trick to multiple lifetimes (and to other positions[^other-positions]). For example:
```rust
fn foo<'a, 'b>(x: &'a (), y: &'b ()) -> impl Sized + Captures<(&'a (), &'b ())> {
(x, y)
}
```
The biggest problem with `Captures` is that it is clunky both to write and to read, and it comes with a high overhead of reasoning about "why" it exists and is necessary.
[^other-positions]: The `Captures` trick is useful in positions other than return position `impl Trait`. For example, if you ever wanted to say that a lifetime outlived a generic type parameter, you probably could have used the `Captures` trick to allow the type parameter to name the lifetime. This is because we allow generics to name lifetimes that appear in their bounds. [Example](https://hackmd.io/zgairrYRSACgTeZHP1x0Zg?view#Refresher-on-the-Captures-trick)
## Inconsistency: Capturing type parameters vs lifetime parameters in RPIT
As mentioned above, capturing applies to type parameters too. Unlike lifetime parameters, in RPIT, opaque types capture **all type parameters in scope**.
This is relevant when the type parameters themselves contain lifetimes.
For example, if we had a `chain` helper defined like this:
```rust
fn chain<T, U, I>(first: T, second: U) -> impl Iterator<Item = I>
where
T: Iterator<Item = I>,
U: Iterator<Item = I>,
{
Chain::new(self, other)
}
```
We could call it and use it like so:
```rust
let a = [1, 2, 3];
let b = [4, 5, 6];
let chained = chain(a.iter(), b.iter());
// Borrows a ^^^^^^^^
// Borrows b ^^^^^^^^
for x in chained {
dbg!(x);
}
```
Moreover, it would be an error to drop `a` or `b` before the for loop, because `chained` captures borrows of each via its type parameters `T` and `U`.
## Overcapturing
What if the rule that causes us to capture all type parameters causes us to capture too much? The solution is type alias `impl Trait` (currently available only on nightly).
As we'll see below, this is how we can also solve the overcapturing of lifetime parameters that happens on stable Rust today with `async fn` and that could occur under RPIT with a revised semantic.
:::info
For an example of overcapturing with RPIT in stable Rust today, see [Appendix D](#Appendix-D-Overcapturing-of-type-parameters-example).
:::
## Summary of problems
In summary, we have identified a number of problems with the state of things today:
1. The RPIT lifetime capture rules are unergonomic and require unobvious "tricks".
2. The RPIT rules for capturing lifetime parameters are inconsistent with those for capturing type parameters.
3. The RPIT rules are inconsistent with the rules for `async fn` and this is exposed to users because of the common need to switch between the two forms.
# The solution
We propose to make the rules for RPIT match the rules for `async fn` exactly in the 2024 edition.
For the upcoming features to be stabilized in the 2021 edition, we propose to align them with the proposed 2024 edition RPIT/`async fn` captures behavior.
## Apply `async fn` rule to RPIT in 2024 edition
Under this proposal, in the 2024 edition of Rust, RPIT would automatically capture all lifetime parameters in scope, just as `async fn` does today, and just as RPIT does today when capturing type parameters.
Consequently, the following examples would become legal:
### Capturing a lifetime from a free function signature
```rust
fn foo<'a, T>(x: &'a T) -> impl Sized { x }
// ^^^^^^^^^^
// ^ Captures 'a and T.
```
### Capturing a lifetime from outer inherent impl
```rust
struct Foo<'a, T>(&'a T);
impl<'a, T> Foo<'a, T> {
fn foo(self) -> impl Sized { self }
// ^^^^^^^^^^
// ^ Captures 'a and T.
}
```
### Capturing a lifetime from a method signature
```rust
struct Foo<T>(T);
impl<T> Foo<T> {
fn foo<'a>(&'a self) -> impl Sized { self }
// ^^^^^^^^^^
// ^ Captures 'a and T.
}
```
## TAIT as the solution to overcapturing
As we saw above, sometimes the capture rules result in unwanted lifetimes being captured. This happens today in Rust due to the existing RPIT rules for capturing lifetimes from all in-scope type parameters and the `async fn` rules for capturing all in-scope type and lifetime parameters. Under the proposal, lifetime parameters could also be overcaptured by RPIT.
The solution to this is type alias `impl Trait` (TAIT). It works as follows. Consider this overcaptures scenario under the proposed semantic.
```rust
fn foo<'a, T>(x: &'a T) -> impl Sized { () }
// ^^^^^^^^^^
// The return type captures 'a and 'b (via T)
// but does not actually use either.
fn is_static<T: 'static>(_t: T) {}
fn bar<'a, 'b>(x: &'a &'b ()) {
is_static(foo(x));
// ^^^^^^
// Error: foo captures 'a and 'b.
}
```
In the above code, we want to rely on the fact that `foo` does not actually use any lifetimes in its return type. We can't do that using RPIT because it captures too much. However, we can use TAIT to solve this problem elegantly as follows:
```rust
#![feature(type_alias_impl_trait)]
type FooRet = impl Sized;
fn foo<'a, T>(x: &'a T) -> FooRet { () }
// ^^^^^^
// The return type does NOT capture 'a or 'b (via T).
fn is_static<T: 'static>(_t: T) {}
fn bar<'a, 'b>(x: &'a &'b ()) {
is_static(foo(x)); // OK!
}
```
## Type alias `impl Trait` (TAIT)
Under this proposal, the opaque type in type alias `impl Trait` (TAIT) will automatically capture all lifetimes present in the type alias. For example:
```rust
#![feature(type_alias_impl_trait)]
type Foo<'a> = impl Sized;
// ^^^^^^^^^^
// ^ Captures 'a.
fn foo<'a>() -> Foo<'a> {}
```
## Associated type position `impl Trait` (ATPIT)
Under this proposal, the opaque type in associated type position `impl Trait` (ATPIT) will automatically capture all lifetimes present in the GAT and in the outer impl.
For example:
```rust
#![feature(impl_trait_in_assoc_type)]
trait Trait<'t> {
type Gat<'g> where 'g: 't; // Bound required by existing GAT rules.
fn foo<'f>(self, x: &'t (), y: &'f ()) -> Self::Gat<'f>;
}
struct Foo<'s>(&'s ());
impl<'t, 's> Trait<'t> for Foo<'s> {
type Gat<'g> = impl Sized where 'g: 't;
// ^^^^^^^^^^
// ^ Captures:
//
// - 'g from the GAT.
// - 'f from the method signature (via the GAT).
// - 't from the outer impl and a trait input.
// - 's from the outer impl and Self type.
fn foo<'f>(self, x: &'t (), y: &'f ()) -> Self::Gat<'f> {
(self, x, y)
}
}
```
## Return position `impl Trait` in Trait (RPITIT)
Under this proposal, the opaque type in return position `impl Trait` in trait (RPITIT) will automatically capture, in the trait definition, all trait input lifetimes, all lifetimes in the `Self` type, and all lifetimes in the associated function or method signature.
In trait impls, *RPIT* will automatically capture all lifetime parameters from the outer impl and from the associated function or method signature. This ensures that signatures are copyable from trait definitions to impls.
For example:
```rust
#![feature(return_position_impl_trait_in_trait)]
trait Trait<'t> {
fn foo<'f>(self, x: &'t (), y: &'f ()) -> impl Sized;
// ^^^^^^^^^^
// Method signature lifetimes, trait input lifetimes, and
// lifetimes in the Self type may all be captured in this opaque
// type in the impl.
}
struct Foo<'s>(&'s ());
impl<'t, 's> Trait<'t> for Foo<'s> {
fn foo<'f>(self, x: &'t (), y: &'f ()) -> impl Sized {
// ^^^^^^^^^^
// The opaque type captures:
//
// - 'f from the method signature.
// - 't from the outer impl and a trait input lifetime.
// - 's from the outer impl and the Self type.
(self, x, y)
}
}
```
## `async fn` in trait (AFIT)
Under this proposal, the opaque type in `async fn` in trait (AFIT) will automatically capture, in the trait definition, all trait input lifetimes, all lifetimes in the `Self` type, and all lifetimes in the associated function or method signature.
In the trait impls, AFIT will automatically capture all lifetime parameters from the outer impl and from the associated function or method signature. This ensures that signatures are copyable from trait definitions to impls.
Signatures are directly copyable from the trait definitions to the impls.
This behavior of AFIT will be parsimonious with the current stable capture behavior of `async fn`.
For example:
```rust
#![feature(async_fn_in_trait)]
trait Trait<'t>: Sized {
async fn foo<'f>(self, x: &'t (), y: &'f ()) -> (Self, &'t (), &'f ());
// ^^^^^^^^^^^^^^^^^^^^^^
// Method signature lifetimes, trait input lifetimes, and
// lifetimes in the Self type may all be captured in this opaque
// type in the impl.
}
struct Foo<'s>(&'s ());
impl<'t, 's> Trait<'t> for Foo<'s> {
async fn foo<'f>(self, x: &'t (), y: &'f ()) -> (Foo<'s>, &'t (), &'f ()) {
// ^^^^^^^^^^^^^^^^^^^^^^^^^
// The opaque type captures:
//
// - 'f from the method signature.
// - 't from the outer impl and a trait input lifetime.
// - 's from the outer impl and the Self type.
(self, x, y)
}
}
```
# Conclusion
The lifetime capture rules in Rust today are inconsistent, unergonomic, and unhelpful to users. Users commonly get this wrong and can't figure out what to do.
We can make a decision today to make these rules more consistent and helpful in the 2024 edition for RPIT.
If we make that decision, then we can unblock current work on upcoming feature stabilizations and allow those upcoming features to align with the future direction of the language's capture rules.
# Appendix A: Other resources
Other resources:
- [Capturing lifetimes in RPITIT](/zgairrYRSACgTeZHP1x0Zg) by compiler-errors and TC
Earlier obsolete drafts:
- [Lifetime Capture Rules 2024](/4NnR4dOWRsCsihYmPs33lg) by TC
- [RPITIT Capture Rules](/h3fCrGqHS6alLLNsPqEcqQ) by tmandry
# Appendix B: Matrix of capturing effects
| | 2021: *Outer LP* | 2021: *Item LP* | 2024: *Outer LP* | 2024: *Item LP* |
|-|-|-|-|-|
| RPIT | N | N | Y | Y |
| `async fn` | Y | Y | Y | Y |
| GATs | Y | Y | Y | Y |
| TAIT | N/A | Y | Y | Y |
| ATPIT | Y | Y | Y | Y |
| RPITIT: trait | Y | Y | Y | Y |
| RPITIT: impl | Y | Y | Y | Y |
:::info
In the table above, "LP" refers to "lifetime parameters".
The 2024 behavior described for all items is the behavior under this proposal.
The 2021 behavior described for RPIT and `async fn` is the current stable behavior. The other 2021 behaviors described are the proposed behaviors that would be implemented for the features ahead of stabilization.
*All* of the feature above automatically capture all lifetimes from all type parameters in scope in both the 2021 and the 2024 editions.
:::
# Appendix C: The need for an inconsistency between inherent RPIT and RPITIT
Under today's semantics, inherent RPITs do not capture any lifetimes automatically.
```rust
struct Foo<'a>(&'a str);
impl<'a> Foo<'a> {
fn into_debug(self) -> impl Debug { self.0 }
//^ ERROR: hidden type for `impl Debug` captures lifetime
// that does not appear in bounds
fn into_debug(self) -> impl Debug + 'a { self.0 }
//^ OK!
}
```
If we were to apply this rule directly to RPITIT, we'd quickly end up in an unworkable situation.
```rust
trait IntoDebug {
fn into_debug(self) -> impl Debug;
}
impl<'a> IntoDebug for Foo<'a> {
fn into_debug(self) -> impl Debug { self.0 }
//^ ERROR
}
```
Where would we put `+ 'a` (or `+ Captures<'a>`) in the above code to make it compile? Notice that the trait has no way of naming `'a` at all, because that is part of the `Self` type it knows nothing about.
Given the current rules, our options are:
1. Allow implicit captures of outer lifetime parameters for all RPITIT, inconsistently with inherent RPIT.
2. Require that only the impl list the outer lifetime parameters it captures. This creates an inconsistency between the trait and impl signatures.
* Strangely in this scenario, copying the signature from a trait to an impl in this situation would amount to a *refinement* of the signature, because the impl would implicitly be saying it does not capture the outer lifetime parameters.
3. Don't allow useful impls of RPITITs on types with lifetime parameters.
The current rules lead to some kind of inconsistency in any case. But the proposed rules would obviate the problem and allow RPIT to be fully consistent, whether it is used in an inherent impl or a trait.
# Appendix D: Overcapturing of type parameters in stable RPIT
As an example of how overcapturing may occur in RPIT in Rust today due to the automatic capturing of type parameters, consider consider this overcaptures scenario:
```rust
fn foo<T>(x: T) -> impl Sized { () }
// ^^^^^^
// The return type captures T but does not actually use it.
fn is_static<T: 'static>(_t: T) {}
fn bar<'a>(x: &'a ()) {
is_static(foo(x));
// ^^^^^^
// Error: foo captures 'a.
}
```
On stable Rust today there's no way to fix that. However, with TAIT, we can write:
```rust
#![feature(type_alias_impl_trait)]
type FooRet = impl Sized;
fn foo<T>(x: T) -> FooRet { () }
// ^^^^^^
// The return type does NOT capture T.
fn is_static<T: 'static>(_t: T) {}
fn bar<'a>(x: &'a ()) {
is_static(foo(x)); // OK!
}
```
# Appendix E: The outlives trick fails with only one lifetime parameter
:::info
This appendix has been added after the meeting in response to discussion during the meeting.
:::
People often think that the outlives trick is OK as long as there is only one lifetime parameter. This is not in fact true. Consider:
```rust
// This is a demonstration of why the Captures trick is needed even
// when there is only one lifetime parameter.
// ERROR: the parameter type `T` may not live long enough.
fn foo<'x, T>(t: T, x: &'x ()) -> impl Sized + 'x {
// ^^
// We don't need for T to outlived 'x, |
// and we don't want to require that, so |
// the Captures trick must be used here. --+
(t, x)
}
fn test<'t, 'x>(t: &'t (), x: &'x ()) {
foo(t, x);
}
```
# Appendix F: Adding a `'static` bound
:::info
This appendix has been added after the meeting in response to discussion during the meeting.
:::
Adding a `+ 'static` bound will work in Rust 2024 in exactly the same way that it works in Rust 2021.
```rust
trait Captures<U> {}
impl<T: ?Sized, U> Captures<U> for T {}
fn foo<'x, T>(t: T, x: &'x ())
-> impl Sized + Captures<&'x ()> + 'static {
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// In Rust 2021, this opaque type automatically captures the type
// `T`. Additionally, we have captured the lifetime 'x using the
// `Captures` trick.
//
// Since there is no `T: 'static` bound and no `'x: 'static`
// bound, this opaque type would not be 'static without the
// specified bound on the opaque type above. *With* that
// specified bound, the opaque type is 'static, and this code
// compiles today.
//
// In Rust 2024, this opaque type will automatically capture the
// lifetime parameter in addition to the type parameter. The
// `Captures` trick will not be needed and will not be present in
// the signature. However, specifically bounding the opaque type
// by 'static will still work, exactly as it does today.
()
}
fn is_static<T: 'static>(_t: T) {}
fn test<'t, 'x>(t: &'t (), x: &'x ()) {
is_static(foo(t, x));
}
```
# Appendix G: Future possibility: precise capturing syntax
:::info
This appendix has been added after the meeting in response to discussion during the meeting.
:::
This proposal intends for people to use TAIT to avoid capturing type and lifetime parameters that should not be captured. If this comes up too often in the future, we may want to consider adding new syntax to `impl Trait` to allow for precise capturing. One proposal for that would look like this:
```rust
fn foo<'x, 'y, T, U>() -> impl<'x, T> Sized { todo!() }
// ^^^^^^^^^^^
// ^ Captures 'x and T in the opaque type but
// not 'y or U.
```
:::warning
This is not part of this proposal.
:::
# Questions / Discussion
(Have a question? Add a new section here.)
### data around existing usage?
nikomatsakis: AFAIK we don't have a TON of data from crates.io, unless I missed something in the doc, but I did do one experiment around this. I searched through `-> impl Trait` in the compiler. I found that literally every usage wanted to capture lifetimes with 2 exceptions, both instances of the same pattern, the [`indices`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_index/struct.IndexVec.html#method.indices) function. Is there more data available?
TC: I spent some hours searching GitHub for instances of `-> impl .* \+ ''` and reading through the findings looking for examples of where it might have been used "correctly" rather than because the author didn't know about the captures trick. I didn't keep count, but I looked through a lot of code but didn't find any examples of where the author shouldn't have just used `Captures` (and I edited and rebuilt each to confirm).
joshtriplett: given that `indices` specifically returns `+ 'static`, that suggests that if we could handle `+ 'static` people could potentially avoid TAIT in that simple case.
compiler-errors: What's the issue with `indices`? Since it explicitly outlives `'static`, changing its captures shouldn't matter. [We can always use a `'static` bound even if we overcapture something](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=75d7bb54c3fbc400be2ad4f7268a9bea).
nikomatsakis: Question for the room: What % of code do you think needs to prefer this to give you pause about changing the default?
joshtriplett: Something like 20%, except that having to do a TAIT is pretty bad, so I wouldn't want more than 1% of code having to do that.
tmandry: I agree TAIT is a bit clunky, but it makes it very easy to reason about the capture rules using the syntax of your code.
pnkfelix: To clarify
TC: I looked at the top crates on Github and in every case they wanted to use `Captures` instead of `+ 'a`.
pnkfelix: It sounds like no one needs `+ 'a` and if there were an ergonomic way to express `Captures` people should use that instead?
nikomatsakis: I can think of places, e.g. in Rayon where you need `+ 'a`
pnkfelix: Seems unfortunate that the slickest syntax has been reserved for something that seems niche, at best. Can't do anything about it.
### timeline for stabilization
nikomatsakis: I am in favor of this proposal. I do have a concern. Specifically, I really want to move TAITs and RPITITs towards stabilization. Suppose we align on the goals of this doc and therefore decide to change TAIT/RPITIT capture rules to "capture everything in scope". Do we feel we need an accepted RFC to modify `-> impl TRait` before we can stabilize those, because the premise is that it will align with the bigger question? What level of assurance do we want. And how quickly can we move towards an accepted RFC, presuming we don't get instantaneous concerns raised.
joshtriplett: Yes, but this doc would be 99% of that RFC; put a bit of RFC substrate around it, file it, FCP it.
TC: If we can get an agreement in principle here, I've offered to write that RFC (I'd guess that tmandry would want to collaborate there), and I agree this document is already most of that.
TC: Also, answering this big but simple question makes a lot of the complex and hairy questions of those feature go away. That can have the net effect of speeding things up.
tmandry: I'm not too worried about RFC'ing but don't want a 1% standard
joshtriplett: To clarify, what I meant was that if it's >1% it would give me pause and I would want a more ergonomic solution. Not that we should block this decision on that.
nikomatsakis: my take would be: that motivates us putting effort, but doesn't block stabilizing what we have.
joshtriplett: we should clarify this in the RFC.
### edition tenets
nikomatsakis: Just for the record, in [RFC 3085], which introduced the Rust 2021 edition, we established some "tenets" around editions. One of them was:
> Uniform behavior across editions
>
> Pursuant to the goal of having Rust feel like "one language", we generally prefer uniform behavior across all editions of Rust, so long as it can be achieved without compromising other design goals. This means for example that if we add a new feature that is backwards compatible, it should be made available in all editions. Similarly, if we deprecate a pattern that we think is problematic, it should be deprecated across all editions.
as well as
> Editions are meant to be adopted
>
> Our goal is to see all Rust users adopt the new edition, just as we would generally prefer for people to move to the "latest and greatest" ways of using Rust once they are available. Pursuant to the previously stated principles, we don't force the edition on our users, but we do feel free to encourage adoption of the edition through other means. For example, the default edition for new projects will always be the newest one. When designing features, we are generally willing to require edition adoption to gain access to the full feature or the most ergonomic forms of the feature.
I believe that these two together imply that we should favor "uniformity across editions" over "uniformity within older editions".
pnkfelix: I agree we should favor consistency across editions
tmandry: it also requires us to answer a lot more subquestions if we try to have uniform behavior within editions rather than across
joshtriplett: I'd be wary of a blanket rule, but I think it applies here because we think it's already the behavior people want.
[RFC 3085]: https://rust-lang.github.io/rfcs/3085-edition-2021.html#uniform-behavior-across-editions
### clarification
nikomatsakis: The text states:
> When an opaque type captures a set of lifetimes, and a caller receives a value of that opaque type and wishes to prove that it outlives some other lifetime, **the caller must prove that each of the captured lifetimes outlives that other lifetime**. This is like saying the opaque type outlives its captured lifetimes, except that types don't outlive lifetimes in Rust.
Although this is indeed the current behavior of the compiler, it is not exactly true. For example, consider this function:
```rust
fn foo(x: &'a u32) -> impl Sized + 'static + Captures<&'a ()> {
()
}
```
We can conceivably prove that this opaque type outlives some lifetime `'x` even if `'a` doesn't outlive `'x`. There's some subtlety here because of the way we currently define outlives in a syntactic way, this *exact* example (with the explicit `Captures`) clause introduces some complications related to RFC 1214 (not sure if that's the right RFC #). This is kind of a side note I think and doesn't impact the main thrust of the point.
compiler-errors: For the record, I clarified this :sweat_smile: [here](https://rust-lang.zulipchat.com/#narrow/stream/315482-t-compiler.2Fetc.2Fopaque-types/topic/lang.20design.20doc.3A.20lifetime.20capture.20rules/near/378846042), either you can outlive all the captured components, or you can use a bound from the opaque's item bounds (or the param-env).
TC: compiler-errors: Could you suggest the correct wording there? I rewrote that section according to your clarification (but of course, all errors are my own).
CE: i did above, it's rough but it's just enumerating the choices one has in [RFC 1214](https://github.com/rust-lang/rfcs/blob/master/text/1214-projections-lifetimes-and-wf.md#outlives-for-projections).
TC: What would fill in for ??? below?
> When an opaque type captures a set of lifetimes, and a caller receives a value of that opaque type and wishes to prove that it outlives some other lifetime, **the caller must prove ???**. This is like saying the opaque type outlives its captured lifetimes, except that types don't outlive lifetimes in Rust.
CE: idk, something like "either all the captured components of the opaque outlive that lifetime, or the opaque has an explicit item bound stating that it outlives that lifetime". trying to state it in a single sentence rather than an enumerable list is the problem here, lol.
TC: Going to edit this in:
> When an opaque type captures a set of lifetimes, and a caller receives a value of that opaque type and wishes to prove that it outlives some other lifetime, **the caller must prove, unless the opaque has a `+ 'other` bound stating the opaque outlives that other lifetime, that all of the captured lifetime components of the opaque type outlive that lifetime**. This is like saying the opaque type outlives its captured lifetimes, except that types don't outlive lifetimes in Rust.
tmandry: I feel like stating the precise rules is beside the main point of the doc, personally.
CE: ~~The precise rules are partly why people can get away with `+'a` in the bounds in some cases.~~ (edit: probably not relevant)
CE: idk, I agree with Niko above that this is an important clarification. Also, the outlives bound thing [works currently](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=75d7bb54c3fbc400be2ad4f7268a9bea).
### history of async fn vs impl trait
nikomatsakis: Another historical side note. The text notes:
> If async had happened first, we could imagine that the lifetime capture rules for RPIT may have gone the other way.
Back when we were designing `async fn`, there was also a question of whether we should do `async fn foo() -> u32` or `async fn foo() -> impl Future<Output = u32>`. I think we probably picked right because the latter is very verbose, but many systems (e.g., C#) have opted to go the other way, and the jury is still out. It's also perhaps inconsistent with what a `try fn` might look like, which doesn't have an obvious "carrier" in the same way. At the time, the argument that I found most persuasive was precisely the fact that `async fn` capture rules were different from `-> impl Future`, which seemed to seal the deal to me.
### random edition thoughts
nikomatsakis: If we were to introduce this change over an edition
### reserving question spot
pnkfelix: (not sure if doc addresses this eventually but I want to ensure I write it down)
pnkfelix: ~~what is the way to opt out from the new implicit captures? e..g if I want the current
semantics, for e.g. `fn foo<'a, T>(x: &'a (), y: T) -> impl Trait { y }`, where I *don't* want `'a` captured by the RPIT, how do I express that?~~ oh, answer is TAIT.
compiler-errors: Yeah, TAIT is probably the suggestion here.
pnkfelix: ~~now wondering about TAIT vs ATPIT.~~ never mind, the `'f` is explicitly wired to the `'g` in the example there.
### Explicit syntax for "captures"?
Would an explicit syntax for captures be useful here? For instance, `-> impl<'a, 'b> Trait`? Premise: if you explicitly name captures you don't get any captures you don't name, so if you want the 2021 behavior you can write `-> impl<> Trait` rather than using a TAIT. (May not be perfect, since there may be lifetimes you can't name.)
### Handling `+ 'a`
joshtriplett: I think it's very likely that people will continue to write `+ 'a`. Should we do something to recognize where doing so may be redundant/unnecessary/harmful?
TC: It'd be interesting to explore heuristics we could use to lint about this.
joshtriplett: Also, can we present a reasonably concise example that *demonstrates* how `+ 'a` (and separately `+ 'a + 'b`) does the wrong thing? This document attempts to explain the difference in terms of the opaque type outliving the lifetimes versus being outlived by the lifetimes, and that `+ 'a` effectively causes *both* which makes the return value *equal* to the lifetimes, but would it be possible to get an example showing code that *doesn't* manage to get workable code by using `+`?
TC: Here's an example from my notes:
```rust
// This is a demonstration of why the Captures trick is needed even
// when there is only one lifetime parameter.
// ERROR: the parameter type `T` may not live long enough.
fn foo<'x, T>(t: T, x: *mut &'x ()) -> impl Sized + 'x {
// ^^
// The Captures trick must be used here. ^
//
// The bound we want to express is that "the lifetime 'x must
// outlive the hidden type, but what `+ 'x` means is that "the
// hidden type must outlive 'x" which is backward from what we
// want.
(t, x)
// ^^^^^^ ...so that the type `T` will meet its required lifetime bounds.
}
fn test<'t, 'x>(t: *mut &'t (), x: *mut &'x ()) {
foo(t, x);
}
```
compiler-errors: An example demonstrating `+ 'a + 'b` (where `'a` and `'b` are unrelated) would be to show that any hidden type would have to outlive `'static` to be valid. Not exactly sure how to show it other than "here's a bunch of code that *doesn't* work", though.
joshtriplett: What happens if you have a thing that outlives `'a` and `'b` but isn't static? Obvious example:
```rust
// intentionally incomplete, needs lifetimes added. Why does `+ 'a + 'b` not work?
fn select<'a, 'b>(which: bool, a: &'a str, b: &'b str) -> impl Display {
if which { a } else { b }
}
```
compiler-errors: Because you need to prove that `&'b str: 'a`, and `&'a str: 'b` in order for the hidden type to be valid. (and `&'a str: 'a`, `&'b str: 'b`, but those are trivial). A hidden type must uphold all of the bounds in the opaque, incl all outlives bounds.
joshtriplett: To ask it another way: what would happen if `impl Display + 'a + 'b` *just* meant "outlives `'a` and outlives `'b`" but didn't mean "is outlived by"?
compiler-errors: By "and", do you mean the lower-bound of those two lifetimes, or the upper-bound? I don't exactly know what you're asking :sweat_smile:
(upper bound; "outlives `'a` and outlives `'b`)
compiler-errors: Or do you mean, what would happen if `+ 'a` means "captures" only? I don't really like the wording "lifetime outlives type", it's confusing and not precise imo.
joshtriplett: let's take this offline to Zulip, perhaps, and focus on the current discussion.
:thumbsup:
### Question: Are we ready to RFC this?
TC: If we were to propose an RFC with the text of this document, cleaned up lightly for the RFC format, and with the addition that Josh proposed (that if in the future we discover that too much code experiences overcapturing and needs to be desugared to TAIT, we should explore more ergonomic solutions), who here would check their box, by a show of hands?
joshtriplett: Hand.
tmandry: Hand.
nikomatsakis: Hand.
pnkfelix: Hand.
(scottmcm was not present in the meeting.)
### EOF
Google Drive
Gist
Clipboard
Sign in with Wallet
