Niko Matsakis
    • Create new note
    • Create a note from template
      • Sharing URL Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Customize slides
      • Note Permission
      • Read
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Write
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Engagement control Commenting, Suggest edit, Emoji Reply
    • Invite by email
      Invitee

      This note has no invitees

    • Publish Note

      Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

      Your note will be visible on your profile and discoverable by anyone.
      Your note is now live.
      This note is visible on your profile and discoverable online.
      Everyone on the web can find and read all notes of this public team.
      See published notes
      Unpublish note
      Please check the box to agree to the Community Guidelines.
      View profile
    • Commenting
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
      • Everyone
    • Suggest edit
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
    • Emoji Reply
    • Enable
    • Versions and GitHub Sync
    • Note settings
    • Note Insights New
    • Engagement control
    • Make a copy
    • Transfer ownership
    • Delete this note
    • Save as template
    • Insert from template
    • Import from
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
    • Export to
      • Dropbox
      • Google Drive
      • Gist
    • Download
      • Markdown
      • HTML
      • Raw HTML
Menu Note settings Note Insights Versions and GitHub Sync Sharing URL Create Help
Create Create new note Create a note from template
Menu
Options
Engagement control Make a copy Transfer ownership Delete this note
Import from
Dropbox Google Drive Gist Clipboard
Export to
Dropbox Google Drive Gist
Download
Markdown HTML Raw HTML
Back
Sharing URL Link copied
/edit
View mode
  • Edit mode
  • View mode
  • Book mode
  • Slide mode
Edit mode View mode Book mode Slide mode
Customize slides
Note Permission
Read
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Write
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Engagement control Commenting, Suggest edit, Emoji Reply
  • Invite by email
    Invitee

    This note has no invitees

  • Publish Note

    Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note

    Your note will be visible on your profile and discoverable by anyone.
    Your note is now live.
    This note is visible on your profile and discoverable online.
    Everyone on the web can find and read all notes of this public team.
    See published notes
    Unpublish note
    Please check the box to agree to the Community Guidelines.
    View profile
    Engagement control
    Commenting
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Suggest edit
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    Emoji Reply
    Enable
    Import from Dropbox Google Drive Gist Clipboard
       Owned this note    Owned this note      
    Published Linked with GitHub
    2
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    --- title: "Temporary lifetime meeting minutes" date: 2023-05-18 - present url: https://hackmd.io/-l7-r7GiSWu2HJrPOI74BA --- ## 2024-02-29 ### Equivalence of match and block with let Mara: With the newly proposed rules, would the following be equivalent? ```rust match $expr { $pat => $body } ``` and ```rust { let $pat = $expr; $body } ``` Niko: yes. Mara: Let's put that clearly in the RFC. Important equivalence. Xiang: to extend it with `let else` there is equivalence between ```rust= match $expr { $pat => $body, _ => { else_and_diverge() } } ``` and ```rust= { super let $pat = $expr else { else_and_diverge() }; // the else block here is not allowed yet but possible $body } ``` ... but wait it is intricate... Niko: consider [this example](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=816fc0457bcd194e44632a86e08532b1)... ```rust let x = identity( match &foo() { x => x, } // <-- if the temporary got freed at end of match ); // `foo()` temporary .. freed at end of `let`? or at end of `match` let x = identity({ let f = &foo(); f }); // foo() result is freed at end of block fn identity<T>(t: T) -> T { t } ``` Niko: equivalence is ```rust match $expr { $pat => $body} // if $expr has an extended temporary lifetime { super let $pat = $expr; $body } // else this { let $pat = $expr; $body } ``` ```rust { super let x = foo(&bar()); // when is bar() freed? ^ x } ``` Mara: it's let, not super let: ```rust let a = match &temp() { _ => (), }; // temp dropped here .. drop(a); // not here. ``` Niko: ```rust let x = match &foo() { t => t, }; drop(x); // error ``` Mara: This errors today: ```rust let x = identity( if let Some(x) = match () { () => Some(&temp()) } { // Debug doesn't x } else { panic!() } ); ``` Conclusion: Mara's proposal is: * expressions have "short" and "long" scopes. * temporary lifetime syntax rules decide the scope for a temporary. (e.g. `[&long(), short().as_ref()]`) * for super let statement: * "short" is till the end of the let statement. * "long" is as long as the "long" scope of the surrounding block. * for let statement: * "short" is till the end of the let statement. * "long" is as long as the binding that's defined. * for match statements (and if let): (change from 2021 rules) * "short" temporaries are dropped at end of condition eval * "long" temporaries are dropped at end of match body * for expression statements: * "short" and "long" is the end of the statement * for tail expressions: * "short" are dropped at the end of the block (change from 2021 rules) * "long" is as long as the "long" scope of the surrounding block. (today only blocks, future also for if and match bodies: https://github.com/rust-lang/rust/pull/121346 ) timeline and questions: * Niko: I think we should cleanup the RFC but leave the treatment of match statements as an unresolved question * is it as written above (as Niko prefers) or is it that short is preserved * Niko: Then we should talk to Tyler and try to convince him on the point of match Mara: Without changing match/if-lets, the only edition change is tail expressions. That's a tiny edition change. Niko: yes but it can cause significant user confusion in my experience. Niko: I think it still makes sense together in one RFC. Mara: super let is simpler to explain when we skip the new match rules, since it only has an effect in a `let` statement. Mara: If we separate https://github.com/rust-lang/rust/pull/121346 out of the RFC, then the RFC is only about redefining what "long" and "short" *are*, not changing how to decide between "long" and "short". ### Splitting off a small part Mara: https://github.com/rust-lang/rust/pull/121346 Niko: doesn't that change beahviour of existing code? Mara: Don't think so. Xiang: it only accepts new code Niko to review. ## 2024-01-25 ### Lifetimes / scopes * Until end of enclosing statement -- intermediate value consumed but not reference by the overall result * `let x = foo.as_ref() // foo is freed "immediately"` * Until end of enclosing block -- intermediate value referenced by the block but does not escape * sometimes you want this * Until "wherever block value is stored" * super -- is this one * Until end of program (static/const/leak/whatever) Rules and * whatever temporary lifetime extension does: (`let { .. }` on Zulip) * `let x = &temp();` and `{ .. match &temp() {}; .. /*here*/ }` ```rust! match temp().lock().some_method() { /* autoref */ Foo(v) => { // v is not a ref into the locked data // unlocked before here. } } ``` ```rust! // // You wanted: "until end of enclosing statement" match &temp() { Foo(v) => { use(v); } } // freed here. ``` ```rust! // Escapes from arm -- error under our proposed rules // // You wanted: "until end of enclosing block" let x = match &temp() { Foo(v) => v, }; ``` ```rust let x = &temp(); f(x); // temp still alive. ``` ```rust let x = { super let lock = temp().lock(); lock.first() }; f(x); // lock still held (on purpose) ``` ### Use cases we are trying to solve ```rust! let x = format_args!(..); match format_args!(..) { _ => {} } // but not let x = match format_args!(..) { x => x /* escapes */ } ``` ```rust! let x = if let Some(s) = something() { s } else { super let temporary = vec![22]; &temporary }; ``` ### Proposed rules ```rust! // equivalent: let a = &temp(); let a = { super let t = temp(); &t }; let a = &super { temp() }; // not equivalent. // equivalent: match &temp() { .. } match { super let t = temp(); &t } { .. } match &super { temp() } { .. } // not equivalent?? ``` * There is a "result scope" and an "intermediate" scope * The *result scope* is the scope of the place where the node's value will be stored. In general, we prefer a smaller result scope. If the result of an expression will *always* be stored into a place with scope R, then the result scope is R. If it may only *sometimes* be stored into a place with scope R, then we would prefer the shorter scope. * The *intermediate scope* is a scope to store temporaries whose results may not need to outlive the expression * We write `ResultScope(E)` and `IntermediateScope(E)` to indicate the result scope of some expression `E` * When you have a reference to a `super {E}` expression, `E` is evaluated into a temporary with "result scope" lifetime * `Let x = Expr` -- result scope is the enclosing block ```rust let x = foo().as_ref(); // error because `foo()` is dropped at end of `let` // let x = $temp { foo() }.as_ref(); match foo().as_ref() { Some(x) => use(x); } // error // match $temp { foo() }.as_ref() { Some(x) => use(x); } // OK ``` ```rust! // instead of bar(foo(E1, E2)), want a way to write // ^^ temporaries in E1 live until after `bar()` is called bar({ let tmp1 = E1; // <-- temporaries in E1 are dropped after this `let` let tmp2 = E2; foo(tmp1, tmp2) }) // <-- but we want them dropped here to do this expansion foo(&$temp { E1 }, &$temp { E2 }) // should be able to do the same. &(&E1, E2) // can trigger temp-lifetime-extension ``` TC: The pin example, for our reference: ```rust let pin = { let x = &mut super { expr() }; // ^^^^^^^^^^^^^^^^ // Promotes such that it can be referenced by the block // enclosing the statement. unsafe { Pin::new_unchecked(x) } }; ``` `extend a = asdf in asdfasdfasdfs(&a)` `{ super let a = asdf; asdfasdfasdfs(&a) }` ```rust let a = &temp(); let a = pin!(temp()); let a = Pin { pinned: &temp() }; let a = Pin::new(&temp()); let a = Pin::new(&templifetimeextend { temp() }); let a = templifetimeextend t = temp() in Pin::new(&t); match &temp() {} match pin!(temp()) {} match Pin { pinned: &temp() } {} match Pin::new(&temp()) {} match Pin::new(&templifetimeextend { temp() }) {} match templifetimeextend t = temp() in Pin::new(&t) {} ``` ```rust match pin!(temp()) { .. } let x = pin!(temp()); // OK let x = match pin!(temp()) { a => a }; // error let x = match &temp() { a => a }; // error drop(x); let x = match match match pin!(temp()) {} {} {} {}; let pin = { match &mut super { expr() } { x => { unsafe { Pin::new_unchecked(x) } } } }; // These are equivalent: let a = &temp(); let a = { super let t = temp(); &t }; // Is equivalent to: let a = { &super { temp() } }; // But not equivalent to: let a = &super { temp() }; /// But is equivalent to?: let a = &let { temp() }; { ... match some syntax ( &temp() ) { ... } ... // we don't need a feature to keep `temp()` alive here. } let y = { ... let x = match &super { temp() } { ... }; ... // we don't need a feature to keep `temp()` alive here. drop(&x); 1 }; // still need to keep the temporary alive here. let y = { let x = &super match &super expr() { ... } x } { ... let some syntax ( &temp() ); ... // we *do* need a featur to keep `temp()` alive here. } let pin = Pin::new(&let { temp() }); let pin = unsafe { Pin::new_unchecked(&let { $expr }); }; // problem with unsafe let pin = { super let t = $expr; unsafe { Pin::new_unchecked(&t); } }; let pin = { let t = &super { $expr }; unsafe { Pin::new_unchecked(&t); } }; let y = { // Does this work? let x = match super { temp() }.as_ref() { a => a, }; &x }; print(y); let y = { super let t = temp(); let x = match t.as_ref() { a => a, }; &x }; print(y); let y = { let t = &super { temp() }; let x = match t.as_ref() { a => a, }; &x }; print(y); let x = &temp(); match &temp() {} let [super a, b] = &[1, 2]; // a and b are &i32, pointing into the [i32; 2] // how long does the [i32; 2] live? let x = match &temp() { a => a, }; ``` ### Formalized version (from below) *Definitions* * AST is a tree of expressions. The subexpressions for a given node can be classified into three categories: * *Intermediate* expressions are the most common. These produce results that can be consumed entirely within the node (though in some cases parts of those results may escape as references). Example: `foo(bar, baz)`. * *Scrutinee* expressions appear in `if`, `if let`, `while` and `match`. These produce results that will be examined to determine which "arms" to execute next. In the case of `match` and `if let`, parts of the scrutinee will always be stored into values that persist into those arms, but parts of the scrutinee may not escape into the overall result of the match. In the case of if/while, the result of the scrutinee is a boolean, so that consideration does not apply. * *Result-producing* indicate a subexpression whose result becomes the result of the parent. These are unusual and occur onnly in a specific set of places: * The *tail expression* of a block is a *result-producing* subexpression * The arms of a match/if/if-let are *result-producing* * Field values from a literal expression like `StructName { f1: $e1, f2: $e2 }` or `($e1, $e2)` are *result-producing* * The referent of a borrow (`&foo()`) is a result-producing expression * **PROPOSAL:** In Rust 2021, `VariantName(x)` and `TupleStruct(x)` are treated like any other function call (i.e., *intermediate expressions*), but they are in fact *result-producing*. We propose to change them in Rust 2024 to be **result-producing**, i.e., `StructName { 0: 22 }` and `StructName(22)` behave the same. There is one potential downside, though, it may mean that changing from tuple struct to a `fn` with same name is a breaking change (is it already?). * A *scope* is a node in the AST. Places are allocated within a given scope; they will be freed (popped from the stack) when that scope is exited. * *Temporaries* are produced when using a *value expression* `$value` (one that does not name a place in memory) in a location where a reference is required: * `&$value` / `&mut $value` -- * `$value.m()` -- when `m` is auto-ref * For each node in the AST, we compute two associated scopes that are used to determine the lifetime of temporaries: * The *result scope* is the scope of the place where the node's value will be stored. In general, we prefer a smaller result scope. If the result of an expression will *always* be stored into a place with scope R, then the result scope is R. If it may only *sometimes* be stored into a place with scope R, then we would prefer the shorter scope. * The *intermediate scope* is a scope to store temporaries whose results may not need to outlive the expression * We write `ResultScope(E)` and `IntermediateScope(E)` to indicate the result scope of some expression `E` * When processing some expression `E`, and recursing into one of its direct subexpressions `S`... * If `S` is categorized as an *intermediate* subexpression: * `ResultScope(S) = IntermediateScope(E)` -- because the value *might not* be referenced in the result * `IntermediateScope(S) = IntermediateScope(E)` * If `S` is categorized as a *scrutinee* subexpression: * `ResultScope(S) = IntermediateScope(E)` -- because the value *does* escape into the bindings for arms, but *might not* escape into the result of those arms * `IntermediateScope(S) = S` -- unlike other subexpressions which use the same intermediate scope as `S`, we prefer to drop temporaries in scrutinees *before* scrutinizing the result or entering the arms (e.g., `match &lock().unwrap() {...}` will drop the lock before matching). This is because of our first tenet: matches and if lets, in particular, often have arms that do a lot of things. Having shorter scopes here is important for reliability and avoids a common source of bugs in Rust today, hence we override our general preference for larger scopes. * If `S` is categorized as a *result-producing* subexpression: * `ResultScope(S) = ResultScope(E)` -- the result of `S` is being stored in the same place * `IntermediateScope(S) = E if E is a block, else IntermediateScope(E)` * In `let $pat = $expr`: * the bindings in `$pat` are given the scope of the enclosing block * `$expr` is evaluated with its: * *result scope* = enclosing block * *intermediate scope* = the `let` statement * In `super let $pat = $expr` * the bindings in `$pat` are given the *result* scope of the enclosing block * `$expr` is evaluated with its: * *result scope* = result scope of the enclosing block * *intermediate scope* = the `super let` statement * When introducing a temporary (via `&temp()` or `temp().m()`)... * if `temp()` appears in a *result-producing* location, it is given the *result scope* * otherwise, it is given the *intermediate* scope ### Use cases we are trying to solve ## 2024-01-18 ### Feedback on `super let` idea Most of the actionable feedback is roughly either: 1. The word "super" is confusing, or 2. "How about an inline syntax like &'super or &'a?", or 3. "How about an inline syntax like super { .. }?". For 1, we need to consider other words. `super` is the only existing keyword that makes sense, but we could add a new keyword For 2, it would look like this: ```rust= let a = 'x: { let pinned = &'x temp(); // lives longer than the `'x` block though. unsafe { Pin::new_unchecked(pinned) } }; // Or: let a = { let pinned = &'super temp(); unsafe { Pin::new_unchecked(pinned) } }; ``` For 3, it would looks like this: ```rust= let a = { let pinned = &super { temp() }; unsafe { Pin::new_unchecked(pinned) } }; ``` Mara's opinion: continue with `super { }` blocks, but perhaps with a different keyword. (Either in stead of or in addition to `super let` statements.) Niko: Agree, with just `super {}` not `super let`, since it's niche. ### RFC and next steps Need to hurry to make it into the 2024 edition. Next steps: RFC Draft: https://hackmd.io/wU_CYnUeT7G7hYazna6vDQ Motivation section not complete/clear yet. Maybe take some stuff from https://blog.m-ou.se/super-let/ Adapt RFC for super blocks instead of super let. How does super expression work? `super{expr} == expr` produces the same value but has different temporary rules `&super{expr}` lives longer `match super{expr} { ref x => .. } // same thing` `super { 1 } + 2` pointless, lint against this `super { Some(22) }.as_ref() == Some(&super{22}) // (more or less equivalent)` ### Status of the PR Hasn't been much review, contains implementation for `super let` PR: https://github.com/rust-lang/rust/pull/119043 ### NEXT STEPS * [ ] Extend RFC motivation -- Mara * [ ] Rewrite RFC details to use `super { }` expressions -- Xiang * [ ] Find a compiler reviewer -- nikomatsakis :) * https://github.com/rust-lang/rust/pull/119043 * [ ] Review RFC progress on Tuesday at ~9:30 Eastern time -- nikomatsakis :) ## 2023-11-30 ### short-circuiting ```rust= if ( a() && b() ) || ( c() && d() ) { expr1 } else { expr2 } // --> let continuation1 = || expr1; let continuation2 = || expr2; if ( a() && b() ) || ( c() && d() ) { continuation1() } else { continuation2() } // --> if a() { if b() { continuation1() } else if c() { if d() { continuation1() } else { continuation2() } } else { continuation2() } } else if c() { if d() { continuation1() } else { continuation2() } } else { continuation2() } ``` Mara: Shouldn't that be equivalent to this? ```rust= if if a() { b() } else { false } || if c() { d() } else { false } { expr1 } else { expr2 } ``` ```rust= #![feature(let_chains)] if test1() && let var1 = init1() && test2() && let var2 = init2() { } ``` Mara: i think super let in init1 and init2 should work (extend to entire body of if), and that super let in test1 and test2 are meaningless (compiler should suggest removing `super`). To be consistent with a regular `if test1() && test2()`. Different than: ```rust= #![feature(let_chains)] if let true = test1() && let var1 = init1() && let true = test2() && let var2 = init2() { } ``` Here, test1() and test2() can have a super let that extends to the entire block? ```rust= if let true = .. { // can one invoke temporary lifetime extension here? } ``` Can't invoke TLE here actually. Because `true` as a pattern has no materialised variable binding. :( ### Alternative to super let: `&'a expr` ```rust let x = 'a: { let file = &'a File::open(..); // <--- Thing::new(&file) }; ``` How about ```rust= let x = { super let thing = Thing {}; if condition { consume(thing); None } else { Some(&thing) } }; ``` can that be written as `&'super`? Don't think so? ## 2023-11-16 [Xiang made progress on RFC](https://hackmd.io/wU_CYnUeT7G7hYazna6vDQ) Refactor will be next https://github.com/rust-lang/rust/pull/111725 ### notes * we want to change short-circuit operators * we probably want to distinguish `if` and `match` scrutinees, such that given `if { super let x = ... }`, the `x` is dropped before entering `if` body (that is, the `super` is meaningless here) * this enables a high fidelity rewrite * we probably want more idiomatic rewrites for `match` and `if let` expressions, using `super let` is weird * `match &foo() { ... }` -- this should work * `match { super let f = foo(); &f } { }` -- therefore this should work * crater run could be useful to collect data on how commonly `match`es and `if let`s depend on today's behaviour. * ### short-circuit operators ```rust if e1 && e2 ... { --- } // ------------ ^ drop all temporaries before entering "then" ``` proposal was to change this to... ```rust if e1 && e2 ... { --- } // -- -- // drop temporaries after evaluating each expression `e1`, `e2` ``` means that: * The MIR built for a `if e1 && e2 {` is now equivalent to the MIR for `if e1 { if e2 {`, which changes how borrowck understands that code, see `tests/ui/rfc-2497-if-let-chains/chains-without-let.rs`. could have some special rules around short-circuit operators... * `a && b` is kind of like `if a { b } else { false }` so...you could justify it this way * `a || b` is kind of like `if a { true } else { b }` so...you could justify it this way `if mutex1.lock().condition() && mutex2.lock().condition() { .. }` -> fine and good to unlock asap. `if cell.write().update_for_a() || cell.write().update_for_b() { ... }` `if foo.lock().acquire() && .... /* expects lock to be held */ {` `if a != null && a.foo() { ` clippy can today already suggest merging two `if`s into a single `if &&`. would be nice if those were exactly identical. issue: https://github.com/rust-lang/rust/issues/111583, https://github.com/rust-lang/rust/pull/111752 observations: the lhs/rhs `&&` and `||` **Conclusion: operands of short-circuit operators should be considered "scrutinee expressions"**, just like an `if` condition. Why? * We know that a reference cannot escape into the result (must be a boolean). Tenets suggest shorter lifetimes are better. * Equivalence between `if a && b {` and `if a { if b }`, which is intuitive but also useful for if-let chains. * clippy suggests rewriting `if a { if b {` into `if a && b {`, for example, but that can be wrong in subtle ways today * Con: the rules are a bit more complex, not just innermost `if` but innermost shortcircuit operator -- but not super-more, there's already a list of things that treat their subexpr as scrutinee and this feels like it belongs in that list * Also: cannot come up with an example where you would want the longer temporary lifetime that seems realistic :) ### edition migration * we have a pre-existing concept for closure migration of "insignificant destructors" * "destructors that are known to only free memory" cases to be careful of... * temporaries in a `match` or `if let`/`while let` scrutinee * maybe something with `while` or `for` * tail expressions in blocks * temporaries in a short-circuit operand (`&&`, `||`) #### examples ##### match ```rust let foo: &Mutex<Vec<String>>; match foo.lock().unwrap().first() { Some(x) => { /* in Rust 2024, this code gets a compilation error */ /* in Rust 2021, `lock` is dropped at the end of the temporary scope for `match` */ } None => {} } ``` ```rust let foo: &Mutex<Vec<String>>; match { super let l = foo.lock().unwrap(); l.first() } { Some(x) => { // This works in all editions, same as Rust 2021 above. } None => {} } ``` ##### short-circuit ```rust let l: &Mutex<Content>; impl Content { fn test(&self) { true } } if foo(l.lock().unwrap().test() && something_else()) { // same for `||` // In Rust 2021, the lock is held while `something_else()` executes and dropped before entering the `if` // In Rust 2024, it is not } if foo({ super let lock = l.lock(); lock.unwrap().test()} && something_else()) { // When does `lock` get dropped? Niko thinks current rules are " before entering the `if` body " // is th } if { super let l = lock(); l.unwrap().test() } && something_else() { // is the lock held here? // or does `super` simply have no effect here? // or does it extend to the entire condition but not the body? } // should be identical to: if { super let l = lock(); l.unwrap().test() } { if something_else() { //.. } } ``` ##### random mara question - `if _ && _ { .. }` in 2024 behaves the same as `if _ { if _ { .. } }` - `let a = _ && _;`, also drops temporaries early? - niko: in 2024, yes. behaves the same as `let a = if _ { _ } else { false }` ##### random niko question To what extent are these equivalent... ```rust if $foo { $bar } else { $baz } match $foo { true => $bar, false => $baz } ``` ...and in particular with respect to super lock... ```rust if { super let x = foo.lock(); x.something() } { $bar } else { $baz } // drops `x` before evaluating `$bar` match { super let x = foo.lock(); x.something() } { true => $bar, false => $baz } // drops `x` after evaluating `$bar` ``` `super` is meaningless in the `if`, would result in a warning. ##### crater run? We need to check how often a `match` or `if let` needs the current temporary lifetime rules. If there are many, we might need quite an advanced migration lint that doesn't generate ugly code. ```rust let x = foo.lock().unwrap(); match x.something() { ... } drop(x); // ? ``` ##### tail expressions in blocks - ### Some() vs Some {} ```rust= let a = Some(&temp()); // not extended (!) let a = Some { 0: &temp() }; // extended ``` if we only look at syntax, we can't distinguish between `Some()` and `f()`. we can fix this though. treat variants differently than functions. niko: we could allow functions to opt-in to this behaviour. ### `super let` lint can we make suggestions to add `super` and remove `super` happen reliably in all cases? ``` help: add `super` keyword before this `let` | 6 | super let file = File::create(…)?; | +++++ ``` example: ```rust= let result = if something() { ... } else { let temp = File::create()?; // <-- want the lint here &temp }; ``` other direction: ``` warning:` super` keyword unnecessary | 6 | super let file = File::create(…)?; | ----- `super` keyword unnecessary 7 | file.into() | ---- file is always moved before enclosing scope ends help: `file` would live just as long with a regular `let` ``` when would this trigger? 1. `if` or other cases where it is does not change the scope at all 2. it changes the scope, but the variable is always dropped/moved before exiting the (smaller) scope anyway ```rust= let result = if something() { ... } else { super let temp = File::create()?; // <-- want the lint here temp.into() // consumes the File }; ``` ## 2023-11-02 ```rust= #![feature(new_temp_lifetime)] fn main() { println!("{}", 10) } ``` ... expands to ```rust= #![feature(prelude_import)] #![feature(new_temp_lifetime)] #![feature(print_internals)] #[prelude_import] use std::prelude::rust_2024::*; #[macro_use] extern crate std; fn main() { ::std::io::_print(format_args!("{0}\n", 10)); // ^~~~~~~~~~~~~~~~~~~~~~~~~ temporary value is freed // at the end } ``` lowers in hir to: ```rust= fn main() { ::std::io::_print( fmt::Arguments::new_v1( &["10\n"], fmt::Argument::none() // returns &[] ) ) /* temporary should be freed here */; } ``` does it work for more complicated ones too? `print!("{0} {0:?}", a)`? that expands to a `match`. *False alarm because it was reported by a buggy implementation* :sweat_smile: ## 2023-10-19 "Value-expression" rules that Xiang does not like: *temporary lifetime extension*: ```rust! let x = &StructLit { y: 0 }; // ^~~~~~~~~~~~~~~~ This StructLit value will get its lifetime extended to the whole block ``` Rust 2021 behavior: * `StructLit { y: 0 }` above gets extended * Per the Oct 5 rules below: * `&<expr>` -- the subexpression `<expr>` is result-producing Nevermind it's fine. :) --- [Draft RFC](https://hackmd.io/wU_CYnUeT7G7hYazna6vDQ) --- * Timeline and upcoming events * Xiang to open PR with feature-gate and add test cases below * cc mara/niko to check that wegot things covered correctly * niko can nominate the feature gate for lang team * Mara authoring blog post describing the problems * RFC open by early November * Niko (maybe) to experiment with a more functional description of the rules --- Experimental description of the Oct 5 rules... AST looks roughly like this: ``` Function = Sig Block Block = { Statement* Expr } Statement = Expr | `let` pattern = Expr | `super let` pattern = Expr Expr = Block | Op(Expr*) // where Op is all the different kinds of "expression operators", e.g., `&`, etc Op = & | Call | + | Tuple | StructName(field...) | Match(Pat*) | ... ``` for a given expression operator `Op` we categorize its subexpressions as ``` Category = Intermediate| ResultProducing | Scrutinee ``` and define `categories(Op) = Category*`, one for each expression. i.e.: * `categories(&) = [ResultProducing]`. * `categories(+) = [Intermediate, Intermediate]`. * `categories(Call(...)) = [Intermediate...]` * `categories(StructName(fieldname...)) = [ResultProducing...]` * `categories(Match(Pat...)) = [Scrutinee, ResultProducing...]`, i.e., the scrutinee is, well, a scrutinee, but each of the arm expressions are result-producing. For a given expression, we can reduce a set `result_producing(E)` of expressions that includes E and all of its (transitive) result-producing subexpressions. So `result_producing(&Foo { x: bar(22) })` would include `&Foo { .. }`, `Foo { x: bar(22) }`, and `bar(22)`. It does not include `bar` or `22`, because all subexpressions of call nodes are intermediate. Some expressions introducer temporaries. Those expressions are always part of some statement or the tail expression of a block. To determine the lifetime of temporaries: * Statement: * `Expr` -- * temporary scope for all subexpressions is end of statement * `let pattern = E_init` -- * temporary scope for `result_producing(E_init)` is end of enclosing block * temporary scope for all other subexpressions is end of statement * `super let pattern = Expr` * temporaries storing the results of expressions in `result_producing(E_init)` are dropped in result-scope of block * all other temporaries dropped at end of statement * Tail expression `E`: * if an expression `E_t` in `result_producing(E)` is stored into a temporary, that temporary lifetime is the result scope of the block. * all other subexpressions are dropped at end of block. ## 2023-10-05 rules Trying to write out the rules below. *Tenets* We believe that... * **When in doubt, shorter scopes are more reliable**. If you give a shorter scope than the programmer needed, they get a borrow check error, and have to insert an explicit `let` that makes the lifetime clear. If you give a *longer* scope than the programmer needs, they may get deadlocks or other surprising runtime behavior. We prefer the former. * **For any expression `E`, it should always be possible to write a block `{ ... }` that behaves the same.** For this to be true, we need to introduce some way within a block to create values that have the same scope as temporaries would in `E`. This design achieves this. Specifically, `{ super let x = temp(); &x }` and `&temp()` are always equivalent. * **Larger scopes are more convenient.** Larger scopes are generally more convenient and give users fewer errors. We try to use the larger scopes when we can, or when it is clearly what the user intended. For example, the temporary in `let x = &temp()` is clearly being stored into `x` For example, we don't muck about with fine-grained scopes within individual statements for the most part * **Scopes should be predictable and stable.** We don't want the scopes of values to depend on type-checking or fine-grained details of what the user wrote. The rules we propose aren't *simple*, they try to do the right thing, but they avoid fine-grained analysis and instead work largely by matching the overall structure of expressions. As one example, we have a set of rules for `$expr` in `let $pat = $expr`, but those rules don't depend on fine-grained details of the pattern `$pat` (so i.e., if any part of a value escapes, we consider that equivalent to all parts escaping; but we do look to see if the value may sometimes never escape). *Definitions* * AST is a tree of expressions. The subexpressions for a given node can be classified into three categories: * *Intermediate* expressions are the most common. These produce results that can be consumed entirely within the node (though in some cases parts of those results may escape as references). Example: `foo(bar, baz)`. * *Scrutinee* expressions appear in `if`, `if let`, `while` and `match`. These produce results that will be examined to determine which "arms" to execute next. In the case of `match` and `if let`, parts of the scrutinee will always be stored into values that persist into those arms, but parts of the scrutinee may not escape into the overall result of the match. In the case of if/while, the result of the scrutinee is a boolean, so that consideration does not apply. * *Result-producing* indicate a subexpression whose result becomes the result of the parent. These are unusual and occur onnly in a specific set of places: * The *tail expression* of a block is a *result-producing* subexpression * The arms of a match/if/if-let are *result-producing* * Field values from a literal expression like `StructName { f1: $e1, f2: $e2 }` or `($e1, $e2)` are *result-producing* * The referent of a borrow (`&foo()`) is a result-producing expression * **PROPOSAL:** In Rust 2021, `VariantName(x)` and `TupleStruct(x)` are treated like any other function call (i.e., *intermediate expressions*), but they are in fact *result-producing*. We propose to change them in Rust 2024 to be **result-producing**, i.e., `StructName { 0: 22 }` and `StructName(22)` behave the same. There is one potential downside, though, it may mean that changing from tuple struct to a `fn` with same name is a breaking change (is it already?). * A *scope* is a node in the AST. Places are allocated within a given scope; they will be freed (popped from the stack) when that scope is exited. * *Temporaries* are produced when using a *value expression* `$value` (one that does not name a place in memory) in a location where a reference is required: * `&$value` / `&mut $value` -- * `$value.m()` -- when `m` is auto-ref * For each node in the AST, we compute two associated scopes that are used to determine the lifetime of temporaries: * The *result scope* is the scope of the place where the node's value will be stored. In general, we prefer a smaller result scope. If the result of an expression will *always* be stored into a place with scope R, then the result scope is R. If it may only *sometimes* be stored into a place with scope R, then we would prefer the shorter scope. * The *intermediate scope* is a scope to store temporaries whose results may not need to outlive the expression * We write `ResultScope(E)` and `IntermediateScope(E)` to indicate the result scope of some expression `E` * When processing some expression `E`, and recursing into one of its direct subexpressions `S`... * If `S` is categorized as an *intermediate* subexpression: * `ResultScope(S) = IntermediateScope(E)` -- because the value *might not* be referenced in the result * `IntermediateScope(S) = IntermediateScope(E)` * If `S` is categorized as a *scrutinee* subexpression: * `ResultScope(S) = IntermediateScope(E)` -- because the value *does* escape into the bindings for arms, but *might not* escape into the result of those arms * `IntermediateScope(S) = S` -- unlike other subexpressions which use the same intermediate scope as `S`, we prefer to drop temporaries in scrutinees *before* scrutinizing the result or entering the arms (e.g., `match &lock().unwrap() {...}` will drop the lock before matching). This is because of our first tenet: matches and if lets, in particular, often have arms that do a lot of things. Having shorter scopes here is important for reliability and avoids a common source of bugs in Rust today, hence we override our general preference for larger scopes. * If `S` is categorized as a *result-producing* subexpression: * `ResultScope(S) = ResultScope(E)` -- the result of `S` is being stored in the same place * `IntermediateScope(S) = E if E is a block, else IntermediateScope(E)` * In `let $pat = $expr`: * the bindings in `$pat` are given the scope of the enclosing block * `$expr` is evaluated with its: * *result scope* = enclosing block * *intermediate scope* = the `let` statement * In `super let $pat = $expr` * the bindings in `$pat` are given the *result* scope of the enclosing block * `$expr` is evaluated with its: * *result scope* = result scope of the enclosing block * *intermediate scope* = the `super let` statement * When introducing a temporary (via `&temp()` or `temp().m()`)... * if `temp()` appears in a *result-producing* location, it is given the *result scope* * otherwise, it is given the *intermediate* scope ### Examples #### Escape-from-if ```rust 'b: { let x = if true { super let a = temp(); &a } else { panic!() }; } // OR 'b: { let x = if true { &temp() } else { panic!() }; } ``` * Here, the *result-scope* of the `if` is equal to the enclosing block `'b` * The arm of the if is a *result-producing* expression, so its result-scope is `'b` * In first example, `a` is a super let, so it is given the result scope of enclosing block, and hence has the scope `'b` * In second example, the same is true of `&temp()` #### Match-drops-lock-before-arm Today the lock is held until end of statement enclosing the match, but we don't want that [Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=263d3223c283449cb1561756a46de214) ```rust use std::sync::Mutex; fn main() { let x = Mutex::new(()); match x.lock().unwrap().something() /* 's, the scrutinee */ { () => (), /* 'a, the arm */ } /* 'm, the match expression */ } //<-- the block `'b` is enclosed by a root scope representing the fn call ``` * Here, the *result-scope* of the main block is `'f`, the scope of the fn * The tail expression `'m` is *result-producing*, so its result-scope is `'f`; intermediate scope is `b` * The *scrutinee subexpression* `'s` has result scope of `'b` and intermediate scope of `'s` * The `something()` call has intermediate subexpression `x.lock().unwrap()`, so its result/intermediate scope is `'s` * The temporary produced by call to `lock()` is therefore dropped at `'s` * **This differs from the behavior today**, where the intermediate scope for match scrutinee is equal to the intermediate scope of the match. #### Match-with-super-let Today the lock is held until end of statement enclosing the match, but we don't want that [Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=263d3223c283449cb1561756a46de214) ```rust use std::sync::Mutex; fn main() { let x = Mutex::new(()); match { super let l = x.lock().unwrap(); l.something() } /* 's, the scrutinee */ { () => (), /* 'a, the arm */ } /* 'm, the match expression */ } //<-- the block `'b` is enclosed by a root scope representing the fn call ``` * Here, the *result-scope* of the main block is `'f`, the scope of the fn * The tail expression `'m` is *result-producing*, so its result-scope is `'f`; intermediate scope is `b` * The *scrutinee subexpression* `'s` has result scope of `'b` and intermediate scope of `'s` * The variable `l` is assigned to the result scope (`'b`), and hence will be dropped on exit from the block * **This is equivalent to behavior of a temporary like `match &temp()` today**. #### Tail-expr-drops-early Today this gives an error, but we don't want one anymore: [Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=263d3223c283449cb1561756a46de214) ```rust use std::sync::Mutex; fn main() { let x = Mutex::new(()); /* 's1, first statement */ *x.lock().unwrap() /* 'e1, the tail expression */ // ------ temporary } //<-- the block `'b` is enclosed by a root scope representing the fn call ``` * Here, the *result-scope* of the main block is `'f`, the scope of the fn * The tail expression `'e1` is *result-producing*, so its result-scope is `'f`; intermediate scope is `b` * The `*` expression's contents are *intermediate* so their result-scope is `'b` * Hence the lock is dropped on exit from the block, before `x` (following the LIFO rule) * **This differs from the behavior today**, where the intermediate scope for the tail expression of a block is effectively the intermediate scope of the block. ## 2023-10-05 * looked at the options to restrict temporary lifetimes, esp. around match scrutinee, as it is a common source of errors * trying to simplify the rules, but also found some breakage in stdlib * `super let` is proposal to explicitly declare the extended temporary lifetimes Intuition: * super let is used to create intermediate values that live "as long as the result of the block" * extended temporaries have to live as long as the result * non-extended temporaries are intermediate values that can be dropped after the result is stored ```rust= { let x = y; // creating a location that will dropped at end of block ... // can only be referenced within the block } ``` ```rust { super let x = y; // creating a location that will be referenced in the result of the block &x // e.g. here } ``` Mara: ```rust let x = if true { super let a = ..; &a // this should work. but this is not a spot where temporary lifetime extension works today. should it? } else { panic!() }; ``` Niko: If possible, these should be the same: ```rust &temporary() { super let x = temporary(); &x } ``` One tricky case: ```rust match &temporary() { // ----------- wanted temporaries to be dropped before entering arms } match { super let x = temporary(); &x } { ... } // equivalent, also errors let x = temporary(); match &x { ... } // works ``` Mara: How about: ? ```rust let a = if true { { super let x = temporary(); &x } } else { .. }; ``` Conclusion: temporary lifetime extension should also work through if-else expressions. Mara: So, this should compile: ```rust let a = if blah { &File::create(..).unwrap() } else { &File::open(..).unwrap() }; ``` ```rust let x = { &foo.lock().compute() ---------- when does this get dropped? }; ``` ## surprising error today [Playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=263d3223c283449cb1561756a46de214) ```rust use std::sync::Mutex; fn main() { let x = Mutex::new(()); *x.lock().unwrap() } ``` ```rust! // rust 2021 fn foo() { let x = Lock::new(); x.lock().foo // temporary gets dropped after x is dropped, today } ``` In Rust 2021, it is equivalent to this today ```rust! fn foo() { let x = Lock::new(); super let l = x.lock(); // error l.foo } ``` and hence you get an error, but in Rust 2024 is equivalent to ```rust! fn foo() { let x = Lock::new(); let l = x.lock(); // OK l.foo } ``` automatic edition upgrade possible by using `super let` for temporaries ```rust! fn foo() { let x = Lock::new(); { super let l = x.lock(); l.foo } } ``` ## Invariants we want * `{super let x = temp(); &x}` and `&temp()` are equivalent * given `{&temp()}`, `temp()` is in a temporary that lives long enough to be stored * `if condition { &temp() } else { &temp() }` is the same * `match lock().foo { }` -- for sure should drop `lock()` before entering arms * what about `match &lock().foo { ... }` ? * `let x = ...` * `x` lives until end of block * the result scope of `...` is the enclosing block * `super let x = ...` * `x` lives until end of result scope of the enclosing block * the result scope of `...` is the result scope of the enclosing block * tail expression of a block: * the result scope is the result scope of the block * block for an if/else: * the result scope is the result scope of the if/else * given a `&temp()`, store it in the result scope Walking through the rules: ```rust 'a: { let x = // Result scope is 'a if true { super let a = ..; // Lives until end of 'a &a // this should work. but this is not a spot where temporary lifetime extension works today. should it? } else { panic!() }; } ``` ```rust // where do we expect `x` to be freed here? 'b: { 's: let y = foo({ super let x = 22; &x }); // answer: end of `'s` } ``` ```rust // where do we expect `x1` and `x2` to be freed here? 'b: { 's: let y = { super let x1 = first_value(); super let x2 = some_composite_thing(&x1.foo); &x2 }; } // dropped at the end of 'b ``` Niko's version: * For each expression, we have a *result scope* and an *enclosing scope* * result scope = scope of where the result will be stored * enclosing scope = the enclosing block, conditional expression, etc ```rust ``` Mara: How about this? ```rust { let a = match { super let b = lock(); &b.foo } { x => { ... x // <- depends on this line??? that'd be weird. }, }; // is the lock dropped here? .. } // or here? ``` How long should `super let` live in match scrutinee? * Definitely not: * As long as the result of the match * Why not? * Because sometimes you * Maybe: as long as match * ```rust match &expr() { ... } { let x = expr(); match &x { ... } // *very similar* } match m!() { } today: m!() expands to: a.lock().foo future: m!() expands to: { super let l = a.lock(); l.foo } or: m!() expands to: ???? ``` some examples around match: [This does not compile](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=e2a68ae5db775807923e3414903bbf7b) ```rust let x = match &temp() { v => v}; println!("{x}"); fn temp() -> u32 { 22 } ``` ```rust // Mara: This should still work: match format_args!("{x}") { f => { write_fmt(f); } } // so, temporaries from format_args!() are dropped here. ``` Only thing that changes behavior: ```rust fn foo() { let temp1 = ...; // This does change behavior: // // temp2 is not a "result producing" location, so it should be dropped at the "intermediate scope", // which is the end of the tail expression. match &temp2() { v => ... } } ``` ## 2023-07-06 - Before I forget... is it still possible to get an appointment on Thursdays in July or August? - Niko working from Boston July 17-19 - Tuesday appt: :x: - Niko working from France July 24-26 - Propose a 30min slot somewhere here? - Niko working from Greece Aug 1 - 4 - Propose a 30min slot somewhere here? - Niko traveling other days until Aug 19 how Niko expected it to work (but it doesn't *quite*?): ```rust! let x = { super let y = 1; &y }; // lifetime of y is equal to // what lifetime of block *would* be if it were // an `&`-rvalue expression (e.g., `&{...}`) f(x); ``` ```rust! fn id<T>(t: T) -> T { t } let x = id({ super let y = 1; &y }) /* y freed here */ ; let x = id(&foo()); // also an error // lifetime of y is equal to // what lifetime of block *would* be if it were // an `&`-rvalue expression (e.g., `&{...}`) f(x); ``` but: - Do we need to inspect if the enclosing block `S_parent` receives lifetime extension? e.g. ```rust 'b: { // Presumably, under the new scoping rule, `guard` should have dropped ... match 's: { super let guard = mutex.lock().unwrap(); &mut *guard } // ... here as soon as 's is out of scope but ... { Some(things) => { // we need the `guard` here to mutate `things` ... things.mutate(); } _ => { } } // ... so it makes more sense if `guard` is dropped here // and that is how `super let` would be useful. do_something_else_without_locking(); } ``` The scope `'s` is the said S_parent scope. It does not receive lifetime extensions. Nonethless, I believe it makes sense to rules for match: * today * temporary lifetimes extend to the innermost statement * e.g. `let x = match foo().as_ref() { x => x };` gives an error * potential revised rule A * temporary lifetimes end after scrutinee evaluation * in particular `match ref_cell.write().bar() { }` drops refcell after calling `bar()` * `match &id(22) { ... }` is an error * `match { super let guard = 22; &guard }` is also an error * potential revised rule B * apply the "extended temporary lifetimes" template to the scrutinee; anything matched temporaries are extended to the temporary lifetime of the match itself * otherwise temporary lifetime end after scutinee evaluation * `match ref_cell.write().bar() { }` unchanged * but in `match &id(22) { ... }` the temp for `id(22)` is freed at end of innermost enclosing statement * or if `let x = match &id(22) { v => v };`, because match has extended temporary lifetime, so does the `id(22)` temporary * `match { super let guard = 22; &guard }` behaves the same * potential revised rule C * apply the "extended temporary lifetimes" template to the scrutinee; anything matched temporaries are extended to the **innermost statement** containing the match * otherwise temporary lifetime end after scutinee evaluation * `match ref_cell.write().bar() { }` unchanged * but in `match &id(22) { ... }` the temp for `id(22)` is freed at end of innermost enclosing statement * `but let x = match &id(22) { v => v };` is an error * `match { super let guard = 22; &guard }` behaves the same ```rust let x = Foo { field: &22 }; match Foo { field: &22 } { f => ... } /* works */ // here a temporary is created for `noisy_drop()` result // from which we then extract the value field; // but `noisy_drop` is not dropped until the end of the // BLOCK... let x: u32 = match Foo { field: &noisy_drop() } => { f => f.field.value }; // ...because you might have done this: let x: &NoisyDrop = match Foo { field: &noisy_drop() } => { f => f.field }; // Here the refcell borrow is stored in an `&mut` that winds up with a lifetime extended until the end of the block let x = match Foo { x: &mut foo.write() } => { f.x += 1; 22 }; ``` you can replace ... `match <expr> { }` ...with ```rust { super let f = <expr>; match f { ... } } ``` this is kind of like the behavior we have today but different. *almost* the behavior we have today except that `f` may live longer in a case like `let x = { super let f = <expr>; match f { ... } }`, as it would be dropped at end of block, but `match <expr> { ... }` would be dropped at end of statement ? Q: What is the behavior of `super let x = &foo()` again? Answer: currently, if they are ordinary temporaries, dropped at the end of the initializer; otherwise, if extended, they live same as the super let bindings. Resolution: this behavior is implemented in the super-let draft today (2023-07-19) ## 2023-06-08 consideration, how should the `assert_eq` macro work today: ```rust match (&$left, &$right) { (left, right) => ... } ``` want to make sure a use like this ```rust assert_eq!(something().as_ref(), something_else.as_ref()) ``` ```rust some_other_thing( { super let left = &$left; super let right = &$right; match (left, right) { ... } } // ideal: temporaries to be dropped as we exit the block ) // alternative: temporaries dropped here ``` Today: * when an expression is evaluted, there are two surrounding scopes * temporary scope * result scope * the scope in which the result will be stored * `let $pat = $expr` * divide the temporaries into two categories, extended + normal * bindings `$pat` are dropped at the end of the block * extended temporaries are dropped after `$pat` is dropped * normal temporaries are dropped after `$pat` is assigned * `super let $pat = $expr` * divide the temporaries into two categories, extended + normal * bindings in `$pat` are dropped at the end of the "result scope" of the surrounding block * extended temporaries are dropped after `$pat` is dropped * normal temporaries are dropped after `$pat` is assigned ```rust // An expression $expr can appear... // ...as the initializer of a `let` let $pat = $expr; // ...as an operand of some other expression foo($expr); (or `$expr + $expr2`, etc) // ...as the tail expression in a block { ...; $expr } // ...as a top-level statement { ... $expr; ... } ``` * define a "drop scope" as either a statement or a block ```rust { let x = &foo().as_str(); // ----- -------- // | | // | extended // normal } ``` We call the "temporary scope" of an expression to be when it would be dropped if it were made into a temporary. For a given let initializer: * Normal temporaries: drop at the end of innermost statement or block * Bindings: drop at the end of the innermost block * Extended temporaries: drop at the end of the innermost block For a given super let initializer: * Call the temporary scope of the enclosing block S_parent * Call the block that encloses the enclosing block B_parent * Normal temporaries: drop at the end of S_parent * Bindings: * if enclosing block has extended temporary lifetime * drop at the end of B_parent * else * drop at the end of S_parent * Extended temporaries: * if enclosing block has extended temporary lifetime * drop at the end of B_parent * else * drop at the end of S_parent ```rust some_other_thing( { super let left = &$left; super let right = &$right; match (left, right) { ... } } ); // <-- temporaries, left, and right dropped here ``` ```rust { let x = &{ super let v = &$v; foo(v) }; // non-extended temporaries in $v are dropped here } // extended temporaries and v are dropped here ``` Intuition: * super let is used to create intermediate values that live "as long as the result of the block" * extended temporaries have to live as long as the result * non-extended temporaries are intermediate values that can be dropped after the result is stored ## 2023-05-18 Two parts * `let super` * narrower match scopes by default * edition rewrite to use let super Question marks: * Does it apply to the `let` or the binding * What happens to the ### Definition The **temporary scope** (need better name) of an expression is * "if that expression is stored into a temporary, the scope of that temporary" Defined as (in Rust 2024)... * innermost statement, match discriminant, block, or if condition; or * (if extended by `super let x`) the scope of the let Expressions are *extended by a let L* if... * (criteria, unchanged from today) `super let` at function scope is an error ```rust if { let super x = 22; &x == 0 } { } match { let super x = 22; &x } { } // works today match &String::new() { foo => drop(foo), } // expect this to work match { let super x = String::new(); &x } { foo => drop(foo); } // in Rust 2024, error match &String::new() { foo => drop(foo), } match {super let x = String::new(); &x} { foo => } match foo.lock().counter { // ^ // lock should be dropped here Some(c) => (), None => (), } match { let tmp = foo.lock(); tmp.counter } { Some(c) => (), None => (), } let value = if foo { let super x = String::new(); &x } else { &self.field }; ``` ### Rule of thumb Role of thumb is that these are equivalent: * `&foo()` * `{super let x = foo(); &x}` --- * `&<expr>` * `{super let x = &<expr>; x}` ### Tenets * macros need the ability to explicitly emulate temporary lifetimes * `&<expr>` and `{super let x = &<expr>; x}` are equivalent * `&<rvalue>` and `{super let x = <rvalue>; &x}` are equivalent, as a corrolary * extend lifetimes when it is necessary to do so to avoid a compilation error * shorter lifetimes are less bug-prone -- therefore we prefer shorter lifetimes otherwise * borrow checker will flag cases where they need to be longer, but cannot help you when `Drop` runs later than you expect ### Litmus tests #### format-args macro expansion ```rust // today let f = fmt::Arguments::new(.., &[fmt::Argument::new(&a)]); dbg!(f); ``` ```rust // with super let let f = { // freed at end of block (just after `f` goes out of scope) super let arg0 = &<expr>; // ? super let args = [fmt::Argument::new(arg0)]; fmt::Arguments::new(.., &args) }; dbg!(f); // using in other position something({ super let args = [fmt::Argument::new(&a)]; fmt::Arguments::new(.., &args) }) // freed at end of statement ``` #### match discriminant ```rust // not extended to temporary scope of match match foo.lock().counter { // ^ // lock should be dropped here Some(c) => (), None => (), } ``` ```rust // not expected to extend, because of the equivalence rule below match &foo.lock().counter { // ^ // lock should be dropped here Some(c) => (), None => (), } // error match { super let counter = &foo.lock().counter; counter } { } ``` ### Question ## Before ### use cases #### The `format_args` macro ```rust= let f = format_args!("{}", a); // Error dbg!(f); ``` ```rust= let f = fmt::Arguments::new(.., &[fmt::Argument::new(&a)]); // ^^ <- error dbg!(f); ``` ```rust= let f = match (a, ) { args => unsafe { fmt::Arguments::new(.., &[fmt::Argument::new(&args.0)]) } }; dbg!(f); ``` ```rust= // let-for idea: let f = let args = (a, ) for unsafe { fmt::Arguments::new(.., &[fmt::Argument::new(&args.0)]) }; dbg!(f); ``` ```rust= // super-let idea: let f = { super let args = (a, ); unsafe { fmt::Arguments::new(.., &[fmt::Argument::new(&args.0)]) } }; dbg!(f); ``` ```rust= // let-lifetime idea: let f = 'a: { let 'a args = (a, ); unsafe { fmt::Arguments::new(.., &[fmt::Argument::new(&args.0)]) } }; dbg!(f); ``` ```rust= let args = &[fmt::Arguments::new(&a)]; // macro can't inject code here. let f = unsafe { fmt::Arguments::new(.., &[fmt::Argument::new(&args.0)]) }; dbg!(f); ``` #### The `pin` macro ```rust= let p = pin!(expr); ``` ```rust= let p = Pin { pinned: &expr }; // not Pin::new() // safety: `pinned` is unstable ``` ```rust= // with unsafe fields: let p = unsafe { Pin { pinned: &expr } }; // now expr is within unsafe {}, also bad. ``` ```rust= // ideal solution?? let p = { magic let pinned = expr; unsafe { Pin::new(&pinned) } }; ``` Or: track unsafeness by the span? ### lifetime in match too long ```rust! match vec.lock().is_empty() { true => foo.lock() /* deadlock */, false => ... } ``` ### lifetime must be extended ```rust! match vec.lock().first_mut() { Some(thing) => { *thing += 1; } false => ... } match let l = vec.lock() for l.first_mut() {} let l = vec.lock(); match l.first_mut() { } ``` ```rust match vec.lock().let.first_mut() { Some(thing) => { *thing += 1; } false => ... } match expr.let { ref pinned => unsafe { Pin::new(pinned)}} // ?? ``` ### Rustc 1. `macro_rules! assert_eq` ```rust= ($left:expr, $right:expr $(,)?) => { match (&$left, &$right) { (left_val, right_val) => { if !(*left_val == *right_val) { let kind = $crate::panicking::AssertKind::Eq; // The reborrows below are intentional. Without them, the stack slot for the // borrow is initialized even before the values are compared, leading to a // noticeable slow down. $crate::panicking::assert_failed(kind, &*left_val, &*right_val, $crate::option::Option::None); } } } }; ``` ```rust= match (a, b, c, ..) { (Some(..), false, ..) => } ``` 2. proc-macro-hack ```rust= match &tokens.next() { } ``` ## move closure capture ```rust= let a = &AtomicUsize::new(0); for i in 0..4 { thread_scope.spawn(move || { dbg!(i); dbg!(a.load(Relaxed)); }); } ``` https://marabos.nl/atomics/atomics.html#example-progress-reporting-from-multiple-threads ## two cases ```rust { let tmp = ...; expr } // tmp is dropped once expr is evaluated, but before it is consumed let tmp = ... for expr // tmp is dropped once expr is consumed { let super tmp = ...; expr } expr(tmp) with tmp = ... ``` ## &dyn if pattern? what people try first ```rust let w: &dyn Write = if use_stdout { let stdout = std::io::stdout(); &stdout } else { let file: File = ...; &file }; ``` what you can do but people never think of it until they are told ```rust= let stdout; let file; let w: &dyn Write = if use_stdout { stdout = std::io::stdout(); &stdout } else { file = ...; &file }; ``` ^ This trick is something many people don't realize is possible. ```rust= let w: &dyn Write = if use_stdout { super let stdout = std::io::stdout(); &stdout } else { super let file: File = ...; &file }; ``` ^ Nice for diagnostics. just "add super". ## RFC 66 ```rust! fn compute_data() -> Vec<u32> { } fn main() { let x = compute_data().last(); // Keep the entire vec alive? or copy the last item and drop the rest? println!("{}", x); } ``` ## surprising error today ```rust! // rust 2021 fn foo() { let x = Lock::new(); x.lock().foo // temporary gets dropped after x is dropped, today } ``` In Rust 2021, it is equivalent to this today ```rust! fn foo() { let x = Lock::new(); super let l = x.lock(); // error l.foo } ``` and hence you get an error, but in Rust 2024 is equivalent to ```rust! fn foo() { let x = Lock::new(); let l = x.lock(); // OK l.foo } ``` automatic edition upgrade possible by using `super let` for temporaries ```rust! fn foo() { let x = Lock::new(); { super let l = x.lock(); l.foo } } ``` ## if let with lock ```rust= if let Some(item) = list.lock().unwrap().pop() { process_item(item); // list should've been unlocked, but is still locked } ``` (https://marabos.nl/atomics/basics.html#lifetime-of-mutexguard) This example should still compile, but now drop the lock right away. (Since pop doesn't return something that borrows the vec, it still compiles. If it was .first(), it'd start to give an error, unlike today.) ## Proposal of sorts * Syntactic rules like today but * match (and if let) drop temporaries before arms are evaluated (like `if`) * tail expression `{ expr }` drops temporaries once block exits * something like `super let` to introduce a value that lives until block is consumed ## What about... ```rust= let _ = (foo(), bar()); // vs let _guard = (foo(), bar()); // vs let (a, _) = (foo(), bar()); ``` what people "intuitively expect"? * line 1 and line 3 are equiv because `let _ = ` is same as `let i = ` * iow, `_` on its own is not a "pattern" but an anonymous identifier * `let $ident = ..`, not `let $pattern =`, because that's how it gets taught. * but line 5 drops `bar()` immediately * now it's clear that it's `let $pattern` (which seems a separate case from `let $ident`) ```rust! let x = { super let (a, _) = (foo(), bar()); Some(&a) }; // niko's semantics is that this would be equivalent to... let tuple = (foo(), bar()); let a = tuple.0; let x = Some(&a); // ...and hence `bar` is dropped at the very end // but if it goes on the binding...? let x = { let (super a, _) = (foo(), bar()); Some(&a) }; ``` ```rust= super let a = &vec![1, 2, 3]; // vec gets lifetime extended? think so ``` ```rust= let f = format_args!("{:?}", vec![1, 2, 3]); // expansion: // let f = { super let args = (vec![1, 2, 3], ); // super let args = &[Argument::new(&args.0)]; // unsafe { fmt::Arguments::new(.., &args) } }; dbg!(f); ``` ## Mutex<()> https://marabos.nl/atomics/basics.html#mutexes-in-other-languages Maybe use `Mutex<Zst>` instead, put the relevant operations on the Zst. ## Mutex::unlock ```rust= Mutex::unlock(x: MutexGuard); // just drop() // opt-in lint to deny/warn about implicitly dropping MutexGuard. ``` (don't like it, but some people talked about this) z

    Import from clipboard

    Paste your markdown or webpage here...

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lose their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template has been removed or transferred.
    Upgrade
    All
    • All
    • Team
    No template.

    Create a template

    Upgrade

    Delete template

    Do you really want to delete this template?
    Turn this template into a regular note and keep its content, versions, and comments.

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password

    or

    By clicking below, you agree to our terms of service.

    Sign in via Facebook Sign in via Twitter Sign in via GitHub Sign in via Dropbox Sign in with Wallet
    Wallet ( )
    Connect another wallet

    New to HackMD? Sign up

    Help

    • English
    • 中文
    • Français
    • Deutsch
    • 日本語
    • Español
    • Català
    • Ελληνικά
    • Português
    • italiano
    • Türkçe
    • Русский
    • Nederlands
    • hrvatski jezik
    • język polski
    • Українська
    • हिन्दी
    • svenska
    • Esperanto
    • dansk

    Documents

    Help & Tutorial

    How to use Book mode

    Slide Example

    API Docs

    Edit in VSCode

    Install browser extension

    Contacts

    Feedback

    Discord

    Send us email

    Resources

    Releases

    Pricing

    Blog

    Policy

    Terms

    Privacy

    Cheatsheet

    Syntax Example Reference
    # Header Header 基本排版
    - Unordered List
    • Unordered List
    1. Ordered List
    1. Ordered List
    - [ ] Todo List
    • Todo List
    > Blockquote
    Blockquote
    **Bold font** Bold font
    *Italics font* Italics font
    ~~Strikethrough~~ Strikethrough
    19^th^ 19th
    H~2~O H2O
    ++Inserted text++ Inserted text
    ==Marked text== Marked text
    [link text](https:// "title") Link
    ![image alt](https:// "title") Image
    `Code` Code 在筆記中貼入程式碼
    ```javascript
    var i = 0;
    ```
    var i = 0;
    :smile: :smile: Emoji list
    {%youtube youtube_id %} Externals
    $L^aT_eX$ LaTeX
    :::info
    This is a alert area.
    :::

    This is a alert area.

    Versions and GitHub Sync
    Get Full History Access

    • Edit version name
    • Delete

    revision author avatar     named on  

    More Less

    Note content is identical to the latest version.
    Compare
      Choose a version
      No search result
      Version not found
    Sign in to link this note to GitHub
    Learn more
    This note is not linked with GitHub
     

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub
        • Please sign in to GitHub and install the HackMD app on your GitHub repo.
        • HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.
        Learn more  Sign in to GitHub

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions
        Include title and tags
        Available push count

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch
        Available push count

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully