Libs-API Meeting 2024-05-14

--- date: 2024-05-14 url: https://hackmd.io/WMhFUuu9QEOD2TLBur_Tug --- # Libs-API Meeting 2024-05-14 ###### tags: `Libs Meetings` `Minutes` **Meeting Link**: https://meet.jit.si/rust-libs-meeting-crxoz2at8hiccp7b3ixf89qgxfymlbwr **Attendees**: Amanieu, David Tolnay, Josh Triplett, Mara, Chris Denton, The 8472, David Barsky, Eric Holk, TC ## Agenda - Triage - Anything else? ## Triage ### FCPs 18 rust-lang/rust T-libs-api FCPs - merge rust.tf/80437 *Tracking Issue for \`box\_into\_inner\`* - (1 checkboxes left) - merge rust.tf/82901 *Tracking Issue for \`Option::get\_or\_insert\_default\`* - (2 checkboxes left) - merge rust.tf/83871 *Tracking Issue for CharIndices::offset function* - (3 checkboxes left) - merge rust.tf/102012 *Tracking Issue for \`const\_waker\`* - (3 checkboxes left) - merge rust.tf/106943 *Implement DoubleEnded and ExactSize for Take\<Repeat\> and Take\<RepeatWith\>* - (3 checkboxes left) - merge rust.tf/62726 *Tracking issue for io\_slice\_advance* - (3 checkboxes left) - merge rust.tf/109402 *Implement owned ops for \`HashSet\` and \`BTreeSet\`* - (3 checkboxes left) - merge rust.tf/116113 * Generalize \`{Rc,Arc}::make\_mut()\` to unsized types.* - (3 checkboxes left) - merge rust.tf/115974 *Split core's PanicInfo and std's PanicInfo* - (2 checkboxes left) - merge rust.tf/117468 *Stabilize Wasm relaxed SIMD* - (4 checkboxes left) - merge rust.tf/101196 *Tracking Issue for \`Ready::into\_inner()\`* - (1 checkboxes left) - merge rust.tf/119131 *Tracking Issue for \`hint::assert\_unchecked\`* - (2 checkboxes left) - merge rust.tf/106418 *Implement \`PartialOrd\` and \`Ord\` for \`Discriminant\`* - (2 checkboxes left) - close rust.tf/56167 *Tracking issue for HashMap::raw\_entry* - (3 checkboxes left) - merge rust.tf/113219 *Tracking Issue for \`const\_cstr\_from\_ptr\`* - (5 checkboxes left) - merge rust.tf/55132 *Tracking issue for thread::Builder::spawn\_unchecked* - (3 checkboxes left) - merge rust.tf/123723 *Make \`std::os::tvos\`, \`std::os::visionos\` and \`std::os::watchos\` public* - (4 checkboxes left) - merge rust.tf/117618 *Tracking Issue for \`duration\_abs\_diff\`* - (3 checkboxes left) [BurntSushi (8)](https://rfcbot.rs/fcp/BurntSushi), [dtolnay (1)](https://rfcbot.rs/fcp/dtolnay), [Amanieu (6)](https://rfcbot.rs/fcp/Amanieu), [m-ou-se (10)](https://rfcbot.rs/fcp/m-ou-se), [joshtriplett (15)](https://rfcbot.rs/fcp/joshtriplett) ### (nominated) rust.tf/79024 *Tracking Issue for #!\[feature(async\_iterator)\]* ### (waiting on team) rust.tf/119550 *Rename \`AsyncIterator\` back to \`Stream\`, introduce an AFIT\-based \`AsyncIterator\` trait* David Barsky: Here's a summary I propose (this hasn't been reviewed by wg-async): Stabilizing `AsyncIterator` as `Stream` (with `poll_next()`) would be really helpful for few reasons: - Allows the ecosystem to converge on a single trait (`std::stream::Stream`) and build abstractions atop of it (e.g., for handling concurrent streams) - Allows people to use `for await` instead of combinators, which is a non-trivial UX improvement. - Improves discovery in libraries (e.g., Tokio) that can provide deeper runtime integrations with streams than those offered by `buffered()`/`.buffered_unordered()` in libraries and offers an ecosystem-wide reset point that delegates concurrent streams to a different trait. - A `merge!` macro could be provided in libraries as an alternative to the footgun-y `loop { select! {} }`  Stabilizing Stream as-is does not close the door on concurrent streams, such as [ConcurrentStream](https://docs.rs/futures-concurrency/latest/futures_concurrency/concurrent_stream/index.html) in libraries. This seems to align with the current thinking in wg-async: https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async/topic/Observations.20on.20starvation. Some details: - I’d call it Stream, not AsyncIterator, since it provides parity with the larger ecosystem and minimizes disruption. - `Stream` should use `poll_next`. - `async fn next()` is not object/dyn-safe and won’t be for several years.' - This also risks suboptimal codegen. - While pinning is terrible, but its downsides can be mitigated through: - `for await` on the consumer side - `gen async` on the producer side of things. - Niko wants an exploration as to how this interacts with push-based streams (“flows”—an async Rayon), but personally, I don’t think this will have an impact on Streams. - Push-based iterators (Niko calls them “Flows”, Yosh calls them “ConcurrentStream”) can be converted to with combinators. - TC has written about starvation. - Full resolution of people’s issues with Streams today requires something like “structured concurrency”, which involves something like `!Drop`/`Leak`, but the opportunity to move away from `futures::stream::StreamExt::buffered/buffered_unordered()` is going to be really valuable. TC: As a procedural matter, this is mostly lang, not libs-api. While there are of course additions to the standard library here, the trait or traits that will be added are going to be lang items that need to strongly align semantically and conceptually with the syntax that we'll be stabilizing. Amanieu: What is lang's current position? JoshT: Waiting on more input from wg-async. Not a strong lang position. Individual members have positions. TC: E.g., tmandry has of course been integral to these discussions. And Niko has been taking a strong interest in this recently. Eric Holk: We are planning to make the async iteration a top-of-mind goal for the upcoming project goals initiative. David Barsky: Is the summary pasted above directionally accurate? Eric: You are right that push-based iteration does play a role. async fn next is the big question keeping us from making forward progress, but we are I think close to resolving it. Josh: The libs team and lang team definitely need to arrive at consensus in order to approve a stabilization, but it's not necessarily the case that we need wg-async to also have consensus. But of course if there is no consensus from wg-async, it makes it less likely that libs or lang will be confident about stabilizing something and override wg-async's lack of consensus. TC: This issue requires a lot of context. We had been hoping to pull together all the work that has been done into, e.g., an RFC that would make a proposal and outline all of the alternatives with elaboration and analysis, and that this document would represent the consensus of WG-async. That document will happen. Eric has been working on a draft that we've been discussing extensively. Whether it will represent the consensus of WG-async or the position of N-1 of the members is yet to be worked out. The 8472: Does the `Leak` trait feed into this at all; are we sure this would be future-compatible with that? TC: Part of the position of the -1 member is that we might want to settle questions such as that before stabilizing async iteration. There are other questions in this category also, e.g. maybe async. The position of the N-1 members is probably that async iteration does not introduce new problems for stabilizing `Leak` that don't already exist in the language. Amanieu: libs-api is being asked to choose between 2 designs (poll_next vs async fn next). If we had received just one of these designs, it would likely have been stabilized already. My preference would be to defer this decision back to lang. We are not in a position to decide on something where the experts disagree. Josh: I'm not sure that's the case, that we would have stabilized something by now if only one design had been presented. Josh: What's the interaction with lending iterators? TC: That probably also falls in the bucket of things that we could choose to work out (or not) before stabilizing. Probaby the position of the N-1 members is again that async iteration doesn't introduce any new problems, and that any solution for lending iterators would apply straightforwardly to async also. Josh: You expect that an AsyncIterator that doesn't handle lending won't close any doors that Iterator hasn't already closed? And that if we can solve compatibility for one we can do it for the other as well? TC: Yes. 8472: Does `for await` already exist as an unstable feature? TC: Yes. For a few months. 8472: Has there been experimentation with that, and is there experimentation yet to do? TC: Within WG-async, we have used it. For example Michael Goulet converted the common future combinators to use `for await` and `async gen`. The interesting part of this exercise are the combinators he wasn't able to write this way. I'm unaware of any experimentation beyond that. 8472: A report on the results from that may be helpful. Josh: Does wg-async have consensus on feature roadmap and timelines for each option, even if they don't have consensus on which option to recommend? Eric: I don't think there is disagreement whether any particular design is months vs years of works, it's more a disagreement about the design itself, desugaring, the algorithm itself. And there is not a lot of appetite from wg-async to do the implementation work that would be needed for the `async fn next` approach. TC: The people who are most capable of writing the implementation are not interested in writing it that way. (Discussion about putting together a statement of the situation that has the consensus of all members.) Josh: The most likely way for libs or lang to arrive at consensus (when wg-async hasn't) is if everyone at least agrees on the implications/roadmaps/timelines of the different options, even if the chosen option is not supported by every member of wg-async. TC: Part of the metaposition of the -1 member is that our time would be better spent right now on other questions that are not this, e.g. that we shouldn't be spending time on this until we settle a number of other questions, so there's some tension between this and putting time into the question to put together materials that represent all the positions here. (Discussion on the timeline.) TC: Even on the `poll_next` version, and among the N-1 members, there are questions that still need to be settled. E.g., tmandry would currently block this on first deciding whether we want to do `poll_progress` (which is probably "no"). It's not that he wants `poll_progress`, necessarily, it's that there are drawbacks to doing it separately later, so we may want to be sure whether or not we need it. (Further discussion of the timeline.) TC: Taking the fullest possible view of the scope of the questions we might conceivably want to settle before approaching this problem, e.g. linear types, effect generics, etc., that could take many years. On the other side, to stabilize what is currently implemented could conceivably happen in months. 8472: It would be on that person to propose a roadmap and carve out any milestones that could be stabilized sooner. TC: The -1 member has written extensively about this, e.g. in blog posts and other work, and so could reasonably feel that this has been done. There are many tens of thousands of words cumulatively. 8472: That ain't it. David Barsky: I am motivated to drive this and coordinate a proposed timeline/milestones for stabilization. Do we agree the summary above is correct? TC: I haven't reviewed it carefully and it should be reviewed. How do you plan to use this? David B: libs and lang need a document that is generally accurate about options and people's perspectives. Can the people here commit to reviewing the above in the next week or two so we can use this to propose the options e.g. to lang and libs-api? TC: That is not going to happen in the next two weeks; a document that would correctly represent the positions of the members to be used in that way would need to be much more extensive. A much more likely next step is to base this on the upcoming RFC from Eric. Eric: David B, how about you collaborate with me on this RFC? I don't want to feel like I am taking a lock on the whole process. Eric: if we were to go with async fn next for the Stream trait, async gen blocks would come later. Need to figure out the new implementation (we believe it's possible, we just haven't done it) Otherwise, it may be possible to stabilize in the shorter term. Action item: Eric Holk and David Barsky to draft a cohesive lang-libs RFC Eric: my personal goal (in line with the project goals initiative) is to work toward some kind of async iteration we can stabilize in a matter of a few months Josh: I'm happy to volunteer to reach out to Yosh and try to figure out if there's any material he could provide to attempt to flesh out the MVP of his path and what that would take, with the idea in mind that if we *don't* have that we're unlikely to be able to consider that path. ### (nominated) rust.tf/93346 *Tracking Issue for panic\_backtrace\_config* 8472: the point of the capture API is to provide behavior that is consistent with panic backtraces (which are controlled by the environment). Mara: If the point is to replace RUST_BACKTRACE, then it should do the same thing.. 8472: what's the benefit of having 2 different knobs? Discussion about counterproposal -- 4 functions, one replacing RUST_BACKTRACE, one replacing RUST_LIB_BACKTRACE, and the 2 getters Amanieu to write up the proposed design ### (nominated) rust.tf/124108 *Add \`Box\<\[T; N\]\>: IntoIterator\` without any method dispatch hacks* FCP finished. Should be merged? scottmcm: Would like to hear your libs-api thoughts on the conversation about whether it's worth trying to have a smaller iterator for the static-capacity box-of-array vs the box-length. Consensus -- separate type for `Box<[T; N]>::IntoIter` vs `Vec<T>::IntoIter` Amanieu to reply ### (nominated) rust.tf/124636 *Make \`std::env::{set\_var, remove\_var}\` unsafe in edition 2024* Josh: do we have a mechanism for "deprecated-safe"? Amanieu: yes, the PR uses `#[rustc_deprecated_safe_2024]` Ready for someone to start FCP -- Amanieu ### (nominated) rust.tf/95629 *Tracking Issue for \`slice\_flatten\`* Mara: I would expect an `as_` prefix, as this is a conversion from reference to reference Amanieu: do we want into_flattened for arrays? Scott: Yes, but the signature (`-> [T; N*M]`) cannot be expressed currently Scott: edge case: `(&[[ZST; usize::MAX]; usize::MAX] as &[_]).flatten()`: the total length will overflow. This is a guaranteed (i.e. not debug-only) panic. Amanieu to propose FCP ### (nominated) rust.tf/124667 *Stabilize \`div\_duration\`* Josh: why is this associated functions instead of a trait impl? Amanieu: because there are 2 possible output types for `impl Div<Duration> for Duration`, f32 or f64. Possible alternative: we could provide just the f64, and expect people to cast to f32 if they want Scott: the implementation is just `self.as_secs_f32() / rhs.as_secs_f32()`, if we only provided f64 anyone who wanted f32 could write that themself Amanieu: this API was excluded from a previous round of stabilizations because it's not clear that this provides sufficient value over just the user writing the as_secs_f** and float division directly 8742: possibly in the future this could be implemented in a more accurate way Mara: if we don't provide an API for duration division, it's tricky because there is another possible implementation as `a.as_nanos() as f64 / b.as_nanos() as f64` which has different accuracy Mara/Josh: if we do add these, we'd want the implementation to be more accurate than the current one Amanieu: at least 2 people asked for this API, so I am happy to start FCP. I don't see a good reason to reject this considering the rest of the float APIs we already have on Duration Josh to propose FCP, Mara to request a fix for the accuracy issue (nonblocking) ### (nominated) rust.tf/124881 *Use ThreadId instead of TLS\-address in \`ReentrantLock\`* This is a correctness fix, although not a soundness fix. The problem occurs if you hold a ReentrantLock, leak the guard, exit that thread, start a new thread which happens to put the TLS at the same address as the previous one -- then you can re-acquire the same ReentrantLock that was previously leaked. There can't be a data race because the previous thread is over. David: We could document that a reentrant lock might or might not be unlocked when a thread exits when it was still locked. Mara: It's not equivalent though.. other threads might not agree that it is unlocked. David: Let's just get it right then? Accept this PR? Mara: Yeah, leaning towards that. (With the seqlock impl fixed/simplified.) Amanieu: that's fine. Platforms people care about are just going to use the 64 bits version. Mara to reply. ### (nominated) rust.tf/124987 *Rename \`\${length()}\` to \`\${len()}\`* Agreed with `len` ### Edition question: Range TC (putting on the edition hat): The RFC for the new range types was accepted. One of the questions raised on doing this for the edition is ensuring that libs-api has enough time to design and review the exact types that would be added to the standard library. How is this to be done, and is libs-api waiting on further work from others before starting on this? Mara/Amanieu: A good first step would be a PR to the library adding these types. Please nominate this PR. TC: Great, thanks. I'll pass that along. ### (new change proposal) rust.tf/libs373 *Expose LLVM's \`or disjoint\` instruction on ints* This was approved in <https://github.com/rust-lang/libs-team/issues/373#issuecomment-2085686057> ### (new change proposal) rust.tf/libs376 *Methods for extracting 1 element out of an owned slice and dropping the rest* ??: what about `.into_iter().nth(i)` or `.drain(..).nth(i)`? Missing motivation David replied ### (new change proposal) rust.tf/libs377 *ACP: \`Sign\` enum for indicating sign of a number* We have <https://doc.rust-lang.org/std/primitive.i32.html#method.signum>, which treats `0` as `0`, neither `-1` nor `+1` ### (new change proposal) rust.tf/libs378 *Add \`get\_unchecked\` to \`Option\` and \`Result\`* scottmcm: see my note in <https://github.com/rust-lang/libs-team/issues/378#issuecomment-2098921092>. `get_unchecked` itself is not useful, but there may be something useful in providing `Option<NonZero<T>>` → `T`. Amanieu: `Option<NonNull<T>>` -> `*const/mut T` as well? Consensus: reject. improve unwrap_unchecked instead (in compiler and/or library) David to reply. ### (new change proposal) rust.tf/libs379 *Combine, an iterator adapter which statefully maps multiple input iterations to a single output iteration* ### (stalled change proposal) rust.tf/libs114 *ACP: Method to split OsStr into \`(str, OsStr)\`* ### (stalled change proposal) rust.tf/libs210 *Add associated consts to f32, f64 for mathematical constants* ### (stalled change proposal) rust.tf/libs255 *Adding \`set\_route\` to \`sys::unix::net\`* ### (stalled change proposal) rust.tf/libs176 *\`Peek\` trait for peekable iterators* ### (stalled change proposal) rust.tf/libs262 *Add infallible variant of RangeFrom::next()* _Generated by [fully-automatic-rust-libs-team-triage-meeting-agenda-generator](https://github.com/rust-lang/libs-team/tree/main/tools/agenda-generator)_