# Reading notes: Iterator, Generator Link to document: https://without.boats/blog/iterator-generator/ ###### tags: `reading-club` Leave questions, observations, discussion topics below. --- ## "pinning" is the "must-not-move" effect yosh: Boats notes that "drop" really ought to have taken `Pin<&mut Self>` as well. But that is only true in the context of pinned references. I don't entirely disagree with that in the sense that yeah you really shouldn't `mem::replace` the value and drop something else in the destructor. But from a soundness perspective it is, in fact, fine to not pin self in destructors - unless you're dealing with pinned references. tmandry: Definitely unintuitive, but it worked backwards-compatibly. Maybe expressing as an effect would be more intuitive, not sure what that looks like. yosh: Thinking of pin as an effect on `Iterator` (tmandry distracted) yosh: MayMove/MustNotMove/MustMove eholk: Could you factor into two effects instead of different degrees yosh: Maybe eholk: Default is MustNotMove, effect is MayMove. Unpin today is basically MayMove eholk: If we can have additive effects it's easier. Negative reasoning is much more complicated. --- ## Does the Update invalidate their argument? eholk: I'd like to see this post rewritten with the idea that there'd be a blanket impl of `Iterator` for `Pin<&mut impl Generator>` (maybe I should write that post...). Could we then change the desugaring of for loops to pin the iterator? Most iterators are probably `Unpin` today, so that wouldn't be super disruptive. It seems like this solves the problem with bridge impls and coherence, and possibly the social costs. Is this approach workable? Are there other problems it introduces? It's too bad we can't have `IntoIterator for impl Generator` use a pinned type as its output iterator, but I don't think we have a way to internally pin like that. tmandry: Someone might have an iterator that is `!Unpin`, but it would be weird eholk: Seems like you'd be already writing unsafe code, or otherwise extremely rare. yosh: Intrusive collections? tmandry: I think Pin is used for the elements, not the Iterator eholk: If you're doing something that would make your iterator !Unpin, you would need to already have it pinned to do anything with it. --- ## The cognitive burden of pin yosh: The post makes a note of the "social cost" of change in the last paragraph. I don't particularly agree with its framing, but that's neither here nor there. Something more interesting to me is the cost that pinning itself has. From talking to users we've repeatedly found that pinning is one of the most complicated aspects of Rust's async system - and even seasoned Rust experts often don't understand it, or get things wrong. I feel like any conversation about pinning should start by acknowledging this. And it should probably highlight that we've made mistakes we need to fix (such as `Unpin` being safe to implement) before we can move to make pinning easier to use (e.g. safe pin projections). yosh: idk, this is missing from the post under discussion - and I think the actual cognitive burden of our current pinning system is something we should keep front of mind when talking about introducing it in more places in the language and libraries. tmandry: It does feel unintuitive. I would be open to a future where we make Pin more builtin to the language in a way that makes it easier to use, or replace it with another abstraction that's shown to be sound (haven't seen one yet). eholk: If we were going to replace Pin we'd need some way of deprecating library APIs across edition boundaries. yosh: Issue is mainly ergonomics. Semantics of Pin are sound, so can we design a nicer to use feature that maps onto the existing Pin trait? tmandry: most things implement unpin --- ## How can we measure the need for self-referential generators? tmandry: How can we decide if the feature is worth the disruption? yosh: Can we do it purely with signals like blog posts, within companies, etc.? eholk: Right now it's really hard to write iterators because you have to manage the state yourself. We don't have self-referential iterators so there's not demand for it. As soon as you can write `yield` in a loop, I have a feeling there will be a demand for self-referential generators pretty soon. But we don't have a way of knowing that until there's a version of generators you can play with. yosh: How strong is the signal for generators? Do we need something like async Drop first? You can write your own Iterator trait that takes `Pin<&mut Self>`, but no one does that. tmandry: My intuition is we'll need self-referential generators once we have generators. eholk: Generators are a language-wide feature, might have broader appeal than async specific features. I personally miss them pretty much every time I write Rust, but I might be the only one. yosh: There are workarounds for lacking generators, there are no workarounds for async Drop and async closures. tmandry: it seems we have general consensus about the prioritization - but the question is that once we have generators whether we want them to be self-referential tmandry: currently leaning towards: "yes"