eh2406

@eh2406

Joined on Jul 27, 2022

  • Cargo is the bundled package manager and build tool for Rust. In practice it is the user interface of Rust. Its stellar ergonomics have made it one of the critical selling points of Rust overall. Most "why I fell in love with Rust" testimonials include praise for Cargo. Cargo is a one-stop shop providing a unified experience from adding a dependency, through identifying available versions, resolving a set of transitive dependencies, to building and testing the actual artifacts. Following Conway's law this unified experience is backed by a monolithic code base. User facing improvements often involve changes across many different modules. Furthermore code often accidentally relies on details of how other parts the code work. Attempts to break up the monolith often get stymied by the number of implicit details that need to be public for the existing code to work correctly. Eventually cargo should be a thin wrapper wiring together independently tested and maintained crates for doing the internal logic. This goal is most helpful because the ever-growing cargo monolith continuously gets harder to maintain and develop. But also, many cargo plug-ins or wrappers are fundamentally "work as if cargo had slightly different functionality" which would be much easier to develop and maintain if the portions of cargo's functionality they want to preserve were available as independent libraries. At this time this goal appears impossibly far off. But this is the kind of work where the goal will seem impossible until suddenly it's almost done. The only thing we can do is find components that can be split out and do so. I am working on the resolver. The dependency resolver solves the NP-Hard problem of taking a list of direct dependencies and index of all available crates and returning an exact list of versions that should be built. This functionality is exposed in cargo's interface as generating/updating a lock file. There are many ways two crate versions can be incompatible. The current resolver relies on concrete datatypes from other modules in cargo to determine if a set of versions have any of these problems. It is therefore difficult to abstract and separate the existing resolver from the code base. On the other hand, because the problem is NP-Hard it is not easy to tell what code changes break loadbearing performance or correctness guarantees. Which makes any change to the current resolver in situ extremely treacherous. Nonetheless, big changes are required: there is lots of new functionality that will affect the resolver and the existing resolver's error messages are terrible. Who could be affected. The previous section mixed together the interests of various different customers that I'd like to try and tease apart here. I'm only gonna focus on those affected by the resolver changes, other work on library-ification may benefit other users. The Cargo Team
     Like 1 Bookmark
  • If code does the wrong thing and nobody uses it, is it really a bug? A software bug is surprisingly hard to define. But it is something like "software that doesn't do what it's supposed to when used. Where 'supposed to' is some kind of documentation between a maintainer and a user." So the obvious case is when the code is documented to "produce X when given L", but when the user gives L they get Z, that is pretty clearly a bug. But that simple example is less helpful than we would like because the situation is normally more like a user gave it I (which they thought should work because it looks like a L under the right circumstances) and got Z. At least three situations can occur in the resulting bug report: The user can be convinced that L and I are fundamentally different. So no bug exists. The report should be closed with "won't fix". The maintainer can be convinced that due to the similarity of L and I the software should be changed to handle them consistently. So there is a bug. The report should result in a PR that fixes it with a test case covered by CI. The maintainer can be convinced that given the potential confusion between L and I the documentation needs to be fixed. There are many kinds of things that could fix this, perhaps the function in questions documentation should be updated, or maybe the entire libraries documentation (the readme should clarify that it works on L = Lennix but not I = Mac), or perhaps in error message should be added that triggers when used with I. As if that wasn't complicated enough, now let's imagine we don't have the user do we still have a bug? One could object, that you can't have software with no users, because the maintainer also acts as a user. This objection feels reasonable. When I say, as I have in the past, "I found a bug in my unreleased code" I might mean that the code I wrote does not behave the way I expect when used the ways I have used it. In that case I am both the maintainer and the user of the code and it's clearly a bug. On the other hand I also find myself saying that when I have not yet had a chance to write any code using the behavior, or where the inputs on which the code would act unexpectedly are not ones I use. So I must mean something more like someone in the future who may or may not be me might use this code with these inputs and then would be surprised by the output. So it seems like there can be bugs that nobody has hit yet, as long as they are reasonably likely to do so in the future and would be very surprised by the outcome. This suggests that if integral(probability_of_use(J) * surprise_of_outcome(J) * dJ, for all J) is high enough then we have a bug even if there are no users.
     Like  Bookmark
  • Some assumptions I have about the RFC 3537 problem space: Some people are stuck on old versions of rustc. Unfortunately, telling them that "they're doing it wrong" does not unblock them to update. No matter how much we would like them to. People are stuck for different reasons and therefore on different versions. There is no "one version we should support" that everyone agrees on. Unfortunately, telling them that "the community has decided on only supporting back to 1.x" does not unblock them if they're stuck on 1.(x-n). No matter how much we would like them to. Just getting everyone to agree on what version to support has not happened despite many years of people trying. Even if someone was capable of "telling the community the right answer", it certainly isn't the cargo team's place to leave that discussion. Almost the entire problem would go away if the community agreed on how quickly to adopt new functionality. Unfortunately, that is unhelpful/irrelevant to this discussion. For people stuck on old versions the infrastructure they are required to maintain currently is untenable, and the tooling (mostly cargo) makes it almost impossible to do without dedicated infrastructure. Specifically the infrastructure we need to set up for ______, and the iterative cargo update -p --precise required to get a set of versions that actually work. The only easy situation for someone stuck on an old version is to have all of your transitive dependencies consciously supporting versions at least as old as yours. Thus if you maintain an intermediate dependency that is intended to be used by people stuck on an old version (given the sorry state of the tooling) you must only depend on libraries that have an even stricter MSRV. Thus if you maintain a foundational crate you should go far beyond the "community norm" for supporting older Rust, just like all the other ways foundational crates are held to a higher standard. Almost everyone stuck on an old version of Rust ends up needing some transitive dependency that does not care about its MSRV (or cares about a newer MSRV then you). The current tooling (in cargo) is actively hostile to dealing with the situation. Most library maintainers (do not maintain foundational crates and) do not care about their MSRV. They do not currently put any thought into supporting older rust versions. Some, but it's not clear how many, actively do not want to think about outdated tooling. If we want to make lives easier for users who are stuck, we need to lower the barriers for maintainers not make more demands on them. We cannot build better tooling without access to accurate metadata. This means that we need to encourage metadata to be published and encourage anything that is published to be correct.
     Like  Bookmark