# Cargo Sustainability Efforts for 2024

For some time, the cargo team has not only been limited in capacity to implement new features but also to mentor and review other people doing so. This has gotten to the point that we've [declared a near-total feature freeze](https://blog.rust-lang.org/inside-rust/2022/03/31/cargo-team-changes.html). Having limited mentorship and review capacity causes long delays between implementing a PR and each round of review feedback which requires a lot more work for everyone to wrap our heads around the problem and the implementation on each iteration as well as is discouraging for all involved. A feature freeze causes stagnation where we aren't progressing in the usability of cargo and aren't adapting to user needs as Rust is adopted into more environments. A feature freeze puts a limit on what is able to be accomplished among other Rust teams when their work intersects with cargo. Most importantly of all, the limited mentorship capacity creates a vicious cycle where we can't as easily grow contributors into being team members that can help share the work.

The cargo team has been brainstorming ideas that we hope will get cargo into a more maintainble state to remove the feature freeze. These are all theoretical and some may not pay off as much as we thought but we'll learn and iterate from there.

## Challenges in Mainaining Cargo

- Lack of people stepping up
- Difficulty in being confident in becoming a maintainer
- As a maintainer, difficult to feel confident to say "yes" or "no" to issues, PRs, RFCs, etc
- Lack of time from existing maintainers to mentor

- A lot of use cases that cross several domains and historical decisions to come up to speed on
- A lot of changes require "global" knowledge, limiting the bar for maintainers to participate
- Big initiatives don't have clear starting points to catch up (build-std, etc)
- Hard to say "no". Like with board game reviewers, its easy to not write a review, its hard to write a positive review, its nearly impossible to write a negative review
- Lack of unified efforts within team or with external contributors
- Large change proposals feel designed in isolation rather than authors working across interested parties to find shared solutions
- PRs linger until people can't commit anymore
- Maintainers are detached from the "customer" for advanced use cases
- Reduce development overhead

complexity
- stability compounds complexity
- intimidate people from doing as much as they could
- Requires more time to get into head space
- lack of comments for non-obvious practices

Developer velocity
- Test suite takes 3 minutes
- docs.rs for cargo-test-support
- windows pathing tripping people up in CI and then being hard to fix without access for iteration

Do our contributing notes recommend incrementally implementing features

## Componentization

**Today**, `cargo` is a monolithic crate with nearly all logic and commands embedded in it. Though it exposes a `[lib]`, the API is not designed and maintained for general use and people are expected to either programmatically use cargo commands or to re-implement it themselves. The cargo team has implemented a lot of logic for itself, including a custom test framework built on top of `libtest`, optimized for cargo's needs.

Being a monolithic binary crate, not all of the logic is exposed in an easily testable and maintainable way. Without clear boundaries, maintainers need to comprehend more of the project, the constituent domain expertise, and the use cases to minimize unintended side effects and align with future plans. Developers can't easily leverage investments that the cargo team has made which then limits contributing and maintenance help with the code base as developers pave their own, redundant path instead.

**Ideally**, cargo would be componentized with clear boundaries, making it easy to test and review each section in isolation and allowing third-party crates to build on top of these. Cargo would build on the work of others in the community to reduce the amount of code that the cargo team is responsible for, allowing all to share in the benefits. Non-core parts of cargo, like testing libraries, would use community-supported libraries to reduce maintenance burden on the cargo maintainers and so everyone benefits from improvements.

**Impeding** the ideal, the cargo team needs process overhead to remain relatively flat with the number of crates created. Automation around multiple crates normally leverages [workspaces](https://doc.rust-lang.org/cargo/reference/workspaces.html) but cargo can't be in one because it is being pulled into the [rust workspace](https://github.com/rust-lang/rust/blob/master/Cargo.toml), as a git submodule.

Even once we can more easily create crates, we might not be able to split out some of the highly desired crates, like for the manifest format. Parsing a manifest currently mixes mechanisms with application policy, like printing warnings on unrecognized fields. Another obvious place to share code is between the `cargo metadata` command generating the structured data and the `cargo_metadata` crate that parses it. However, `cargo metadata` is write-only and only needs to concern itself with the needs of the current version of cargo while `cargo_metadata` needs to deal with old versions of cargo that might generate less data.

**Moving forward**, we need a way for cargo to be in a workspace, whether through [nested workspaces](https://github.com/rust-lang/cargo/issues/5042) or [other ways of structuring rust](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Workspace.20for.20Cargo.20itself). At that point, it will be easier to work towards lower level pieces being broken out into their own crate. As we do so, we can iterate and experiment with ways to structure crates that currently would have mixed logic and presentation or have different concerns inside cargo as they would outside.

Through a combination of `[lib]` and `[[bin]]` APIs, we could experiment with moving built-in cargo commands into separate crates and maybe even separate repos. This narrows the focus of `cargo` and each `[[bin]]`s crate, making them easier to maintain and review, providing faster build and test times when working on a specific section of the project, allows more specialized reviewers, and dog foods the APIs to make sure they are sufficient.

Independent of splitting cargo into multiple crates, we can also work on developing or improving external crates for use within cargo, replacing home-grown code and leveraging efforts of existing maintainers, including:

- Cargo is working to adopt [`snapbox`](https://docs.rs/snapbox), a testing library modeled off of `cargo-test-support`, though [a lot of work remains](https://github.com/rust-lang/cargo/issues/10635) before we can remove the related functionality from `cargo-test-support`.
- More polish is needed on [clap's new completion engine](https://github.com/clap-rs/clap/issues/3166) but it has the potential to replace cargo's hand-written completions, reducing the toil when adding to cargo's CLI among [other benefits](https://github.com/rust-lang/cargo/issues/6645).
- Continuing research on switching the dependency resolver to [pubgrub](https://github.com/rust-lang/cargo/issues/5284), a generic version solver. Leaving the correctness of version solving to pubgrub and focusing our efforts on transforming our version solving needs to pubgrub's API reduces a significant source of complexity within cargo's code base and allows for more focused testing. For more information, see Eh2406's [presentation on this effort](https://www.youtube.com/watch?v=XyJpYFnl18U).
- Close [the gap](https://github.com/Byron/gitoxide/issues/106) between git and [gitoxide](https://github.com/Byron/gitoxide), a Rust implementation of git, so cargo can be ported to gitoxide. Currently, cargo interacts with repos primarily through [git2](https://github.com/rust-lang/git2-rs), a Rust wrapper around the C `libgit2` library. Being able to switch to gitoxide would reduce the load on the cargo team members who maintain git2, remove the complications associated with using a non-Rust library, and potentially improve performance.

## Documentation

**Today**, Cargo's documentation suffers from the curse of knowledge: those who are most likely to find areas to be improved are the least likely to be able to capture nuance that can be needed in the documentation. This puts the burden of documentation improvements on the maintainers. One particular area of concern is on [semver compatibility](https://github.com/rust-lang/cargo/issues/8736). [rust-semverver](https://github.com/rust-lang/rust-semverver) exists today but it focuses on the API and not the whole crate (including feature flags) and relies on nightly Rust to do its checks which makes it more difficult to develop and deploy.

**Ideally**, cargo would create tools that would help guide the user where possible, codifying deeply nuanced documentation.

**Moving forward**, we'll mentor efforts on [cargo crate-api](https://github.com/crate-ci/cargo-api) as a tool to report on the holistic semver compatibility of a crate based on the json output from rustdoc. Though this is a nightly feature for now, it has a lot more potential for being able to run on stable. See also [rust-lang/cargo#374](https://github.com/rust-lang/cargo/issues/374)

**TODO** Why cargo-crate-api over rust-semverver
- Focus instead on need (easy user setup, easy diffing, more than rust api) and give cargo-crate-api as

## Inter-Organization Collaboration

**Today**, there are different organizations adapting cargo to meet their needs. For example, as Rust adoption continues to grow, we'll need to deal with integrating into existing business's processes. They come to the cargo team with their specific solutions and the cargo team has to take on the work of identifying related needs and trying to find ways to unify them to keep cargo cohesive and light.

**Ideally**, we have a forum, like a special-interest group, where we bring organizations together from companies to open source projects, to understand each others needs, and find a shared vision for how to get there that will then scale as we Rust gets adopted by the next set of organizations.

## Conclusion

While we hope this will put us on a path for the cargo team to be taking on more design work, the challenge will be doing so without overburdening the existing maintainers.

## TODO

- Explore idea of tracing as another approach for understanding cargo, how parts interact, and implementing / debugging a feature