# Cargo Garbage Collection

## Summary

This is a proposal to extend Cargo to allow it to automatically and manually remove cache files from disk.

## Overview of Existing Caches

The following directories are relevant for cache cleanup:

- CARGO_HOME:
- `git/db` — Git repositories for git dependencies.
- `git/checkouts` — Clones of specific versions of the git repositories from the `db` directory.
- `registry/index` — The registry index (a git repository with a cache, or just a cache for the http protocol).
- `registry/cache` — `.crate` files downloaded from a registry.
- `registry/src` — Extracted contents of the `.crate` files.
- local `target` directory:
- `target/tmp` — Temporary files used by tests.
- `target/doc` — Output from running `cargo doc`.
- `target/...` — Output artifacts from a build.
- `target/*/build` — Build script OUT_DIR outputs, and build script executables.
- `target/*/deps` — Intermediate build files and final outputs.
- `target/*/examples` — Build outputs and intermediate files for examples.
- `target/*/incremental` — rustc incremental caches.
- `target/*/.fingerprint` — Cargo change detection database.

## Proposed Solution

This is a large project, so it is being broken down into multiple steps.
The general outline is:

1. Start tracking last-use data for CARGO_HOME caches.
This will allow knowing the date and time the last time some piece of cache information was used.
This information can then be used for automatically deleting data that has not been used in some time.

Last-use information will be tracked in a sqlite database.
This information will include timestamps of the last time the following were "used":

- `git/checkouts/<name>/<hash>/`
- `git/db/<name>`
- `registry/index/<name>`
- `registry/cache/<registry>/<name>.crate`
- `registry/src/<registry>/<name>`

The timestamp is updated whenever something is initially added, or something reads from it.
Every build will ensure that every `.crate` entry and `src` entry are updated together.
Similarly git `db` entries are updated together with `checkouts` entries whenever a build is done that has those dependencies.

Timestamps are also recorded for the last time certain cleaning operations were performed to avoid needing to clean too often.

Care will need to be taken to ensure that this does not have too much of a performance impact, that it is resilient in the case of errors, and can handle backwards and forwards compatibility.
Initial experiments showed about a 35ms performance hit with a HDD with a project with about 200 dependencies, though more investigation will be needed.

1. Implement a garbage-collection system for the cache data.

This system will provide a means to automatically delete unused data from CARGO_HOME.
It will likely have some tunable parameters which may or may not ultimately be exposed via the config system.

The exact rules are to-be-determined.
This will need to consider various use cases and performance concerns.
Some example considerations:

- Don't clean too often.
- Don't clean when offline.
- Don't clean too aggressively to consider users who go offline.
- Docker caching scenarios which could cause thrashing.
- Caching systems (such as CI).
- One consideration may be thinking about ways to avoid instantly cleaning old caches that were restored from another cache.
We don't want every initial execution of `cargo` on GitHub Actions to start deleting files.

Some example tunable parameters may be:
- `auto-clean-time = "1 day"` — How often the automatic gc runs.
- `max-src-age: "1 month"`
- `max-crate-age = "3 months"`
- `max-index-age = "3 months"`
- `max-git-co-age = "1 month"`
- `max-git-db-age = "3 months"`

1. Implement a UI for the user to manually trigger garbage collection.

This UI will need to consider a variety of use cases and future extensions, such as:

- Manually triggering a gc.
- Manually forcing the deletion of caches that are newer than the default gc age.
- Controlling which caches are being cleared.
- Supporting a future extension with multiple target artifact directories (such as global shared build caches).

This proposes to extend the `cargo clean` command to handle cleaning of caches via various subcommands.
A rough sketch of this may look like:

- `cargo clean gc` — Deletes unused data in both the local and global caches.
This is essentially manually performing the same task that cargo does automatically once a day (or whatever `auto-clean-time` ends up being).
- This will have various options to choose how aggressive it works, specifying specific caches to clean, dry-run support to inform what it will do, etc.
- `cargo clean gc --removed-toolchains` — If rustup is installed, will query which toolchains are installed, and delete artifacts for those that are not installed (or when stable/beta channel has been updated). (See cargo-sweep.)
- `cargo clean gc --max-age=now` — Deletes all caches.
- `cargo clean query` — Various options for querying how much cache data is being used.

The default behavior of `cargo clean` without a subcommand is the same as today (cleans the local build directory).

1. (Needs investigation) Move the registry index cache files to use sqlite instead of the binary sparse files it currently uses.
These cache files don't represent a very significant amount of disk usage (tens of megabytes at most), but seeing if we can leverage a database may help here.
This needs some investigation to see if sqlite can use less space and have similar performance and reliability.
I'm not sure if it will be desired to age out data from this database or not.
This sqlite database could also be shared with the last-use database.
<https://github.com/weihanglo/cargo/commit/4e277122b> contains an initial investigation into this.

1. Implement a new fingerprint database with sqlite.

The current fingerprinting information does not contain enough information to accurately track which artifacts exist and what they were used for.
There are several long-standing feature requests that are blocked on having this information, so this is expected to bring a lot of value for other features.

1. Implement last-use tracking for the local target directory.

Similar to the global registry cache, this will track the last time each artifact was "used".
That is, any build that required the given artifacts to exist.

1. Implement a garbage-collection system for the target directory.

This is similar to the package cache, and will likely use similar configuration parameters.

1. Extend the UI to also handle local target directory garbage collection.

### Deployment

Since this will likely be implemented in stages, the deployment may also be done in stages.
In particular, we want to quickly start gathering data about the reliability and performance of the sqlite databases without fully engaging the cleaning system.

A potential rollout could be:

1. Implement last-use tracking and garbage collection for registry and git caches.
This will only be available on nightly with a `-Z` flag.

2. Stabilize last-use tracking, but *not* garbage collection.
The intent is to get as much real-world information about sqlite support as quickly as possible.
Problems reading or writing the database will be reported as warnings instead of errors to prevent disrupting users.
Otherwise, this has no change to the user.

3. (Optional) Implement sqlite for the index caches.

4. Implement the new fingerprint database.
This will be immediately stable, and replace the existing fingerprint system.

5. Implement last-use tracking for the local target directory.
Depending on the schedule, this could go immediately to being stable.

6. Implement local target directory garbage collection.
This will initially only be available on nightly with a `-Z` flag.

7. Stabilize all garbage collection options.

Possibly with automatic garbage deletion disabled, with a future release turning it on by default.
That would further provide more opportunities for test.

## Prior Art

Cargo-related tools:
- [`cargo cache`](https://crates.io/crates/cargo-cache)
- [`cargo sweep`](https://crates.io/crates/cargo-sweep)

Other tools:
- [`npm cache`](https://docs.npmjs.com/cli/v8/commands/npm-cache)
- [`git gc`](https://git-scm.com/docs/git-gc)
- [`buck clean`](https://buck.build/command/clean.html)
- Similarly, `buck2 clean` (no online docs)
- [`ninja clean`](https://ninja-build.org/manual.html#_extra_tools)
- [`meson compile --clean`](https://mesonbuild.com/Commands.html#compile)
- [`go clean`](https://pkg.go.dev/cmd/go@go1.20.4#hdr-Remove_object_files_and_cached_files)
- [`nuget locals`](https://learn.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-locals)
- [`bundle clean`](https://bundler.io/v2.2/man/bundle-clean.1.html)
- [`pip cache`](https://pip.pypa.io/en/stable/cli/pip_cache/)
- [`dart pub cache`](https://dart.dev/tools/pub/cmd/pub-cache)
- `swift package clean` (no online docs)
- [`dub clean-caches`](https://dub.pm/commandline.html#clean-caches)
- [`apt clean`](https://wiki.debian.org/AptCLI) and `apt autoclean`

## Alternatives

- Instead of extending `cargo clean`, additional top-level commands could be introduced.
`cargo clean` was chosen because it already exists for "cleaning" caches (just the local target directory).
Having multiple commands that perform similar actions could be confusing.
Additionally, some third-party commands like `cache` are in semi-popular use, and cargo doesn't handle repurposing commands very well.

- `cargo clean` could have additional subcommands with clearer intent than just "gc".
For example, there could be a `cargo clean registry` could be used for deleting registry data.
Or `cargo clean git` would delete git repositories.
- Or aliases could be added for common actions.
For example, `cargo clean all` could be the same as `cargo clean gc --max-age=now`.

- The last-use database for the package caches and the cache for the index could potentially be combined into a single cache.
At this time, I would lean towards keeping them separate since they have somewhat different use characteristics.
It's also unclear what the performance implications would be.
However, combining them could have benefits.

## Unresolved questions

- Investigation will be needed to determine the safest yet most performant way to use sqlite.
My understanding is that it does not handle concurrent writes very well on network filesystems that don't provide proper locking.
Cargo needs to be able to work in these scenarios without breaking.
We may need to handle corrupted databases gracefully.

- `target/*/deps` can be filled with cgu's from cancelled builds that may not be easily tracked.
This becomes even more difficult with split-debuginfo.
There are other miscellaneous files that could potentially accumulate that are hard to track.

- Similarly, the incremental caches can be hard to track since they use their own hashing scheme that cargo doesn't understand.
This may require changes in rustc.

- Automatic gc probably won't handle things not tracked in its last-use database (like registry files downloaded by older versions of cargo).
Should these only be cleaned manually?

- There might be some circumstances where cargo could potentially "know" that certain artifacts should definitely be deleted.
For example, if you change the package version, name, or other properties that impact the filename hashes, or removing or changing a dependency.
However, tracking that seems complicated, so deleting by age might be the best solution.
Cleaning dependency artifacts that are no longer in `Cargo.lock` may be feasible, but that may need to be a manual operation only.

## Future possibilities

- At this time, there is no intent to provide a public API for external tooling to access the cache information.
The `cargo clean query` command will provide human-readable summaries.
- Support a `--recursive` option like <https://github.com/rust-lang/cargo/issues/11305>.