# Nested workspaces

## Summary
This adds new fields to `[workspace]` that allow workspaces to be nested inside of each other.

## Motivation

[rust-lang/cargo](https://github.com/rust-lang/cargo) contains multiple crates but they do not exist in a workspace because [rust-lang/rust](https://github.com/rust-lang/rust) pulls `cargo` into its workspace via a git submodule. This requires manually running tools across each crate in the rust-lang/cargo repo, increasing the risk that something will fall through the cracks in updating the processess (CI or manual) and increasing the friction for splitting content out into additional crates, as desired.

> **NOTE(weihanglo)**: we need to figure out some more compelling motivations. Something only nested workspaces can solve. As we haven't yet make a consensus on Cargo adopting workspace.

Open Questions
- Is top-level control (dependency versions, profiles, etc) a requirement for rust-lang?
- Does anyone require bottom-up control?
- See [nested profiles issue](https://github.com/rust-lang/cargo/issues/8264)
- ["have a workspace, and one of the crates is a WASM application which should be compiled with `panic = "abort"`"](https://github.com/rust-lang/cargo/issues/8264#issuecomment-751931213)

## Guide-level explanation

This is intended to go into the [workspace reference](https://doc.rust-lang.org/cargo/reference/workspaces.html)

### Updates to the `[workspace]` table

#### For the root workspace

##### `workspace.members`
To nest a workspace, the root workspace would add it to `members`. Any of the nested workspaces members would automatically be included as a member of the root workspace. If a member of a nested workspace is added to the root workspaces `members` it will be ignored. This allows the nested workspace to freely add, remove, or move crates and no updates are needed in the root workspace, this is very helpful for git submodules.

##### `workspace.default-members`
When using `default-members` in the root workspace and a nested workspace is added, all of its `default-members` are added. If the nested workspace does not specify `default-members` and its a [virtual workspace](https://doc.rust-lang.org/cargo/reference/workspaces.html#virtual-workspace) all of its `members` are added. If the nested workspace does not specify `default-members` and it's a non-virtual workspace, only the root crate is added. This mirrors how [package selection](https://doc.rust-lang.org/cargo/reference/workspaces.html#package-selection) currently works within a workspace.

| `default-members` | Virtual workspace | Non-virtual workspace |
| ----------------- | ---------------------- | ---------------------- |
| Specified | Uses `default-members` | Uses `default-members` |
| Unspecified | All workspace members | Only the root crate |

#### For a nested workspace

##### `workspace.nested`
Any worksapce that is going to be nested must set the new field `nested` inside of `[workspace]`. It can be specifed in a few ways:

[workspace]
nested = true

When the parent workspace might not be available, such as when the parent workspace pulls in the nested workspace through a git-submodule, the nested workspace can declare the root workspace as `optional`:

[workspace]
nested = { optional = true }

Just as workspace members need to specify `workspace = "path"` when they are not hierarchically below the workspace, nested workspaces must also specify the path when they are not below the root workspace:

[workspace]
nested = { path = "../parent" }

This new fields lets `cargo` know to keep searching for the root workspace, since it normally would stop searching for the root workspace at the first one encountered.

#### `workspace.name`
One major concern is the workflow of using nested workspaces. To make it so you can run `cargo` commands on a specfic nested workspace all nested workspaces and the root workspace must be named. You can set the name of a workspace by setting the new `workspace.name` field.

[workspace]
name = "cargo"

These names will need to be distinct from one another but will be in a differnt namespace than packages.

### CLI

#### `--workspace`
It was considered to make `--workspace` work similar to [`--package`](https://doc.rust-lang.org/cargo/commands/cargo-check.html?highlight=--package#package-selection), where if it is blank a list of possible worksspaces would be displayed. It was decided against this as it would be a breaking change. To have a way to still be able

#### `--nested`
`--nested` is a new CLI flag that can be used with the new `workspace.name` field. It is meant to be used in the same way as [`--package`](https://doc.rust-lang.org/cargo/commands/cargo-check.html?highlight=--package#package-selection). When `--nested` is blank it will show a list of possible nested targets. When a workspace name is specified (`--nested cargo`) the command will only be ran on that workspace. Packages within the workspace to operate on will adhere to the guidelines described in [Package selection](https://doc.rust-lang.org/cargo/reference/workspaces.html#package-selection)

### When using nested workspaces

- The root workspace's `Cargo.lock` will be used
- The root workspace's [output directory](https://doc.rust-lang.org/cargo/guide/build-cache.html) will be used
- The root workspace's `workspace.resolver` will be used
- Entries in `[patch]` and `[replace]` will be ignored in any nested workspace
- If there is something that is needed for a nested workspace to work correctly, then the entries should be copied to the root workspace
- Entries in `[patch]`, `[replace]`, and `[profile]` sections will be overwritten by their corresponding sections in the parent workspace, if present. The nested workspace will not have access to entries that only exist in the root workspace
- **TODO:** look into patch, replace, and profile inheritance. Errors if `optional = true`
-
## Reference-level explanation

## Drawbacks

- It leads to more confusion over different inheritance rules between Cargo's configuration (.cargo/config.toml) and manifest (Cargo.toml).

## Rationale and alternatives

- Instead of a united workspace tree, we could have a confederation of workspaces where the nested workspaces interact at only the most limited ways possible
- We resolve and build based on each binary's profile
- This would explode the number of targets getting built, the target directories, etc
- This is a larger lift to implement as it runs counter to how cargo works today
- This would make it harder for `rust-lang` to control the behavior for all binaries
- When a behavior needs to be consistent (resolver), we make the parent overwrite the child workspace
- Interactions between workspaces was selected to preserve the explicit nature of inheritance (**TODO** link to RFC 2906) and to work well when `optional = true`
- Nesting information needs to be included in the leaf workspace so that it knows to search up (think running `cargo check` in a crate, how does it know what workspaces to use?). The parent workspace will discover the child workspace like any other workspace member.
- We could make things nested by default
- This will be a breaking change if it follows the current package <-> workspace relationship because a workspace in a parent directory would automatically be considered the parent workspace
- This will slow down due to extra walking
- This will be another case running into ceiling directory issues
- We could make the `nested = {}` table have fields to control how to merge patch, replace, and profile

### Alternatives

- `CARGO_TARGET_DIR` for setting a share build cache directory.
- `.cargo/config.toml` for setting sharing rustflags and other options. Currently supports
- [`[patch]`](https://doc.rust-lang.org/nightly/cargo/reference/config.html#patch)
- [`[profile]`](https://doc.rust-lang.org/nightly/cargo/reference/config.html#profile)
- **TODO**: any alternative to share Cargo.lock without nested workspace?
- **TODO**: any alternative to reduce boilerplate of defining dependencies across workspaces?

## Prior art

epage worked on a proprietary build system that was a confederation of workspaces for mostly compiled code (non-Rust). Each package had its own equivelant of requirements and locked versions and its own target directory and each binary package had its own equivalent of profiles. You could build the package on its own. If a workspace was built, it would update the lockfile of the package but the most nested workspace had highest precedence. In a cross-team, cross-product setup, this allowd more local control to adapt to more specialized needs than global control would allow, in dependencies, profiles, etc.

Past discussions
- [Nested workspaces issue #5042](https://github.com/rust-lang/cargo/issues/5042)
- [nested workspaces](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/nested.20workspaces) zulip thread
- [Workspace for Cargo itself](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/Workspace.20for.20Cargo.20itself) zulip thread

## Unresolved questions

## Future possibilities

## Performance impact