Try   HackMD

Cargo team meeting notes

2023-04-25

  • Patch release for stable regression on Windows: https://github.com/rust-lang/cargo/pull/12017
    • Issue for figuring out how to prevent this in the future: https://github.com/rust-lang/cargo/issues/12033
    • cargo-release <- suggested by epage
      • cargo-edit has cargo-set-version, not on a path to stability yet
    • If you bump the version in master, that can cause problems with [patch] when pointing to git.
    • question about pushing version updates directly to master.
    • Report to user what the version is changed from publish
    • Could always bump patch
    • Put on contributor to bump
      • Maybe a bot command to edit the branch
    • Problem is knowing to bump other stuff
    • jacob: CI job that does a git diff, and requires the version to be updated (even if it has already been updated)
    • eric: will look at adding some kind of CI check to validate if a dependency is modified that the version gets bumped, and posts comments to the PR. Will look for more sophisticated support (like more automation) in the future.
  • home release
    • Updated to use windows-sys, ehuss to release today
  • PR author pinged me a couple of times in a week: https://github.com/rust-lang/cargo/pull/12005#pullrequestreview-1397575771
    • If an individual recurrently is too persistent, then we can have a direct conversation with them.
    • Don't let that level of persistence influence your priority.
    • This will happen again in the future, since some people arrive with different expectations and are not familiar with how things work (or they are just excited and eager). A response like weihang's is perfect in those cases.
    • Understood that out-of-band pings can be uncomfortable. Feel free to reach out on the private channel or direct message another team member if you need it.
  • Any input on cargo-script Pre-RFC to help with next steps?
    • https://github.com/epage/cargo-script-mvs/blob/main/0000-cargo-script.md
    • Reviewing what the major decision points are.
      • The syntax for the embedded manifest.
      • How to handle locked dependencies.
      • How to handle the edition field.
      • eric: Curious about the launching mechanism.
    • eric: Thinking about the significance of this feature, that it could be one of the largest changes in the way Rust and Cargo is used in a long while.
    • eric: recommended to move forward with an RFC that makes it a little clear that this as a general concept is what we want to work on, but the details may change (current RFC text seems to acknowledge this pretty well with the different options and variants).
      • Left up to ed if they want to use "eRFC" terminology or not.
      • Trying to nail down all the details in an RFC would likely be too painful.
    • There is a prototype, but it has to do a bunch of hacks since the cargo-lib API doesn't expose enough. Having it as part of cargo would allow integrating it better.

2023-04-18

  • Scott: Allow named debuginfo options in Cargo.toml
    • The PR adds support for "line-directives-only" and "line-tables-only" in profile.<>.debug as well as "none", "limited", and "full"
      • This matches rust#109808
      • "The motivation is not just to match rustc, but also to make it easier for us to change the default for 1 over an edition to mean "line-tables-only" instead of "limited". If we allow limited as an alias for 1, people will be able to use it before and after the change in defaults to mean the same thing; if we disallow it and only allow it in the new edition, it will be impossible to have the same code work with both new and old versions of cargo."
      • Enabling these flags in cargo is required before bootstrap can start passing them through to rustc (which I'd like to do so we can reduce the size of the generated artifacts: rust-lang/rust#104968 (comment)).
    • Scott: I am unsure what stabilization should look like
      • Insta-stable: cargo would have to stick with "line-directives-only", "line-tables-only", "none", "limited", and "full" being options with no nightly time.
      • Nightly: Would delay usage in rust, and seems like a relativly small change
    • Reason this was added: https://github.com/rust-lang/compiler-team/issues/613
      • eric: We don't know what these options mean. It isn't critical for this PR, though.
    • eric: Would like to see tests. Since it isn't available on stable, it will need to do runtime detection.
    • No particular objections to the instant stabilization since it is already stabilized in rustc.
    • eric: Uncertainty around changing the default of "1" (in edition). It's unclear if anybody actually uses 1.
  • Issue status labeling
    • https://github.com/rust-lang/cargo/issues/11788#issuecomment-1511899339
    • eric: concern about being too fussy or complicated
    • eric: cocnern about naming
      • mentor: shephard, driver
    • Don't necessarily need to sync with other rust-lang projects
    • eric: concern about not being clear about which issues are soliciting public involvement, where previously E- labels were holding that role.
      • Hopefully with updates to the contributing guide, the links to "here are things to help with" should make it clearer.
  • epage: Office hours
    • hi-rustin showed up, got a better understanding of what they are wanting
    • hi-rustin doesn't want any design work (not relaxing post-work) but to just code
    • Open to any thing we want to point at them (pointed cargo new doing workspace inheritance to them)
  • epage: lints RFC seems to settle down in new information and is just re-iterating personal preferences now

priority w/ later adding warnings/sorting

[lints.rust]
all = { level = "warn", priority = -1 }
allow_dead_code = "allow"
unsafe_code = "deny"
  • flip1995 prefers priority over RUSTFLAGS style (still prefers another schema)

RUSTFLAGS-style

[tools.rust]
lints = [
    { warn = "all" },
    { allow = "allow_dead_code" },
    { deny = "unsafe_code" },
]
  • Preference to not repeat the level.
  • RUSTFLAGS style top-level table was generalized to tools to avoid lints.rust.lints or lints.lints
  • RUSTFLAGS could go back to clippy:: rather than a clippy table but that might be harder to read if/when tool-level configuration is added
[tools]  # now `tools` name is a bit weird with top-level `lints`
lints = [
    { warn = "all" },
    { allow = "allow_dead_code" },
    { deny = "unsafe_code" },
    # ... plus more

    { warn = "clippy::all" },
    { allow = "clippy::cyclomatic_complexity" },
    # ... plus more

    { warn = "rustdoc::all" },
]

[tools.clippy]
cyclimatic_complexity_threshold = 50
warn-on-all-wildcard-imports = true
allow-expect-in-tests = true
allow-unwrap-in-tests = true
allow-dbg-in-tests = true
  • Question about performance between the different styles: unlikely to have a noticeable overhead.
    • Untagged unions can potentially have noticeable performance issues.
  • Question about how to get testing on nightly. Can this just be ignored on stable?
    • There are some complications, such as making sure the schema can change, or lint levels being required for correctness (lowering a deny-default level).
  • epage: Input for https://internals.rust-lang.org/t/pre-rfc-cargo-script-for-everyone/18639
    • In background, think about what would we block on RFC
  • weihang: adding some xtask-style tasks 








        flags::XtaskCmd::Install(cmd) => cmd.run(sh),
        flags::XtaskCmd::FuzzTests(_) => run_fuzzer(sh),
        flags::XtaskCmd::Release(cmd) => cmd.run(sh),
        flags::XtaskCmd::Promote(cmd) => cmd.run(sh),
        flags::XtaskCmd::Dist(cmd) => cmd.run(sh),
        flags::XtaskCmd::PublishReleaseNotes(cmd) => cmd.run(sh),
        flags::XtaskCmd::Metrics(cmd) => cmd.run(sh),
        flags::XtaskCmd::Bb(cmd) => {

2023-04-11

  • beta:
    • Network retry delays: https://github.com/rust-lang/cargo/pull/11922
    • Path fuzzy query permutation fix: https://github.com/rust-lang/cargo/pull/11940
    • -C and rustup toolchains: https://github.com/rust-lang/cargo/issues/11957
      • Cargo could re-execute itself in the new directory.
        • Would need to tell the child process to ignore the -C value since cargo wouldn't know where the arg is in the args list after clap processing.
          • Or child process just compares -C with current_dir and do nothing
      • Handling via rustup would be difficult, it would need to parse the command-line args.
        • Wouldn't be available to non-rustup users.
      • ed: Follow up with compenguy
      • eric: Ask rustup maintainer input
      • Original issue author indicated they wanted it to honor rustup overrides.
      • eric: will look at reverting in beta (with -Z unstable) to give us a little more time to make a decision (since we have about 2 days to work with).
  • jon: weihang's post-mortem
    • Is the focus for the process or the post-portem itself?
      • Kinda both, but the process may be a higher focus.
    • COE: https://aws.amazon.com/blogs/mt/why-you-should-develop-a-correction-of-error-coe/
    • jon: have the cargo team go through the assessments first before publishing, is it accurate, etc.
      • May also be leasons to learn here.
      • publish on internals? Or if you want a wider audience, inside rust blog.
      • Doesn't necessarily need to reference Amazon. The way we handle it doesn't necessarily need to look the same.
    • eric: question about some of the recommendations didn't seem quite feasible. Is the intent to have concrete recommendations, or preferences for an ideal world?
      • Example is the limitation of CI for running larger crater-like runs.
      • cargotest is currently very limited, and could potentially be extended, but would need to figure that out, work with other teams, etc.
    • eric: recommend posting a blog post on insiders, focusing on the process and sharing with other teams which can think about if it would be useful for them.
    • weihang: follow up with blog post
    • team should read and give feedback on the document
    • property testing of cargo
      • Generating valid Cargo.toml with complex properties.
  • eric: Design help with cargo doc and forced-target
    • https://github.com/rust-lang/cargo/issues/11728
    • Problem with navigation, there is no way to link between the targets, so the user can only see one target, and the others are essentially invisible.
    • Loop in the rustdoc team.
    • Will propose documenting only for a single target. Need to also check with docs.rs behavior (would that break anything?).

2023-04-04

  • eric: beta backport of adding delays to network retries
  • lint RFC
    • https://github.com/rust-lang/rfcs/pull/3389
    • Question: How to handle prioritization?
    • Question: How to handle lint configuration?
    • Question: Syntax for
    • With a long list of lints, it may not be clear which section it is in (the "allow" or "warn" section, etc.).
    • Workspace inheritance could make "merging" arrays more difficult.
    • Clippy may want to avoid having configuration values tied to specific lints. It is more of a global setting, since the settings could be used across multiple lints.
    • Allowing arbitrary config settings in a per-lint thing would prevent extending it in the future.
    • Concern over behavior within a priority
    • Priority
      • auto-sort
      • explicit (sort within a priority)
        • flip1995 "puts burde on user"
        • "but its set and forget"
        • flip1995: "transitioning away would be maintenance burden for cargo"
        • Seems like its a fairly low code to support it
        • Could we make the first pass of heuristics to just error if priority is needed or if there is an intersection
          • Lint groups may evolve to needing disambiguiting
          • So not hard error, just warning
      • array-of-tables

Now

[lints.rust]
all = { level = "warn", priority = -1 }
allow_dead_code = "allow"
[lint.clippy]
cyclomatic_complexity = "allow"
  • Similar to rustflags
  • People lookup by lint name, not level
[lints.rust]
warn = [
    { level = "all", priority = -1 }
]
allow = [
    "allow_dead_code"
]
[lints.clippy]
allow = [
    "cyclomatic_complexity"
]
  • Less verbose
  • Get lost in what section you are in
  • Gets weird if we allow overiding what you inherit
    • Could just have rustc deal with it
[lints]
warn = [
    { level = "all", priority = -1 }
]
allow = [
    "allow_dead_code",
    "clippy::cyclomatic_complexity"
]
  • Like above, no translation needed by users

Inline

[lints]
rust = [
    { lint = "all", level = "warn" },
    { lint = "allow_dead_code", level = "allow" },
]

Standard

[[lints.rust.all]]
level = "warn"
[[lints.rust.allow_dead_code]]
level = "allow"

Short-hand?

[lints]
rust = [
    "Wall",
    "Aallow_dead_code",
    "A=allow_dead_code",
    "-Aallow_dead_code",
    "--warning=cyclomatic_complexity",
    { lint = "cyclomatic_complexity", level = "warn", threshold = 50 }
]
  • Would users try to sort these and it mostly work but break in some cases?
  • The W/A/D/F single letter prefix isn't very clear, looks kinda strange.
    • Generally short-flags are meant for humans, long flags are encouraged automation























[lints]
rust = [
    {allow="all"},
]

clippy = [
    {warn="pedantic"}
]

# ???
clippy = [
    # TOML does not yet support multi-line inline tables, 
    # so you can't break arrays up in this scenario.
    {warn=["unused_self", "almost_swapped", "..."]},
    {warn=["pedantic"]},
    {allow=["..."]}
]

rust = [
    "allow = all",
    "warn = clippy::pedantic",
    "warn = clippy::unused_self",
]
  • "level = lint" (vs lint=level) so the keys are a known, finite set
  • Are we going down this path just because we started with the CLI/.rs syntax?
  • Reach out to Embark about their experience?






[lint.rust.-1]
all = "warn"
[lint.rust.0]
allow_dead_code = "allow"
[lint.clippy.0]
cyclomatic_complexity = "allow"






[lints.clippy.groups]
pedantic = "warn"

[lints.clippy.lints]
unused_self = "allow"
  • User might mis-assign
    • Require new CLI behavior if linter wants to warn
    • Could have the linter just warn if the order is weird
      • The more naive, the more warnings will appear when using RUSTFLAGS
  • require hard coding of "all"



















# Syntax similar to how Rust defines them.
# #![warn("clippy::pedantic")]
# #![allow("clippy:unused_self")]
# #![deny("")]
[[lints.clippy]]
warn = [
    "clippy::pedantic"
]

[[lints.clippy]]
allow = [
    "clippy::unused_self"
]

[[lints.rust]]
deny = [...]

[[lints.rust]]
allow = [...]

Potential Future

[lint.rust]
all = { level = "warn", priority = -1 }
allow_dead_code = "allow"

[lint.rust.config]
[lint.clippy.config]
cyclomatic_complexity_threshold = 50
rust-version = "1.30.0"

[lint.rust]
warn = [
    { level = "all", priority = -1 }
]
allow = [
    "allow_dead_code"
]
[lint.rust.config]
[lint.clippy.config]
cyclomatic_complexity_threshold = 50
rust-version = "1.30.0"

[lint]
warn = [
    { "all", priority = -1 }
]
allow = [
    "allow_dead_code"
]
[lint.rust.config]
[lint.clippy.config]
cyclomatic_complexity_threshold = 50
rust-version = "1.30.0"
  • epage: office hours
    • First: just Jacob
    • Todays:
      • Byron shooting the breeze
      • Someone with build-std workflow not yet supported
      • "be-ing" asking about --out-dir some more
        • Talked about alternatives
        • Stepped through cargo build --dependencies-only as a template for how they could move forward. In particular, encouraged them to work within the cxx community rather than coming to use with a bespoke route

2023-03-28

  • eric: dedicated meeting notes scribe
  • eric: cargo install alias
    • https://github.com/rust-lang/cargo/pull/11845
    • https://github.com/rust-lang/cargo/issues/11838
    • weihang: Same opinion with Turbo87. ❌
    • Install means something different than the npm ecosystem. Good to emphasize that it
    • cargo i could recommend the command.
    • Make it a more explicit operation.
    • Index of explicit decisions of what not to do.
      • Maybe a "rejected" label?
    • Will close it out as not accepted.
    • Typo suggestions are not particularly good. They could be improved to handle things like prefixes (cargo i -> cargo install).
    • Aliases that could directly error, and tell you the correct flag to use. Weihang opened an issue on clap for native support for that.
  • eric: continue cfg validation discussion
    • Clarification for previous discussions, validation is applied to command-line options.
    • eric: Concern on amount of time this putting on the team for something that isn't a high-priority for the team.
      • Push back for writing design.
        • Examples, --dependencies-only or cargo clean custom flag.
      • eric: Personal concern is about this creating a first-class system that is similar to features.
    • eric will respond about getting more discussion on internals, and letting them know it isn't a priority for us.
  • FYI: sigstore RFC: https://github.com/rust-lang/rfcs/pull/3403
  • eric: issue labeling/disposition
    • https://github.com/rust-lang/cargo/issues/11788
    • weihang: I have no strong opinion on any option. Feel free to choose one.
    • Who it is currently waiting on, and what it needs. The phase for setting expectations. In the guide, can document what those phases mean. Set expectations for users. Can remind them to consider prior art, etc.
    • Lean away from multi-dimensional status
    • Users often don't look at labels (or know what they mean).
      • But we can point them via links "here are labels that are ready", etc.
    • E-triage, E-design, E-impl, etc.
      • E is intended for things that users can help with
      • S-waiting-on-shephard, S-accepted-for-impl
        • S-needs-shephard would remove the E in terms of it is not open for people to contribute
    • Having specific individuals assigned to help mentor or review?
      • It can be difficult to balance the load. We won't know who will actually start on an issue, and won't know ahead of time if a team member will have the capacity to help.
      • Could be silent at first, not explicit on the issue.
        • The person needs to know who to reach out to.
        • Starting slow, one or two, to see how it goes.
      • Having office hours would help making a fixed/explicit opportunity for engagement.
        • Making a wiki page for easy edits.
    • eric: will post a proposal
  • weihang: Edition 2024 Planning
    • just want to make sure we still on track not lagged behind
    • No significant movement on anything. If anyone wants to shephard/champion a change, they can push on it at any time.
    • Not absolutely certain there will be an edition. That is the tentative plan, but AFAIK nothing has actually made it to be accepted in the compiler/language, yet.
    • When in the year?
      • No fixed schedule
        • Previously, december was the release
        • Aim for June (2024) for mostly done
    • https://github.com/orgs/rust-lang/projects/30
    • https://github.com/rust-lang/cargo/labels/A-edition-next
    • target-applies-to-host

2023-03-21

  • eric: Index rate limiting.
    • We have the request ID. Can we ask AWS support to answer why it 503'd?
      • Without logs, some of the information may not be available.
    • Logs would be very helpful.
    • Seeing S3 500 error rates would be helpful.
    • S3 partitioning
      • Is the prefix everything up to the last slash?
      • When it detects something that is hot, it will split it up (based on some prefix string).
      • Does the partitioning happen within a short period of time?
      • Was this an initial growing pain of getting things partitioned?
      • Is the paritioning per-region? Should be just in the one region where the S3 buckets live.
    • How long to cache 500's from S3?
      • Default is 10s
      • Maybe change to 1s?
      • Negative TTL and coaelescing interaction?
    • Exponential backoff
      • https://github.com/rust-lang/cargo/pull/9925
      • Lots of different tweakable values
      • Probably don't need to make these controllable by the user.
      • Global count of backoffs (or per registry).
        • If everything is erroring, it likely means "stop trying". This increases the load on the server, possibly making it worse.
      • Adding jitter?
        • eric doesn't feel it will be necessary (because the concurrent request patterns happen in CI which often have so much noise in starting that they should already be offset), but it also probably doesn't hurt to add it.
    • Importance of this issue?
      • Only one report, very likely a rare fluke?
      • eric: If we can add backoff with a small amount of code, then it shouldn't be difficult to include. If it is more complicated, then let's discuss it more.
  • eric: HTTP header diagnostics.
    • Is there any potential concern about leaking sensitive information? (nothing we can think of).
      • There is a concern about displaying all headers, since we may not be able to predict which headers might contain sensitive information.
    • eric is interested in pop, request-id, and x-cache (hit vs miss).
    • What headers does fastly set?
    • Concern about special-casing AWS

2023-03-14

  • Add experimental [cfgs] section to decribe the expected cfgs
    • https://github.com/rust-lang/cargo/pull/11631
    • cfg only intended for internal use, by documenting in Cargo.toml this may imply it is something that should be externally used.
      • Perhaps have some indication that it is internal? This would generate a warning indicating that it is internal. Includes a message to be included in the warning.
    • Scoping by platform?
      • Concern about previous tables that some people wanted to be platform specific.
      • This is mainly aimed for catching typo issues, so may not necessarily be as important for being per-platform.
    • Is there a way to not warn on all unknown names?
      • -A unexpected-cfgs will disable all warnings (probably too blunt)
    • How does it work for:
      • mycrate -> foo
      • setting some cfg on foo
      • unexpected-cfgs won't trigger on foo since it has cap-lints.
      • tokio_unstable, emits low-level metrics, primarily for users of tokio.
      • RUSTFLAGS="–cfg tokio_unstable" is going to trigger warnings in my crate. (Correction: This is not what it does.)
        • Concern about features are part of the stable API.
        • https://github.com/rust-lang/cargo/issues/10881 <- unstable features
        • Enabling features on transitive dependencies can be awkward since you have to depend on it yourself (and make sure the versions match, etc.).
        • Other enhancements to features could cover some of the cfg use-cases (like global features that can't be set via dependency declarations, unstable features, hidden features, etc.)
      • Would it be possible to unify this across the entire dependency tree?
        • How would this handle conflicts?
        • Since this is primarily for typos, not being perfect or exhaustive should be ok.
    • There are two typo sources.
      • RUSTFLAGS="–cfg tokeo"
      • #[cfg(tokeeio)]
      • This is primarily addressing the later.
    • eric to ask questions for:
      • How is this primarily different from defining in a build script?
      • Clearer descriptions of the use cases.
      • Difference between source code and command-line.
        • If we want to check command-line in the future, if this doesn't handle it, that would be an issue.
        • CORRECTION: check-cfg handles both command-line misspellings and source misspellings.
  • weihang: --ignore-rust-version for cargo install
    • https://github.com/rust-lang/cargo/issues/11718
    • Was likely just an oversight, there wasn't any intent (we remember) for restricting it.
    • This flag is intended for low-level overrides for exceptional circumstances, not something it is expected people use much.
  • eric: rustup optimization
    • An issue around rustup circumvention. A script/plugin calling cargo via an explicit path could end up calling a different rustc. Does this scenario need to be supported?
      • Scratch that, eric has found a possible solution.
    • Will follow up with PR after doing more performance analysis.
  • jon: weihang's post-mortem
    • (Will discuss more next week.)

2023-03-07

  • https://github.com/rust-lang/cargo/issues/11188
    • Is it ok for a checksum to change in a local registry? (when the version doesn't change)
    • When a git dependency is vendored into a local registry, it is no longer a "git dependency".
    • When the checksum changes, should Cargo do the right thing?
    • https://github.com/rust-lang/cargo/issues/10071
    • Paths:
      • Disallow checksum changes
      • When vendoring a git dependency, modify its version number to make it different
      • Cargo are allowed to changed, cargo should deal with any caching issues.
    • Stuck in limbo, don't want to casually tell the author to do something that we later realize isn't what we want
    • Jon will leave a comment
    • Do we want to allow any checksum change to be allowed? That breaks the concept of an immutable registry.
    • Odd that git depednencies are mutable but allowed in local registry which is immutable, leading to conflicting workflows
  • Jon: mentoring discussion next steps?
    • Generally, how to get people to help contribute to cargo? What is the overall experience?
    • What does a good path look like? Where to start, what is useful, how to submit PR, what's the process afterwards, how to engage with the cargo team, how to get to the point you can review (even if not a member).
    • entry labels
    • Previously, we talked about using the PR label config to find ways to organize the code
    • How to acknowledge people who do work?
      • Example: hi-rustin
      • A separate contributor team.
    • Improving the contributor guide can help.
    • Difficulty in understanding cargo
      • Comments help with code in isolation
      • But we need to help with interactions
      • Assumes you know everything.
      • Good: fingerprint
      • Bad: sources
        • epage did several complete implementations of wait-for-publish that kept using the wrong layer of the stack
    • Highlighting where to jump in. And what is useful.
      • In theory, label cleanup that weihang is doing would help
      • Having someone available to ask questions, someone who understands it and has time.
      • What about open forum on zulip?
        • Depends on personalities
    • How to make sure a team member is associated with an issue and contributor enough that they can be helped.
    • Focus areas and priorities
    • Make it clear what an issues needs, what needs to happen next.
    • Thinking about the distinction of people who come in interested in open source work, and more flexible in what they do, and people who come in to only fix or add a specific thing.
    • The roadblock of "sorry, can't review", but then have nothing to be able to actually help with that.
    • Tokio sponsorships
      • The foundation is possibly a way to channel sponsorships.
      • Larger companies can pay people to work on things nearly full time, whereas smaller companies don't have the resources to do that.
      • If someone was paid full time for several months to do refactorings in Cargo, would we even be able to review that work (probably not).
    • Having a written-down path to being a contributor might encourage more people to follow it.
    • Contributor team
      • Acknowledge limiting to cap of people.
      • Need someone to lead that team, define what that team is
    • Hosting office hours
      • Scott: can get derailed easily, going on tangents
      • 1 hour/week
      • Zulip is available for text-based semi-async
      • Text or video
      • Good place to identify what to improve in documentation.
      • Advertise in GitHub readme, or issues
      • Customize the welcome bot
        • Not currently possible, but would be very easy to add (eric can mentor or help if desired)

2023-02-28

  • Brief update on lint RFC
    • Comfortable but not solid on
      • Only applies to the package, not deps, affecting future-incompat
      • Priority or have rustc figure it out?
    • Push out location reporting
    • Wait 2 weeks of quiet to propose
  • Workspace update
    • Dedicated workspace
      • Distinct lock file
    • What changed was removing RLS
    • Don't feel this requires nested workspaces
      • [profile] becomes complicated
    • We can still do nested workspaces in the future if the need is there.
  • namespaced packages
    • Could use someone to help push it over the finish line, would be very helpful and desired
    • "I think the main issue was/is that few of the team members had the time/energy to really dig in on this RFC. unless we're in a big rush I'd defer this until the crates.io position at the Foundation is filled, which should make things a lot easier for the rest of the team." - Tobias
  • Sparse update
    • Switching the default to crates.io next week?
    • Updating the publish text: https://github.com/rust-lang/cargo/pull/11713
      • Doesn't necessarily need to wait for that
    • Interested in knowing the load on crates.io and GitHub
      • GitHub says they have last 3 months rolling
    • Eric: Concern about people behind a firewall that update and suddenly can't use Cargo
      • Reason: They specifically allow trusted sites. Answer: They just need to add index.crates.io to firewall. Try to be understanding. They can temporarily change the default until they can get it approved.
      • A custom message for crates.io error message to let the user know, and that there is an option to change it?
        • eric: uncertain if needed. Maybe documentation will be enough for people to figure it out?
  • Arlo: Should we allow authenticated registres without requiring asymmetric tokens?
    • Summary
    • asymmetric-tokens RFC
    • Zulip: Cargo registry OIDC
      • GitHub has a system where a GitHub action can provide a JWT signed with a well-known public key that proves it's running inside a given GHA. This could be used to publish crates without storing any secrets in GHA. Asymmetric tokens are not easily compatiable with this.
    • Zulip: asymmetric token escape hatch
      • An "escape hatch" for embeddeding symmetric tokens inside the non-signed portion of asymmetric tokens was suggested. Consensus was that we should not move forward with this approach.
    • Zulip: Consider Stabilizing registry-auth
      • Request from 3rd party registry (shipyard.rs) to stabilize registry-auth as-is.
    • Arlo working with a registry implementor. They have a scheme used by other registry protocols. Cargo's asymmetric tokens would be difficult for them.
    • Why is it painful to support asymmetric?
      • Workaround is to stuff the token in a field. What's wrong with that? Does it need to be in a standardized header? Does it just feel bad?
      • Gives a sense of security that isn't there. Don't want to encourage that.
    • OIDC, GitHub Actions gives a short-lived signed token, crates.io can validate that.
      • You can say this GitHub Action has exclusive permission to publish this crate.
      • GitHub OIDC should work, but uncertain about exactly how it works. It is optional to have a "scope", it would be nice to enforce all registries use a scope. But we can't do that. Cargo can't enforce that, but the registry (like crates.io) can do that. Makes it easy to make a poorly secured registry.
    • Pick one of the four options (or propose a fifth)
      • Josh: 3 or 4
    • Is GitHub OIDC unique to GitHub?
      • Sort of. They are using it in a way that OIDC was not intended. Other services like Azure are picking up their way of doing it. Others may be using the same structure, but a different key. You came from CircleCI using a "GitHub-like" key.
      • Ideally this should have been standardized.
    • Don't want long-lived tokens anywhere.
      • Can a short-lived asymmetric token work?
        • Unknown. New authentication mechanisms are very hard to adopt.
    • Josh: Happy with a mechanism that is capable of being scoped (and crates.io can require that scoping), and encourage all registries to require scoping. That is the main requirement, to prevent a token for a different registry being used (if it is leaked to the wrong service). Assuming the registries enforce that scoping.
    • Jacob: Implementing the asymmetric token support, hacked in to support existing format. Encrypted by key only used by authentication server.
      • Various systems are based on JWT and JW?, one of which is signed, one is encrypted. Reasonable specification, but requires the user to get right. That is risky from a security standpoint. Change Cargo's format from PASETO to JW? would be difficult to get right, and get people to get it right.
        • JWS=signed, JWE=encrypted
        • Doesn't allow an external process to get the JWT.
    • Jon: The token has to declare what kind it is. Bless a set of things we want to support. github-oidc:…, cargo will check that it is github-oidc and validate it (or azure or aws or asymmetric PASETO or whatever).
      • Would require some public definition of what their format is.
      • Don't want to favor specific providers. Drop the company name?
      • Arlo: Looks like should be able to validate.
        • Although enforcing the scope would only be able to say "it is scoped", but otherwise not able to say how it is scoped.
    • Does this need an RFC?
      • The RFC explicitly says it does not specify the format.
      • Likely will need a few RFCs (another for credential process?).
      • Arlo will look at writing some followups.

2023-02-21

  • Arlo: Authenticated registres and asymmetric tokens
    • Jacob and Arlo to do summary writeup to bring team up to speed
  • epage: check-in on https://github.com/rust-lang/rfcs/pull/3389
    • epage: main open question is on precedence (-Aclippy::all -Wclippy::doc_markdown and -D future-incompatible -A semicolon_in_expressions_from_macros)
      • Don't know about groups
      • Can't use levels as the above example shows different levels might need to be first
      • Priority?
        • Is high or low first?
        • Restrict it to u8 or even 4?
        • Start with priority: bool and open up later
          • If we plan for int, start there?
        • Highest priority overrides all, so comes last
        • High numbers is high priority as it also maps to command-line ordering
        • If priority is a number, default = 1
          • Allow -Aclippy::all first
        • To avoid ordering issues, sort within same priority
          • HashMap: could break fingerprinting?
        • Fine grained not needed
        • Really just "put this group first and then let me override"
      • List: in command-line order
      ​​​​[lints]
​​​​clippy = [
​​​​   { all = "Alow" },
​​​​   { doc_markdown = "Worn" },
​​​​]

      ​​​​[[lints.clippy.all]
​​​​level = "Alow"
​​​​[[lints.clippy.doc_markdown]
​​​​level = "Worn"
    • epage: lint vs rules
      • lint: don't speculate
      • rule: allow unstable experimentation with rustfmt being included so we can learn more
    • epage: some what the table inverted but the current shape makes more sense to me and allows for per-lint config
    • epage: while I suggest we don't support rustfmt out of the gate, I do wonder about following eslint's approach and calling these rules which would allow us to generalize in the future
    • epage: For another day: Personally, I find clippy's stance weird that there is no lint configuration. That feels more like an implementation detail and that most configuration is lint-specific.
    • Cargo lints!
      • Also clippy does some cargo lints and rustfmt hopes to do some formatting
      • Deprecations!
      • Configure future incompat here?
      • Would this encourage "rules"?

2023-02-14

2023-02-07

  • Publish timeout
    • staging sparse registry?
    • Only 1 case?
    • Was it even using sparse
    • Alex hit timeout when publishing 20
      • Gets longer while publishing (gets backed up)
      • Is it invalidation backing up?
      • Rate limiting? probably not
    • Noisy and unclear
      • Lots of layers
      • Lots of return paths
    • eric to consider doubling or more
  • Welcome Arlo and Scott as official team members.
  • Branch protections
    • Locking the master branch. There is some reason it is disabled, need to investigate what that is.
    • Conversations must be resolved
    • ehuss: will follow up on this
  • Any changes for the next Edition?
    • Reminders:
      • Edition changes are supposed to be automatiable
        • build.rs dependencies should always work and not cause noise
        • Workflow changes are not likely to be accepted
    • epage: Should we have a label for this?
      • Yes, let's add one. A-edition-2024 or A-edition-next?
        • Rolling label?
        • Label lifetime for when things are open
        • Milestone?
          • Not more than one on a Issue/PR
          • Or project board
      • weihanglo: we may also want labels lkes needs-lockfile-version-bump and needs-index-version-bump.
    • https://github.com/rust-lang/cargo/issues/10556 (weak feature syntax)
    • jon: build.rs directives (cargo: syntax)
      • Had decided not to use edition boundary
      • Maybe on edition we warn on old syntax?
      • Warnings would still be against edition spirit
    • jon: old RFC precise pre-release deps
    • jon: enable-features?
      • But ehuss' RFC
    • josh: deprecate default-features = false
      • josh: We still haven't written the needed RFC, let alone the code, though.
    • jacob: build.rs re-runs
      • Require people to opt-in to all rules, rather than doing it implicitly too much
      • Correct by default vs fast by default
      • If we had a helper…
      • build-std
        • or lang is considering global resource providing
    • weihanglo: run tests in parallel
      • https://github.com/rust-lang/cargo/issues/5609 or cargo-nextest
        • epage: imo cargo-nextest does a lot of things I don't think belongs at the runner level. I'm mapping out a libtest alternate that I think would replace most of nextest
          • Could we make libtest/cargo use jobserver?
          • Would that need an edition?
          • Handling of output is biggest issue
      • -Zpanic-abort-tests
    • Implicitly create single test binary
    • MSRV resolver…
      • In theory, is fine?
        • Not too worried about unvalidated input
      • Removes upgrade pressure…
      • Tell people on cargo update
      • UX might require an edition
      • --resolve-to to allow experimenting (including "msrv" placeholder)
  • epage: Path for MSRV

2023-01-31

  • Process for stale pull request assigned to someone else.
    • Check with the asignee (perhaps on a less noisy channel like Zulip), and if no response, reassign.
  • Handling the firehose of notifications from the cargo repo.
    • People should feel welcome to not watch the entire repo.
      • Can subscribe to individual labels if you want a subset.
    • As long as at least some people are following for moderation and triage.
  • Labeling pull requests?
    • The benefit:
      • People can subscribe to labels.
      • May help with encouraging better code structure that makes the labels more correct.
    • The process:
      • Fine to add auto-labels.
      • eric: Concern about adding too many labels, particularly for a small refactor that touches lots of files.
    • Task: Determine which labels are useful for auto-labeling.
    • Jon: Curious about what a focused week of cleanup could help. (Focused on just moving things around.)
      • Ed: Can move code into top-level modules as an initial step before moving into separate crates.
      • To have things be in more files and modules (which can make labeling easier).
      • Say auto-label A-cli for src/bin, docs, and src/cargo/util/command_prelude.rs
      • Maybe a sign we should move command_prelude.rs
    • Remove A-install-upgrade (done: removed)
    • A-features2 into A-features
      • Eric: There are some issues that are specific to the new behavior, and the A-features is a bit overwhelming. Unfortunately GitHub doesn't handle dependent labels.
    • Consolidate O- to A-os?
    • Scott may have a branch with some auto-label configurations.
      • 160 lines just for commands.
    • That might help guide things that need better code organization.
  • Process for PR that you are not confident in.
    • rfcbot fcp merge?
      • Not explicitly written down what is needed for FCP
      • one-way door vs two-way door, is it hard to undo, use an FCP
      • Person who starts FCP can't uncheck their own box
        • Can immediately file a concern
        • It's ready for FCP, but you don't quite have the time to check your own box, yet.
      • Not waiting for it to conclude the 10? day process
    • ping or r? another reviewer?
      • People should be totally fine with asking someone else for a review.
    • Eric to post PR suggesting FCP guidelines and we can discuss further
  • Handling support requests.
    • Redirect to a forum?
    • eric: if it takes just a few minutes, I just answer
    • jacob: always redirect to a forum.
    • eric: two kinds of concerns with just bouncing people:
      • It's not clear if this is a bug in Cargo or not
      • If the user was confused about something, that indicates that Cargo's error messages or documentation might be lacking, and could be improved.
    • Close and label for going back ("confused" label). Can go back later to those labels.
    • Posting a common text for which forums to try.
    • Give a quick answer, for more ask over here.
    • Write a template, and keep it in the repo somewhere. The "saved replies" feature is per-user.
  • Mentorship
    • Writing down the instructions for an E-easy task often takes just as long or longer than just doing the work.
    • For E-Medium or harder issues, the Cargo Team hasn't discussed what the expected design should be, which makes it harder to open them up for contributors.
    • Is there a better framework for mentorship?
    • underestimating potential contributors
      • Let them dig and figure out on their own. Don't write down the specific steps.
    • How much work is this expected?
      • Less scary if someone says it is "easy", then they should be able to expect to be able to tackle it.
    • Is there consensus on what it should be?
      • How much ambiguity is there? Don't tackle something, writing a PR, and then told to do something different or write an RFC.
    • Won't always predict accurately, but should be able to do it more often than not.
    • Do you help a contributor as they work?
    • Aiding in self discovery helps. If the code is navigatable, they can accomplish the task more easily.
    • Documenting the internals.
    • Tracing (logging) can help make the code more introspective. Switching to tracing crate can provide more detail and power.
  • jon: Stream building a crate that does .crate -> publish JSON -> index entry. Useful?
    • Pipeline is split. Make .crate. Create a JSON payload. They internally turn that into a database and index entry. Then Cargo downloads that.
    • The definitions are split in several places.
    • Live stream showing creating these definitions.
    • Would this be something valuable for Cargo and crates.io to take on?
    • JSON payload for publish.
      • crates_io crate already contains this, but doesn't contain the definitions or functions for translating to an index entry.
      • Sharing with registries may be difficult, because they have unique requirements. Example: crates.io doesn't directly translate publish API -> index. It has an intermediate database type with its own diesel tables and such. Its JSON deserialized type is substantially different from Cargo's (it uses newtypes, custom deserializers, built-in validation, etc).
    • The conversion from that payload to an index entry.
    • arlo: Wish the registry would just extract the information from the .crate file.
    • jon: A good learning experience regardless if it gets used in cargo
    • Some interdependencies may be difficult to disentangle, like Cargo's use of InternedString.
    • Jon to open discussion on Zulip for more.

2023-01-24

  • Reminder: Release 1.67 is this week
  • sparse registry blog post
    • Interested in an inside blog post to get people to test early
    • Post to users and internals and reddit
    • When we turn it on by default: we should post that on the main blog.rust-lang.org
    • Credit for the various organizations involved, GitHub donating for hosting the index, Amazon for the sparse index (and others), the various people that worked on implementing it.
      • "We are deeply grateful for GitHub for hosting the obscene amount of resources the Git index uses…"
      • Don't call attention to something they don't want attention to.
      • Good cooperative press
    • Unfortunately opt-in requires -Z flag on stable.
      • eric: Document process unstable configuration options.
    • forum for feedback:
      • Try to keep the friction low, users may not have accounts on various services.
      • Report bug in GitHub
      • Post experiences in internals.
  • epage and Scott: CARGO_WORKSPACE_ROOT: https://github.com/rust-lang/cargo/issues/3946
    • expectest, insta, snapbox, "self-modifying code"-style codegen could benefit
    • concern: exposing to production code for dev purposes might be a trap for people writing published code
    • concern: for registry crates, using the registry path wouldn't expose anything relevant outside of CARGO_MANIFEST_DIR. The only thing would be the changing path of the lock file but that is irrelevant for building [lib]s.
    • concern: for registry crates, using the final artifact's workspace root invert relationships
    • For development-only tasks like copying files, the user can use .. to their hearts content; this would just be a convenience for them
    • A way to re-frame this is "root for making file! be absolute" but what to call it ("CARGO_COMPILE_ROOT"), removing behavior ambiguity
      • In contrast to current_dir which is always pkg.root
    • Options
      • Only expose to tests
        • Precedence: CARGO_TARGET_TMPDIR
          • This has problems with libraries that want to access it (like snapbox, cargo-test-support).
      • Make a runtime variable for cargo test (and maybe cargo run)
        • First one that isn't available through env!? How do we document it to be clear
        • Lots of commands to patch up to make this work?
        • cargo nextest wouldn't work along with directly running
        • Instrumenting coverage might run it directly
    • file! meant for debug only
    • Alt: macro for absolute path?
      • Seems weird to make this only work for tests
      • CARGO_WORKSPACE_ROOT in config
      • Shell out to cargo metadata
      • CARGO_TARGET_DIR: not reliable, people can move it
    • For production, reproducible build people matter
    • Name
      • CARGO_WORKSPACE_ROOT
        • When documented for tests, the scope makes it clear
        • Breaks if we expand it
        • Sounds overly broad, relying too much on docs?
      • CARGO_DEV_WORKSPACE_ROOT
        • Clear enough?
      • CARGO_COMPILE_ROOT
        • Too broad of name
    • Jon: Not necessarily an absolute path, but a relative path (from CWD) to access the file pointed to at file!.
      • Reduces path lengths. Fewer objections from placing absolute paths into a program. Already an issue for CARGO_MANIFEST_DIR.
      • To emphasize this is local.
    • Instead of "Here's a path you can embed in your binary", look at what people are doing with it, with a minimally error-prone fashion. They want a path they can open, not necessarily an absolute path.
    • Fix file!?
      • str
      • undefined value. May need some discussion with the libs team to figure out if and how to define what file! means. Perhaps not critical for the intended purpose of "dev" environments.
  • Meeting process.
    • People may need more time to understand and digest an issue. Ideas for improvements?
    • Have the agenda available earlier
      • Topics added after some threshold will be put in next meeting.
      • Notification for meeting topics due in Zulip
        • Threads for things to add
      • Agenda generator scripts
      • Nominating an issue ensures that the issue will be on the agenda.
    • Start of the meeting is dedicated to reading the document, people leave comments/questions directly in it, and then spend the rest of the meeting discussing those comments.
    • Spend 3-5 minutes in silence for people to read.
      • How to indicated when everyone is ready?
      • Jitsi raise hand or poll
    • more focus on writing meeting notes for those that would prefer to see things written down?
      • Don't expend too much effort, get the major points.
    • jon: Unlikely to get people to prepare written summaries for issues ahead of time.

2023-01-17

  • jon: https://github.com/rust-lang/cargo/issues/10094
    • Interested in moving this forward
    • https://github.com/rust-lang/cargo/pull/10780 an earlier attempt to address this
    • PATH: if [env] sets path, should that affect where subcommands are found?
      • Proposed: it will use the new PATH for locating the subcommand.
    • CARGO_HOME: If [env] sets this, should the subcommand use the new CARGO_HOME?
      • Proposed: Forbid to set CARGO_* flags.
      • This seems like the correct behavior.
      • CARGO_WORKSPACE_ROOT being set in [env]
      • josh: at the very least, disallow CARGO_HOME
    • Handling of global flags?
      • How can subcommands tie into the overarching cargo invocation?
        • The versions might not match.
        • Global CLI flags don't get passed through.
    • Should this be added with an opt-in?
      • Would this actually break someone?
      • josh: flip the default, early in a nightly cycle, post a note about it, respond to any issues.
    • use cases
      • binutils
      • cargo in different directory, wanting to pull in old env
        • Wouldn't help with rest of config
      • Ensuring that the subcommand can be found by setting PATH to find it.
    • Re-framing: third-party subcommands are acting like first-party, they don't have env set on them. The problem is how to access the config, especially [ENV].
      • This is an implicit solution for a sub-set of config
      • We should do a rough enumeration of other solutions to make sure we are ok with this
      • Should we set_env at process start?
    • What happens for a recursive cargo invocation?
    • Should fix cargo to read-through config env rather than std::env::var
    • Weihang: Can information be embedded in the ELF executables to indicate the behavior they want?
    • jon: When executing a custom subcommand, don't propagate CARGO_* from [env].
    • alternative: exposing config to third-party subcommands so they can fetch the [env] table themselves.
    • dogfooding with first-party external subcommands would help
    • Start from the known problems to see if it clarifies the problem for us
    • don't allow setting CARGO_HOME
      • It seems like the wrong behavior, affecting recursive invocations
      • Want more control on this
      • disallow until we decide on the semantics
      • an alternative is to have cargo switch its home directory (at the start)
    • fix the calls that directly read from std::env to read from config instead
    • consider merging [env] into the config snapshot
    • Evaluate if further env variables need to be blocked from config
  • add gitlab.com ssh host key https://github.com/rust-lang/cargo/pull/11564
    • policy for adding new keys
    • We added github.com to accommodate people who have insteadOf.
    • Consider removing builtin keys after changing default sparse index.
  • arlo: stabilize sparse registry: https://github.com/rust-lang/cargo/pull/11224
    • do we need to wait for all FCPs? Risk missing 1.68 stable.
    • May be difficult to land before beta branch this weekend
    • Testing of 403 response, how to configure cloudfront

2023-01-10

  • FCP review reminder

    • Add default-features to TomlWorkspaceDependency
      https://github.com/rust-lang/cargo/pull/11409#issuecomment-1374225756
      • Still additive
      • When workspace disables and package defaults, is it confusing?
        • This was RFC specified behavior
        • Most common case is <dep>.workspace = true so shouldn't be a problem
      • When workspace disables and package enables
        • Seems reasonable
    • Omit checksum verification for local git dependencies
      https://github.com/rust-lang/cargo/pull/11188#issuecomment-1317837836
      • Special case for cargo local-registry.
      • jon: Concern that cargo won't pick up changes. Updating a git dependency (whose version does not change). Then re-vendor. Cargo's src cache wouldn't get updated.
        • The problem is that Cargo.lock does not contain a checksum for git dependencies. This is using source replacement to a local-registry. The code for registries requires a checksum, but the lock file does not have one.
        • weihang to follow up to see if the cache handles that
        • jon thought there is already a special case to ignore checksums if they aren't in Cargo.lock
        • https://github.com/rust-lang/cargo/issues/10071 issue about checksum changing

  • Can't use unstable feature in ~/.cargo/config without enabling it in Cargo.toml

    • https://github.com/rust-lang/cargo/issues/9926
    • eric: In the past we have just done -Z flags for unstable things in config.toml.
    • https://github.com/rust-lang/cargo/issues/11552
    • First issue's use case has already been stabilized.
    • Second one is profile.rustflags.
    • eric: I don't think we need a general purpose way of enabling cargo-features everywhere.
    • scott: Can we warn when config isn't being used.
      • eric: That can make it difficult for experimenting with nightly features. Using a stable channel would end up warning all the time, which would be noisy.
    • Move forward with -Z flag to enable profile rustflags in config.

  • Cargo Roadmap Take 2

  • cargo update -w and consistency: https://github.com/rust-lang/cargo/issues/11554

    • Using -w for any other meaning might be difficult now

  • Lookback at what has improved?

  • Creating a Zulip stream for meeting discussions

    • Consider a contributors? team for tracking the members

  • Stabilize showing cargo fix for warnings

    • Has been in nightly since September
    • Haven't received any particular feedback about people having trouble.
    • Would be nice to have a better user experience with cargo fix, but doesn't necesarrily need to block stabilizing this particular thing.

2023-01-03

  • Build script errors

  • Question: move to a new hackmd doc?

    • Will rotate in a few months, in order to make it easier to search recent stuff.

  • Regression with multiple registries config: https://github.com/rust-lang/cargo/issues/11524

    • Part of the new registry authentication RFC (reference) changed the behavior so that the following is an error:

      ​​​​​​[registries.foo]
​​​​​​index = "https://github.com/rust-lang/crates.io-index.git"

​​​​​​[registries.bar]
​​​​​​index = "https://github.com/rust-lang/crates.io-index.git"

      The reason is that in some cases Cargo only knows the registry URL. However, the credential information is stored based on the registry name. The RFC changed it so that it is an error to have multiple registries with the same URL so that when Cargo looks up which credentials to fetch, it only gets one answer.

      • Searching credentials for both named registries. Deterministically pick one, print a message.
      • Only knows registry URL for transitive dependencies (cross-registry dependencies), --index CLI argument.
      • May want separate credentials for the same registry. "Here's my credentials for this team, here's for another team."
        • Independent username per registry.
      • Concern about config merging could bring multiple definitions together unexpectedly.
      • eric: propose to wont-fix for now, unless there is more feedback
      • make sure this is noted in the changelog, relnotes

  • credentials.toml default filename: https://github.com/rust-lang/cargo/issues/11509

    • Cargo supports both credentials and credentials.toml. If credentials.toml exists, then cargo login will write to that filename. Otherwise cargo login will create a new file named credentials. The issue is asking if it should instead create credentials.toml when both files do not exist?
    • Reading credentials.toml was added in 1.39
    • josh: Hypothetically could transition in the future to not reading credentials, but seems low benefit. Could theoretically associate that with new fields (such as asymmetric tokens) that old Cargo doesn't support?

  • sparse-index: waiting on documentation, then will do FCP. Invalidation seems to be working.

    • Separating the FCP process (to avoid all but 2 slowing it down).
Last changed by 
 
rust-cargo-team
Cargo team for the Rust Programming Language
0
137

Read more

2025 All Hands Topics

Discuss https://hackmd.io/-jpsOWYoRACRNF3G80Hkuw?edit

Apr 2, 2025
Cargo team meeting notes

https://meet.jit.si/CargoTeamMeeting

Apr 1, 2025
Build Times: Cargo Perpsective

Notes to prepare for 2025H1 Project Goal planning

Mar 19, 2025
This Development-cycle in Cargo: 1.86

This is a summary of what has been happening around Cargo development for the last 6 weeks which is approximately the merge window for Rust 1.84.

Feb 25, 2025
Read more from rust-cargo-team
Published on HackMD