Try   HackMD

Cargo team meeting notes 2022 H2

2022-12-27

CANCELLED

2022-12-20

CANCELLED

2022-12-13

  • @weihanglo: Reasons for rebuilding
    • https://github.com/rust-lang/cargo/pull/11407
    • If Dirty messages were only displayed with --verbose, then they can be difficult to find in the sea of Running messages.
      • ehuss: suggestion that Dirty is only displayed with --verbose, it shouldn't be too difficult to sift through the output when doing an investigation.
    • Also questions about how to display it in terms of multiple lines.
      • "From", "To", "Because" are printed as own status, rather than part of the "Dirty" status
      • Current PR has removed some of the From/To lines from the original proposal. Also, those aren't terribly useful due to the way fingerprints work (they often don't show up).
      • The screenshotted "because" also seems overly verbose, making it harder to scan for important information
      • Make additional information a separate PR to not block the current one
      • Verbose vs log?
        • log us hard to discover
        • "start with trace and prune down"
          • "Common to know" vs "trace events"?
        • Rebuild information is always too late…
        • Always log to file?
          • Uncertainty about what kind of performance impact that could have.
          • env logger might not work well for this
          • tracing might help
        • Log meta information so a command can print what happened
  • @weihanglo: Moving contributing docs into rustdoc
  • Holiday meeting schedule?
    • ehuss gone on 27th
    • jon out for next two weeks
    • Jacob is traveling
    • Ed will be watching kid out of school
    • Only skip 20th and 27th, meet on 3rd
    • And add Jon and Arlo to calendar event?
      • Contact Jacob
  • Sparse registry
    • ehuss going to add labels for cross-team approval
    • Split up PR
    • crates.io outage: they shut off invalidation to try to alleviate it, but the problem is more fundamental to how crates.io processes one thing at a time.
  • Proxy HTTP auth
    • https://github.com/rust-lang/cargo/pull/11433
    • Can the auth be part of http_proxy env var?
    • concern about including password in config
    • verifying that http_proxy works with username/password
    • narrowing the scope, finding out what auth scheme this user specifically wants
    • eric will respond
  • Josh brought up "interop" proposal, and the potential long-term future involvement with cargo https://github.com/rust-lang/rust/pull/105586#issuecomment-1349332850
  • Jacob did some investigation into crate compression, which ended up being predicated on a misunderstanding of the AWS bill, so not being pursued.

2022-12-06

@weihanglo: Unavailable. Conflict with other meeting

  • epage: https://github.com/rust-lang/cargo/pull/11312#

    • cargo:warning+=second line syntax support (epage is in favor)
      • Running things in parallel?
      • Print as unit with locked stdio would fix it
      • Could instead escape newlines
        • Breaking change?
      • Guarentee warning ordering (1 warning with 5 lines vs 5 warnings)
      • Need to precisely specify how we modify the line (prefix on first line)
        • Only guarentee alignment for lines 2..?
        • No prefix for multiline?
    • Should cargo:error= automatically cause the build script to error or not (previously, we said it should)
      • Make it clear in the documentation under which circumstances error should be used (under no circumstances can it proceed)
    • How to add new build.rs directives due to cargo:KEY=VALUE support
      • Assume what we add is unlikely to be used (minor break, rather than major break)
        • cargo:error=
        • cargo:metadata=KEY=VALUE
        • Do in new edition
          • Dependencies that aren't latest edition
          • Tell build script what the edition is: $CARGO_EDITION
          • But crates in package can be different editions
          • But we can just use top level for build.rs, scoping it to directives, rather than all code-gen
          • build.rs is tied to [package], not target, so can assume a single edition
        • Have directive to opt-in
        • cargo::
      • Can we add new namespace?

  • Question about sparse registry feedback. Eric to follup up with the summary.

  • arlosi: registry-auth without asymmetric tokens: zulip

    • Should we allow authenticated registries without asymmetric tokens?
    • Delaying the rollout of authenticated registries waiting for asymmetric tokens.
    • Time of rollout may be the primary concern.
    • Picking a timeline for re-evaluation of whether or not to require asymmetric.
    • Ask to try implementing asymmetric tokens. Other implementors shoehorn into things that are not designed for asymmetric tokens.
    • Current solutions can have worse security implications.
    • If asymmetric support was available in 6 months, how many implementors would still want symmetric tokens?
    • Can the work be split into smaller pieces to make the review/merge/implementation go more smoothly?
    • Having registry implementations would be very valuable for moving towards stabilization.
    • Some confusion due to missing information in the RFC about the details (headers, how to have a "dumb" implementation, etc.).
    • Start on the documentation in parallel with the implementation.
      • A specification for how to implement the protocol.
    • Jacob needs questions to be asked publicly.
    • Can we ask registry implementors (possibly the same set of people that @ehuss asked about sparse?)
      • "We want to do asymmetric authentication, this is our proposal, would this work for you, or would you rather have a different asymmetric implementation?"

2022-11-29

  • @weihanglo: Error when cargo update --precise or --aggressive without -p flag
    • https://github.com/rust-lang/cargo/pull/11349
    • Hit stable in 1.65.
    • Has the warning period been long enough?
    • Current behavior: It ignores the --precise field and updates everything.
    • --agressive is a bit different from --precise. --precise takes a field, which has no meaning without -p. --aggressive is the "default" behavior, so adding it really doesn't cause any harm.
      • jon: Should --aggressive even be a warning?
    • eric: would prefer to just make --precise an error now.
      • --aggressive could be left as a warning
  • @weihanglo: Aware of compression ratio for unpack size limit
  • Josh: Q: What would it take to support other compression algorithms in .crate files?
    • Probably add new copies in the other format (with different extension, such as .crate.xz or .crate2 or .cr8 or whatever).
  • Cloudfront invalidation:
    • Higher retries seems to be working.
    • Current TTL is 5min. Can raise more.
    • arlo: probably wouldn't want to go more than 1 day.
    • 1 hour might be a good balance in the short term.
  • crates.io renormalization:
    • May cause a large diff to the index
    • May be the first commit after a squash.
      • jacob: recommend doing the commit before the squash (to avoid fetching both sets of data)
      • Can also just reset the ref to the new normalization commit so that it becomes the first commit.
    • jon: consider keeping the previous branch a little longer (~2 weeks)
  • RFC review: https://hackmd.io/@rust-cargo-team/BJp45vnXo
    • https://github.com/rust-lang/rfcs/pull/3177 Patch dependencies using unidiff patchfiles
      • A simple patch can work easily with updating dependencies (assuming the patch still applies)
      • Depending on a git repo for a patch can be somewhat heavy (needs to download the whole repo, etc.).
      • Patches using git were aimed for a workflow where once you have a git clone, you're one step away from posting a PR upstream. This would side-step that.
      • guardrails for the format of the patch (minimum number of lines, offset, fuzz, etc.)
        • Supporting other formats in the future?
        • git extensions to unidiff (rename x->y, then change this one line, binary files, etc.)
      • patching Cargo.toml could change the dependency requirements, requiring re-running the resolver.
        • Will require multiple passes.
          • Run resolver without the patches, apply the patches as new path-dependencies, run resolver again, that may change dependencies that now match other patches, which need to be applied, run resolver again, etc.
        • jon: put a restriction in the first iteration that doesn't allow patching Cargo.toml, and possibly add support for that later. It may add significant complexity to cargo, compared to the rest of this RFC.
      • Josh (and hopefully others 😊) will follow up with comments on some of the particulars. Overall in agreement with the general concept.

2022-11-22

  • reminder: https://github.com/rust-lang/cargo/pull/11188#issuecomment-1317837806 (Omit checksum verification for local git dependencies) is waiting for FCP checkmarks
  • eric: Nominate to close #3686: Alternative to RUSTFLAGS that allows spaces within arguments.
    • This is now handled with CARGO_ENCODED_RUSTFLAGS and there is also --config.
    • No objections
  • eric: Appetite for renaming "Cargo target"?
    • question is about what to use.
      • "artifact"
      • Something related to cargo/crate/manifest/shipping
    • --all-targets will probably need to be renamed
    • eric will come back with some ideas
  • Continue RFC review: https://hackmd.io/@rust-cargo-team/BJp45vnXo/edit
    • https://github.com/rust-lang/rfcs/pull/3263
      • Precise pre-release deps
      • Breaking changes between certain steps (like alpha->beta) causes problems. The SemVer spec implies "anything goes" during pre-releases.
      • There are situations where things are likely to break (like during the alpha phase), but less likely in a later phase (like "rc" release candidates).
      • If you really want equal, you can write equal.
        • cargo add could add pre-releases with an equal by default
      • dtolnay's suggestion was
        • ^1.0.0-beta.1 works as it does today.
        • 1.0.0-beta.1 is treated as =1.0.0-beta.1
      • resolver="3" is a global thing which might not be the right way to opt-in to this.
        • But an edition might work since it is local to the Cargo.toml file.
      • Another option is: not specifying an operator is an error, and force the user to pick one. (in an edition)
        • Backwards compatible, less ambiguous
        • Perhaps sometime in the next year, can start a warning before the edition release to get people moving.
      • Will recommending = make it too difficult to update dependencies due to small differences?
        • Users will likely want a way to force an update (override the =), but we currently don't support that.
      • Should we (continue to) recommend that packages be a pre-release if any dependency is a pre-release?
        • Possibly adding a speed bump to cargo publish, with an option to override.
      • eric: Wondering about having built-in capabilities for discovering new pre-release versions. cargo outdated would provide that (and also cargo upgarde's dry-run?). Would that maybe help alleviate the difficulty of using = pre-releases? Because otherwise, the existence of new pre-releases isn't very visible.
      • Maven: snapshots. Snapshots can be mutated.
  • Cloudfront invalidation
    • Now in production again with more retries
    • Long-term known issue
    • Concern about moving to a third registry format (merkle), what to do with the serving of sparse registries? Can we just remove the invalidation (since it is mostly important to publishing)?
      • Can also switch to git index for publishing if using an old version or whatever.
    • Third-party registries will definitely prefer the current sparse-registry. Merkle may be unlikely something that they will want to (or can) use.
    • config.json could be a mechanism for opting-in to the merkle structure. Branching and depth factors could be parameters in the config.json.

2022-11-15

  • Beta backport https://github.com/rust-lang/cargo/pull/11366
    • Fix for safe.directory
  • RFCs: https://hackmd.io/@rust-cargo-team/BJp45vnXo/edit
    • Per-target dependencies (two RFCs):
      • https://github.com/rust-lang/rfcs/pull/2887
        • Allows specifying dependencies directly in target tables like [[bin]]
      • https://github.com/rust-lang/rfcs/pull/3020
        • Allows specifying features directly in target tables like [[bin]], which can then be used with optional dependencies to implement the same behavior as 2887.
        • That is not as intuitive, but covers more use cases, and would be easier to implement, and have a smaller impact.
        • Can't repupose required-features, since it has a different, existing use-case of "don't build this if not asked for".
          • josh: some people think required-features enables features.
          • Is there any way we can avoid having two fields? Like in an edition or something?
          • enable-features = […] force enables the given features when the binary is included.
          • force-enable-features = true would change the meaning of required-features
          • Some precedent: select vs depends in the Linux kernel's Kconfig language
          • Some way to build, but without a binary that enables features you don't want.
          • Instead of generating an error with --bin and required-features, change it so that it enables the features instead of generating an error.
            • The binaries are only built if expecitly requested (--bin) or the required-features are enabld otherwise.
          • artifact dependencies counts as an explicit request
          • default-install-features = […]
          • cargo run, there is an implicit binary associated with that
          • cargo test and cargo bench can build binaries, and there isn't an explicit choice for those binaries being given
        • Is there a concern about using this for optional dependencies will make those optional dependencies visible to all targets (like the lib)?
        • Something like cargo test will enable additional features for binaries, which isn't the set of features that are default for the library. cargo test --lib would be a workaround, but may not be clear to users
        • Improving error messages and diagnostics. For example cargo install saying what is excluded due to required-features.
          • The error message "error: target myexample requires the features: myfeature, another" could suggest using enable-features instead. (That error message is not good, and we should possibly continue to think about just removing it and imply --features, but that seems somewhat independent.)

2022-11-08

  • Backport cargo-remove GC to 1.66 beta?
  • Scott: Add default-feauresto workspace dependencies derived from issue #11329
    • Issue: a bug that foo = {workspace=true, default-features=false} is not generating a warning about specifying default-features=false. Scott is working on a fix for that.
    • decision: allow default-features=false, and if you inherit a workspace dependency, the default-features needs to match what is specified in [workspace.dependencies], and if it doesn't, warn.
      • Should have test cases for both warning scenarios (workspace-dep has default-features=false and inheritance of it doesn't, and vice versa).
  • libgit2 performance issue with delta resolution
    • eric identified threading support as a huge performance hit, discussed various alternatives for that.
  • sparse registry status
    • Cloudfront invalidation: Turned on in prod yesterday, but caused issues.
      • Alro working on invalidation issues
      • Jacob started discussion about Fastly with infra
      • Invalidations are causing 503 errors
    • publish issues
      • Blocking is getting some testing now, but there might be some issues.
      • concern: crates.io regularly has a 4 minute outage (daily plus on every deploy)
        • Uncertain how cargo's timeout should deal with that.
        • Would be good to investigate why their git clone is so slow.
    • Need positive confirmation from crates.io/infra that they are ready to support it.
    • Stabilization will be opt-in at first.
      • Turn it on in nightly-only by default to force getting more exposure and testing and feedback.
    • Eric sent out requests for feedback from registry implementers. Feedback has been coming in, and eric will try to summarize it.

2022-11-01

2022-10-25

epage: unavailable

  • @weihanglo: interaction between --package and feature resolve v2
  • @arlosi: Should we add --token to other subcommands?
  • @arlosi: What's still needed for merging the auth support into nightly
    • https://github.com/rust-lang/cargo/pull/10592
    • no specific blockers, just needs review time
    • discussed relationship with asymmetric tokens PR and the window between them. Should be fine to land on nightly for experimentation.
    • Josh: don't want to every stabilize support for symmetric shared-secret (non-asymmetric) tokens with sparse registry, but it's fine if they go in as nightly features sequentially
    • Have not brought up asymmetric tokens with crates.io. Would like to start those conversations. May need an RFC due to the details.
      • Jacob to follow up with them to start understanding how we can get started.
  • @weihanglo: Cannot build vendored git dependency if it inherits info from a workspace
    • https://github.com/rust-lang/cargo/issues/11192
    • Rewrite Cargo.toml when vendoring to fix things
    • Jon: Essentially do cargo package and extract the result.
      • May be a little slower.
    • Scott to look into the change to see how difficult it will be.
    • Josh: Consider separate PRs, one for the process of gathering the package data, a separate for the vendor support.
  • @jonhoo: $CARGO when Cargo is used as a library
    • https://github.com/rust-lang/cargo/issues/10119
    • Cargo uses current_exe() which is the wrong thing from a library.
    • Some build scripts don't use $CARGO and blindly call cargo
    • Proposal: Don't change $CARGO if it is already set. This allows the library user to override it.
    • Josh: Concern about using it to abuse it by making it a wrapper around cargo to change its behavior.
    • Josh: Alternative, adjusting PATH so that everything just calls the right thing. Make a separate directory that just contains cargo (so as not to affect other things). Tradeoff: could make existing build scripts work that way, but would be more complex.
      • Jon: Concern about fragility of changing PATH, and $CARGO is already a well-known method.
    • Subcommands: A subcommand that uses cargo as a library, when that subcommand does something (like running a build), $CARGO points to cargo-mysubcommand, and the build script will run the wrong thing.
      • Concern about this changing behavior. The outer cargo sets $CARGO, the subcommand then performs some operation with $CARGO set to itself. With this change, it will now point ot the outer cargo (~/.cargo/bin/cargo).
      • Adding a new environment variable could help with subcommands that don't get updated on user systems.
      • eric: Not certain there is enough of a problem here to justify adding more environment variables.
    • cargo-the-binary could override $CARGO to itself (like it does today)
    • Jon to open a PR implementing the "don't override" option.
    • Jon to open a discussion for a separate PR to set CARGO_BIKESHED (for some value of BIKESHED)

2022-10-18

  • Merging publish delay when?
    • Now seems fine, no need to wait.
    • Sparse may need to wait till the next branch.
  • Muscraft: Is suggesting cargo fix something that should be added now?
    • https://github.com/rust-lang/cargo/pull/10989
    • Adding the suggestion could cause lots of new issues to be filed
    • Could drain already low team resources
    • cargo fix workflow can be a little bumpy. Particularly, --allow-dirty is a roadbump.
      • Should the warning tell you to pass --allow-dirty, or should --allow-dirty be the default?
      • Interactive mode could be a nicer workflow.
      • Are we OK with the number of issues it could create?
    • warning: `foo` (lib) generated 1 warning (run `cargo fix --lib -p foo` to apply 1 suggestion)
    • What about only suggesting on nightly?
      • Use input from issues to drive what users are having problems with. Iterate and improve based on that.
    • Verbosity of the suggestion:
      • Showing a summary at the end, versus one per crate.
      • Is just running cargo fix (without flags) what users would normally run?
      • With one-per-crate, can show the suggestion. If a final summary, and there is a failure, then it won't get displayed.
    • Let's try to run with the current PR, on nightly only, and see what kind of feedback comes in.
  • Migrating home to rust-lang org
  • eric: Are changes to dependencies that cause new lints to fire a semver compatible change? I think the answer is "yes", but want more feedback.
    • Example: Removing unsafe from a function can cause unused_unsafe lint to fire. If the package has deny(unused_unsafe), it will then fail to compile.
      • Some other examples would be deprecated and must_use, though I think those almost certainly should be allowed.
    • There are lots of lints, and some have some subtle nuances that might not be obvious. Should we consider it on a lint-by-lint basis, or is it ok to just make a blanket statement?
    • epage: See also deprecations (https://github.com/rust-lang/cargo/pull/10742)
    • Some very popular crates may be more sensitive to breaking dependents.
    • cap-lints makes this not a breaking change
    • Macros are a little complicated, since they suppress some lints (don't remember the rules for which ones).
    • Will move forward with the assumption that introducing any new lint is OK. May add some wording around the potential risks so that library authors can understand them.
  • epage: Should we clear out the RFC backlog?
    • https://github.com/rust-lang/rfcs/labels/T-cargo
    • josh: probably don't systematically close them (don't be stale bot). But we can work on them in smaller batches. Use the meetings to discuss them.
    • Some of the RFCs look very useful.
    • Hard to say just based on age what to do with some of these.
    • Deferring blocked RFCs. Can reopen at some point in the future.
    • T-cargo "issues" should probably be closed. Write a stock paragraph to indicate that this isn't the place for proposing changes.
    • ed will look at summarizing the current status
    • Definitely move to close all the "issues". Think about how to redirect them?

2022-10-11

  • Muscraft, @weihanglo: moving forward with nested workspaces
    • Does it solve the correct problems?
    • Could it solve this issue with rust analyzer needing a Cargo.lock for /library in the rust repo
    • Is it worth the review time?
    • Moving forward: Post on internals for feedback
      • Talk to people with large workspaces, and see what the workflows are like and what issues they have.
    • What is the value for a cargo workspace?
      • It is awkward to run tests for multiple packages.
      • Sharing target directory is awkward. Locking doesn't get shared, either.
      • Lower overhead (actual, psychological) for evolving package structure of cargo
  • jonhoo: should cargo move built-in subcommands to separate binaries?
    • One big binary
      • Rewrite to merge into cargo
      • Native subcommand get access to internal APIs
      • No dog fooding improving things for both public API and contributors
    • Make all binaries in cargo package
      • Use public API
      • Faster cargo-the-library builds
      • Longer link times
      • Larger binary sizes in total
      • Adds a fork/exec overhead to more commands
        • Maybe keep plumbing internal
    • Make all binaries in cargo separate packages
      • Moving shims out doesn't do much, we need to move ops
      • Separate package allows still exposing the ops
      • Isolates tests (faster build/test times when working on a subset)
    • Distributing external commands
      • Have to update rustup on each command being added
    • If we just rely on PATH, there will be a change in behavior
    • package-as-namespace: https://github.com/rust-lang/rfcs/pull/3243#
    • This will add half a gig to rustup
      • Will building only what you need make much of a difference
      • Duplicate clap, etc
      • check, build, etc would duplicate the core of things
    • Linking against cargo is a bad idea
      • Couples them to a specific parser version
      • Always breaking the API
      • Maintenance burden
      • People using unstable features
    • Split out crates
      • Remove cargo metadata overhead
        • fork/exec
        • openssl loading
        • parse full workspace
        • parse config
        • serialize/deserialize
      • CLI processing, including selecting workspace members
      • Manifest editing
    • Unstable crates?
      • As long as we are clear on stability
      • cargo-semver-checks will be a big help for users
      • "I'd rather them be pub and break on every release than copy/paste"
    • Compatibility issues?
      • For bins, we already handle forward / backwards compatibility for on-disk
      • Could cargo-as-a-lib expose things to alow people to break it
    • Baby step
      • Clean up modules which is blocked on movign contrib docs so we can use intra-doc links and update them when broken

2022-10-04

  • epage: wait on publish: https://github.com/rust-lang/cargo/issues/9507
    • https://github.com/rust-lang/cargo/pull/11062 — Current PR
    • Wait before publish or after?
    • "good defaults"
    • Record publishes that happened
      • Block on these by having the resolver check these and report special error
      • Stateful errors are hard for users to deal with
      • Instead, on error check the global state
        • Also cargo-update, cargo-upgrade
        • Don't need to introspect the error, just assume it is
      • Can we check crates.io?
        • no: crates.io doesn't know when GitHub is updated any faster than cargo does
      • Have publish check the list
    • Block in crates.io?
      • Heroku won't let us, built-in timeout
      • Websocket or retry with long-poll could
    • Can we do immediate invalidation
      • CloudFront isn't optimized for immediate
    • Provide porcelain for this
    • We could easily check before if they are path dependencies
    • Option to control whether or not to wait for publish to finish.
      • Concern it will take time to figure out the exact details of how it works, why someone would decide to use it, etc.
    • Check source's status API to see if there is a major problem to shortcircuit the polling
    • Timeout?
      • Currently, 300 seconds
    • Tell user explicitly about ctrl-c
    • Contact axum to see how this affects them
    • Jacob reached out to large ones, thought they were careful with waits, but weren't but doing what it does now
    • Plan
      • Move forward with PR + ctrl-c message + unstable config
      • Allow as much window as possible before stable
      • Post in TWiR for testing
      • Reach out to big projects
  • Status of sparse registry
  • Path sanitization

2022-09-27

  • RFC updates
  • style team established, may have some intersection with Cargo, rustfmt evolving based on edition, formatting of Cargo.toml itself
  • RFC 3287: Add native code coverage support
    • https://github.com/rust-lang/rfcs/pull/3287
    • Scott: Native would make the ecosystem better versus third-party. Would be used more readily. Does incur burden
    • Ed: What would we expect developers to be doing (workflow)?
      • We make some workflows easy (fmt, clippy), this is another one
      • Include survey of coverage in other ecosystems. What lessons can we learn from them? Don't design into corner
    • This is LLVM-backend only
      • Theoretically the data could be collected through other mechanisms, and processed with the LLVM tools, but that doesn't exists (AFAIK).
      • LLVM_PROFILE_FILE
    • Eric: comments from nick are compelling in terms of the applicability of coverage.
    • Coverage thresholds can cause difficulties / drive wrong behavior
    • Coverage is to give insight to author/reviewer
    • Compatibility issues?
      • Report is considered transient, meant for that cargo version the test was run from
        • Like another report tool…
      • But all of the useful flags
        • Put rest behind -Z
      • Stabilizing some simpler data format (for example, integrating with IDE)
      • Generate file only, no html output, leave that to another tool
        • Too weak?
        • All CI needs
        • Lot of users just look at fist pass of information
      • Uploading raw data to codecov.io or similar tools; generating a report ourselves may not be the common case
    • RUSTFLAGS for roots: https://github.com/rust-lang/rfcs/pull/3310
      • Ed feels like first-class support for code coverage is still worth it, to reduce the complexity and make it more accessible
    • Focus on setting coverage for roots?
      • Performance
      • Filtering out the information that is relevant
    • Focus on rustflags so this can be iterated on externally first
      • Use cargo-nextest as a testbed
    • RUSTFLAGS passing
      • more control via config
        • (preference for config over CLI)
      • Or in profile
        • [profile.codecov]…
        • Have a built-in profile for coverage.
        • Josh: doesn't particularly want to see rustflags in Cargo.toml (prefers config), not in profile

2022-09-20

2022-09-13

  • Blocking on publish: https://github.com/rust-lang/cargo/pull/11062
    • Config/flag and name? Ok for blocking to insta-stable but not the config/flag?
    • Timeout flag that can be overloaded as "don't wait"
    • Config gives env variabler automatically
      • Supports command-line
      • Can emulate project config
    • Test: put in git repo first
    • Test: just do it right
    • PR to update the match case to not be no-op, sharing as much code as possible
    • Workaround
      • Ctrl-c
      • Timeout
      • We feel ctrl-c is good enough
    • opt-in vs opt-out
      • Start as a config that people can opt-in to and then switch to opt-out
      • But then we need to name things
      • Blocker for stablization of sparse registry
      • The earlier we get this, the more that can use sparse registry
      • Is risk high enough to warrant phased stabization?
      • Will people give us feedback?
      • If we miss the next cut to beta, then that gives us extra time for feedback (targeting Dec 15th)
      • TWiR testing
  • Nested workspace?
    • Update Pre-RFC with phased stablization plan
  • pubgrub progress
    • Blocked on readme
    • Pre-release handling causes challenges
    • Working on repo that can handle it
    • Need fuzz / regression testing

2022-09-06

2022-08-30

  • sparse index followups
    • invalidation: https://github.com/rust-lang/crates.io/issues/4913
      • Checking with Arlo on working on this. Will follow up with
    • Index normalization
      • How these changes are announced.
      • Squashing was announced on the main blog.
      • How to announce? Inside rust or twitter.
    • publish delays: https://github.com/rust-lang/cargo/issues/9507
      • owned by epage
      • Delays will be worse
      • Current tooling polls using git
    • registry api
      • crates-index uses git
      • what do we do in the future?
      • crates.io will continue to support git
      • but maybe not third party registries
      • crates-io crate?
      • More generally provider wrappers?
      • Expose types for serialialization?
        • Problem is slightly different shapes
        • Share types between crates io and cargo?
  • eddyb: cargo build -O, see https://github.com/rust-lang/rust/issues/57968#issuecomment-1208772020tf
    • Something that is quicker to specify than --profile dev-opt.
    • Uncertainty if this would mean "override the current profile" vs "replace the current profile"
    • concern: how often is this needed?
      • Game developers?
      • Makes it a little trickier to explain when it is needed
    • Wokarounds: can create an alias for shorter flag
    • ehuss to respond
  • Continue discussion from 2022-08-16 about policy on warning on specific crates.
    • Changing the layout of network structures (libc) 1.64
      • They are yanked
      • Would people using the yanked not noticed?
      • Talk of something stronger than yanked
      • libs wanted it if convenient, but fine without it
      • Already in beta, don't want to rush something anyways
    • Why not warn on build?
      • Its allowed, so noisy
      • Help if there were lint controls
      • Use lock for one-time warning?
      • But it mutates lock
    • "yanked" is checked on cargo install or cargo publish
      • How to fail on cargo publish?
    • general database like cargo-audit
    • Another example: communicating that strucopt is dead
      • It isn't "unmaintained", but if you want to update from clap 2, then clap 3 takes over
      • rustsec tracks "unmaintained" packages
      • There is a maintenance status in the badges table
    • Mutable state in registry?
    • For the net changes, the crates have already been yanked.
    • The "yanked" status doesn't help with a Cargo.lock and is difficult to get visibility.
  • Feature migration
    • Enough consensus on removing default feature activations to move forward?
    • See https://github.com/rust-lang/cargo/issues/3126#issuecomment-1190785350
    • Concern about this being confusing about subtracting features
    • Motivation:
      • Take exiting behavior and put it behind a feature flag
        • This is a breaking change for default-features=false users
      • Libraries are actively doing this without bumping major
      • Ideally do this without a breaking change
      • 2 RFCs and unconf on it, there is interest
      • Bumping major
        • Disruptive for everyone, even if not needed
        • Warp, tokio, etc probably wants larger windows without breakin
      • Usability for clap is terrible

2022-08-23

  • epage: Follow up on conversation with Jon
    • tl;dr of conversation with Jon: give him 1 dev year of projects to free up team to focus on growing itself with no concern for company politics. They'll then choose how to split it up and best fits among people. If its successful, he'll more easily get management to do another year etc.
    • Ideas
      • Issue triage (ensuring reproductions, etc)
      • Reduce support burden, e.g.
      • Reduce distractions
        • Help with HTTP work
        • Help with cargo-edit
        • Help with directory walking security issue
      • Reduce toil
        • Allow rust-lang and cargo to have independent workspaces
        • cargo-semver-checks
    • Measure of success
      • not necessarily something the cargo team needs to worry about
      • Show worth of investment
      • Person helping able to show their worth for their career
      • Lasting impact, "patching holes in boat, not just bailing"
      • Can free up team to do the patching but team needs to have concrete plan for improving
    • Need for review handled carefully
      • Must own
      • Explicit non-goal is feature project
    • Healthy state
      • Before, all on alex
      • what is the path to becoming a reviewer?
        • Free us up to mentor people to have more reviewers
        • non-goal: amazon people joining team
    • Mentoring
      • alex knew enough to work on or mentor in all areas
      • systemic knowledge?
        • split out
        • increased test coverage
        • evaluate when to bring in someone specialized
        • josh: wants to have things more isolated so that changes are not hard to evaluate their widespread impact
        • weihang: documentation improvements to make it easier to dig in and be able to understand and review
        • What things have people been uncomfortable with?
          • ed: example: "fuzzy" parameter was not clear
        • registry, manifest, etc
      • How possible is it to have anyone except Jacob work on PubGrub?
        • One artifact will be documentation
        • Work rotation?
          • Not first time Jon has heard it
        • More sustainable to have Jacob mentor?
          • Curse of knowledge affects Jacob
      • Compilation mode?
      • If there is something unclear about architecture and documentation, please file an issue on ping eric to write that.
    • Path forward:
      • Based on things cargo team wants help with, choosing the right person
      • josh: prefers structural work for maintainability (pubgrub, process steps, etc.)
    • ed: Moving architecture to rustdoc
      • publishing master for low latency
    • Organize ideas on specific things to work on and priorities and feasibilities
  • Follow-up on sparse index configuration
    • https://hackmd.io/@rust-cargo-team/B13O52Zko
    • issue: dependency on an alt registry which has a dependency on a second alt registry. Today, the second registry doesn't have to be configured locally. In the future, if the second registry shuts down their git index, then that second registry wouldn't be accessible.
    • uncertainty
    • concern: in order to know the canonical URL, Cargo needs to download config.json ahead of time to know that mapping.

2022-08-16

  • epage: Should we put more than ehuss into the review cycle?
    • Proposal: put all available but ehuss, we then escalate PRs to him when we feel uncomfortable with it
    • People available: epage, weihanglo, josh after spain
      • Use CODEOWNERS
    • ehuss will point to process
  • epage: rotate release assignments to help all learn, try to improve
    • People available: epage, weihanglo
    • ehuss enjoys it!
  • Sparse registry UX
    • https://github.com/rust-lang/cargo/issues/10964
    • https://github.com/rust-lang/cargo/issues/10965
    • Can auto-detection be feasible?
    • Transition plan for alt-registries
      • This will likely be rare. Most will be content being in git.
      • New registries can stick to one or the other.
      • Challenge: companies that need to rebuild software from years ago
        • Can take transition cost
        • Would be alt-registry
      • For those that want to transition, there are a few options:
        • Serve both temporarily, and have users update their config, and then turn off git once everyone is transitioned.
        • "Yank the band-aid", where there is a single day transition
          • This is very difficult. Requires updating all Cargo.lock and .crate files.
      • Some are using alt-registries to "mirror" crates.io with modifications.
    • config.json could indicate what the "lock" URL is.
      • When configured in git, cargo would then say "oh, ok, I'll use sparse".
        • Cargo could then remember that it needs to talk to sparse.
        • Drawback: Users would clone the repo every time they start from scratch (like uncached CI or new environments).
      • Undoing the operation: If the git repo has a config -> use this instead, and that is used unconditionally, then there is no way for the user to go back to the git repo.
        • Could have a TTL or something.
    • Replacing crates.io with "modifications":
      • [source.crates-io] points to private
      • Trying to build without [source] config would be broken.
      • This new functionality provides another way to do that.
      • The local sparse registry has config.json with index="http://example.com". Now, all Cargo.lock files have example.com in them instead of crates.io.
    • Possibility of auto-detection via some kind of probing. For example, having a custom git server that can respond in special ways.
    • auto-detection:
      • if just http://example.com, just ask for the config file.
        • If 404, then assume it is git.
    • firewall issues might be rare (adding index.crates.io should be "easy").
    • At least for crates.io, being able to switch between the two is desired.
    • Preference on making it easy to configure?
      • slight preference towards easy toggle
    • Auto-detection
      • "Can't talk to index.crates.io". Provide a message on how to configure? The "preferred" flag.
      • Similar to the fetch-with-cli
      • The number of problems, risks, complexities with automatically switching back to GitHub seems risky, so it's probably not worth trying for now.
    • Can the redirect be a little more generic (not just "sparse", but "use this other URL").
    • ehuss will write up a more coherent proposal with how these transitions can be used.
    • Can Cargo just try to use the URL as a sparse index, and if it fails then fall back to git.
      • Today: if it fails, it is just an error
      • Possibility: Try to download config.json first, and if that fails, assume it is git.
      • Auto-detection might have difficulty in the face of authentication.
        • arlo will investigate this more.
  • Policy on Cargo warning about specific crates?
    • Warning about yanked crates in general?
    • General yank warning on publish/install
    • rustc has some hard coding
    • cargo only has diesel
    • More of it?
    • Idea: General list of crate warnings that is configurable
    • Idea: expand existing warning, annoying for publish to get message then
      • Lint levels
    • Idea: if only we could work more closely with cargo-audit
    • Idea: don't force registry update, warn with what we have
      • Reproducibility between users
    • Requires network connection even if we don't need it otherwise, see https://github.com/rust-lang/cargo/issues/10984
    • Today: You get a warning for an yanked crate in cargo install and cargo publish
    • Can we get more visible warnings?
    • Is it OK to just add more special cases into Cargo?
    • Example: mem::transmute on private libstd types (std::net types)
      • If people have the old version in their lock file, they won't know that there is an issue.
      • Was this a security issue? Not really. The structure will be mostly corrupted, and the kernel should reject it.
      • Ability to have patches
    • Something like cargo-audit would be nice for surfacing these issues
      • They are particular about what exactly they accept.

2022-08-09

  • Explicit resolver and MSRV, see https://github.com/rust-lang/cargo/issues/10954
    • 1.56 stabilized 2021
    • 1.51 stabilized resolver=2
      • Unstable from 1.50 to …
    • Don't add resolver if edition<2021 and resolver=1
    • Ed will ping nemo, otherwise will own it
  • epage: Cargo Contributors?
    • Inviting people to the meeting
    • Distinction between contributor and maintainer
    • Provide recognition and higher-bandwidth access
      • And path to maintainer
    • No formal definitions / charter
      • ehuss has a charter from prior work with core, but that project stalled
    • Core talked about teams by participation level
    • Like existing informality (no requirements to uphold)
    • Downside to teams: overhead
    • Suggestion: create list of new invitees
    • https://github.com/kubernetes/community/blob/master/community-membership.md#community-membership
    • formal definition of what is required for a membership?
      • Trusted for RFCs?
      • But popularity contest, lack of diversity
      • Untangle formal definition from getting more people involved
      • Wait on Josh for feedback from libs-api vs libs-contrib
    • Arlo, Muscraft?, Jon?, Byron?
      • Ask josh on zulip with sync conversation next meeting
    • Meeting size?
      • "Lovely problem to deal with"
      • Formalize agenda process and people skip if needed
  • epage: Unconf build system integration conversation
    • Until governance can address SIGs, should we at least get a zulip stream / hackmd area for this?
    • Mismatch in velocity. Companies often need solutions yesterday, and may fork things to move faster.
    • Help companies be vulnerable, don't want to share their forks
    • Create zulip/ hackmd or github repo
    • Zulip first (epage will ask josh)

2022-08-02

  • No Ed, conflict with family event
  • RustConf
  • cache invalidation
    • currently: TTL is 1minute
    • plan: switch to invalidation on publish, and have a much higher TTL
      • Usually fast, but sometimes up to 5 minutes (at least their guarantee that every server is updated)
      • Uncertain how to implement (direct network request, lambda trigger, etc.)
    • Cargo: On publish, wait until it sees the index updated
    • crates.io can't block until the invalidation is finished (heroku doesn't allow long-running connections)
    • Should this be a blocker?
      • sparse index is a big win for more people
      • Invalidation is a problem for publish of multiple packages
      • It might be good to know how much of an impact this will have on real users
  • merkle trees
    • There is some nontrivial complexity on design
    • Don't want to block on completing that design
  • Source ambiguity https://github.com/rust-lang/cargo/pull/10907
  • Improving MSRV experience
    • https://github.com/rust-lang/cargo/issues/7169 (cargo install —locked by default)
      • eric: arguments for keeping it unlocked:
        • When a fix is published, that requires all reverse dependencies to publish a new version. For security fixes, this is more important because people often don’t publish very often.
        • It puts pressure on the community to keep compatibility, and to expose incompatible changes quickly. (At least, the exposure seems to happen faster with cargo install.)
    • https://github.com/rust-lang/cargo/pull/10891 (Error message for rust-version)
    • Should we change the lockfile recommendation to include libs to help with pinning to MSRV until the resolver can handle it?
    • Others
    • How to organize discussion?
      • Crossing multiple teams
    • Concern about the sole focus on "MSRV". There could be a more wholistic view of what the user experience should be.
    • The rust-lang org only supports latest stable. Implying with MSRV that users can or should be running older versions doesn't match with our current policy that only stable is supported.
    • Documenting rust-lang's support
      • Cargo: on the rust-version field
      • Release: Forge?
    • Populating rust-version into the index.
      • Resolver support.
      • cargo add or cargo update or whatever, provide better error messages (or behavior) when comparing the rust-version.
      • crates.io filtering the index based on version being requested
      • supporting third-party tooling, such as mirrors filtered for a specific version
      • concern about this requires almost everyone to manually enter their rust-version and track it. If just one package doesn't follow this, then the premise that rust-version will address people's use cases may be false.
        • And even projects that have rust-version may be incorrectly stated (since it isn't tested on publish).
        • Compiler warnings for using API items annotated with a higher version?
      • Josh: compressing the manifest and include it with the index so that it has all information
    • Resolver support.

2022-07-26

  • hackmd
    • Will switch before next week
    • Will look into team sponsorship
  • https://github.com/rust-lang/cargo/pull/10844
    • negative -j
    • -j cpus-2
    • -J
    • 0=all In some contexts, -1 means auto
    • move forward with negative numbers, -1 means max-1
    • Performance impact of including in help?
      • Probably u secs, so once per proc is probably fine, even for third-party commands
    • What about default for hetergenous (big/little core, performance/efficiency core, etc) CPUs?
      • For now, it’s usually ok to ignore the differences, but eric is concerned that it will become more of an issue in the future as they become more common (and more complex). A bigger concern might be memory usage when spawning too many jobs, which is a problem today.
  • https://github.com/rust-lang/cargo/pull/10787
    • build-override precedence
    • future-globs concern: https://github.com/rust-lang/cargo/pull/10787#issuecomment-1170646225
      • Intuition or mental model may be confusing or complicated
      • What if * means something different in the future.
      • [profile.build] would that be any less confusing?
        • The idea is to have a single place to set build settings (instead of needing to duplicate in multiple profiles), and to try to avoid complexities around shared dependencies (profile.build is always used). In theory to simplify the logic. build-override should then almost never need to be used.
        • But, adding a whole new mechanism could add to the complexity, not reduce it.
      • A warning to let them know if package.* and build-override are both set, that one is overriding the other.
    • No conclusion yet, needs more thought and consideration.

2022-07-19

  • safe.directories
    • Should we trust the directory we’re directly in?
      • Concerns:
        • Inconsistency with rustup
        • Still has a potential scenario where cargo is run in an automated fashion (e.g. metadata, locate-project)
        • Don’t want to worry about security boundaries in cargo itself.
        • Inconstent behavior in a workspace, if there is mixed permissions among the members (think a weird git submodule checkout)
      • Can it be ignored as running as root?
        • People run su and probably have more complex root prompts than they should
        • Another docker detection case? Probably not, let’s not be Docker-specific.
    • git recently added more escape hatches and checks, eric will post those to the rfc
      • A user running sudo on their files now works
  • hidden features
    • https://github.com/rust-lang/cargo/pull/10878
    • Concern about moving forward with this change if this isn’t something that we’ve discussed. It may create a convention that we may not necessarily want users to adopt.
    • Was mirroring the behavior in docs.rs.
    • Use cases
      • CI or dev related, like deny warnings, docs-specific features
      • Optional dev dependencies
      • Ivy Java dependency manager has this
    • Semantic behaviors:
      • Theoretical: Prevent activating a feature (across a registry? or in Cargo.toml? CLI only?)
      • Actual: hide from documentation and other user-things (like cargo add display)
    • There may be multiple states that we may want to add for a feature, such as public/private,
    • stable/unstable, deprecated, etc. Having one of these use a naming convention would be a little inconstent.
    • Example of what this would look like:
      [features.myfeat]
      description = "This feature makes things awesome."
      public = true # or hidden?
      unstable = true
      deprecated = "use xyz instead"
      include = ["anotherfeature", "dep:somedep", "anotherdep/featxyz"]
    • deprecated features isn’t ideal right now
      • Limited to build.rs warnings, in one case its the only reason for the build.rs
    • Feature migrations
    • _feature-foo underscores and kebab-case. Might pressure someone to use underscores instead of dashes.
    • Survey how it is being used today.
    • Hold off for it now
  • Feature migrations
  • cargo fix apply to cargo.toml?
  • https://github.com/rust-lang/rust/issues/84673#issuecomment-1179096971
    • terminal-width
    • rustc can’t determine width, needs cargo to auto-configure it
    • Should we allow the user to set hard coded width?
    • Ed: meh, let’s wait until someone asks
    • Tests need to control width to ensure compatibility with different environemts (like clap does)
    • eric will stabilize as-is with automatic selection, and open an issue for further control ideas
  • paper vs hackmd vs something else
    • This is last use of paper by Josh
    • hackmd gives us more control (like setting language for code blocks)
    • hackmd’s permissions?
      • Might not be a problem for us
      • Supports user permissions, just need to avoid hitting the “publish” button.

2022-07-12

  • No Weihang again (conflict with other recurring meeting 🥲)
  • https://github.com/rust-lang/cargo/issues/8415
    • Workspace inheritance
    • What is the bar to start stabilization?
    • [workspace.package]
      • Deviation from the RFC
      • Can’t set the “workspace edition”, which means users continue to get confused about not setting resolver = "``2``"
        • cargo new for workspace could set the default
        • is there some way to detect the scenario and suggest a fix?
          • If all members are 2021, and resolver is unset?
      • This helps decouple when using non-virtual workspace
      • eric: only thought is that package doesn’t scream that this means “these can be reused in the workspace”, and might be confusing (subtle difference between [workspace.package] and [package] that is very different)
      • josh: in the future, it would be nice to be able to inherit multiple fields from the [workspace.package] table in one go (or all dependencies, etc.)
      • josh: would this help with rustc-workspace-hack. No, not without several more changes.
  • https://github.com/rust-lang/rfcs/pull/3289
    • Source replacement
    • FCP can start soon
  • https://github.com/rust-lang/rfcs/pull/3127
    • Path sanitization
    • They’d like to start an FCP, which will include three teams (including Cargo); are we fine with that?
    • eric: concern that the motivation is for privacy, but this is not sufficient. There are multiple ways that paths and other environment information can leak into a binary, and I do not want to give the false sense of security to someone using it.
      • Build scripts
      • environment variables (env!). For example, cargo-bisect-rustc embeds the absolute path of RUST_SRC_REPO.
      • Things out of rustc’s control (linker scripts, linkers, etc.)
      • This provides some incremental improvement to privacy, but not absolute privacy
    • Passing the information down the build scripts so cc-rs could pass --remap-path-prefix to clang or gcc
    • josh: concern of too many variants
      • none, all, all-except-debug
      • gcc:
        • -fdebug-prefix-map
        • -fmacro-prefix-map: Similar to debug, but handles __FILE__ macros
        • -ffile-prefix-map: alias for debug and macro
      • jacob: maybe rustc has more options than cargo?
    • gdb common remapping from different build farm environments
    • effects with dsymutil, pdb
      • whether or not dsymutil fundamentally supports this, or if the paths can be fully removed
  • https://rust-lang.zulipchat.com/#narrow/stream/296752-t-cargo.2Fprivate/topic/Path.20to.20Sustainability
    • smaller components can be easier to reason about things, and to specialize in.
    • incremental improvements
    • reducing distractions
    • cross-cutting concerns are difficult to understand or unclear what the consequences are
    • how to prioritize what additional people could focus on?
    • rust-edu
      • what would help improve the usability for rust in education
        • better error messages would be great
    • What do we like working on least? 😊
    • identify “what needs a refactor to accomplish some new feature?”
    • issue triage
      • backlog issues to cargo tests
    • nested workspaces
    • cargo semver compatibility checks

2022-07-05

Last changed by 
 
rust-cargo-team
Cargo team for the Rust Programming Language
0
126

Read more

2025 All Hands Topics

Discuss https://hackmd.io/-jpsOWYoRACRNF3G80Hkuw?edit

Apr 22, 2025
Cargo team meeting notes

https://meet.jit.si/CargoTeamMeeting

Apr 22, 2025
Cargo team meeting notes 2025 Q1

2025-03-25 eric: Check if anyone is interested in helping with blanket source replacement?https://rust-lang.zulipchat.com/#narrow/channel/246057-t-cargo/topic/Possibility.20to.20replace.20any.20source.20in.20config.2Etoml https://github.com/rust-lang/cargo/issues/11406 jacob: There was some related work on the git/sparse equivalence. There are some landmines here. ed: Doesn&apos;t know how it would work with git, so that seems like that needs some work. But seems like there is merit. How do you break out of the proxy, where there is a mix (inclusion/exclusion rules). Could put on them for determining use cases. Reach out on the integration stream to find what requirements clients have. jacob: Nuget (or some other) has something more extreme, where it is substituting networking at a low-level, and you can customize what you want to do. &quot;Here&apos;s URL, give me a file.&quot; A low-level system could help avoiding missing things in the future. weihang: Why can&apos;t implement outside of cargo? An http proxy should be able to handle any request. eric: Will try to follow up with some questions and level set some expectations. epage: review open questions on build.build-dir, see https://github.com/rust-lang/cargo/issues/14125

Apr 8, 2025
Build Times: Cargo Perpsective

Notes to prepare for 2025H1 Project Goal planning

Mar 19, 2025
Read more from rust-cargo-team
Published on HackMD