LLD x64 GNU Linux stabilization report

# Use LLD on `x86_64-unknown-linux-gnu` by default, and stabilize `-Clinker-features=-lld` and `-Clink-self-contained=-linker` This PR proposes making LLD the default linker on the `x86_64-unknown-linux-gnu` target for the artifacts we distribute, and also stabilizing the `-Clinker-features=-lld` and `-Clink-self-contained=-linker` codegen options to make it possible to opt out. LLD has been used as the default linker on nightly and CI on this target since May 2024 ([PR](https://github.com/rust-lang/rust/pull/124129), [blog post](https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html)), and it seems like it is working fine, so we would like to propose stabilizing it. The main motivation for using LLD instead of the default BFD linker is improving [compilation times](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=instructions%3Au&tab=compile). For example, in the linked benchmark, it makes incremental recompilation of `ripgrep` in `debug` more than twice faster. Another benefit is that Rust compilation becomes more consistent and self-contained, because we will use a known version of the LLD linker, rather than "whatever GNU ld version is on the user's system". Due to the performance benefit being so huge, many people already opt into using LLD (or other fast linkers, such as mold) using various approaches ([1](https://github.com/search?type=code&q=%2Flinker-flavor%5B%3D+%5Dgnu-lld-cc%2F), [2](https://github.com/search?type=code&q=%2Flinker-features%5B%3D+%5D%5C%2Blld%2F), [3](https://github.com/search?type=code&q=language%3Atoml+%22-fuse-ld%3Dlld%22), [4](https://github.com/search?type=code&q=language%3Arust+%22-fuse-ld%3Dlld%22)). By making LLD the default linker on the `x86_64-unknown-linux-gnu` target, we will be able to speed up Rust compilation out of the box, without users having to opt in. ## What is being stabilized - `rust-lld` being used as the default linker on the `x86_64-unknown-linux-gnu` target. - Note that `rust-lld` is being enabled by default in the compiler artifacts distributed by our CI/rustup. It is still possible to use the system linker by default using `rust.lld = false` in `bootstrap.toml`, which can be useful e.g. for some Linux distros that might not want to use the LLD we distribute. - This is done by activating the LLD linker feature and using the self-contained linker on that target. Both of which are also usable on the CLI, if some opt outs are necessary, as described below. - `-Clinker-features=-lld` on the `x86_64-unknown-linux-gnu` target. This codegen option tells rustc to disable using the LLD linker. - Note that other options for this codegen flag (`cc`) remain unstable. - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using the flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of LLD on stable, which would it turn also opt out of using the self-contained linker (since it's an LLD). - `-Clink-self-contained=-linker`. This codegen option tells rustc to use the self-contained linker. It's not particularly useful to turn it on by itself, but when enabled and combined with `-Clinker-features=+lld`, rustc will use the `rust-lld` linker wrapper shipped with the compiler toolchain, instead of some `lld` binary that the linker driver will find in the `PATH`. - Note that other options for this codegen flag (other than the previously stable `y/yes/n/no`). - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using this flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of using self-contained linking on stable. Doing this would then fall back to using the system `lld`. To opt out of using LLD, `RUSTFLAGS="-Clinker-features=-lld"` would be used. To opt out of using `rust-lld`, falling back to the LLD installed on the system, `RUSTFLAGS="-Clink-self-contained=-linker"` would be used. ## Tests When enabling `rust-lld` on nightly, we also switched x64 linux to use it at stage >= 1, meaning that all tests have been running with lld since May 2024, on CI as well as contributors' machines. (Post opt-dist tests also had been using it when running their test subset earlier than that). There are also a few tests dedicated to the CLI behavior, or ensuring the default linker is indeed the one we expect: - [link-self-contained-consistency](tests/ui/linking/link-self-contained-consistency.rs): Checks that `-Clink-self-contained` options are not inconsistent (i.e. that passing both `+linker` and `-linker` is an error). - [link-self-contained-unstable](tests/ui/linking/link-self-contained-unstable.rs): Checks that only the `-linker` and `y/yes/n/no` options for `-Clink-self-contained` are stable. - [linker-features-unstable-cc](tests/ui/linking/linker-features-unstable-cc.rs): Checks that only the non-lld options of `-Clinker-features` are unstable. - [linker-features-lld-disallowed](tests/ui/linking/linker-features-lld-disallowed.rs): Checks that `-Clinker-features=-lld` is only stable on `x86_64-unknown-linux-gnu`. - [link-self-contained-linker-disallowed](tests/ui/linking/link-self-contained-linker-disallowed.rs): Checks that `-Clink-self-contained=-linker` is only stable on `x86_64-unknown-linux-gnu`. - [no-gc-encapsulation-symbols](tests/ui/linking/no-gc-encapsulation-symbols.rs): Checks that that linker encapsulation symbols are not garbage collected by LLD, so that crates like [linkme](https://github.com/dtolnay/linkme) still work. - [rust-lld](tests/run-make/rust-lld): Checks that LLD is actually used when enabled with `-Clinker-features=+lld` and `-Clink-self-contained=+linker`. - [rust-lld-x86_64-unknown-linux-gnu](tests/run-make/rust-lld-x86_64-unknown-linux-gnu): Checks that LLD is used by default on `x86_64-unknown-linux-gnu` when the bootstrap `rust.lld` config option is `true`. - [rust-lld-x86_64-unknown-linux-gnu-dist](tests/run-make/rust-lld-x86_64-unknown-linux-gnu-dist): Dist test that checks that our distributed `x86_64-unknown-linux-gnu` archives actually use LLD by default. ## Ecosystem impact As already stated, LLD has been used as the default linker on x64 Linux on nightly for almost a year, and we haven't seen any blockers to stabilization in that time. There were a handful of issues reported, these are discussed later below. Furthermore, two crater runs ([November 2023](https://crater-reports.s3.amazonaws.com/pr-117684-2/index.html), [February 2025](https://crater-reports.s3.amazonaws.com/pr-137044-3/index.html)), were performed to test the impact of using LLD as the default linker. A triage of the earlier crater run was previously done [here](https://hackmd.io/OAJxlxc6Te6YUot9ftYSKQ), but all the important findings from both crater runs are reported below. Below is a list of compatibility differences between BFD and LLD that we have encountered. There is a more thorough list of differences in [this post](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) from the current LLD maintainer. From that post, "99.9% pieces of software work with ld.lld without a change". --- ### `.ctors/.dtors` sections [#128286](https://github.com/rust-lang/rust/issues/128286) reported an issue where LLD was unable to link certain CUDA library was using these sections that were using the `.ctors/.dtors` ELF sections. These were deprecated a long time ago (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46770), replaced with a more modern `.init_array/.fini_array` sections. LLD doesn't (and won't) support these sections ([1](https://github.com/llvm/llvm-project/issues/68071), [2](https://github.com/llvm/llvm-project/issues/30572)), so if they appear in input object files, the linked artifact might produce incorrect behavior, because e.g. some global variables might not get initialized properly. However, the usage of `.ctors/.dtors` should be very rare in practice. We have performed a [crater run](https://github.com/rust-lang/rust/pull/137044) to test this. It has identified only 8 crates where the `.ctors/.dtors` section is occurring in the final linked artifact. It was caused by a few crates using the `.ctors` link section manually, and by using a very (~6 year) old version of the [ctor](https://crates.io/crates/ctor) crate. <details markdown="1"> <summary>Crater run analysis</summary> - The [ctor](https://github.com/mmastrac/rust-ctor) crate used `.ctors` in versions <= [`0.1.10`](https://github.com/mmastrac/rust-ctor/pull/19) - https://github.com/rustify-emacs/fuz.el/tree/fee874aa35d2ee6b12b836290b5c8eaa44175a28 depends on `ctor 0.1.9`. - https://github.com/thebracket/rustyroguelike/tree/06f6ef51829b07962e58896e0fad13a78bd1a92d depends on `ctor 0.1.9`. - https://github.com/ckaran/noisy_float_issue/tree/3cb2921e931a065c49caf46e96eea0c448c75c2e depends on `ctor 0.1.8`. - https://github.com/museun/shaken/tree/6bbb6b6aea60d22f87c5c601105959bde5e859d5 depends on `ctor 0.1.8`. - https://github.com/remexre/mpisa/tree/7e4fbd9fd04a9836f55c52041001b157085c9dd0 depends on `ctor 0.1.7`. - Manual usage of `.ctors` - [sharedlib-address-dumper](https://github.com/SammCheese/sharedlib-address-dumper/blob/40ea1bbf715bac745eadec6feb82a824a5dc23de/src/lib.rs#L7) - [rust-ctor](https://github.com/mmastrac/rust-ctor/blob/459eb2147ff8938b9f398243b27d06cceff76ed5/tests/src/lib.rs#L37) (manual usage of `.ctors` in a single test) - [glycin_krita](https://github.com/vurentjie/glycin_krita/tree/e5c3abdb14200fbb700f082ec93fb2a28bfbc18e) (manual usage of `.ctors` in the [`glycin-utils` dependency](https://gitlab.gnome.org/GNOME/glycin/-/blob/main/glycin-utils/src/instruction_handler.rs#L162)) </details> **Possible workaround** It is possible to [detect](https://github.com/rust-lang/rust/commit/e5e2316712e19036eb40e0bf59150f25e35f9880) if `.ctors/.dtors` section is present in the final linked artifact (LLD will keep it there, but it won't be populated), and warn users about it. This check is very cheap and doesn't even appear on [perf](https://github.com/rust-lang/rust/pull/112049#issuecomment-2661125340). We have benchmarked the check on a 240 MiB Chrome binary, where it took 0.8ms with page cache flushed, and 0.06ms with page cache primed (which should be the common case, as the linked artifact is written to disk just before the check is performed). In theory, this could be also solved with a linker script that moves `.ctors` to `.init_array`. We think that these sections should be so rare that it is not worth it to implement any workarounds for now. --- ### Different garbage collection behavior [#130397](https://github.com/rust-lang/rust/issues/130397) reported an issue where LLD (correctly) prunes a local symbol, so it is missing in the linked artifact. However, BFD keeps the same symbol, so it is a regression. This is caused by a difference in linker garbage collection. Rust uses `--gc-sections` and puts each function into a separate linker section, which prunes unused code. In some rare situations, code that uses so called linker encapsulation symbols might not be [annotated properly](https://lld.llvm.org/ELF/start-stop-gc.html#z-start-stop-gc), which causes it to break if `--gc-sections` is used. Note that this is essentially a bug with the object file. BFD (2.37+) uses a conservative linking mode that works around this issue, which might slightly increase binary size of the linked artifact. LLD does not use this workaround by default, but it can be made to use the conservative mode using `-z nostart-stop-gc`. Perhaps more importantly, the default behavior of LLD also breaks the somewhat popular [linkme](https://github.com/dtolnay/linkme) crate. To avoid this issue, we told LLD to use the [conservative mode](https://github.com/rust-lang/rust/pull/137685), which maintains backwards compatibility with BFD. We found that it has [no effect](https://github.com/rust-lang/rust/pull/112049#issuecomment-2666028775) on compilation performance and binary size in our benchmark suite. With this change, `linkme` works. Since then, rust-lang/rust#140872 removed `linkme` distributed slice's dependence on conservative GC behavior, so this PR also removes that conservative mode: no transition period is necessary, as the PR immediately fixed the crate with no source changes. <details markdown="1"> <summary>Crater run analysis</summary> There are the 20 occurrences of the GC/linkme errors: 1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Archisman-Mridha.rust-microservice-template/log.txt 2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/beoboo.monkey-rust/log.txt 3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/blackbox_cast-0.1.1/log.txt 4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/companion-service-0.1.0/log.txt 5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/derive-codegen-0.0.4/log.txt 6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/elfo-core-0.2.0-alpha.11/log.txt 7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/eventheader-0.3.2/log.txt 8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/ffizz-header-0.5.0/log.txt 9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Gohla.serde_flexitos/log.txt 10. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/H4m5t3r.cses-cli/log.txt 11. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/intertrait-0.2.2/log.txt 12. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/linkme-0.3.17/log.txt 13. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mdm.confidante/log.txt 14. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/metriken-0.3.3/log.txt 15. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mm-zk.vise/log.txt 16. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mrobakowski.odra/log.txt 17. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pliron-0.1.7/log.txt 18. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/RustyYato.test-harness/log.txt 19. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/serde_flexitos-0.1.0/log.txt 20. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/SergeevPavel.morph-endo/log.txt </details> --- ### Various uncommon issues Here we show a bunch of issues that only occurred in a handful of instances, and it is unclear if LLD is at fault or if there is some other issue that was not detected with BFD. #### Undefined references due to `--no-allow-shlib-undefined` This issue is caused by libraries having undefined references to some of their dependent libraries. This has been reported [upstream in LLD](https://github.com/llvm/llvm-project/issues/54700), where it was noted as "not a bug". The library should be properly linked to its dependencies, without depending on undefined symbols. It manifests e.g. in `py-spy 0.3.14` ([log](https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-0.3.14/log.txt)) with this error: ``` = note: rust-lld: error: undefined reference due to --no-allow-shlib-undefined: lzma_stream_buffer_decode >>> referenced by /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libunwind-ptrace.so ``` Probably because it's made to be used for the musl target, but as a dynamically linked library, building the crate has implicit requirements to both [`libunwind`](https://github.com/benfred/py-spy/issues/334), and `liblzma` which we can see failing with `lld`. Adding both explicitly as link-args fixes the issue. With that it is possible to build with `ld`, system `lld`, and `rust-lld`, and tests behave the same with all 3. Mentioned in [this issue](https://github.com/benfred/py-spy/issues/334), and in the [rstack project](https://github.com/sfackler/rstack/issues/21) (where the author mentions it as being a bug in `libunwind`). It seems to have been [fixed](https://github.com/libunwind/libunwind/issues/586) in `libunwind` 1.8.0 (released in April 2024), but systems can have the older versions installed. <details> <summary>Crater run analysis</summary> There are 7 occurrences of this error (4 times as `py-spy`): 1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/apachecn-archive.py-spy/log.txt 2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/benfred.py-spy/log.txt 3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/lxtrace-0.1.0/log.txt 4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-0.3.14/log.txt 5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-for-datakit-0.3.16/log.txt 6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rsps-0.4.0/log.txt 7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rstack-self-0.3.0/log.txt </details> #### Undefined symbol `__gxx_personality_v0` This was reported in [#135257](https://github.com/rust-lang/rust/issues/135257), and was also found in previous crater runs. Reproduction can be found in https://github.com/hasenbanck/rust_lld_bug. This might be related to https://github.com/llvm/llvm-project/issues/54701, where it was marked as "usage error", and within tolerance of linker incompatibilities. It seems that this code should be linked with the C++ standard library, that the linker usually takes care of. The difference with `ld` is unclear, and the LLVM people mention that this could be an `ld` bug that this works. In other contexts, it can happen when using `gcc` by mistake instead of `g++`, and the same is true when using the `cc` crate to build a C++ library. For example, in `libsvm-sys-3.24.0`, changing the build script to [opt-in to c++ library compilation](https://docs.rs/cc/latest/cc/#c-support), fixes the issue locally. In these cases, one could also manually link the appropriate target-specific C++ standard library via `-Clink-arg`. <details> <summary>Crater run analysis</summary> There were 9 occurrences of this error: 1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/angrylion-rdp-plus-sys-0.2.0/log.txt 2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/daniel-dimovski.poe-bundle-reader/log.txt 3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/expenses.ktx2-tools/log.txt -- though the cause is in https://github.com/Traverse-Research/intel-tex-rs-2 (probably related to https://github.com/rust-lang/rust/issues/135257). 4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/libsvm-sys-3.24.0/log.txt 5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/n-elderbroom.wtx_tools/log.txt 6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/poe_bundle-0.1.5/log.txt 7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/pollend.angelscript-rs/log.txt 8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/srplab.starcore_for_rust/log.txt 9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/xc3_model-0.1.0/log.txt </details> #### Duplicate symbol errors For example, `duplicate symbol: regerror` in https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt ``` = note: rust-lld: error: duplicate symbol: regerror >>> defined at regposerr.c:88 (oniguruma/src/regposerr.c:88) >>> regposerr.o:(regerror) in archive /opt/rustwide/target/debug/deps/libonig_sys-e3f0e36df31d8de7.rlib >>> defined at regcomp.c:547 (libgit2/deps/regex/regcomp.c:547) >>> regex.o:(.text.regerror+0x0) in archive /opt/rustwide/target/debug/deps/liblibgit2_sys-e9010b2c607c511f.rlib collect2: error: ld returned 1 exit status ``` This looks like a legit linker error using `lld`. <details> <summary>Crater run analysis</summary> There are 4 occurrences of this error: 1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt 2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ecumene.spin_pg/log.txt 3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/isgasho.openforum/log.txt 4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ns6251.spin-cookie-token-sample/log.txt </details> #### "is incompatible with elf_x86_64" errors Found in a crater run. Appears as "incompatible with elf_x86_64" and "... with elf64-x86-64". ``` rust-lld: error: /opt/rustwide/target/debug/deps/libnrfxlib_sys-a5ab3d8e5beeb8f3.rlib(rpc_framework.c.obj) is incompatible with elf64-x86-64 ``` <details> <summary>Crater run analysis</summary> There are 3 occurrences of this error: 1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pros-sys-0.3.0/log.txt 2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/nrfxlib-0.6.0/log.txt 3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/varesa.robot-iounit/log.txt </details> #### Uncategorized issues - "Invalid local symbol" error was reported in [#128938](https://github.com/rust-lang/rust/issues/128938). It seems to occur only in specific LLD versions, and in general seems to be a LLD issue. - Also occurred once in [crater](https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/jestarray.fmod_linking_test/log.txt). - Linking error with manual `#[link_name]` was reported in [#126280](https://github.com/rust-lang/rust/issues/126280). This turned out to be a user error. --- ### Missing jobserver support LLD doesn't support the jobserver protocol for limiting the number of threads used, it simply defaults to using all available cores, and is one of the reasons why it's faster than BFD. However, this should mostly be a non-issue, because most of the linking done during high parallelism sections of `cargo build` is linking of build scripts and proc macros, which are typically very fast to link (e.g. ~50ms), and a potential oversubscription of cores thus doesn't hurt that much. When the final artifact is linked (which typically takes the most time), there should be no other sources of parallelism conflicts from compiling other code, so LLD should be able to use all available threads. That being said, it is a difference of behavior, where previously a `-j` flag was generally not using more cpu than the specified limit. It can be impactful in some resource-constrained systems, but to be clear that is already the case today due to [cargo parallelism](https://github.com/rust-lang/cargo/issues/9157). This could be one reason to opt out of using `rust-lld` on some systems. LLD has support for limiting the number of threads to use, so in theory rustc could try to get all the jobserver tokens available and use that as lld's thread limit. It'd still be suboptimal as new tokens would not be dynamically detected, and we could be using less threads than available. <details> <summary>A practical example of this parallelism increase</summary> We have benchmarked the compilation of [hyperqueue](https://github.com/It4innovations/hyperqueue), a mid-size (~200 dependencies) crate that invokes the linker ~40 times during `cargo build` (except the final artifact, everything else are build scripts and proc macros). The experiment was performed on an AMD Zen3 x64 Linux machine with 8 cores enabled. We have tried three modes: - `LLD_PARALLELISM_MODE=single` forcefully tells LLD to link using a single thread using `--threads=1`. - `LLD_PARALLELISM_MODE=leaf` does the same, but only for build scripts and proc macros. - Normal compilation with LLD using all available threads. We have also included a benchmark where jemalloc is compiled in the crate graph, to add more parallelism to the mix. The results show that there is no real difference between these modes: ``` /pr/it/hyperqueue [main/u]$ hyperfine --prepare "rm -rf target" --warmup 1 --runs 2 "LLD_PARALLELISM_MODE=serial cargo +stage1 build" "LLD_PARALLELISM_MODE=leaf cargo +stage1 build" "cargo +stage1 build" "LLD_PARALLELISM_MODE=serial cargo +stage1 build --features jemalloc" "LLD_PARALLELISM_MODE=leaf cargo +stage1 build --features jemalloc" "cargo +stage1 build --features jemalloc" Benchmark 1: LLD_PARALLELISM_MODE=serial cargo +stage1 build Time (mean ± σ): 60.401 s ± 0.530 s [User: 200.743 s, System: 15.753 s] Range (min … max): 60.026 s … 60.776 s 2 runs Benchmark 2: LLD_PARALLELISM_MODE=leaf cargo +stage1 build Time (mean ± σ): 60.590 s ± 0.015 s [User: 204.620 s, System: 15.596 s] Range (min … max): 60.580 s … 60.601 s 2 runs Benchmark 3: cargo +stage1 build Time (mean ± σ): 60.831 s ± 0.065 s [User: 204.392 s, System: 15.938 s] Range (min … max): 60.786 s … 60.877 s 2 runs Benchmark 4: LLD_PARALLELISM_MODE=serial cargo +stage1 build --features jemalloc Time (mean ± σ): 72.319 s ± 0.261 s [User: 304.779 s, System: 36.013 s] Range (min … max): 72.134 s … 72.504 s 2 runs Benchmark 5: LLD_PARALLELISM_MODE=leaf cargo +stage1 build --features jemalloc Time (mean ± σ): 71.804 s ± 0.695 s [User: 305.534 s, System: 36.125 s] Range (min … max): 71.312 s … 72.296 s 2 runs Benchmark 6: cargo +stage1 build --features jemalloc Time (mean ± σ): 72.475 s ± 0.231 s [User: 304.899 s, System: 36.344 s] Range (min … max): 72.312 s … 72.639 s 2 runs Summary LLD_PARALLELISM_MODE=serial cargo +stage1 build ran 1.00 ± 0.01 times faster than LLD_PARALLELISM_MODE=leaf cargo +stage1 build 1.01 ± 0.01 times faster than cargo +stage1 build 1.19 ± 0.02 times faster than LLD_PARALLELISM_MODE=leaf cargo +stage1 build --features jemalloc 1.20 ± 0.01 times faster than LLD_PARALLELISM_MODE=serial cargo +stage1 build --features jemalloc 1.20 ± 0.01 times faster than cargo +stage1 build --features jemalloc ``` The linking time for the final artifact (a ~150 MiB unoptimized binary with debuginfo) was ~0.7s with one thread is and ~0.4s with eight threads. In theory, we could use some heuristic to e.g. link build scripts and proc macros with only one LLD thread, but it does not seem to be worth it. </details> --- ### Opting out of LLD in the ecosystem We have also examined repositories where people opted out of LLD on nightly, using [this GitHub query](https://github.com/search?q=%22linker-features%3D-lld%22&type=code). The summary can be found below: <details> <summary>Summary of LLD opt outs</summary> > This examination was performed on 2025-03-09. Here we briefly examine the most common reasons why people use `-Zlinker-features=-lld`, based on comments and git history. - Nix/NixOS ([1](https://github.com/rszyma/vscode-kanata/blob/59d703dff5a238b14ab3759cac27f73fb34bbcfe/flake.nix#L33), [2](https://github.com/sbernauer/breakwater/blob/3cc3449fc126c5c99d4a971733fd32be589884e0/.cargo/config.toml#L4), [3](https://github.com/tiiuae/ebpf-firewall/blame/32bdb17cedd1c9bea1ab3482623de458d95da7d0/.cargo/config.toml#L2), [4](https://github.com/jules-sommer/wavetheme-gen/blob/f5f657d014d4a30607625afb70f810c229c0294e/Cargo.toml#L4), [5](https://github.com/LayerTwo-Labs/zside-rust/blob/e4266f5c5571a1b180a9c70cf0939c7070e410c7/.cargo/config.toml#L10), [6](https://github.com/przyjacielpkp/zkml/blob/22a4aef24e9d2c77789229d7c634fc67e9eb1184/README.md?plain=1#L78), [7](https://github.com/LayerTwo-Labs/thunder-rust/blob/2222d53474c8d2d0428b4c56f8157095dced6d5a/.cargo/config.toml#L2), [8](https://github.com/enesoztrk/nixos-tc-aya-test/blob/b2ffa59d3eba8b60fd04b0a4c8bbe047400eb981/.cargo/config.toml#L4), [9](https://github.com/lowRISC/container-hotplug/blob/3ead4ef9c7f79c303392178c99677dbecff1aea6/.cargo/config.toml#L2), [10](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/lsp-client/src/extension.ts#L94)) - There was an [issue](https://github.com/NixOS/nixpkgs/issues/312661) with LLD, which seems to have been fixed with https://github.com/NixOS/nixpkgs/pull/314268. It's unclear whether that fixed all the Nix issues though. - Issues with linkme ([1](https://github.com/0xPolygonZero/zk_evm/blob/ef388619ffbd5305209519a3a5bc0396185d68ac/.cargo/config.toml#L4), [2](https://github.com/conjure-cp/conjure-oxide/blob/be0fc5827ff90e8486d416cc184b6ce24f73bf01/README.md?plain=1#L20), [3](https://github.com/clchiou/garage/blob/c5d8444d56bb6ee24ca95e5c6b9880ed996f4918/rust/.cargo/config.toml#L6), [4](https://github.com/PonasKovas/craftflow/blob/5b4cc1a5196e08a975368399fefda4b71f3a2f6f/.cargo/config.toml#L3), [5](https://github.com/kezhuw/zookeeper-client-rust/blob/4e27c7de2a7cc5e709af012b791c8fea9bb47f1f/.github/workflows/ci.yml#L82), [6](https://github.com/niklasdewally/conjure-oxide/blob/8fe60c12bca7011a2f9eded4b7c95ad0e77b6f44/.github/workflows/code-coverage.yml#L48), [7](https://github.com/kezhuw/spawns/blob/c8b468379805de9df3287c01b94b4ed3db6b61ed/.github/workflows/ci.yml#L74)) - These should be resolved with the conservative garbage collection ([#137685](https://github.com/rust-lang/rust/pull/137685)). - Bazel ([1](https://github.com/google-parfait/confidential-federated-compute/blob/1823f69ed8f5f4f819f7bfa21da1ca629fdc826b/.bazelrc#L71)), WASM ([1](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/wasm-build.sh#L37), [2](https://github.com/yacineb/pgrx-wasi-test/blob/2bf99037ca1b650b2cbc35f1257a87fb6ead0920/build.sh#L21)), uncategorized ([2](https://github.com/nbdd0121/r2vm/blob/5118be6b9e757c6fef2f019385873f403c23c548/.cargo/config.toml#L3), [3](https://github.com/Wyvern/Img/blame/45020c7e1dc4926c8129647014c708db0c13f463/.cargo/config.toml#L209), [4](https://github.com/arnaudpoullet/leptos-i18n-compile-error/blob/042eb835f7ca0dc36be67cf7fe65b35b22b6059f/README.md?plain=1#L89), [5](https://github.com/JonLeeCon/numerical-rust-cpu/blob/fd0b3006768ed81c56147044dc05c92b11b7b6f0/exercises/.cargo/config.toml#L13), [6](https://github.com/PonasKovas/shallowclone/blob/be65f2ec923cac6ceedbc8db520c89969ebfce7c/.github/workflows/rust.yml#L20)) - Reason unclear. </details> ## History The idea to use a faster linker by default has been on the radar for quite some time ([#39915](https://github.com/rust-lang/rust/issues/39915), [#71515](https://github.com/rust-lang/rust/issues/71515)). There were [very early attempts](https://github.com/rust-lang/rust/pull/29974) to use the gold linker by default, but these had to be [reverted](https://github.com/rust-lang/rust/pull/30913) because of compatibility issues. Support for LLD was implemented back in [2017](https://github.com/rust-lang/rust/pull/40018), but it has not been made default yet, except for some more niche targets, such as [WASM](https://github.com/rust-lang/rust/pull/48125), [ARM Cortex](https://github.com/rust-lang/rust/pull/53648) or [RISC-V](https://github.com/rust-lang/rust/pull/53822). It took quite some time to figure out how should the interface for selecting the linker (and the way it is invoked) look like, as it differs a lot between different platforms, linkers and compiler drivers. During that time, LLD has matured and achieved [almost perfect compatibility](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) with the default Linux linker (BFD). Most of the implementation and reviews required to get to today's state were done by @lqd and @petrochenkov. - [#56351](https://github.com/rust-lang/rust/pull/56351) stabilized `-Clinker-flavor`, which is used to determine how to invoke the linker. It is especially useful on targets where selecting the linker directly with `-Clinker` is not possible or is impractical. - December 2018, author @davidtwco, reviewer @nagisa - [#76158](https://github.com/rust-lang/rust/pull/76158) stabilized `-Clink-self-contained=[y|n]`, which allows overriding the compiler's heuristic for deciding whether it should use self-contained or external tools (linker, sanitizers, libc, etc.). It only allowed using the self-contained mode either for everything (`y`) or nothing (`n`), but did not allow granular choice. - September 2020, author @mati864, reviewer @petrochenkov - [#85961](https://github.com/rust-lang/rust/pull/85961) implemented the `-Zgcc-ld` flag, which was a hacky way of opting into LLD usage. - June 2021, author @sledgehammervampire, reviewer @petrochenkov - [MCP 510](https://github.com/rust-lang/compiler-team/issues/510) proposed stabilizing the behavior of `-Zgcc-ld` using more granular flags (`-Clink-self-contained=linker -Clinker-flavor=gcc-lld`). - Initially implemented in [#96827](https://github.com/rust-lang/rust/pull/96827), but @petrochenkov [suggested](https://github.com/rust-lang/rust/pull/96827#issuecomment-1208441595) a slightly different approach. - The PR was split into [#96884](https://github.com/rust-lang/rust/pull/96884), where it was decided what will be the individual components of `-Clink-self-contained=linker`. - And [#96401](https://github.com/rust-lang/rust/pull/96401), which implemented the `-Clinker-flavor` part. - The MCP was finally implemented in [#112910](https://github.com/rust-lang/rust/pull/112910). - [#116514](https://github.com/rust-lang/rust/pull/116514) then removed `-Zgcc-ld`, as it was replaced by `-Clinker-flavor=gnu-lld-cc` + `-Clink-self-contained=linker`. - April 2022 - October 2023, author @lqd, reviewer @petrochenkov - Various linker handling refactorings were performed in the meantime: [#97375](https://github.com/rust-lang/rust/pull/97375), [#98212](https://github.com/rust-lang/rust/pull/98212), [#100126](https://github.com/rust-lang/rust/pull/100126), [#100552](https://github.com/rust-lang/rust/pull/100552), [#102836](https://github.com/rust-lang/rust/pull/102836), [#110807](https://github.com/rust-lang/rust/pull/110807), [#101988](https://github.com/rust-lang/rust/pull/101988), [#116515](https://github.com/rust-lang/rust/pull/116515) - The implementation of linker flavors with LLD was causing a sort of a combinatorial explosion of various options. [#119906](https://github.com/rust-lang/rust/pull/119906) suggested a different approach for linker flavors (described [here](https://github.com/rust-lang/rust/pull/119906#issuecomment-1894088306)), where the individual flavors could be enabled separately using `+/-` (e.g. `+lld`). - After some back and forth, this idea was moved to `-Clinker-features` (see [comment 1](https://github.com/rust-lang/rust/pull/119906#issuecomment-1895693162) and [comment 2](https://github.com/rust-lang/rust/pull/119906#issuecomment-1980801438)), which was implemented in [#123656](https://github.com/rust-lang/rust/pull/123656). - April 2024, author @lqd, reviewer @petrochenkov - [#124129](https://github.com/rust-lang/rust/pull/124129) enabled LLD by default on nightly. - April 2024, author @lqd, reviewer @petrochenkov - [#137685](https://github.com/rust-lang/rust/pull/137685), [#137926](https://github.com/rust-lang/rust/pull/137926) enabled the conservative gargage collection mode (`-znostart-stop-gc`) to improve compatibility with BFD. - February 2025, author @lqd, reviewer @petrochenkov (implementation), author @kobzol, reviewer @lqd (test) - [#96025](https://github.com/rust-lang/rust/pull/96025) (April 2022), [#117684](https://github.com/rust-lang/rust/pull/117684) (November 2023), [#137044](https://github.com/rust-lang/rust/pull/137044) (February 2025): crater runs. ## Unresolved questions/concerns - Is changing the linker considered a breaking change? In (hopefully very rare) cases, it might break some existing code. It should mostly only affect the final linked artifact, so it should be easy to opt out. - Similarly, is the single-threaded behavior of such tools encompassed in our stability guarantee: it can be observed via the `-j` job limit (though I believe we have/had some open issues on sometimes using more CPU resources than the job count limit implied). As mentioned above, LLD does not support the jobserver protocol. - A concern [was raised](https://github.com/rust-lang/rust/issues/71515#issuecomment-2612370229) about increased memory usage of LLD. We should probably let users know about the possibly increased memory usage, and jobserver incompatibility: we did so when announcing this landing on nightly. - LLD seems to produce [slightly larger](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=size%3Alinked_artifact&tab=compile) binary artifacts. This can be partially clawed back using Identical Code Folding (`-Clink-args=-Wl,--icf=all`). - Should we detect the outdated `.ctors/.dtors` sections to provide a better error message, even if that should be rare in practice? --- ### Next steps After the FCP completes: - we should land this PR at the beginning of a beta cycle, to maximize time for testing - keep an eye on the beta crater run results for possible linker issues (or do a dedicated beta crater run with only this change) - release a blog post announcing the change, and asking for testing feedback of the appropriate beta

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.