changed a year ago  
Linked with GitHub

crater triage for -Clinker-features=+lld -Clink-self-contained=+linker

Summary:

  • PR #117684 from november 12, 2023, 45 regressions (full report), build-only run
  • 44 regressions during linking ("error: linking with cc failed: exit status" that only happen with lld).
  • Out of those, 16 are occurrences of 2 issues tracked by LLVM, and closed as "working as intended".
  • Overall, between 29 and 36 errors are expected or can be avoided.

Flakiness?

There's 1 regression marked as "build compiler error", but doesn't seem to be related to linking.

error[E0275]: overflow evaluating the requirement `BitWriter<endio_bit::endian::BigEndian, _>: std::io::Write`

I can't reproduce this locally. The same commit builds and links without issues, tests all pass, with both linkers.

7 "undefined reference to lzma_*" errors due to --no-allow-shlib-undefined

I believe this is one the 2 lld issues mentioned by ARM in the MCP, and opened on the LLVM project. Closed as "not a bug".

Example with py-spy 0.3.14, log
= note: rust-lld: error: undefined reference due to --no-allow-shlib-undefined: lzma_stream_buffer_decode
    >>> referenced by /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libunwind-ptrace.so

Locally, py-spy's tests are a bit flaky; there's apparent non-determinism, they launch subprocesses, etc.

Probably because it's made to be used for the musl target, but as a dynamically linked library, building the crate has implicit requirements to both libunwind, and liblzma which we can see failing with lld. Adding both explicitly as link-args fixes the issue.

With that I can build with ld, system lld, and rust-lld, and tests behave the same with all 3.

Mentioned in this issue, and in the rstack project (where the author mentions it as being a bug in libunwind).

There are 7 occurrences of this error (4 times as py-spy):

  1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/apachecn-archive.py-spy/log.txt
  2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/benfred.py-spy/log.txt
  3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/lxtrace-0.1.0/log.txt
  4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-0.3.14/log.txt
  5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-for-datakit-0.3.16/log.txt
  6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rsps-0.4.0/log.txt
  7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rstack-self-0.3.0/log.txt

9 "undefined symbol: __gxx_personality_v0" errors

I believe these are occurrences of the second LLVM issue mentioned by ARM: tracked here. Closed as "usage error", and within tolerance of linker incompatibilities.

It seems that this code should be linked with the C++ standard library, that the linker usually takes care of. The difference with ld is unclear, and the LLVM people mention that this could be an ld bug that this works.

In other contexts, it can happen when using gcc by mistake instead of g++, and the same is true when using the cc crate to build a C++ library.

For example, in libsvm-sys-3.24.0, changing the build script to opt-in to c++ library compilation, fixes the issue locally.

In these cases, one could also manually link the appropriate target-specific C++ standard library via -Clink-arg.

There are 9 occurrences of this error:

  1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/angrylion-rdp-plus-sys-0.2.0/log.txt
  2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/daniel-dimovski.poe-bundle-reader/log.txt
  3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/expenses.ktx2-tools/log.txt – though the cause is in https://github.com/Traverse-Research/intel-tex-rs-2
  4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/libsvm-sys-3.24.0/log.txt
  5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/n-elderbroom.wtx_tools/log.txt
  6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/poe_bundle-0.1.5/log.txt
  7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/pollend.angelscript-rs/log.txt
  8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/srplab.starcore_for_rust/log.txt
  9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/xc3_model-0.1.0/log.txt

4 "duplicate symbol" error

For example, duplicate symbol: regerror in https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt

= note: rust-lld: error: duplicate symbol: regerror
    >>> defined at regposerr.c:88 (oniguruma/src/regposerr.c:88)
    >>>            regposerr.o:(regerror) in archive /opt/rustwide/target/debug/deps/libonig_sys-e3f0e36df31d8de7.rlib
    >>> defined at regcomp.c:547 (libgit2/deps/regex/regcomp.c:547)
    >>>            regex.o:(.text.regerror+0x0) in archive /opt/rustwide/target/debug/deps/liblibgit2_sys-e9010b2c607c511f.rlib
    collect2: error: ld returned 1 exit status

Looks like a legit error involving a few C libraries? I'm not sure what happens with ld with such collisions, maybe the last symbol wins.

There are 4 occurrences of this error:

  1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt
  2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ecumene.spin_pg/log.txt
  3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/isgasho.openforum/log.txt
  4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ns6251.spin-cookie-token-sample/log.txt

20 "undefined symbol: à la __start_linkme_*" errors

The crate linkme-0.3.17 https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/linkme-0.3.17/log.txt triggers --gc-sections-related errors for multiple symbols:

= note: rust-lld: error: undefined symbol: __start_linkme_FUNCTION_DESCRIPTIONS
    >>> referenced by autometrics.88bc4422b03cff89-cgu.4
    >>>               autometrics-fe9866b0d32715da.autometrics.88bc4422b03cff89-cgu.4.rcgu.o:(autometrics::__private::FUNCTION_DESCRIPTIONS::hf4420c3da1a7a8b3) in archive /opt/rustwide/target/debug/deps/libautometrics-fe9866b0d32715da.rlib
    >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)

Self-mentioned as "linker shenanigans", linkme relies in practice on a difference of opinions on --gc-sections precision, with ld and lld having the opposite default.

lld provides an argument to flip the default, as mentioned in the error message above, and -Clink-arg=-Wl,-z,nostart-stop-gc does fix the issue for me locally. Maybe we can make a PR to add this information to linkme's readme.

This error also appears in multiple issues in the linkme repository, and apparently its used_linker feature could also fix the issue, but this would requires dependents to use the unstable #![feature(used_with_arg)] feature which is less than ideal.

The error also appears as another undefined symbol pattern, __start_linkm2_*. And, the same mechanism is also present in another crate that doesn't depend on linkme but relies on the same ld default.

There are the 20 occurrences of these errors:

  1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Archisman-Mridha.rust-microservice-template/log.txt
  2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/beoboo.monkey-rust/log.txt
  3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/blackbox_cast-0.1.1/log.txt
  4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/companion-service-0.1.0/log.txt
  5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/derive-codegen-0.0.4/log.txt
  6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/elfo-core-0.2.0-alpha.11/log.txt
  7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/eventheader-0.3.2/log.txt
  8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/ffizz-header-0.5.0/log.txt
  9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Gohla.serde_flexitos/log.txt
  10. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/H4m5t3r.cses-cli/log.txt
  11. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/intertrait-0.2.2/log.txt
  12. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/linkme-0.3.17/log.txt
  13. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mdm.confidante/log.txt
  14. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/metriken-0.3.3/log.txt
  15. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mm-zk.vise/log.txt
  16. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mrobakowski.odra/log.txt
  17. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pliron-0.1.7/log.txt
  18. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/RustyYato.test-harness/log.txt
  19. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/serde_flexitos-0.1.0/log.txt
  20. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/SergeevPavel.morph-endo/log.txt

3 "is incompatible with elf_x86_64" errors

Appears as "incompatible with elf_x86_64" and "… with elf64-x86-64".

rust-lld: error: /opt/rustwide/target/debug/deps/libnrfxlib_sys-a5ab3d8e5beeb8f3.rlib(rpc_framework.c.obj) is incompatible with elf64-x86-64

There are 3 occurrences of this error:

  1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pros-sys-0.3.0/log.txt
  2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/nrfxlib-0.6.0/log.txt
  3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/varesa.robot-iounit/log.txt

1 "invalid local symbol '*' in global part of symbol table" error

= note: rust-lld: error: /opt/rustwide/workdir/fmod-sys/libfmodL.so: invalid local symbol '__bss_start' in global part of symbol table

Occurs in https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/jestarray.fmod_linking_test/log.txt

Errors encountered in prior crater runs:

LLD error:

Hard to reproduce: the crate depends on r3_core-0.1.4 which relies on nightly features (const traits) with a pinned nightly-2023-03-28, and doesn't compile on more recent rustc (and maybe not on the pinned nightly). The latest r3_portkit-0.2.3 still depends on it. There's not been a newer release of r3_core either.

Behavior change about backwards-compatibility:

Trying to register functions called before main, for static initializers, using the old section .ctors. My understanding (gathered from this blog post explaining the history) is that it's deprecated for .init_array.

ld has a link script to place these in the new section, gold has --ctors-in-init-array, both the keep some backwards-compatibility. lld is more recent and doesn't emulate this ld behavior, as .ctors is rare.

In general with lld, one can use a link script (shown in the blog post) to keep compatibility. In this case, changing #[link_section = ".ctors"] to #[link_section = ".init_array"] fixes the issue and works with both linkers. (I can make a PR, but the crate is a 30-lines project, abandoned when published 7 years ago.)

Select a repo