crater triage for `-Clinker-features=+lld -Clink-self-contained=+linker`

---
#### Summary:

- PR #117684 from november 12, 2023, [45 regressions](https://github.com/rust-lang/rust/pull/117684#issuecomment-1807208716) ([full report](https://crater-reports.s3.amazonaws.com/pr-117684-2/index.html)), build-only run
- 44 regressions during linking ("error: linking with `cc` failed: exit status" that only happen with `lld`).
- Out of those, 16 are occurrences of 2 issues tracked by LLVM, and closed as "working as intended".
- Overall, between 29 and 36 errors are expected or can be avoided.

---

#### Flakiness?

There's 1 regression marked as "build compiler error", but doesn't seem to be related to linking.

- `lcdr.lu_packets`: https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/lcdr.lu_packets/log.txt

error[E0275]: overflow evaluating the requirement `BitWriter<endio_bit::endian::BigEndian, _>: std::io::Write`

I can't reproduce this locally. The same commit builds and links without issues, tests all pass, with both linkers.

---

#### 7 "undefined reference to `lzma_*`" errors due to `--no-allow-shlib-undefined`

I believe this is one the [2 `lld` issues mentioned by ARM](https://github.com/rust-lang/compiler-team/issues/510#issuecomment-1102741002) in the MCP, and opened on the [LLVM project](https://github.com/llvm/llvm-project/issues/54700). Closed as "not a bug".

##### Example with `py-spy 0.3.14`, [log](https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-0.3.14/log.txt)

= note: rust-lld: error: undefined reference due to --no-allow-shlib-undefined: lzma_stream_buffer_decode
    >>> referenced by /usr/lib/gcc/x86_64-linux-gnu/11/../../../x86_64-linux-gnu/libunwind-ptrace.so

Locally, `py-spy`'s tests are a bit flaky; there's apparent non-determinism, they launch subprocesses, etc.

Probably because it's made to be used for the musl target, but as a dynamically linked library, building the crate has implicit requirements to both [`libunwind`](https://github.com/benfred/py-spy/issues/334), and `liblzma` which we can see failing with `lld`. Adding both explicitly as link-args fixes the issue.

With that I can build with `ld`, system `lld`, and `rust-lld`, and tests behave the same with all 3.

Mentioned in [this issue](https://github.com/benfred/py-spy/issues/334), and in the [rstack project](https://github.com/sfackler/rstack/issues/21) (where the author mentions it as being a bug in `libunwind`).

There are 7 occurrences of this error (4 times as `py-spy`):

1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/apachecn-archive.py-spy/log.txt
2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/benfred.py-spy/log.txt
3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/lxtrace-0.1.0/log.txt
4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-0.3.14/log.txt
5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/py-spy-for-datakit-0.3.16/log.txt
6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rsps-0.4.0/log.txt
7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/rstack-self-0.3.0/log.txt

---

#### 9 "undefined symbol: __gxx_personality_v0" errors

I believe these are occurrences of the second LLVM issue mentioned by ARM: [tracked here](https://github.com/llvm/llvm-project/issues/54701). Closed as "usage error", and within tolerance of linker incompatibilities.

It seems that this code should be linked with the C++ standard library, that the linker usually takes care of. The difference with `ld` is unclear, and the LLVM people mention that this could be an `ld` bug that this works.

In other contexts, it can happen when using `gcc` by mistake instead of `g++`, and the same is true when using the `cc` crate to build a C++ library.

For example, in `libsvm-sys-3.24.0`, changing the build script to [opt-in to c++ library compilation](https://docs.rs/cc/latest/cc/#c-support), fixes the issue locally.

In these cases, one could also manually link the appropriate target-specific C++ standard library via `-Clink-arg`.

There are 9 occurrences of this error:

1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/angrylion-rdp-plus-sys-0.2.0/log.txt
2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/daniel-dimovski.poe-bundle-reader/log.txt
3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/expenses.ktx2-tools/log.txt -- though the cause is in https://github.com/Traverse-Research/intel-tex-rs-2
4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/libsvm-sys-3.24.0/log.txt
5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/n-elderbroom.wtx_tools/log.txt
6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/poe_bundle-0.1.5/log.txt
7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/pollend.angelscript-rs/log.txt
8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/srplab.starcore_for_rust/log.txt
9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/xc3_model-0.1.0/log.txt

---

#### 4 "duplicate symbol" error

For example, `duplicate symbol: regerror` in https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt

= note: rust-lld: error: duplicate symbol: regerror
    >>> defined at regposerr.c:88 (oniguruma/src/regposerr.c:88)
    >>>            regposerr.o:(regerror) in archive /opt/rustwide/target/debug/deps/libonig_sys-e3f0e36df31d8de7.rlib
    >>> defined at regcomp.c:547 (libgit2/deps/regex/regcomp.c:547)
    >>>            regex.o:(.text.regerror+0x0) in archive /opt/rustwide/target/debug/deps/liblibgit2_sys-e9010b2c607c511f.rlib
    collect2: error: ld returned 1 exit status

Looks like a legit error involving a few C libraries? I'm not sure what happens with `ld` with such collisions, maybe the last symbol wins.

There are 4 occurrences of this error:
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/angeldev7.bat/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ecumene.spin_pg/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/isgasho.openforum/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/ns6251.spin-cookie-token-sample/log.txt

---

#### 20 "undefined symbol: à la `__start_linkme_*`" errors

The crate [`linkme-0.3.17`](https://github.com/dtolnay/linkme) https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/linkme-0.3.17/log.txt triggers `--gc-sections`-related errors for multiple symbols:

= note: rust-lld: error: undefined symbol: __start_linkme_FUNCTION_DESCRIPTIONS
    >>> referenced by autometrics.88bc4422b03cff89-cgu.4
    >>>               autometrics-fe9866b0d32715da.autometrics.88bc4422b03cff89-cgu.4.rcgu.o:(autometrics::__private::FUNCTION_DESCRIPTIONS::hf4420c3da1a7a8b3) in archive /opt/rustwide/target/debug/deps/libautometrics-fe9866b0d32715da.rlib
    >>> the encapsulation symbol needs to be retained under --gc-sections properly; consider -z nostart-stop-gc (see https://lld.llvm.org/ELF/start-stop-gc)

Self-mentioned as "linker shenanigans", `linkme` relies in practice [on a difference of opinions on `--gc-sections` precision](https://lld.llvm.org/ELF/start-stop-gc), with `ld` and `lld` having the opposite default.

`lld` provides an argument to flip the default, as mentioned in the error message above, and `-Clink-arg=-Wl,-z,nostart-stop-gc` does fix the issue for me locally. Maybe we can make a PR to add this information to `linkme`'s readme.

This error also appears in multiple issues in the `linkme` repository, and apparently its `used_linker` feature could also fix the issue, but this would requires dependents to use the unstable [`#![feature(used_with_arg)]`](https://github.com/rust-lang/rust/issues/93798) feature which is less than ideal.

The error also appears as another undefined symbol pattern, `__start_linkm2_*`. And, the same mechanism is also present in another crate that doesn't depend on `linkme` but relies on the same `ld` default.

There are the 20 occurrences of these errors:

1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Archisman-Mridha.rust-microservice-template/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/beoboo.monkey-rust/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/blackbox_cast-0.1.1/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/companion-service-0.1.0/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/derive-codegen-0.0.4/log.txt
2. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/elfo-core-0.2.0-alpha.11/log.txt
3. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/eventheader-0.3.2/log.txt
4. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/ffizz-header-0.5.0/log.txt
5. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/Gohla.serde_flexitos/log.txt
6. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/H4m5t3r.cses-cli/log.txt
7. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/intertrait-0.2.2/log.txt
8. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/linkme-0.3.17/log.txt
9. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mdm.confidante/log.txt
10. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/metriken-0.3.3/log.txt
11. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mm-zk.vise/log.txt
12. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/mrobakowski.odra/log.txt
13. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pliron-0.1.7/log.txt
14. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/RustyYato.test-harness/log.txt
15. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/serde_flexitos-0.1.0/log.txt
16. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/SergeevPavel.morph-endo/log.txt

---

#### 3 "is incompatible with elf_x86_64" errors

Appears as "incompatible with elf_x86_64" and "... with elf64-x86-64".

rust-lld: error: /opt/rustwide/target/debug/deps/libnrfxlib_sys-a5ab3d8e5beeb8f3.rlib(rpc_framework.c.obj) is incompatible with elf64-x86-64

There are 3 occurrences of this error:
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/pros-sys-0.3.0/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/reg/nrfxlib-0.6.0/log.txt
1. https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/varesa.robot-iounit/log.txt

---

#### 1 "invalid local symbol '*' in global part of symbol table" error

= note: rust-lld: error: /opt/rustwide/workdir/fmod-sys/libfmodL.so: invalid local symbol '__bss_start' in global part of symbol table

Occurs in https://crater-reports.s3.amazonaws.com/pr-117684-2/try%2353e03135c36e913ff664ce8329041b7e3c2fd358/gh/jestarray.fmod_linking_test/log.txt

---
#### Errors encountered in prior crater runs:

LLD error:

- doctest, error: relocation R_X86_64_64 cannot be used against local symbol; recompile with -fPIC https://crater-reports.s3.amazonaws.com/pr-96025/try%231b74e096b9bfb06f84a3007193dcd2f059cbdf6a/reg/r3_portkit-0.2.1/log.txt

Hard to reproduce: the crate depends on `r3_core-0.1.4` which relies on nightly features (const traits) with a pinned `nightly-2023-03-28`, and doesn't compile on more recent rustc (and maybe not on the pinned nightly). The latest `r3_portkit-0.2.3` still depends on it. There's not been a newer release of `r3_core` either.

Behavior change about backwards-compatibility:
- `thread 'test::works' panicked at 'assertion failed: unsafe { RAN }', src/lib.rs:95:9` in https://crater-reports.s3.amazonaws.com/pr-96025/try%231b74e096b9bfb06f84a3007193dcd2f059cbdf6a/reg/constructor-0.1.0/log.txt and https://crater-reports.s3.amazonaws.com/pr-96025/try%231b74e096b9bfb06f84a3007193dcd2f059cbdf6a/reg/contructor-derive-0.1.1/log.txt

Trying to register functions called before main, for static initializers, using the old section `.ctors`. My understanding (gathered from [this blog post](https://maskray.me/blog/2021-11-07-init-ctors-init-array) explaining the history) is that it's deprecated for `.init_array`.

`ld` has a link script to place these in the new section, `gold` has `--ctors-in-init-array`, both the keep some backwards-compatibility. `lld` is more recent and doesn't emulate this `ld` behavior, as `.ctors` is rare.

In general with `lld`, one can use a link script (shown in the blog post) to keep compatibility. In this case, changing [`#[link_section = ".ctors"]`](https://github.com/purpleposeidon/constructor/blob/a0b677bfde76aea42123fb923fb024547b12f695/src/lib.rs#L54) to `#[link_section = ".init_array"]` fixes the issue and works with both linkers. (I can make a PR, but the crate is a 30-lines project, abandoned when published 7 years ago.)