d# Target modifiers

## Summary

* We introduce the concept of "target modifier". A target modifier is a flag where may be unsound if you link together two compilation units that disagree on the flag.
* We fail the build if rustc can see two Rust compilation units that do not agree on the exact set of target modifier flags.
* There are already several existing flags that could fall into this category. There are also hypothetical new flags that do.
* The error can be silenced using the `-Cunsafe-allow-abi-mismatch` escape hatch.
* Not having a stable way to build stdlib crates does not block stabilization of target modifiers.
* As a future extension we may be able to relax the rules to allow some specific kinds of mismatches.
* This RFC does not stabilize any target modifiers. That should happen in follow-up MCPs/FCPs/RFCs/etc.

## Motivation

As Rust expands into low-level domains, there will be a need for precise control over how code is compiled. This often manifests as a new compiler flag. Some of these flags trigger undefined behavior if used incorrectly, which is in tension with Rust's safety goals. This RFC proposes a new mechanism to allow use of such flags while also preventing undefined behavior.

The primary goal of this RFC is to unblock *stabilization* of target modifier flags. Adding them as unstable (and unsound) flags is already happening today without this RFC.

### The Linux Kernel

The Linux Kernel has run into a handful of cases where it is necessary to tweak the ABI used in the kernel. Often, this is done conditionally depending on a configuration option. A few examples:

* When using `CONFIG_SHADOW_CALLSTACK` the x18 register must be reserved in the ABI with `-Zfixed-x18`.
* The `-Ctarget-feature=-neon` flag is used to prevent use of floating points on arm.
* On 32-bit x86, `-Zreg-struct-return` and `-Zregparm=3` are used.
* When using `CONFIG_CFI_CLANG` the kCFI sanitizer is enabled with `-Zsanitizer=kcfi`. Unlike most other sanitizers, this sanitizer is used in production.
* In several different cases, `-Zpatchable-function-entry` is used to add nops before or after the function entrypoint. When mixed with `-Zsanitizer=kcfi` this causes special considerations as kcfi works by placing a tag before the function entrypoint.
* To support `MITIGATION_RETPOLINE` and `MITIGATION_SLS`, `target.json` is used on x86.

We expect there to be more examples in the future.

### Sanitizers

There is [an ongoing effort to stabilize some of the sanitizers](https://github.com/rust-lang/rust/issues/123615). However, this effort explicitly aims to stabilize sanitizers that can be used without rebuilding the stdlib. With this RFC, that is no longer a blocker as the remaining sanitizers can be classified as a target modifier and then stabilized.

### Embedded Targets

Currently, embedded platforms such as `thumb*` or `rv*` use separate targets for configuration with significant ABI changes. For `thumb*` targets, this is currently limited to Hard- vs Soft-Float which can cause issues when linked. However for RISC-V targets, the F (32-bit hardware float), D (64-bit hardware float), and Q (128-bit hardware float) extensions all can [potentially change the ABI](https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#named-abis), which would increase the number of required targets.

### Existing -C flags that are unsound

It has recently been discovered that several existing `-C` flags modify the ABI, making them unsound. Examples:

* [`-Csoft-float`](https://github.com/rust-lang/rust/issues/129893)
* [`-Ctarget-feature=-neon`](https://github.com/rust-lang/rust/issues/131058)
* [`-Clinker-plugin-lto`](https://github.com/rust-lang/rust/issues/127979)
* [`-Cllvm-args`](https://github.com/rust-lang/rust/issues/131800#issuecomment-2418595757)
* Possibly `-Ccode-model` and `-Crelocation-model`

This problem is a new discovery and it's still not clear how to solve it. Target modifiers will not solve all of the flags; [some flags are just unfixable and need to be removed](https://github.com/rust-lang/rust/issues/130968). But I expect that target modifiers will be the solution for some of these flags.

## Guide-level explanation

The Rust compiler has many flags that affect how source code is turned into machine code. Some flags can be turned on and off for each CU (compilation unit) in your project separately, and other flags must be applied to the entire application as a whole. Usually flags are in the latter category because they change some aspect of the ABI. For example, `-Zreg-struct-return` changes how to return a struct from a function call, and both the caller and callee must agree on how to do that even if they are in different CUs.

The Rust compiler will detect if you incorrectly use a flag that must be applied to the application as a whole. For example, if you compile the standard library with `-Zreg-struct-return`, but don't pass the flag when compiling a dependency, then you will get the following error:

error: mixing -Zreg-struct-return will cause an ABI mismatch

help: This error occurs because the -Zreg-struct-return flag modifies the ABI, 
      and different crates in your project were compiled with inconsistent
      settings.
help: To resolve this, ensure that -Zreg-struct-return is set to the same value
      for all crates during compilation.
help: To ignore this error, recompile with the following flag:
      -Cunsafe-allow-abi-mismatch=reg-struct-return

As an escape hatch, you can use `-Cunsafe-allow-abi-mismatch=reg-struct-return` to disable the error. Using this flag is unsafe as ABI mismatches are undefined behavior. However, there may be some cases where the check is too strict, and you can use the flag to proceed in those cases.

The requirement that all CUs agree includes stdlib crates (core, alloc, std), so using these flags usually requires that you compile your own standard library with `-Zbuild-std` or by directly invoking `rustc`. That said, some flags (e.g., `-Cpanic`) have mechanisms to avoid this requirement.

## Reference-level explanation

A compiler flag can be classified as a _target modifier_. When a flag is a target modifier, it can be undefined behavior to link together two CUs that disagree on the flag.

To avoid unsoundness from mixing target modifiers, rustc will store the set of target modifiers in use in the crate metadata of each crate. Whenever rustc is invoked, it will inspect the crate metadata of all crates that are visible to it (usually the current crate and its direct dependencies) and emit an error if any of them have mismatched target modifiers in use.

The `-Cunsafe-allow-abi-mismatch=flagname` flag can be used when compiling a crate to indicate that it should not be included in the list of crates when checking that all crates agree on `flagname`.

### Stabilization of target modifiers

Target modifiers can be stabilized even if there's no way to use them without also using an unstable feature such as `-Zbuild-std`.

## Drawbacks

### Teaching

We should be careful to not introduce too many concepts that end-users have to learn.

It is intentional that the guide-level section of this RFC does not use the word "target modifier". The "target modifier" name is not intended to be used outside of the compiler internals and very technical documentation. Compiler errors should not say "error: trying to mix target modifiers" or something like that; rather the error should just say that mixing `-Cfoo` may cause ABI issues.

For similar reasons, the flag for silencing the error is called `-Cunsafe-allow-abi-mismatch` with the word "ABI" to avoid having to teach the user about mismatched flags or target modifiers.

## Rationale and Alternatives

### Why not just add flags like normal?

Preventing undefined behaviour is an important goal of the Rust project. If we add flags that change the abi, then that is in direct opposition to that goal, as mixing them would lead to UB.

### Why not just add new targets?

The flag that started this entire discussion is `-Zfixed-x18`. This flag changes the ABI by changing the x18 register from a caller-saved temporary register to a reserved register. At the time, people suggested adding a new target (e.g., `aarch64-unknown-none-fixed18`), instead of adding a dedicated `-Zfixed-x18` flag.

The primary benefit of adding a new target is that it's a workaround for `-Zbuild-std` being unstable. Each new target will get a prebuilt stdlib, which sidesteps the need for building your own stdlib.

This RFC does not propose this solution because:
1. The primary benefit is not being blocked on stabilization of `-Zbuild-std`. However, I don't think we really are blocked on stabilization of `-Zbuild-std` in the first place. See the stabilization of target modifier flags section below.
2. Target modifiers help with other problems such as unblocking the stabilization of sanitizers as well as existing `-C` flags that are unsound due to ABI issues. Adding new targets would leave these issues unsolved.
3. Adding new targets risks an exponential number of targets. In the kernel on x86 we would need 8 different targets to support the different possible kernel configurations. It's not hard to imagine that number growing to 16 or 32 targets in the near future, especially once you consider that other embedded projects may have their own set of target modifier flags.
4. Adding new prebuilt stdlibs does not actually help the projects that need these flags. Even if a prebuilt stdlib is provided for every combination of ABI-affecting flags that the kernel may need, the kernel has other reasons that require building a custom `core`.

### Why not use `target.json`

Because the `target.json` feature is perma-unstable, and this RFC primarily concerns itself with unblocking the _stabilization_ of these flags. Adding target modifiers as unstable flags is already happening today.

One possible alternative would be to stabilize a subset of `target.json`. However, I don't think there's much benefit to this. It just means that you now have to learn two different ways of passing flags to the compiler. See the Teaching section above.

It would also be inconvenient to use in external build systems. Right now, the kernel passes the `-Zfixed-x18` flag like this:

ifeq ($(CONFIG_SHADOW_CALL_STACK), y)
KBUILD_CFLAGS    += -ffixed-x18
KBUILD_RUSTFLAGS += -Zfixed-x18
endif

If `-Zfixed-x18` had to be specified in a `target.json` file, it would need to happen in an entirely different part of the kernel build system. It is better to specify the rustc flag together with the clang flag.

### Stabilization of target modifiers

Using a target modifier without rebuilding the Rust stdlib is often not possible. This means that some target modifiers can only be used in tandem with `-Zbuild-std`, which is currently unstable.

However, there's no reason we _have_ to block the stabilization of target modifiers on the stabilization of `-Zbuild-std`. If a target modifier `-Cfoo` is stabilized, then you can break users of `-Cfoo` with the reason "we changed the way you pass flags to `core`", but you can't break users with the reason "we renamed `-Cfoo` to `-Cbar`; this is okay because you're also using `-Zbuild-std` even though the rename is unrelated to `-Zbuild-std`".

### Not all mismatches are unsound

This RFC says that mismatching target modifiers in any way results in a build error. However, there are a lot of cases where the real rules are more complicated than that. For example, with the following three CUs:

* CU A Compiled with `-Zfixed-x18 -Zsanitizer=shadow-call-stack`.
* CU B Compiled with `-Zfixed-x18`.
* CU C Compiled with neither flag.

It is unsound to link together CUs A and C, but linking A with B or B with C is sound.

That said, real-world use-cases for mismatching a target modifier are pretty rare. The only case I'm aware of where people would actually want to mismatch a target modifier is the runtime for a sanitizer. For example, when ASAN detects a failure, it calls into a special ASAN-failure handler function. The function for handling ASAN-failures should not be sanitized.

Making the compiler accept specific mismatches that are sound is out of scope for this RFC. Such decisions will be made on a flag-by-flag basis in follow-up decisions (most likely an MCP). Until then, end-users can use `-Cunsafe-allow-abi-mismatch` to proceed in such cases.

### Problems with mixing non-target-modifiers

It's rather common to want a flag to be applied everywhere, even if it is not ABI breaking. This section discusses some of these cases.

#### Speculation and vulnerability mitigations

There are some mitigations that are used to mitigate CPU speculation vulnerabilities (e.g. spectre) or used to make exploitation of vulnerabilities harder (e.g. [ROP](https://en.wikipedia.org/wiki/Return-oriented_programming)), which work by telling the compiler to never generate certain "gadgets" (sequences of instructions). These mitigations usually don't change the ABI, as they just change how code is generated within functions.

The problem is that the attacks you are trying to mitigate involve jumping to an arbitrary attacked-controlled address. This jump is can be a real jump (with ROP) or a jump that is only speculated (with spectre-like issues). Either way, since the jump is to an attacker-controlled address, if you have an instance of the gadget _anywhere in your program_, you are vulnerable. This means that if you only apply the mitigation to some CUs, then the CUs that lack the mitigation will probably have the problematic gadget, and the mitigation doesn't work at all.

#### Sanitizers

This case is rather similar to mitigations.

Some sanitizers can be mixed and matched between CUs without breaking the ABI. For example, on the Android aarch64 target, the shadow call stack sanitizer does not change the ABI, and can be freely mixed between CUs. However, the sanitizer does not catch violations in CUs that don't enable the sanitizer.

For sanitizers used in production (such as shadow call stack or kcfi) this is particularly problematic, as a vulnerability in sanitized code may allow you to jump into unsanitized code.

#### .note.gnu.property

In the case of BTI (`-Zbranch-protection=bti`), the mitigation relies on the kernel's ELF loader setting a special bit in the page table. However, setting this bit is only valid if BTI is enabled everywhere. The compiler will use a section called `.note.gnu.property` to tell the linker whether BTI is enabled, and the linker only propagates `.note.gnu.property` if all CUs agree on it. This means that if one CU is missing BTI, the linker will disable it for the entire executable, and the kernel's ELF loader will not set the bit in the page tables when loading the machine code, rendering BTI ineffective.

#### Performance

Another reason is performance. One some targets, the precompiled stdlib always comes with panic landing pads, even if you're using `-Cpanic=abort`. It's also usually compiled with a very minimal set of target features for greater compatibility. These discrepancies can have an unacceptable impact on performance.

#### Code patching

You might use `-Zbranch-protection=pac-ret` or `-Zpatchable-function-entry` to insert special instructions at the beginning/end of all functions so you can use runtime code-patching to replace them later. It is only because of the runtime code-patching logic that these flags need to be used everywhere.

#### Debugging information

Mixing CUs with different options for `-Cforce-unwind-tables`, `-Zdwarf-version` or `-Zdebuginfo-compression` may result in a binary that you consider to be invalid as you may be unable to read the debugging information. But it would not be an ABI issue.

#### Detection for non-target-modifiers

We will most likely want some way to detect mismatches of some of the cases above; especially the mitigations. However, as these cases are not ABI breaking, the flag for silencing the error/warning should not include the word "unsafe".

## Prior art

### The panic strategy

The Rust compiler already *has* infrastructure to detect flag mismatches: the flags `-Cpanic` and `-Zpanic-in-drop`. The prebuilt stdlib comes with different pieces depending on which strategy is used, although panic landing flags are not entirely removed when using `-Cpanic=abort`, as only part of the prebuilt stdlib is switched out.

### Global target modifiers

A suggestion that has come up several times ([1](https://github.com/rust-lang/rust/issues/116972), [2](https://github.com/rust-lang/rust/issues/116973), [3](https://github.com/rust-lang/rust/issues/121970#issuecomment-1978605782)) is to have a variation of `-Ctarget-feature=` that must be applied globally, which could be called `-Cglobal-target-features=`. This is very similar to this RFC, though it is broader as the "target modifier" concept can apply to any flag and not just to one `-Cglobal-target-features=` flag.

### Stabilization of things that require nightly features

This RFC proposes that we shouldn't block stabilization of target modifiers on a stable way to build libcore. There is precedent in the rust project for unblocking stabilizations in this manner: When `#![no_std]` was stabilized, [the RFC](https://rust-lang.github.io/rfcs/1184-stabilize-no_std.html) said the following:

> As mentioned above, there are three separate lang items which are required by the libcore library to link correctly. These items are:
>
> * `panic_fmt`
> * `stack_exhausted`
> * `eh_personality`
>
> This RFC does not attempt to stabilize these lang items for a number of reasons:
>
> * The exact set of these lang items is somewhat nebulous and may change over time.
> * The signatures of each of these lang items can either be platform-specific or it’s just “too weird” to stabilize.
> * These items are pretty obscure and it’s not very widely known what they do or how they should be implemented.
>
> Stabilization of these lang items (in any form) will be considered in a future RFC.

This means that no-std can't actually be used without providing these symbols in some other way. Doing so is unstable.

### .note.gnu.property

The `.note.gnu.property` section discussed previously is an example of C code detecting mismatches of a flag at link time.

## Unresolved questions

This RFC does not stabilize any target modifiers. Such decisions should be made as a follow-up to this RFC on a flag-by-flag basis using the usual process for stabilizing a compiler flag.

The `-Cunsafe-allow-abi-mismatch` flag will be stabilized when the first target modifier is stabilized.

## Future possibilities

A possible future extension could be to detect inconsistencies between the ABI of C code an Rust code. This would be an interesting extension, but it is not critical for target modifiers as calling into C is inherently unsafe to start with.