- Feature Name: `generic_pointer_casts`
- Start Date: 2024-01-20
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)

# Summary
[summary]: #summary

- Add a new built-in trait to express (potentially fat) pointer-to-pointer casts in the type system.
- Use this new trait to type check pointer-to-pointer casts, allowing them in more places.
- Relax the trait bounds on the `ptr.cast()` methods to allow casting to fat pointers.

# Motivation
[motivation]: #motivation

The use of `as` casts for pointer conversions is discouraged in favor of several standard library functions that communicate intent more precisely and are less error-prone:

- `ptr::from_ref(r)` / `ptr::from_mut(r)` for reference->pointer casts
- `ptr.cast()` for pointer type conversions, preserving mutability
- `ptr.cast_const()` / `ptr.cast_mut()` for pointer mutability conversions, preserving types
- `ptr.addr()` / `ptr.expose_addr()` for pointer->address casts (unstable)
- `ptr::invalid(addr)` / `ptr::from_exposed_addr(addr)` for address->pointer casts (unstable)

However, the `ptr.cast()` method currently does not support conversions between fat pointers. These kinds of casts are often used to convert from a reference to a reference of a wrapper type. Currently, this can only be achieved with an `as` cast:

#[repr(transparent)]
struct Wrapper<T: ?Sized>(T);

fn from_ref<T: ?Sized>(value: &T) -> &Wrapper<T> {
    unsafe {
        &*(core::ptr::from_ref(value) as *const Wrapper<T>)
    }
}

In this RFC we propose a mechanism that allows using the `.cast()` method instead:

#[repr(transparent)]
struct Wrapper<T: ?Sized>(T);

fn from_ref<T: ?Sized>(value: &T) -> &Wrapper<T> {
    unsafe {
        &*core::ptr::from_ref(value).cast::<Wrapper<T>>()
    }
}

Another example that currently requires `as` casts is extending the lifetime or auto-traits of a trait object. And it gets especially annoying when dealing with `NonNull` fat pointers.

use core::cell::Cell;
use core::ptr::{self, NonNull};

trait Trait {}

thread_local! {
    static TLS: Cell<Option<NonNull<dyn Trait>>> = const { Cell::new(None) };
}

// Store a trait object in a thread-local variable.
// The caller must ensure that the thread-local is
// unset before the trait object lifetime expires.
unsafe fn set_tls_old(val: &dyn Trait) {
    // excuse me what?
    let maybe_ptr = NonNull::new((ptr::from_ref(val) as *const (dyn Trait + 'static)).cast_mut());
    TLS.set(maybe_ptr);
}

unsafe fn set_tls_new(val: &dyn Trait) {
    // so much more readable
    let ptr = NonNull::from(val).cast::<dyn Trait + 'static>();
    TLS.set(Some(ptr));
}

# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

Pointers in Rust are made out of two parts: A *data pointer*, which is platform-dependant and includes the memory address, and a *metadata*, which depends on the pointee type and can store additional information in the pointer. For example, `*mut i32` has a metadata of `()` (no additional data), `*mut [i32]` has a metadata of `usize` (the slice length), and `*mut dyn Trait + Send` has a metadata of `DynMetadata<dyn Trait + Send>` (the vtable). Pointers with a metadata of `()` are called *thin pointers* and pointers with other metadata are called *fat pointers*.

Casting between pointers with `as` is allowed if and only if the the metadata of the source pointer can be *converted to* the metadata of the target pointer. Any metadata can be converted to itself. Furthermore, any metadata can be converted to `()`, which means that any pointer can be cast to a thin pointer.

[RFC 2580](https://rust-lang.github.io/rfcs/2580-ptr-meta.html) added the `Pointee` trait, which can be used to express the metadata of pointers in the type system. This information is used to check whether casting between two pointers `as` is allowed:

use std::ptr::Pointee;

fn example_casts<T, U, V>(fixed: *mut [u8], generic: *mut T)
where
    T: ?Sized + Pointee<Metadata = usize>,
    U: ?Sized + Pointee<Metadata = usize>,
{
    // These are allowed, because `V` is `Sized` and therefore
    // has a metadata of `()`:
    let _ = fixed as *mut V; // old
    let _ = generic as *mut V; // old

    // These are allowed, because pointers to `[u8]`, `[i8]`,
    // `T`, and `U` all have the same metadata of `usize`:
    let _ = fixed as *mut [i8]; // old
    let _ = generic as *mut [i8]; // new
    let _ = fixed as *mut U; // new
    let _ = generic as *mut U; // new
}

We can generalize the example from above to write a function that casts from any pointer to any other pointer as long as both have the same metadata:

fn cast_same_meta<T, U>(ptr: *mut T) -> *mut U
where
    T: ?Sized,
    U: ?Sized + Pointee<Metadata = <T as Pointee>::Metadata>,
{
    // old:
    let (addr, meta) = ptr.to_raw_parts();
    let _ = ptr::from_raw_parts_mut::<U>(addr, meta);
    
    ptr as *mut U // new
}

That alone, however, does not allow arbitrary pointer-to-pointer casts in a generic context, because we cannot express "source and target have the same metadata *or* target is thin". Additionally, we cannot express casts between pointers to trait objects that add or remove auto trait bounds or extend the trait object lifetime, such as the following:

// The metadata of `dyn Trait` is `DynMetadata<dyn Trait>` and the
// metadata of `dyn Trait + Send` is `DynMetadata<dyn Trait + Send>`,
// but we can still cast between them.
fn add_send(ptr: *mut dyn Trait) -> *mut (dyn Trait + Send) {
    ptr as *mut (dyn Trait + Send) // old
}

To solve this problem, we introduce a new trait `MetadataCast<U>`, where `T: MetadataCast<U>` means "the metadata `T` can be converted to the metadata `U`". This trait can then be used to express arbitrary pointer to pointer casts in a generic context:

fn cast_compatible_meta<T, U>(ptr: *mut T) -> *mut U
where
    T: ?Sized,
    U: ?Sized,
    <T as Pointee>::Metadata: MetadataCast<<U as Pointee>::Metadata>,
{
    ptr as *mut U // new
}

Because the required trait bound in the example above is a lot to write, we add a trait alias `PointerCast<U>`, where `T: PointerCast<U>` means "a pointer to `T` can be cast into a pointer to `U`". With this trait alias, the example above can also be written as the following:

fn cast<T, U>(ptr: *mut T) -> *mut U
where
    T: ?Sized + PointerCast<U>,
    U: ?Sized,
{
    ptr as *mut U // new
}

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

## Valid metadata conversions

The trait bound `T: MetadataCast<U>` is satisfied if and only if any of the following is true:

1. `U` is `()`
- This rule allows casting any pointer to any thin pointer.
2. `T` is `DynMetadata<DynT>` and `U` is `DynMetadata<DynU>`, where `DynT` and `DynU` are trait objects that either have the same principal (non-auto) trait or both have no principal trait.
- This rule allows arbitrary changes to the trait object lifetime and auto-traits, but not the principal.
- Removing the principal is not allowed (for now).
- This rule reflects the existing rules for casting between pointers to trait objects and this RFC does not intend to change them.
3. `T` is a subtype of `U`
- In particular, this rule includes `usize: MetadataCast<usize>`.
- This rule is checked after the `DynMetadata` rule, so there is no subtyping relation between `DynMetadata<DynT>` and `DynMetadata<DynU>` (as long as both `DynT` and `DynU` are trait objects, otherwise we do require `DynT` to be a subtype of `DynU`).

Note that `dyn MetadataCast<usize>` does not implement `MetadataCast<usize>`, but does implement `MetadataCast<()>` and `MetadataCast<dyn MetadataCast<usize>>`.

### Rationale for subtyping

There are currently no types with metadata where subtyping is relevant. However, with this RFC it becomes possible write functions that cast between pointers to such imaginary types. Since it is already possible to convert the metadata into a supertype by breaking the pointer into its parts, doing a coercion on the metadata and then reassembling the pointer, it should be possible to cast the metadata to a supertype with `as` casts as well.

fn subtyping<'short, 'long: 'short, T, U>(ptr: *mut T) -> *mut U
where
    T: ?Sized + Pointee<Metadata = &'long ()>,
    U: ?Sized + Pointee<Metadata = &'short ()>,
{
    // Because this allows subtyping ...
    let (addr, meta) = ptr.to_raw_parts();
    let _ = ptr::from_raw_parts_mut::<U>(addr, meta);

    // ... this should allow it as well.
    ptr as *mut U
}

## Language/Compiler changes

`MetadataCast` is added as a language item.

Type checking of pointer to pointer cast expressions is adjusted to require the trait bound `<Src as Pointee>::Metadata: MetadataCast<<Dst as Pointee>::Metadata>` instead of just looking at the types of `Src` and `Dst`.

For items that do not have the `Pointee` trait (or the aliases `Thin` / `PointerCast`) anywhere in their signature, the set of allowed pointer to pointer cast expressions is unchanged.

Pointer to pointer cast expressions will be borrow checked, because these expressions can introduce subtype relations.

## Library changes

The traits `MetadataCast` and `PointerCast` are defined in `core::ptr` and re-exported to `std::ptr`.

The trait bounds on `<*const T>::cast::<U>`, `<*mut T>::cast::<U>`, and `NonNull<T>::cast::<U>` are relaxed from `T: ?Sized` and `U: Sized` to `T: ?Sized + PointerCast<U>` and `U: ?Sized`. Note that `U: Sized` implies `T: PointerCast<U>` for all `T`.

### API overview

// core::ptr

/// A marker for valid metadata conversions during pointer-to-pointer casts.
///
/// A type `T` implements `MetadataCast<U>` if and only if pointers with metadata `T`
/// can be cast to pointers with metadata `U`.
#[lang = "metadata_cast"]
pub trait MetadataCast<U: ?Sized> {}

/// A marker for valid pointer-to-pointer casts.
///
/// A type `T` implements `PointerCast<U>` if and only if a pointer to `T` can be
/// cast into a pointer to `U`.
pub trait PointerCast<U: ?Sized> = Pointee<Metadata: MetadataCast<<U as Pointee>::Metadata>>;

impl<T: ?Sized> (*const T | *mut T | NonNull<T>) {
    // relaxed from (implicit) `U: Sized`
    pub const fn cast<U: ?Sized>(self) -> (*const U | *mut U | NonNull<U>)
    where
        T: PointerCast<U>,
        // ^ equal to <T as Pointee>::Metadata: MetadataCast<<U as Pointee>::Metadata>
    {
        self as _ // for raw pointers
        unsafe { NonNull::new_unchecked(self.as_ptr() as _) } // for NonNull
    }
}

# Drawbacks
[drawbacks]: #drawbacks

> Why should we *not* do this?

It adds complexity to the language for something that can already be expressed differently.

It adds a potential footgun to `.cast()` for casting between slices of differently sized types (e.g. `*const [u8]` to `*const [u32]`), which is almost always incorrect.

Currently, the diagnostics are sub-optimal. While technically correct, they don't really explain the problem:

fn cast<T: ?Sized, U: ?Sized>(x: *mut T) -> *mut U {
    x.cast()
}

error[E0277]: the trait bound `<T as Pointee>::Metadata: MetadataCast<<U as Pointee>::Metadata>` is not satisfied
  --> src/lib.rs:2:7
   |
2  |     x.cast()
   |       ^^^^ the trait `MetadataCast<<U as Pointee>::Metadata>` is not implemented for `<T as Pointee>::Metadata`
   |
   = note: required for `T` to implement `PointerCast<U>`
note: required by a bound in `std::ptr::mut_ptr::<impl *mut T>::cast`
  --> /rust/library/core/src/ptr/mut_ptr.rs:73:12
   |
71 |     pub const fn cast<U: ?Sized>(self) -> *mut U
   |                  ---- required by a bound in this associated function
72 |     where
73 |         T: PointerCast<U>,
   |            ^^^^^^^^^^^^^^ required by this bound in `std::ptr::mut_ptr::<impl *mut T>::cast`
help: consider further restricting the associated type
   |
1  | fn cast<T: ?Sized, U: ?Sized>(x: *mut T) -> *mut U where <T as Pointee>::Metadata: MetadataCast<<U as Pointee>::Metadata> {
   |                                                    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

For more information about this error, try `rustc --explain E0277`.

# Rationale and alternatives
[rationale-and-alternatives]: #rationale-and-alternatives

An alternative approach is skipping `MetadataCast` and making `PointerCast` a built-in trait directly. However, that would complicate the implementation and reduce the expressiveness of the language. Changing the definition of `PointerCast`, for example by forbidding casts from `*const [u8]` to `*const [u32]`, would mean that we can no longer use it to type check `as` casts without breaking backwards compatibility.

Another approach is just defining `cast_same_meta` in the standard library and adjusting the metadata of trait objects to allow casts such as `*mut dyn Trait` -> `*mut (dyn Trait + Send)`. However, that would still not allow most of the examples in the guide-level explanation to compile, even though the compiler has enough informaiton to make them compile.

# Prior art
[prior-art]: #prior-art

There are some similar built-in traits for properties that can otherwise not be expressed in the type systems, for example `Unsize` and `PointerLike`.

# Unresolved questions
[unresolved-questions]: #unresolved-questions

- Should the traits be defined in `core::ptr` or `core::marker`? They are marker traits, but strongly related to `Pointee`, which is defined in `core::ptr`.
- Should we relax the bounds of `.cast()` directly or add new methods `.cast_unsized()` with the relaxed bounds? New methods could avoid potential footguns (and not be insta-stable).

# Future possibilities
[future-possibilities]: #future-possibilities

Pointer-to-address and address-to-pointer casts can be relaxed in a similar way, such that they only require `T: Pointee<Metadata = ()>`. But with this RFC these are already expressible as `ptr as *const () as usize` or `addr as *const () as *const T` respectively.

Conversions between `DynMetadata` types can be implemented in the standard library (e.g `DynMetadata<dyn Trait>` <-> `DynMetadata<dyn Trait + Send>`). This would use `MetadataCast` directly.