# Arbitrary Self Types, VTable calls, and Object Safety

disclaimer: when i wrote this I did not think through the fact that we allow for `<Self as SuperTrait>::Assoc` in dyn-compatible traits' methods. I don't think this really matters because the exact type the alias normalizes to is preserved in the dyn type and is in an invariant position.

## Soundness requirements of VTable calls

Calling trait methods on trait objects involves converting the reciever involving the trait object, to the equivalent underling type. For example when calling some method on `&dyn Trait` it is converted to a `&Foo` at some point as that is the reciever that was used in the trait implementation.

This operation is not visible to the type checker as (rather fundamentally to the concept of trait objects) we do not actually know what the underlying type of a trait object is.

As we never see the underlying type of a trait object when calling methods, we are unable to check that any where clauses on the method involving `Self` hold for the underlying type. Due to this there is a fundamental property of the language that must be upheld in order for trait objects to be sound:

Given some impl of `Trait` for `SelfTy`, for every method on the trait, the where clauses on `dyn Trait`'s builtin implementation's method must be sufficient to prove all where clauses on `SelfTy`'s method and that the reciever is "well formed".

If this is not upheld then it is not possible for type checking to ensure that calling a trait method on a trait object is sound. It *must* be true that it being valid to call a trait method on a trait object implies it would be valid to call the underlying type's trait method. It is the job of the object safety rules to ensure this rule is upheld.

The combination of the `arbitrary_self_types` and `derive_coerce_pointee` features means that we now have to be sure that the object safety rules are sufficient for any possible ADT that can be written in the language not *just* the "well behaved" types in std.

Specifically, what we are looking to determine is that given some arbitrary user defined type `Ptr<P1...PN, P>` and some type `Bar<...>` that implements `Trait<...>`, where `Trait` defines a method with a reciever of `Ptr<..., Self>`, is the following guaranteed:
All bounds on `<dyn Trait<...> as Trait<...>>::method::<...>` holding implies that all bounds hold on `<Underlying as Trait<...>>::method::<...>` and that in turn implies that `Ptr<..., Self>` is well formed.

## Does this requirement hold

There are ~4 ways for bounds to be introduced that we must ensure hold for the underlying type when calling `<dyn Trait as Trait>::bar`:

impl<...> Trait<...> for Bar</* ... */>
where
  // *1
  /* ... */,
{
  fn bar(
    self: Ptr</* ... */, Self>
  )
  where
    // *2
    Self: SomeTrait,
    // *3
    Self: 'a,
    // *4
    Self: AutoTrait,
  
}

- `*1` is checked when coercing `Foo<...>` to `dyn Trait`.

Trait objects are invariant over the trait parameters so it is not possible to use subtyping to change a type/lifetime present in `Trait<...>` to one that would result in bounds no longer holding.

The self type is erased so it is not possible to use subtyping to change any lifetimes or types present in `Foo<...>`.

Are there any asteriks here involving raw pointer recievers as those would allow arbitrarily changing trait parameters. E.g. `*mut dyn Trait<'static> as *mut dyn Trait<'a>`? No, we forbid pointer casts involving arguments to object types' traits

- `*2` Arbitrary where clauses on `bar` involving `Self` are outright forbidden, there are however two exceptions to this rule, lifetime bounds and auto trait bounds are allowed to reference `Self`. Whether `Self` is `dyn Trait` or `Foo<...>` does not affect the behaviour of a bound not referencing `Self` in any way.

Are there any asteriks here involving `T: Trait<Assoc = Self>` bounds on the impl? Don't think so, predicates of the trait are required to be proven when calling methods on trait objects which requires `T: Trait<Assoc = dyn Trait>` to hold which is not possible if `T: Trait<Assoc = Underlying>` also holds (which it must).

- `*3` Type outlives bounds involving `Self` are okay if an object type's lifetime is always outlived by the underlying type, as outlives are transitive. `dyn Trait<P1..PN> + 'a: 'b` holding implies `Underlying: 'b` if `Underlying: 'a` holds.

Object types are covariant over the object lifetime so it is only possible to shrink the lifetime of the object type which cannot cause more `Self: '...` bounds to hold. If an object type is used in a contravariant position this is not necessarily true as `fn(&dyn Trait + 'a)` can be turned into `fn(&dyn Trait + 'static)`.

It is therefore important that `DispatchFromDyn` does not allow dispatching from trait objects in contravariant positions. I do not believe it is possible to write a type that is contravariant over a type parameter while also using it in a way that allows accessing a vtable. Having a vtable requires a pointer/reference/ownership of the object type which forces covariance (which turns to invariance in the presence of a PhantomData asking for contravariance)

Unfortunately, raw pointers allow for *safe* casting of object type lifetimes which allows for an object type to have its underlying type *not* outlive the object type's lifetime bound. It must be `unsafe` to cast the lifetime of an object type, i.e. `*const dyn Trait + 'a -> *const dyn Trait + 'static` cannot be a safe operation in the general case as there may be a `Self: 'static` bound on a method. [#136702](https://github.com/rust-lang/rust/issues/136702)

- `*4` Auto trait bounds with `Self` as the the self type are also allowed. This is only okay if an object type implements auto traits if (and only if) the underlying type also implements the auto trait.

This is enforced by forbidding manual implementations of auto traits for object types, i.e. `unsafe impl Send for dyn Trait` is a hard error. Also, when coercing from the underlying type to the object type, auto traits are checked for the underlying type.

This, however, can by bypassed with raw pointer recievers as you may safely cast from `*const dyn Trait` to `*const dyn Trait + Send` and then call a method with a `Self: Send` bound. [#127323](https://github.com/rust-lang/rust/issues/127323)

Having gone through all of that I feel somewhat convinced that the object safety rules (once fixed for pointers) do behave correctly under `arbitrary_self_types/derive_coerce_pointee` and allow us to ensure all where clauses on the underlying type's method hold.

Then all that is left is to actually check that in the context of `<Bar<...> as Trait<...>>::method`, `Ptr<..., Bar<...>>` is actually well formed. If it is then, in theory, proving all bounds on `<dyn Trait<...> as Trait>::method` implies that all bounds on `<Underlying as Trait>::method` hold which implies that `Ptr<..., Bar<...>>` is well formed.

It being required *for soundness* to check in the impl (or trait definition) that the reciever is well formed is a new constraint on the language as far as I know. Previously we could just look at the list of types in std that are valid types to do vtable calls from and tell that none of them bound `Self` in any interesting ways.

Now, however, types with *arbitrary* where clauses including those that may hold for `dyn Trait` but not the underlying type of the object type are able to exist and be used for vtable calls. I do not believe this poses any kind of problem for the language but I do think it is *interesting*.

## Why are pointer casts only a problem *now*

Something that is interesting to think about is why raw pointers allowing safe casting of object types hasn't caused problems before `arbitrary_self_types/derive_coerce_pointee` ([#127323](https://github.com/rust-lang/rust/issues/127323), [#136702](https://github.com/rust-lang/rust/issues/136702)). The answer to this, I believe, is just that all types in std that allow performing vtable calls also require unsafe to be constructed from a raw pointer.

For example while you *can*:
- Create some `Box<Foo>`
- Coerce it to `Box<dyn Trait>`
- Call `Box::into_raw`
- Cast to`*const dyn Trait + Send + Sync + 'static`
- Call `Box::from_raw`
- Do a vtable call to some method that required `Self: Send + 'static`

The `Box::from_raw` step requires unsafe, and this is true of all smart pointers in std that allow for vtable calls. What `arbitrary_self_types`/`derive_coerce_pointee` do is allow you to *safely* construct a smart pointer type that can make vtable calls.

If it's going to be safe to make vtable calls, and safe to construct types that can make vtable calls, then it just has to be *unsafe* to have an object type that allows methods to be called on its vtable that shouldnt be called[^1].

In some sense raw pointers and vtable calls have *never* been sound, but as the unsoundness only affects implementors of `std`/`core` and the traits involved were relatively obviously "special" it just didn't matter very much.

[^1]: An alternative might be for `derive(CoercePointee)` to require the field containing the vtable to be unsafe (though such a feature is not even implemented unstabley). This would effectively force construction of the smart pointer to be unsafe (slash safely encapsulated)