Arbitrary Self Types, VTable calls, and Object Safety

disclaimer: does reciever mean the type of the self function parameter?? thats what i wrote this assuming but suddenly I am doubting myself

Calling trait methods on trait objects involves converting the reciever involving the trait object, to the equivalent underling type. For example when calling some method on &dyn Trait it is converted to a &Foo at some point as that is the reciever that was used in the trait implementation.

This operation is not visible to the type checker as (rather fundamentally to the concept of trait objects) we do not actually know what the underlying type of a trait object is.

As we never see the underlying type of a trait object when calling methods, we are unable to check that any where clauses on the method involving Self hold for the underlying type. Due to this there is a fundamental property of the language that must be upheld in order for trait objects to be sound:

Given some impl of Trait for SelfTy, for every method on the trait, the where clauses on dyn Trait's builtin implementation's method must be sufficient to prove all where clauses on SelfTy's method and that the reciever is "well formed".

If this is not upheld then it is not possible for type checking to ensure that calling a trait method on a trait object is sound. It must be true that it being valid to call a trait method on a trait object implies it would be valid to call the underlying type's trait method. It is the job of the object safety rules to ensure this rule is upheld.

The combination of the arbitrary_self_types and derive_coerce_pointee features means that we now have to be sure that the object safety rules are sufficient for any possible ADT that can be written in the language not just the "well behaved" types in std.

Specifically, what we are looking to determine is that given some arbitrary user defined type Ptr<P1...PN, P> and some type Bar<...> that implements Trait<...>, where Trait defines a method with a reciever of Ptr<..., Self>, is the following guaranteed:
All bounds on <dyn Trait<...> as Trait<...>>::method::<...> holding implies that all bounds hold on <Underlying as Trait<...>>::method::<...> and that in turn implies that Ptr<..., Self> is well formed.

There are ~4 ways for bounds to be introduced that we must ensure hold for the underlying type when calling <dyn Trait as Trait>::bar:

impl<...> Trait<...> for Bar</* ... */>
where
  // *1
  /* ... */,
{
  fn bar(
    self: Ptr</* ... */, Self>
  )
  where
    // *2
    Self: SomeTrait,
    // *3
    Self: 'a,
    // *4
    Self: AutoTrait,
  
}

*1 is checked when coercing Foo<...> to dyn Trait.

Trait objects are invariant over the trait parameters so it is not possible to use subtyping to change a type/lifetime present in Trait<...> to one that would result in bounds no longer holding.

The self type is erased so it is not possible to use subtyping to change any lifetimes or types present in Foo<...>.

Are there any asteriks here involving raw pointer recievers as those would allow arbitrarily changing trait parameters. E.g. *mut dyn Trait<'static> as *mut dyn Trait<'a>? No, we forbid pointer casts involving arguments to object types' traits
*2 Arbitrary where clauses on bar involving Self are outright forbidden, there are however two exceptions to this rule, lifetime bounds and auto trait bounds are allowed to reference Self. Whether Self is dyn Trait or Foo<...> does not affect the behaviour of a bound not referencing Self in any way.

Are there any asteriks here involving T: Trait<Assoc = Self> bounds on the impl? Don't think so, predicates of the trait are required to be proven when calling methods on trait objects which requires T: Trait<Assoc = dyn Trait> to hold which is not possible if T: Trait<Assoc = Underlying> also holds (which it must).
*3 Type outlives bounds involving Self are okay if an object type's lifetime is always outlived by the underlying type, as outlives are transitive. dyn Trait<P1..PN> + 'a: 'b holding implies Underlying: 'b if Underlying: 'a holds.

Object types are covariant over the object lifetime so it is only possible to shrink the lifetime of the object type which cannot cause more Self: '... bounds to hold. If an object type is used in a contravariant position this is not necessarily true as fn(&dyn Trait + 'a) can be turned into fn(&dyn Trait + 'static).

It is therefore important that DispatchFromDyn does not allow dispatching from trait objects in contravariant positions. I do not believe it is possible to write a type that is contravariant over a type parameter while also using it in a way that allows accessing a vtable. Having a vtable requires a pointer/reference/ownership of the object type which forces covariance (which turns to invariance in the presence of a PhantomData asking for contravariance)

Unfortunately, raw pointers allow for safe casting of object type lifetimes which allows for an object type to have its underlying type not outlive the object type's lifetime bound. It must be unsafe to cast the lifetime of an object type, i.e. *const dyn Trait + 'a -> *const dyn Trait + 'static cannot be a safe operation in the general case as there may be a Self: 'static bound on a method. #136702
*4 Auto trait bounds with Self as the the self type are also allowed. This is only okay if an object type implements auto traits if (and only if) the underlying type also implements the auto trait.

This is enforced by forbidding manual implementations of auto traits for object types, i.e. unsafe impl Send for dyn Trait is a hard error. Also, when coercing from the underlying type to the object type, auto traits are checked for the underlying type.

This, however, can by bypassed with raw pointer recievers as you may safely cast from *const dyn Trait to *const dyn Trait + Send and then call a method with a Self: Send bound. #127323

Having gone through all of that I feel somewhat convinced that the object safety rules (once fixed for pointers) do behave correctly under arbitrary_self_types/derive_coerce_pointee and allow us to ensure all where clauses on the underlying type's method hold.

Then all that is left is to actually check that in the context of <Bar<...> as Trait<...>>::method, Ptr<..., Bar<...>> is actually well formed. If it is then, in theory, proving all bounds on <dyn Trait<...> as Trait>::method implies that all bounds on <Underlying as Trait>::method hold which implies that Ptr<..., Bar<...>> is well formed.

It being required for soundness to check in the impl (or trait definition) that the reciever is well formed is a new constraint on the language as far as I know. Previously we could just look at the list of types in std that are valid types to do vtable calls from and tell that none of them bound Self in any interesting ways.

Now, however, types with arbitrary where clauses including those that may hold for dyn Trait but not the underlying type of the object type are able to exist and be used for vtable calls. I do not believe this poses any kind of problem for the language but I do think it is interesting.

Why are pointer casts only a problem now

Something that is interesting to think about is why raw pointers allowing safe casting of object types hasn't caused problems before arbitrary_self_types/derive_coerce_pointee (#127323, #136702). The answer to this, I believe, is just that all types in std that allow performing vtable calls also require unsafe to be constructed from a raw pointer.

For example while you can:

Create some Box<Foo>
Coerce it to Box<dyn Trait>
Call Box::into_raw
Cast to*const dyn Trait + Send + Sync + 'static
Call Box::from_raw
Do a vtable call to some method that required Self: Send + 'static

The Box::from_raw step requires unsafe, and this is true of all smart pointers in std that allow for vtable calls. What arbitrary_self_types/derive_coerce_pointee do is allow you to safely construct a smart pointer type that can make vtable calls.

If it's going to be safe to make vtable calls, and safe to construct types that can make vtable calls, then it just has to be unsafe to have an object type that allows methods to be called on its vtable that shouldnt be called^[1].

In some sense raw pointers and vtable calls have never been sound, but as the unsoundness only affects implementors of std/core and the traits involved were relatively obviously "special" it just didn't matter very much.

An alternative might be for derive(CoercePointee) to require the field containing the vtable to be unsafe (though such a feature is not even implemented unstabley). This would effectively force construction of the smart pointer to be unsafe (slash safely encapsulated) ↩︎