# New design overview

The core idea of this new design (developed in a conversation together with Alice & Josh & Ding at RustWeek) is to reuse the existing borrow checker logic for shared and mutable references. These directly translate to shared and exclusive projections.

The syntax also needs a change, since we need to know which kind of operation the user is asking for (a type can support both at the same time, just like `Deref` and `DerefMut`). The new (not-yet-bikeshed) syntax is very much inspired by field reborrowing (`&base.field`/`&mut base.field`): `@base->field` and `@mut base->field`.

The rules are exactly the same as with field reborrowing: a field may either be shared-projected multiple times simultaneously, or exclusive-projected once.

I already have implemented this design using macros: <https://github.com/Rust-for-Linux/field-projection/tree/new>. It has a couple rough edges, requiring a derive macro on all structs & writing `start_proj!` whenever one wants to project a value, as well as having to wrap projection operations in the `p!()` macro. But aside from that it works surprisingly well.

[experimental implementation]: https://github.com/Rust-for-Linux/field-projection/tree/new

I've had problems with writing v3 of the RFC, certain details were not clear at all to me. Thus I created the implementation above (it showed me for example that the `Project::project` function needs a lifetime, but more on that later). After finishing, I tried again to write v3 of the RFC. This time though it was difficult creating an understandable narrative/text for the feature. I get the feeling that I'm too deep in the weeds to explain it to someone not familiar with the concept. Therefore this is a pre-RFC intended to outline the new design and some open design issues. I hope to get some help writing the actual text of the RFC :)

# Examples

The examples given below are also [implemented](https://github.com/Rust-for-Linux/field-projection/blob/new/tests/ui/compile-fail/pre_rfc.rs) using the [experimental implementation].

Structs used throughout the examples:

struct Foo {
    field: usize,
    bar: Bar,
}

struct Bar {
    x: u32,
}

Here is a very simple first example with type annotations:

fn example1(foo: &mut MaybeUninit<Foo>) -> usize {
    fn validate(_: &MaybeUninit<usize>) {}

    let field: &MaybeUninit<usize> = @foo->field;
    // we are allowed to share-project `field` again:
    validate(@foo->field);
    let bar: &mut MaybeUninit<Bar> = @mut foo->bar;
    let x: &mut MaybeUninit<u32> = @mut bar->x;
    x.write(42);
    unsafe { field.assume_init_read() }
}

Exclusive projections are... well exclusive, so this doesn't compile:

fn example2(foo: &mut MaybeUninit<Foo>) {
    let a = @mut foo->field;
    //      --------------- first exclusive projection occurs here
    let b = @mut foo->field;
    //      ^^^^^^^^^^^^^^^ second exclusive projection occurs here
    MaybeUninit::write(a, 42);
    //                 - first projection later used here
}

When a value is being projected in any way, it must not be accessed normally:

fn example3(foo: &mut MaybeUninit<Foo>) -> usize {
    let field = @foo->field;
    //          ----------- shared projection occurs here
    let value = Foo {
        field: 42,
        bar: Bar { x: 72 },
    };
    MaybeUninit::write(foo, value);
    //^^^^^^^^^^^^^^^^^^^^^^^^^^^^ projected value used here
    unsafe { field.assume_init_read() }
    //       ----- shared projection used here
} 

---

`<tangent>`

This is currently not enforced by the [experimental implementation] for shared projections and shared borrows, so this compiles fine:

fn example3_allowed_in_exp_impl(foo: &mut MaybeUninit<Foo>) -> usize {
    start_proj!(foo);
    let field = p!(@foo->field);
    let _ = unsafe { foo.assume_init_read() };
    unsafe { field.assume_init_read() }
}

This question is discussed in more detail below.

`</tangent>`

---

# Proposal details

The next few sections are focusing on the language & library additions educated by the [experimental implementation].

# Field types

The compiler generates (on-demand, so only if they are actually used to preserve compile time) a type for every field of every struct. They can be named by using the `field_of!` macro:

pub macro field_of($Container:ty, $($fields:expr)+ $(,)?) {
    /* built-in */
}

It uses the same syntax as `offset_of!`, but resolves to the respective field type instead of a `usize`.

The type is defined locally where the struct is, so authors of the struct can implement any trait for the field representing types.

## Field traits

Field types implement the following trait:

/// Type representing a field of a `struct`, `union` or tuple.
///
/// # Safety
///
/// Given a valid value of type `Self::Base`, there exists a valid value of type `Self::Type` at
/// byte offset `OFFSET`.
pub unsafe trait UnalignedField: Sized {
    /// The type of the base where this field exists in.
    type Base: ?Sized;

    /// The type of the field.
    type Type: ?Sized;

    /// The offset of the field in bytes.
    const OFFSET: usize;
}

### Aligned fields

Field types of non `repr(packed)` structs also implement the `Field` trait:

/// Type representing an aligned field of a `struct`, `union` or tuple.
///
/// # Safety
///
/// Given a well-aligned value of type `Self::Base`, the field at `Self::OFFSET` of type
/// `Self::Type` is well-aligned.
pub unsafe trait Field: UnalignedField {}

### Pinnable fields

To support pin-projections, we need another trait in `core`:

/// Type representing a field of a `struct`, `union` or tuple with structural pinning information.
///
/// # Safety
///
/// `Self::Projected<'a>` either is `Pin<&'a mut Self::Type>` or `&'a mut Self::Type`. In the first
/// case the field is structurally pinned.
pub unsafe trait PinnableField: UnalignedField {
    /// The pin-projected type of `Self`.
    ///
    /// Either `Pin<&'a mut Self::Type>` or `&'a mut Self::Type`.
    type Projected<'a>
    where
        Self::Type: 'a;

    /// Sets the correct value for a pin projection.
    ///
    /// # Safety
    ///
    /// The supplied reference must be derived from a `Pin<&mut Self::Base>`.
    unsafe fn from_pinned_ref(r: &mut Self::Type) -> Self::Projected<'_>;
}

It is only implemented for fields if
* at least one field in the struct is annotated with `#[pin]`, or
* the struct is annotated with `#[derive(PinnableFields)]`.

(in both cases all fields of the struct implement the trait)

There also is the alternative to implement it for all structs, but this will probably break assumptions by `unsafe` code. For example:

struct TrackedFuture<F> {
    fut: F,
    poll_count: usize,
}

impl<F> TrackedFuture<F> {
    pub fn poll_count(&self) -> usize {
        self.poll_count
    }
}

impl<F: Future> Future for TrackedFuture<F> {
    type Output = F::Output;
    
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        let this = unsafe { Pin::into_inner_unchecked(self) };
        this.poll_count += 1;
        let fut = unsafe { Pin::new_unchecked(&mut this.fut) };
        fut.poll(cx)
    }
}

If we now introduce pin-projections for all structs, then `TrackedFuture<F>` would allow projecting `Pin<&mut TrackedFuture<F>> -> &mut F` which is wrong, since it's considered structurally pinned by the `poll` implementation.

We can change this behavior (a "by-default" `derive(HasFields)`) across an edition boundary.

# Projections traits

There are four new traits added to `core::ops`:
* `Projectable` - Type supporting field projections.
* `Project` - Shared projection operation `@base->field`.
* `ProjectMut` - Exclusive projection operation `@mut base->field`.
* `SafeProject` - Marks project operations on `Self` as safe.

## `Projectable`

First we need the `Projectable` trait:

/// Type supporting field projections.
///
/// The exact kind of field projection is governed by [`Project`] and [`ProjectMut`]. This trait
/// only gives the compiler access to the `Self::Inner` type which is the type containing the
/// potentially projectable fields. So given an expression `base` of type `Self`, then the `field`
/// ident in the expressions `@base->field` and `@mut base->field` refer to a field of the type
/// `Self::Inner`.
///
/// If the projection `@base->field` is available still depends on weather `Self` implements
/// `Project<field_of!(Self::Inner, field)>`.
pub trait Projectable: Sized {
    type Inner: ?Sized;
}

It is used to tell the compiler of which type the field representing types should be used. All types supporting projections must implement it.

## `Project`

This trait governs the shared projection operator `@base->field`.

/// Shared projection operation `@base->field`.
pub trait Project<F>: Projectable
where
    F: UnalignedField<Base = Self::Inner>,
{
    /// The output of this projection operation.
    type Output<'a>
    where
        Self: 'a;

    /// Projects the base to the output.
    ///
    /// # Safety
    ///
    /// * `this` must be a valid pointer pointing at a valid value of `Self`.
    /// * for the duration of `'a`, the value at `this` is only used by other projection
    ///   operations.
    /// * for the duration of `'a`, the value at `this` is not mutably projected with `F`.
    /// * Implementers may impose additional safety requirements. These must be documented on the
    ///   implementation of this trait.
    unsafe fn project<'a>(this: *const Self) -> Self::Output<'a>
    where
        Self: 'a;
}

The lifetime declared on `project` is the lifetime of the projection. Note that `Output` can depend on the field type and the implementer can also decide to impose additional bounds on `F` or only implement projection for one single concrete field.

## `ProjectMut`

This trait governs the exclusive projection operator `@mut base->field`.

/// Exclusive projection operation `@mut base->field`.
pub trait ProjectMut<F>: Projectable
where
    F: UnalignedField<Base = Self::Inner>,
{
    /// The output of this projection operation.
    type OutputMut<'a>
    where
        Self: 'a;

    /// Projects the base to the output.
    ///
    /// # Safety
    ///
    /// * `this` must be a valid pointer pointing at a valid value of `Self`.
    /// * For the duration of `'a`, the value at `this` is only used by other projection
    ///   operations for fields other than `F`.
    /// * Implementers may impose additional safety requirements. These must be documented on the
    ///   implementation of this trait.
    unsafe fn project_mut<'a>(this: *mut Self) -> Self::OutputMut<'a>
    where
        Self: 'a;
}

Note that the output type is `OutputMut` a new associated type, this way the output can be different for shared and exclusive projections for the same field.

## `SafeProject`

By default all projection operations are `unsafe`. When implementing this trait, they all become safe.

/// Marks project operations on `Self` as safe.
///
/// # Safety
///
/// * The [`Project::project`] and [`ProjectMut::project_mut`] functions implemented for `Self`
///   must not have additional safety requirements.
pub unsafe trait SafeProject: Projectable {}

I think it's a bad idea to allow implementers to depend on the field type for changing the safety of the operation. It's already pretty difficult to find the safety requirements of the projection operation for a type.

---

# Open questions

## Reject interleaved shared projections & shared usage of the projected value

Originally, I intended to forbid any usage of projected values while projections are active. For exclusive projections this is of course required, since they need exclusivity. But as seen from the [experimental implementation], shared projections can theoretically coexist with shared usage of the projected value (and I don't see a way to prevent this without giving up ergonomics in the experiment).

A lot of types naturally allow this interaction (`&UnsafeCell<T>`, `&mut MaybeUninit<T>`, `&Untrusted<T>`, `&RcuMutex<T>`, `&SeqLock<T>`, all other examples from RfL...), so we might be able to allow it after all. We need a decision here, since either way we do it is incompatible with the other:
* we allow simultaneous shared projections and shared usage of the projected value: then people can rely on that fact and design APIs that require projections & usage of the value
* we disallow them: then people designing smart pointers can take advantage of this fact and write `unsafe` code that relies on the value not being accessed while projected, for example store an additional `UnsafeCell` for each field and update that when a projection starts (the type itself must be `!Sync`).

From the perspective "just do what references do", it also makes sense to allow this, since references also allow using the entire value when a field is share-reborrowed.

(after having finished this section, I am convinced that we should allow it, but I'm leaving it here for clarity)

## Context-aware projection kind selection

Field borrowing supports the following: `self.field.fun()` and `fun` can either be `fn fun(&self)` or `fn fun(&mut self)` and the compiler decides the correct borrow for `self.field`.

Field projections probably can also support this, but the fact that exclusive projections can return an entirely different type might make it more difficult. That's because `fun` in `self->field.fun()` could refer to two different `fun` from two different types. Otherwise the syntax would be `(@self->field).fun()` or `(@mut self->field).fun()` respectively.

This is something that we could easily add backwards-compatibly in the future, since it's new syntax.

## Best way to make projections available on `&mut MaybeUninit<T>`

The current design (and [experimental implementation] also) use the following impls to make projections available for `&mut MaybeUninit<T>`:

impl<T> Projectable for &mut MaybeUninit<T> {
    type Inner = T;
}

// SAFETY: no additional safety requirements on `Project[Mut]::project[_mut]`.
unsafe impl<T> SafeProject for &mut MaybeUninit<T> {}

// No additional safety requirements for `project_mut`.
impl<'a, T, F> ProjectMut<F> for &'a mut MaybeUninit<T>
where
    F: Field<Base = T>,
    F::Type: Sized + 'a,
{
    type OutputMut<'b>
        = &'b mut MaybeUninit<F::Type>
    where
        Self: 'b;

    unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'a>
    where
        Self: 'b,
    {
        let ptr: *mut MaybeUninit<T> = unsafe { this.read() };
        unsafe { &mut *ptr.byte_add(F::OFFSET).cast() }
    }
}

(the implementation of `Project` is not shown for brevity, it looks almost identical)

Now this implementation works, because there is no

impl<'a, T, F> ProjectMut<F> for &'a mut T
where
    F: Field<Base = T>,
    F::Type: Sized + 'a,
{
    type OutputMut<'b>
        = &'b mut F::Type
    where
        Self: 'b;
    /* ... */
}

That implementation isn't really useful in a non-generic context, since people can just do `&mut base.field` instead of `@mut base->field`. This way it's less confusing for people when to use reborrowing vs projecting for references. It also allows projections for `&Custom<Struct>` to return non-reference values like `Custom2<'_, Field>` or `usize` etc.

But it also prevents references being used in a generic interface that asks for a type that can be projected.

I haven't yet written any such APIs, but in theory nothing would stop one from doing so. (I'm not even sure what one would do with such a function)

One solution to still have projections for references **and** not the syntax is yet another trait that enables the projection syntax, or just special case it in the lang item for references. But if we do that, the above implementation for `&mut MaybeUninit<T>` will overlap with it and we need a different solution.

One such solution would be to give `MaybeUninit<T>` the fields of `T` with the type of the field wrapped in `MaybeUninit`. That was originally part of the design of the RFC v2 and v1. It also captures a sentiment shared by Alice in a meeting that "`&'a Custom<T>` and `Custom<'a, T>` are fundamentally different". But as I explained that is only the case if we make `&` implement projections.

Another disadvantage of not implementing projections for references is that `Pin<&mut MaybeUninit<T>>` will not be projectable. If we were to make `MaybeUninit<T>` have fields like `T`, then under the current implementation, one could project `Pin<&mut MaybeUninit<T>>`. It would also allow projections like `Pin<&mut MaybeUninit<UnsafeCell<MyCustomWrapper<T>>>>`. Here is a sketch of how we could modify the field types (you might need to look at the [field types section](#Field-types) first):
* require users to explicitly enable field types on structs (eg using a `derive(HasFields)`)
* add an associated constant to `UnalignedField` for the compiler to resolve visibilty of a field
* allow custom impls of `UnalignedField`, then one could eg do for `MaybeUninit`:

  pub struct MaybeUninitField<F>(F);
  
  unsafe impl<F> UnalignedField for MaybeUninitField<F>
  where
      F: UnalignedField,
      F::Base: Sized,
      F::Type: Sized,
  {
      type Base = MaybeUninit<F::Base>;
      type Type = MaybeUninit<F::Type>;
      
      // MaybeUninit is `repr(transparent)`, otherwise we'd need some more help from the compiler
      // here? or could we do `offset_of!(Container, contained) + F::OFFSET` if the container also
      // contains other fields?
      const OFFSET: usize = F::OFFSET;
      
      const VISIBILITY: Visibilty = F::VISIBILITY; // preserve visibilty
  }
  
  unsafe impl<F> Field for MaybeUninitField<F>
  where
      F: Field,
      Self: UnalignedField,
  {}

So to summarize, if we go with no `Projectable for &T` (top level bullet points) or with `Projectable for &T` (bullet points one level indented), then:
* `&Custom<Struct>` can project to any type (`Custom2<'_, Field>`)
* `&Custom<Struct>` can only be projected to a reference to a type (`&Custom2<Field>`). And this only works if `Custom<Struct>` contains a field of type `Custom2<Field>`
* `Pin<&mut Custom<Struct>>` cannot be projected if `Custom` doesn't expose its fields
* `Pin<&mut Custom<Struct>>` can be projected to `Pin<&mut Custom<Field>>`, similarly any other nested containers & projectables (eg `&mut MaybeUninit<UnsafeCell<Struct>>`)
* `@base->field` is unavailable when `base: &Struct` and the compiler can suggest to replace with `&base.field`
* `@base->field` works when `base: &T`, but depends on `T`. The compiler could check if `field` is a "real" field and reject if `&base.field` would also work.
* references cannot be used in generic projection interfaces like `fn foo(_: impl Project<field_of!(MyStruct, field)>)`
* references can be used in generic projection interfaces
* all structs are available for projections even ones that existed before the introduction of field projections and since edition 2015 (if the fields are accessible)
* only structs annotated with `derive(HasFields)` support being projected

## "Moving" projections

In the [experimental implementation], I have a workaround that I called `move` projections. For example:

fn move_projections<'a>(foo: &'a mut MaybeUninit<Foo>) -> &'a mut MaybeUninit<Bar> {
    start_proj!(move foo);
    p!(@move mut foo->bar)
}

This is because the following doesn't compile:

fn move_projections_motivation<'a>(mut foo: &'a mut MaybeUninit<Foo>) -> &'a mut MaybeUninit<Bar> {
    start_proj!(mut foo);
    //------------------ `foo` is borrowed here
    p!(@mut foo->bar)
    // ^^^^^^^^^^^^^ returns a value referencing data owned by the current function
}

There are two options for the language-level implementation of field projections: either it "just works" or it has the same issue. I'm not sure yet, would need some comments.

# Notes & TODO

## Unsafety of `project`

Sadly I couldn't come up with a design that makes the `project[_mut]` functions safe. This makes them significantly worse for use in a generic context. I don't yet have any use-cases for those except for implementing other projection operations, so maybe the normal projection syntax should have support for generic projections? Or there should be a macro?