---
title: "Design meeting 2025-08-13: Field projections"
tags: ["T-lang", "design-meeting", "minutes"]
date: 2025-08-13
discussion: https://rust-lang.zulipchat.com/#narrow/channel/410673-t-lang.2Fmeetings/topic/Design.20meeting.202025-08-13.3A.20Field.20projections/
url: https://hackmd.io/PgVxFwBDQlGXPGTQrI0i3A
---
# Field projections design meeting
## Before We Start...
### Purpose of this Meeting
The design space is very big, lots of mutually exclusive changes as well as hard to balance interests between expressiveness and ergonomics. For this reason, this meeting shouldn't try to produce a working solution. The purpose is to perform vibe checks and get everyone on the same page as to what problems should be addressed by a field projection language feature. Additionally, the meeting should discern which design axioms should guide the development of the feature. We should also evaluate the current approach and green-light it for a lang experiment. Finally, we can try to tackle some of the open design dilemmas that the current approach has surfaced.
To summarize, this meeting should:
* Determine the use-cases a field projection language feature must be able to solve.
* Select design axioms for creating the field projection language feature.
* Decide if the current approach is ready for a lang experiment.
* Provide guidance on the open design dilemmas.
### Project Goal
This design meeting is the first for the [field projection project goal 2025H2]. Note that this document goes into more details than the goal, so only the table at the bottom is important to read.
[field projection project goal 2025H2]: https://rust-lang.github.io/rust-project-goals/2025h2/field-projections.html
## Motivation
*Field projections* are available on references: given `x: &Struct`, we can create a reference to a field (`&Field`) with `&x.field`. A field projection language feature makes this kind of operation available to all types. The syntax of a projection is similar to reborrowing: `@expr->field` and `@mut expr->field`.
### Custom Reference Types
Many of the types that benefit from field projections fall into the category "custom reference types". These are types that behave similarly to `&`/`&mut`, for example:
* `&mut MaybeUninit<T>` the referent may be uninitialized,
* `cell::Ref[Mut]<'a, T>` carries runtime borrow info to access the `RefCell<T>`,
#### `&mut MaybeUninit<T>`
While `&mut MaybeUninit<T>` already is a reference, it doesn't provide field projections to `&mut MaybeUninit<U>`. There is an unstable function `transpose(MaybeUninit<[T; N]>) -> [MaybeUninit<T>; N]` that does this projection for arrays, so it's only natural to also want to have it for structs.
Projections for `&mut MaybeUninit<T>` would allow initializing a struct step-by step more safely:
```rust
struct Point {
x: i32,
y: i32,
}
struct Player {
hp: usize,
pos: Point,
}
impl Player {
fn init(this: &mut MaybeUninit<Self>) -> &mut Self {
let hp: &mut MaybeUninit<usize> = @mut this->hp;
(@mut this->point->x).write(0);
// ^^^^ this is inferred like `&mut` would be, you can also write:
this->point->y.write(0);
unsafe { this.assume_init_mut() } // the only unsafe block needed
}
}
```
In this small example just assigning `Foo { ... }` directly would of course also work, but there are several reasons when that's not possible:
* different functions initialize different fields,
* only some parts of the struct should be initialized,
* the entire struct would be too big for the stack, but every field fits.
#### `cell::Ref[Mut]<'a T>`
Both `cell::Ref[Mut]<'a, T>` already provide a [`map` function][1] that is specifically designed to give users access to field projections. However that is not enough: if one wants to project two fields simultaneously, `map` cannot be used, as it consumes the `Ref[Mut]` by-value. For this reason the [`map_split` function][2] exists; but it too only permits to project two fields simultaneously.
[1]: https://doc.rust-lang.org/std/cell/struct.RefMut.html#method.map
[2]: https://doc.rust-lang.org/std/cell/struct.RefMut.html#method.map_split
Field projections allow any number of different fields to be projected simultaneously, allow interleaving with control flow and even allow the use of the projected value afterwards if all field borrows end before then:
```rust
#[derive(Debug)]
struct Data {
x: usize,
y: i64,
z: bool,
w: usize,
}
fn handle(mut data: RefMut<'_, Data>, mut out: &mut usize) {
let mut x: RefMut<'_, usize> = @mut data->x;
if *x == 0 {
*out = 0;
out = &mut *x;
}
compute(@mut data->y);
if *@mut data->z {
// ^^^^^ we cannot use `Deref` here, as that would invalidate the projection
// to `x` that we want to use via `out` below.
let w = @mut data->w;
mem::swap(out, &mut *w);
}
// All projections of fields end before here, so we can use the entire struct again.
info!("{data:?}");
}
```
#### `NonNull<T>` and `*{const,mut} T`
While these types aren't references, for field projections they behave exactly the same way as `&`/`&mut`. So given a valid `*const Struct`, one is able to project to `*const Field` with unsafe means today. However, doing so is very verbose.
```rust
struct Point {
x: i32,
y: i32,
}
unsafe fn project_x(point: NonNull<Point>) -> NonNull<i32> {
unsafe { NonNull::new_unchecked(&raw const (*point.as_ptr()).x) }
}
// with field projections:
unsafe fn project_x2(point: NonNull<Point>) -> NonNull<i32> {
unsafe { @point->x }
}
```
These projections need to be `unsafe`, because `NonNull` could wrap around the address space. For raw pointers, see [this comment][4].
[4]: https://github.com/rust-lang/rfcs/pull/3735#discussion_r1922808515
#### Other Reference Types
There are lots of other types that could benefit from field projections in the same way as the previous examples of this section. A non-exhaustive list:
* `pyo3::pycell::PyRef[Mut]<'_, T>`: there is an [open issue][3] to add a `map` function.
* `Cpp[Mut]Ref<T>`: a pointer to a C++ value.
* `&[mut] Cell<T>`: to write only part of a bigger struct, making it available for bigger, possibly non-`Copy` types.
* `&Atomic<T>`: if `T` can be a struct made up of primitives or a tuple of primitives.
* `ArcRef<T>`: an `Arc<T>` with separate pointers to the refcount and data, permitting projecting to fields.
* `VolatilePtr<'_, T>`: a pointer that always uses volatile operations ([wanted by the general community][volatile] & RfL to remove the [`dma_read` macro] (also `dma_write`)).
[volatile]: https://github.com/rust-lang/unsafe-code-guidelines/issues/411#issuecomment-2894600985
[3]: https://github.com/PyO3/PyO3/issues/2300
Rust for Linux also has several custom reference types that would benefit from field projections:
* `&[mut] Untrusted<T>` used to mark data as untrusted (it needs to be validated before it can be read, but writing it is fine).
* `SeqLockRef<'_, T>`: similar to `ArcRef`, have separate pointers for the sequence count and the data, allowing to lock & only care about a field of the entire stored data.
* `UserPtr<T>` a pointer into userspace that must not be conflated with a normal pointer.
* `Ptr<'_, T>` essentially just an `&Opaque<T>`, so a valid pointer whose pointee might be uninitialized & concurrently modified.
[`dma_read` macro]: https://rust.docs.kernel.org/kernel/macro.dma_read.html
### Pin ergonomics
While pin ergonomics are most likely going to be hard-coded by the compiler, field projections should be able to simulate pin ergonomics. Users might want to have custom pinned references and equip that type with field projections. For example, the `Mutex<T>` type in RfL is structurally pinning its contents and thus `MutexGuard<'_, T>` must behave like `Pin<&mut T>` for accessing fields. We would like to provide field projections on that type to allow convenient access.
Both `Unpin`-guided pin ergonomics as implemented in the current lang experiment and marked-fields pin ergonomics will should be supported by field projection. An example for the marked field approach:
```rust
struct FairRaceFuture<F1, F2> {
fair: bool,
#[pin]
fut1: F1,
#[pin]
fut2: F2,
}
```
With structurally pinned fields `fut1`, `fut2` we get the following projections when given `f: &pin mut FairRaceFuture<F1, F2>`:
* `@mut f->fair` has type `&mut bool`,
* `@mut f->fut1` has type `&pin mut fut1`,
* `@mut f->fut2` has type `&pin mut fut2`,
There also are projections like `@f->fair` to `&bool`.
### RCU (Read-Copy-Update)
Rust for Linux has a very exotic use-case for field projections: creating a safe abstraction for RCU. This use-case is very important to RfL, because RCU is used fairly often in the kernel and without field projections there most likely is no way to write a safe abstraction for it. It also is [included in C++26][4] so it could see more uses in other projects the future.
RCU is an acronym for "read copy update" and it is an efficient locking mechanism for rarely written data. It synchronizes between readers and writers; however, it cannot synchronize between multiple writers. For this reason it must combined with an additional locking mechanism that only the writers use. This external lock can be a mutex, a spinlock or another type of lock.
[4]: https://en.cppreference.com/w/cpp/header/rcu.html
Now the problem for a Rust abstraction is that in RfL we also adopted the design that locks contain the data they protect. But in the RCU case, only the writer is supposed to take the external lock, the reader is only takes the RCU lock, circumventing the external lock. This is where field projections come in. They enable us to project through the mutex in the reader case. The writer just normally locks the mutex.
```rust
// The data stored by the driver. Usually as `Arc<rfl::Mutex<Data>>`
struct Data {
// RCU-protected data must be stored in a pointer type wrapped by RCU.
// internally, this is just an `AtomicPtr<Config>`
#[pin]
cfg: Rcu<Box<Config>>,
// Data that isn't protected by RCU is just stored normally.
other: i32,
}
struct Config {
size: usize,
name: &'static CStr,
}
// Reader case:
fn size(data: &rfl::Mutex<Data>) -> usize {
// `&rfl::Mutex<T>` allows projecting to fields of type `Rcu<U>`,
// but not to other fields (that would be unsound).
let cfg: &Rcu<Box<Config>> = @data->cfg;
// now we begin the critical read section of RCU.
let rcu = rcu::read_lock();
let cfg: &Config = cfg.get(&rcu);
cf.size
}
// Writer case:
fn set_config(data: &rfl::Mutex<Data>, config: Config) {
// We normally lock the mutex.
let mut data: rfl::MutexGuard<'_, Data> = data.lock();
// Normal data can just be handled as usual using field projections.
// (remember: the RfL mutex pins its data, so we need projections here)
data->other = 42;
// Maybe somewhat surprisingly, we can obtain a `Pin<&mut Rcu<...>>`:
let cfg: Pin<&mut Rcu<Box<Config>>> = @mut data->cfg;
// This is fine, because it will use `UnsafePinned` to remove `noalias`,
// and it's API cannot be abused by having both `&pin mut` and `&`
// concurrently existing. The pinning is needed to prevent `mem::swap`
// being used to change the value without using atomics.
// `Rcu::set` has `Pin<&mut Rcu>` as the receiver, so it can only be called,
// if the external lock is taken.
let _old = cfg.set(Box::new(config));
// When `_old` is dropped, `synchronize_rcu` is executed, waiting for a
// grace period to end (ie all currently active critical read sections
// must end). This guarantees that any readers still holding onto a
// pointer to the contents of `_old` have a valid pointer.
drop(_old);
}
```
### Field Reflection Use-Cases
The current approach uses at its core a kind of field reflection to inform generic code about which field is being projected. Outside of field projection, this kind of field reflection also is useful. In Rust for Linux, there are types that currently use a `const ID: usize` generic to allow having multiple differentiable fields with the same type (well except for the ID of course). With field reflection, we could use the field type/identifier itself as the ID and remove the generic improving the ergonomics of the API.
## Design Axioms
The project goal lists these two design axioms:
* **Simple & easy-to-remember syntax.** Using field projections in a non-generic context should look very similar to normal field accesses.
* **Broadly applicable solution.** Field projections should be very general and solve complex projection problems such as pin-projections and `Rcu<T>`.
To better explain the intentions behind these axioms here is some more prose text: the ergonomics of using field projections really is the most important piece. People already write `map` functions today to perform projections when references are backing the actual data. And as later mentioned it is possible to implement unergonomic projections with macros. But adding them with a native operator is the biggest benefit that comes from a language feature. The second axiom ensures that the feature is beneficial to as many applications of field projections as possible. If we're introducing a language feature for field projections, it better serve all cases of the concept.
What we're willing to compromise on -- and the current approach heavily compromises this aspect -- is the ergonomics of adding projection support to a type (so implementing the traits for the projection operator). Since we're trying to fit so many use-cases into a single feature & want to have nice ergonomics for the usage site, this part necessarily has to suffer. This aspect reminds me of the current API design of the `Try` & `Residual` traits, they also are very ergonomic to use and serve many different use-cases; but they also pay the same price of complexity at the implementation site.
Aside from these two axioms I don't think there are others that we should adhere to, since most other decisions are going to be a balancing act and there will not really be an easy answer. But if you have any suggestions feel free to post them.
## The Current Approach
A discussion at RustWeek resulted in the following overarching design idea:
* Make the behavior of `&`/`&mut` w.r.t. field projections available for all types.
This means that projections follow the same borrow checker behavior and can reuse the existing compiler code. They also are very familiar to Rust developers and thus relatively easy to teach. In this section, we go over a high level overview of the current design.
### Library Implementation
Note that a [library implementation] of this approach exists. It has bad ergonomics due to several reasons, but it showcases the various traits for the operator. Additionally many of the motivational examples are implemented there, so it gives a good impression of what it would be like to have it as a language feature.
[library implementation]: https://github.com/Rust-for-Linux/field-projection/tree/new
The ergonomic shortcomings of the library solution are:
* structs must be annotated with `#[derive(HasFields)]` to be available for projecting
* field types aren't defined locally next to the struct, but by the `field-projection` crate, thus one can't implement traits on them, preventing extensions to the field types
* can only project through a single field
* can only project variables and not arbitrary expressions
* need to wrap the projection expression with the `p!` macro
* in order to be able to have projections on `var`, one needs to write `start_proj!(var);`
* cannot omit `@[mut]` when calling a function on a projected value
* lifetime problems when returning a projected value, for this reason there is a special [`move` projection](https://github.com/Rust-for-Linux/field-projection/blob/new/examples/kernel_mutex_rcu/main.rs#L41) that needs to be used in such a case (note the unfortunate naming, as this is something different from "Moving projections" that I have mentioned sometimes)
An example showing the last five issues:
```rust
fn buffer_config<'a>(&'a self, rcu_guard: &'a RcuGuard) -> &'a BufferConfig {
let buf: &'a RcuMutex<Buffer> = &self.buf;
start_proj!(move buf);
p!(@move buf->cfg).read(rcu_guard)
}
// with a language feature it could look like this:
fn buffer_config<'a>(&'a self, rcu_guard: &'a RcuGuard) -> &'a BufferConfig {
(&self.buf)->cfg.read(rcu_guard)
}
```
### Field Reflection (`Field` trait & generated types)
To make field information available in generic code, every field of every struct is associated with a field representing type. It can be named via the `field_of!(Struct, field)` macro that uses the same syntax as `offset_of!`. A field representing type implements the `Field` marker trait:
```rust
pub unsafe trait Field: UnalignedField {}
// fields of `repr(packed)` structs only implement this trait
pub unsafe trait UnalignedField {
/// The type that this field is a part of.
type Base: ?Sized;
/// The type of this field.
type Type: ?Sized;
/// The offset of this field in bytes.
const OFFSET: usize;
}
```
The associated types and the constant must be correct (that's why the trait is `unsafe`). So one can rely on them and write code like:
```rust
fn project_ref<F: Field>(r: &F::Base) -> &F::Type
where
// needed for the `.cast` call below
F::Type: Sized,
{
unsafe { &*ptr::from_ref(r).add_bytes(F::OFFSET).cast::<F::Type>() }
}
```
Field representing types are also generated for nested fields, so `field_of!(Struct, bar.baz)` will return a field representing type that has `Struct` as the base, the offset will be that of `bar` plus that of `baz` inside of `bar` and the type will be that of `bar`.
### Syntax & Operator Traits
We add a new operator: `@[mut] $base:expr$(->$field:ident)+` which is governed by three traits:
* `Projectable`: this trait is a supertrait of the next two and is not generic over the projected field. It is similar to `Receiver`, since it stores of which struct type we project the fields of.
* `Project<F>`: this trait is used for `@base->field` desugaring,
* `ProjectMut<F>`: this trait is used for `@mut base->field` desugaring,
```rust
pub trait Projectable: Sized {
type Inner: ?Sized;
}
```
`Projectable` is used to figure out which field type to use in the expression `@[mut] $base:expr$(->$field:ident)+`:
* the expression `$base` must be of a type that implements `Projectable`,
* the compiler then can look up the `Inner` type and then use `field_of!(Inner, $($field).+)` as the field type
* then `Project[Mut]<F>` is used for the desugaring where `F` is the field type from above.
```rust
pub trait Project<F>: Projectable
where
F: UnalignedField<Base = Self::Inner>,
{
type Output<'a>
where
Self: 'a;
unsafe fn project<'a>(this: *const Self) -> Self::Output<'a>
where
Self: 'a;
}
```
`ProjectMut` looks identical except the `Output` and `project` functions have a `Mut`/`_mut` suffix.
Finally there is one last marker trait:
```rust
pub unsafe trait SafeProject: Projectable {}
```
That makes all projection operations on that type not require an `unsafe` block.
#### Desugaring
Before `@[mut] base->field` is desugared, the compiler performs a borrow-check:
* `@base->field` is treated exactly as `&base'.field` where `base'` is a new variable of type `&<typeof(base) as Projectable>::Inner`. And the borrow lasts for exactly the same length as is required by the projection (`@base->field` is of type `Project<...>::Output<'a>`).
* `@mut base->field` is treated exactly as `&mut base'.field`
* accessing `base` in any other way counts as using the whole of `base'`, so field projections and any other access cannot be combined (only if the field projection ends before the other operation).
After this borrow check passes, the following desugaring is used for `@[mut]base->field`:
```rust
Project[Mut]::<
field_of!(
<typeof(base) as Projectable>::Inner,
field
),
>::project[_mut](&raw {const,mut} base)
```
When the projection goes through multiple fields, so `@base->field1->field2`, then those fields are just appended in the `field_of!` invocation:
```rust
Project::<
field_of!(
<typeof(base) as Projectable>::Inner,
field1.field2
),
>::project(&raw const base)
```
### Solving the Motivation
We're only going to inspect a few example implementations for time reasons.
See the [library implementation] for the details, as it implements a lot of the motivational examples (look in the `examples/` directory and the `src/projections/` directory).
#### `&mut MaybeUninit<T>`
Using the current approach to implement field projections for `MaybeUninit` would look like this:
```rust
impl<T> Projectable for &mut MaybeUninit<T> {
type Inner = T;
}
unsafe impl<T> SafeProject for &mut MaybeUninit<T> {}
impl<'a, T, F> ProjectMut<F> for &'a mut MaybeUninit<T>
where
F: Field<Base = T>,
F::Type: Sized + 'a,
{
type OutputMut<'b>
= &'b mut MaybeUninit<F::Type>
where
Self: 'b;
unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'b>
where
Self: 'b,
{
let ptr: *mut MaybeUninit<T> = unsafe { this.read() };
unsafe { &mut *ptr.byte_add(F::OFFSET).cast() }
}
}
```
*(The library implementation also implements `Project`, but it looks almost identical to `ProjectMut`)*
#### `Pin<&mut T>`
##### With Marking Structurally Pinned Fields
We need a new field subtrait:
```rust
pub unsafe trait PinnableField: UnalignedField {
/// Either `Pin<&'a mut Self::Type>` or `&'a mut Self::Type`.
type Projected<'a>
where
Self::Type: 'a;
unsafe fn from_pinned_ref(r: &mut Self::Type) -> Self::Projected<'_>;
}
```
It is implemented via a derive macro in the crate, but with an edition change it could become the default. Then this is how the traits implemented on `Pin<&mut T>` look like:
```rust
impl<T> Projectable for Pin<&mut T> {
type Inner = T;
}
unsafe impl<T> SafeProject for Pin<&mut T> {}
impl<'a, T, F> ProjectMut<F> for Pin<&'a mut T>
where
F: PinnableField<Base = T> + Field<Base = T>,
F::Type: Sized + 'a,
{
type OutputMut<'b>
= F::Projected<'b>
where
Self: 'b;
unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'b>
where
Self: 'b,
{
let r = unsafe { Pin::into_inner_unchecked(this.read()) };
let ptr: *mut T = r;
let ptr = unsafe { ptr.byte_add(F::OFFSET).cast() };
unsafe { F::from_pinned_ref(&mut *ptr) }
}
}
```
##### Current `pin_ergonomics` lang experiment version
Simulating pin projections using the `Unpin` rules that the current `pin_ergonomics` lang experiment define would look like this:
```rust
impl<T> Projectable for &pin mut T {
type Inner = T;
}
unsafe impl<T> SafeProject for &pin mut T {}
impl<'a, T, F> ProjectMut<F> for &'a pin mut T
where
T: !Unpin,
F: Field<Base = T>,
F::Type: Sized + 'a,
{
type Output<'b>
= &'b pin mut F::Type
where
Self: 'b;
unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'b>
where
Self: 'b,
{ /* ... */ }
}
// sadly this impl is overlapping...
impl<'a, T, F> ProjectMut<F> for &'a pin mut T
where
F: Field<Base = T>,
F::Type: Sized + Unpin + 'a,
{
type Output<'b>
= &'b pin mut F::Type
where
Self: 'b;
unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'b>
where
Self: 'b,
{ /* ... */ }
}
```
This doesn't completely work due to the overlapping traits, it could be solved by having a marker trait that is allowed to overlap (the marker trait is implemented for `F: Field` where `F::Base: !Unpin` or `F::Type: Unpin`). "Alternative B" proposed in the [tracking issue][6] would work out of the box, since there projection is always allowed. The `ProjectMut` impl would look like this:
[6]: https://github.com/rust-lang/rust/issues/130494
```rust
impl<'a, T, F> ProjectMut<F> for &'a pin mut T
where
T: IsMarkedPinV2,
F: Field<Base = T>,
F::Type: Sized + 'a,
{
type Output<'b>
= &'b pin mut F::Type
where
Self: 'b;
unsafe fn project_mut<'b>(this: *mut Self) -> Self::OutputMut<'b>
where
Self: 'b,
{ /* ... */ }
}
```
### Key Features
This approach informs us about several key features that we want for field projections given that we want to solve the examples from the motivation. Those key features are:
* Support projections on custom wrapper/pointer types (not just `&T` and `&mut T`),
* this one is hopefully obvious: RfL, pyo3, c++ interop & many other projects will want access to projections
* support *selective* projections in generic wrapper/pointer types via where clauses on the struct and field types,
* RfL's `Rcu<U>` use-case needs this as well as pin projections of any flavor
* support distinguishing between shared and exclusive projection at the usage site,
* needed by mutable pointer types that also have a shared variant, so `[Py]Ref[Mut]` or `Cpp[Mut]Ref`
* support distinguishing between user-defined and reference projection at the usage site
* there already exist `Deref` operations on several types that would like to have field projections, but those projections don't return references. Thus rendering `Deref` and the projections incompatible & adding the need to distinguish between them
* support `unsafe` projections
* sadly raw pointers and `NonNull` won't have projections otherwise. we could bite the bullet on this one and just not have their projections, instead we add a new pointer type `AllocatedAndAligned<T>` that always points at a valid allocation, it could have safe projections
* support projections to different types based on which field is being projected
* this is required by pin projections when using the field-marking approach
In addition, as a "sniff test", the design should be able to generalize over all types of projection currently supported; i.e., reference and pin projection can be implemented in terms of this (whether or not they actually are), as well as all the use cases we can think of for RFL and interop.
## Design Guidance
There are several mutually exclusive interests at play as with any design. This section contains the most pressing issues that need to be resolved in order to move forward. While reaching a consensus on every single one is unlikely (& we want to also get to any questions/discussions posted by participants of the meeting), it would still be great to have some guidance in coming to a decision on these for the lang experiment.
### Projecting References of Custom Types
**Decision:** should `&T` and `&mut T` implement `Projectable` & `Project<F>` for any field `F`?
**Conflicting Interests:**
* If they do implement it, it allows using them in APIs asking for `impl Project<F>`.
* If they don't, it allows users to have custom projections on `&Custom<T>` (e.g. `&mut MaybeUninit<T>`).
(we need to have a decision here, as `&T` and `&mut T` are fundamental and thus either have to implement the trait from the get-go, or never implement the trait in the future)
**Verdict:** *(preliminary) having projections for references to custom types is essential for the RCU use-case. It also is very useful in practice and as such we should allow them. This sadly means that `&[mut] T` cannot be used in generic projection contexts. But since we won't add those in the beginning and we might never add them, we can't let them get in the way of the best design now.*
### `unsafe` Projections
**Decision:** should it be possible to have an `unsafe` projection operation?
**Conflicting Interests:**
* `NonNull<T>` and raw pointers need their projections to be `unsafe`.
* But this would be the first `unsafe` operator of Rust that can be overloaded. This might make the unsafety requirements not as obvious as with functions. How would users properly document the safety requirements?
**Verdict:** *(preliminary) we should have `unsafe` projections, as without them, we lose the ergonomic gain for `unsafe` Rust.*
### Optimizing the Last Projection
**Decision:** should the last projection be handled specially?
**Conflicting Interests:**
* if yes, it would allow for example `ArcRef<T>` to prevent an additional increment and decrement of the refcount if the original value is not used afterwards:
```rust
fn coords(point: ArcRef<Point>) -> (ArcRef<i32>, ArcRef<i32>) {
let x = @point->x; // increments the refcount
let y = @point->y; // increments the refcount
(x, y)
// now `point` is dropped, decrementing the refcount, but it could have
// just been consumed by the `->y` projection instead of incrementing
// there & decrementing here.
}
```
* if no, it would improve the simplicity of the API, as every projection is the same.
**Verdict:** *(preliminary) no, we need to keep the API as simple as possible, as it already is pretty complex.*
### Syntax
**Decision:** what syntax should the field projection operator use? The concrete symbols can be decided later when bikeshedding, it's more about where do we place the expression, identifier and extra symbols.
The syntax used in the current approach tries to avoid parenthesis when combining it with other Rust syntax & ensures that it's not ambiguous with other syntax. It also copies the syntax of references:
* it supports projecting through multiple fields: `@foo->bar->baz`
* the reason for using `->` instead of `.` is to be able to omit the leading `@` & not use any parenthesis when
* calling a function after projecting: `foo->bar.baz()`,
* writing to a field after projecting: `foo->bar.baz = Baz::new()`,
While writing this section, I'm no longer convinced that this is the best syntax, since it might run into ambiguity in the function call & field write cases. So it might be that one of these alternatives fits better:
* use postfix syntax to disambiguate exclusive projections:
* `base.@field.mut` and `base.@field.const` (also showing different sigils for the infix operator)
* have different operators for shared/exculsive projections `base.@field` and `base.mut@field`
* this is adding a lot of characters to type in the mut case: `foo.mut@bar.mut@baz` (especially when projecting multiple fields), which I don't like, but using something else would probably be confusing
# Discussion
The first four discussion topics are the main output of this meeting. We should
* Decide on the use-cases that are supported by the field projection language feature.
* Decide on the design axioms used for the development.
* Green-light the current approach for a lang experiment or suggest changes.
* Express an opinion on the design questions (or decide which direction to go).
## Use-Cases
Which of the use-cases explained above should field projections support?
* [ ] `&mut MaybeUninit<T>` (also covers `&[mut] Cell<T>`, `&Atomic<T>`, `&[mut] Untrusted<T>`, `Ptr<'_, T>`)
* [ ] `cell::Ref[Mut]<'_, T>` (also covers `ArcRef<T>`, `pyo3::cell::PyRef[Mut]<'_, T>`, `SeqLockRef<'_, T>`)
* [ ] `NonNull<T>` (also covers `*{const,mut} T`, `VolatilePtr<'_, T>`, `UserPtr<T>`)
* [ ] `Pin<&mut T>` (also covers `rfl::MutexGuard`)
* [ ] `Rcu<T>`
## Design Axioms
Which of the following design axioms should be used to guide the lang experiment?
* [ ] **Simple & easy-to-remember syntax.**
* [ ] **Broadly applicable solution.**
* [ ] Should any other axioms be added?
## Lang Experiment Based on the Current Approach
Which changes should be made to the current approach before starting a lang experiment?
## Design Guidance
For which of the design questions have we discussed?
* [ ] Projecting References of Custom Types
* [ ] `unsafe` Projections
* [ ] Syntax
* [ ] Optimizing the Last Projection
---
# Discussion
## Attendance
- People: Josh, Tyler, TC, Benno Lossin, Gary Guo, Miguel, Tomas, Zachary Sample, Rusty Yato, Xiang, Eric Holk, Yosh, Boqun Feng
## Meeting roles
- Driver:
- Minutes: Tomas
Benno: I have the purpose of this meeting section at the top of the doc. That's exactly what I'd like to achieve here. The last section (Discussion) to check whether everyone aligns with what the document has to say. Decide on the use-cases, design axioms, if the current approach is a good start for a lang experiment (or have changes suggested) adn get more opinions on the design guideance section.
## Vibe Checks
### tmandry
I'm excited to have a design that's vetted through so many use cases. This seems like a useful feature in many contexts, and we haven't even started talking about how it could be used for `Option` and `Result`. I also see generalizing what `&T` can do as an important step for having an extensible and general-purpose systems language.
I'm in favor of an experiment, and I agree with the preliminary decisions. I think we'll have to do some exploration of the interaction with related features, like autoref, autoreborrow, and coercions to see whether all of the key features are necessary or whether we can handle the use cases with those other features. As far as syntax goes, I'm leaning in the direction of something like `foo.@bar`. I also don't personally see anything wrong with `foo.@mut bar`. (edit: I'd like to extend this to `foo.&bar` as well.)
### Josh
Josh: Enthusiastically in support of having field projections, and all the things that'll enable (e.g. `Arc` projections).
Very much appreciate that the question about `&T` and `&mut T` needs to get settled up front. Would like to explore that further and see if it's possible to have a design that makes `&T` and `&mut T` work with projection, so that projections of types that contain intermediate references work well.
Would like to see an ACP to T-libs-api early for `ArcRef` (and `RcRef` presumably), so we can get that concept reviewed and approved early to go into nightly.
Have thoughts about syntax but broadly speaking the "shape" seems approximately correct. This may interact with the discussion about applying projections serially rather than all-at-once. Like Tyler, I do like the concept of the `.@` approach as well.
### TC
TC: Great work. Let's experiment here. I particularly like language features that reduce the need for library surface area, and this is one of those.
We'll need to keep comparing notes as it relates to pin ergonomics, and the discussions we'd had so far have been insightful and have raised interesting and useful points, e.g. about the `!Unpin` bound approach. I have an open item to talk that one over with Niko.
As mentioned below, I intuitively suspect that building `pin` into the language, as it relates to projections, will imply that we want `@pin mut x->f` projections for the same sort of reasons that we want `@mut x->f` projections, but need to look into this further.
It's worth noting too that this is a long document and a lot of design questions to consider here, and I certainly have only skimmed these and will need to have a more careful look later.
### Yosh (not T-Lang)
Yosh: I'm excited to see the use cases listed. I was worried that this would be primarily useful for pin projections, but the proposal feels like it will apply to a broad range of use cases.
Intuitively I feel like syntax might end up being the biggest sticking point here; more so than in many other proposals. While not something that we need to get into in this meeting, that's probably not something that should be left as the last item to get to either.
### Eric (not on T-Lang)
eholk: Definitely happy to see experimentation here. It seems like it has to potential to open up a lot of ergonomic opportunities for libraries and such going forward. I'm not sure the syntax is exactly right yet, but that's probably more on me needing to get familiar with it.
Also, it's very cool to see how RfL is driving a lot of language design. Languages are at their best when they have a kill app in mind.
### Scott
Short because I'm late getting here, but I've often pondered what Rust would have been like if `.` was always projected this way (instead of to places), so I think having it around as a primitive thing makes a ton of sense to me. I'm particularly thinking that having a projection as a thing we can talk about would be great for `MaybeUninit` usage and usage of a hypothetical `Packed<_>` type, as well as replacing a bunch of `offset_of` manual addressing, since staying in value-land (avoiding references entirely) has a whole bunch of good underlying advantages in other ways.
## Use cases
Benno: Should all the use cases in the document be something we are interested in supporting?
TC: Are you asknig whether this is an *exclusive* set of use cases? Or whether each of these is valid?
Benno: The latter. Let's agree on a minimum set of cases we want to support.
Tyler: I think all of these seem quite reasonable. None stand out as something we don't want to support. The more use cases we can support, the better. Might benefit having an RFC earlier to help collect the use cases early.
TC: One thing we talked about is RFC-ing a set of requirements / use cases separately.
Tyler: I had trouble with that in the past because I think it can quickly become vapid. But in this case we do have a very concrete solution and problem statement in mind.
TC: To make it not vapid, we could RFC specific requirements that a later RFC would be evaluated against. IETF did that.
Tyler: I think that could be useful here.
TC: Let's talk about the use cases:
- Mutable reference to `MaybeUninit<T>`
- Raw pointers that and `NotNull`
- `cell::Ref[Mut]<'_, T>` (also covers `ArcRef<T>`, `pyo3::cell::PyRef[Mut]<'_, T>`, `SeqLockRef<'_, T>`)
- Pin (depending on the property of the field you may have to change the output type)
- RCU (this restricts which fields you're allowed to do based on type)
Josh: Those last two cases (Pin and RCU) sound closely related.
TC: The only one I have a question about is the fourth one (the pin one). This gets into the question of treatment of pin in the language. We treat Pin as a library type rather than something like &mut. So this leads us to treating it like a library type (like a RefCell). The concept of Pin ergonomics is making it a real builtin type rather than just a library type. I'll focus on that one and we should talk about it.
Benno: In Rust for Linux, the MutexGuard has the inside pinned. So it is a pinned mutable reference. We'd like to have the same projection here for pin. So here, the question is about: do you think the usecase should be same as what field projection would solve.
Josh: :+1: for these use cases. I very much want to see the RCU usecase covered. Would also like to discuss the `Option<T>` use case.
Josh: There's a case and language feature worth bringing up because it interacts with this: Projecting through `Option`. If you have `Option<&T>` you can project through that to a field of `T`. But if you have an option of a struct (not a reference). You can't materialize that Option through anywhere -- it doesn't exist in memory. But we talked about having a special kind of reference that could let us say that "this logically exists even though I can't create an actual reference here". That would let us project through Option in general.
TC: I don't recall the discussion but what you said makes sense and is worth pursuing. The other context is match ergonomics. The natural ways of composing things lead to types that haven't materialized so that may be another place where this may be applicable.
Scott: I don't remember this a discussion for a type. But I do remember discussions about fields.
Josh: Those two things go hand in hand. If you have it as a field type, the compiler can optimize it because it doesn't exist in memory. If you have it as a special kind of reference, you can take that reference and pass it around.
Scott: But only if they're encoded in the same way.
Josh: Only if they're encoded in a fashion similar to an impl subject to monomorphisation.
Scott: Ah so the reference would have to carry metadata like vtable.
Josh: It would have to be metadata in the style of dyn or impl.
If you have `Option<Struct>`, you can project to `&magic Option<Field>`.
You don't have the `&Option<Field>` materialized anywhere. But in reality you can treat it that way.
Josh: The reason I'm bringing this up now is that having this "magic" reference type could interect with how we implement the concept of projection. It generalizes what you project through. This feels similar to what Benno talked about earlier about being to bring up a wrapper. There may be a design opportunity there if we have something like these move-only references where you could make the projections more orthogonal.
tmandry: Fwiw, I think I would spell it `Option<&move Field>`.
Josh: You can get `Option<&Field>` already. But you can't get `&Option<Field>`, because there's no `Option<Field>` in memory to reference. That's the motivation for having `&magic` (whatever we call it), so you can get `&magic Option<Field>`.
TC: I propose we say the lang team reviewed the usecases and didn't immediately see any problems with these usecase being motivating. We'll look later, but we're excited about the experiment and addressing about each of these usecases.
(general agreement)
Benno: Sounds like a lang experiment has been approved?
TC: We're absolutely doing a lang experiment. What you need to do next is create a tracking issue for the lang experiemnt. Link the tracking issue to the project goal too. Ask Tomas if you have any questions. We're championing this. Tyler this goal, I the ping ergonomics. We'll keep talking.
Benno: I think the conflict for `&T/&mut T` projections and custom projections is good.
Benno: If we want to have custom projections on reference types, we're not allowed ot implement `@mut` for the reference. The trait implementations are overlapping so we can only have one of the two. If we go the route of allowing it for references, we get the generit benefit but we lose custom projections -- for MaybeUninit, Cell types, RCU.
Either: projections on references (`&T`) or allow implementing projections for `&Custom<T>`.
```rust
impl<T> Projectable for &T {}
impl<F, T> Project<F> for &T
where
F: Field<Base = T>
{}
```
or
```rust
impl<T> Projectable for &Custom<T> {}
impl<F, T> Project<F> for &Custom<T>
where
F: Field<Base = T>
{}
```
Tyler: Another way of framing the choice is that either we can have `@foo.bar` mean the exact same thing as `&foo.bar` when `foo: &Struct`, OR we can support projections like `&mut MaybeUninit<Struct>` to `&mut MaybeUninit<Field>`. I find the second very motivating.
TC: Could we have both with specialization?
Benno: No we cannot, we also have to add the `Projectable` impls (added above in the code)
Josh: Don't want to open the whole rabbit hole again here (we can handle it async), but if we can project using `Custom<T>` (without having to use `&Custom<T>`), then we could independently project through the `&` and the `Custom`.
TC: Clearly we shouldn't depend on specialization. I'm pursuing this line of inquiry to understand better how fundamental this dilemma is. If it's not fundamental, sometimes we can come up with an argument that we can convince the compiler of about why to accept this, and then maybe we can make a general language feature out of it.
Benno: The current design couldn't work even with specialization.
Gary: I don't think it's fundamentally unworkable in the compiler, because I draw parallelism to `Deref`. Compiler can resolve whether you're referencing on a field on a struct or you're referencing a field of the `Deref`'d target, so it shouldn't fundamentally not work. It *might* be hard to make it work with trait resolution, but technically if you add an extra generic parameter if can solve conflicting impl issue and you can have compiler magically insert the special generic argument for you, it can work.
Xiang: First a question, does some type `F` such that `F: Field<Base = T>` for any type `T`, or a small but diverging set of type `T`s, even exist?
(The meeting ended here.)
---
## Understanding the conflict between `&T`/`&mut T` projections and custom projections
(suggestion: handle several shorter items first)
Josh: I'd like to better understand the conflict described between supporting projections on references and supporting custom projections as needed for RCU. The nature of the conflict is not obvious to me, and I'm not seeing where in the document that's discussed (other than saying they conflict). Some further elaboration on that would help.
Benno: The problem is overlapping implementations: if we add `impl<F, T> Project<F> for &T where F: Field<Base = T>` , then it would overlap with the RCU projections `impl<F, T, U> Project<F> for &Mutex<T> where F: Field<Base = T, Type = Rcu<U>>`. For mutable references it would also overlap with projections on `&mut MaybeUninit`.
Josh: So, when you mention custom projections on references, you're talking about distinguishing projections on the basis of the type of the field? This seems like something we could potentially overcome by using different types or transparent wrappers; for instance, could we have rfl's `Mutex` type have a `.rcu()` method that returns a transparent wrapper, and then write a different projection on that transparent wrapper? I absolutely want to support the RCU use case; I'd *also* like the orthogonality of having projections on references Just Work.
Benno: That'd be annoying, since it also affects `&mut MaybeUninit<T>` or `&UnsafeCell<T>` etc.
Josh: Can you elaborate on the `MaybeUninit` case, please? (I don't want to break the ergonomics of common cases, but I also think reference projections are important.)
Benno: Sure, if we had a `ProjectMut<F>` impl for `&mut T`, then we can't have one for `&mut MaybeUninit<T>`. Since again they would overlap
Josh: It feels like there's a problem of levels of indirection here, where there should be a projection through `&mut` and separately a projection through `MaybeUninit` and they should be possible to combine.
Benno: In an earlier RFC, I had these kinds of projections for structs where you could essentially declare "hey I have this field". So essentially we could say that we are allowed to write `field_of!(MaybeUninit<Struct>, field)` and it would resolve to `impl Field<Base = MaybeUninit<Struct>, Type = MaybeUninit<Field>>`. But this will mean that you can only ever project `&Custom<T>` to `&U` and this `U` must be contained within `Custom<T>` at `offset_of!(T, u)`, Which is pretty restrictive IMO.
Gary: Is this possible to use associated types and have trait resolver understand that `F: Field<Base = A>` and `F: Field<Base = B>` cannot possibly overlap if A != B? In the case above, the impl for `&_` would have `F: Field<Base = Mutex<T>>` on `&Mutex<T>`, while for the impl for `&Mutex<_>` it would have `F: Field<Base = T>`.
Benno: in that case we sadly would have matching bases (mutex doesn't have any accessible fields to the outside world & we want projections on the fields of `T`)
Josh: It feels like there's some overlap here with the conversations we've had about match ergonomics, where you're kinda "matching under a reference", and that leads to the resulting match having an implicit `ref`. To elaborate on why reference projection seems important, it seems important to be able to project through, for instance, `Arc<Something<&T>>` and have the `&` work just as easily as the `Arc` and the `Something`.
Benno:
* I haven't added this to the doc, but I have some ideas for a `@match` operator that would essentially just project ever field that you mention in there (it also allows to support enums).
* How would you imagine the `Arc<Something<&T>>` projection to look like? Ie what's the output type and which impl would be doing the projection.
* I would be very careful with projections of the kind `impl<P: Project<F>> Project<F> for MyType<P>`, since that involves a lot of `unsafe` stuff.
Josh: So, part of the problem here is that I'm wondering if the projection-of-a-projection case actually belongs in the type/trait system, or if it belongs in the compiler that can apply one projection after another without necessarily having to have a transitive impl. (I do appreciate that transitive trait impls can create a lot of problems.)
You need to know how to project from T to U, and from U to V, and I'm wondering if the compiler can treat that as two steps with two separate trait impls, rather than looking for a single impl that says how to project from T to V.
## Pin Ergonomics
eholk: I'm interested in the pin ergonomics aspects of this feature. In generaly, I'd say one feature that serves two use cases perfectly is better than two features that each serve one use case perfectly, so if field projections can subsume pin ergonomics, that seems like a win.
Benno: I agree, pin ergonomics probably still want to add `&pin` references though, so additional syntactic sugar on top of the ones added through field projections.
## (Resolved) Is it possible to project from one type to another?
Josh: For instance, can we project directly from `Arc` to `ArcRef`, without having to convert the top-level `Arc` to an `ArcRef` first?
Benno: Yes that's possible.
## `@pin mut x->f`
TC: Need to analyze this further, but it occurs to me that we might want `@pin mut x->f` for the same reasons that we want `@mut x->f` and not just `@x->f`. The underlying truth behind the pin ergonomics work is that `pin` is essentially similar to `mut` in being a separate non-fixed property of a place tracked by the borrow checker. So everywhere that we put `mut` with respect to references, it ends up making sense to put `pin` as well. I'd suspect that'd end up being true here too, but as said, it needs specific analysis.
Benno: I don't really follow. If you want to pin your type `T`, just wrap it in `Pin<T>` and provide pinning projections on `Pin<T>` & normal ones on `T`?
TC: The analysis here involves if and where this analogy breaks down: "if you want to mutate (or force exclusive access to) your type `T`, just wrap it in `&mut T` and provide exclusive projections on `&mut T` and normal ones on `T`."
tmandry: One relevant question is whether we want `Pin<MyPtr<T>>` to be meaningful, and whether we want to support projections through that.
Benno: I think that should be the decision of the author of `MyPtr<T>`.
## (Resolved, can summarize) "Optimizing the Last Projection"
Josh: Is it possible to *explicitly* do this, by writing it out explicitly, even if the compiler isn't helping you by doing that automatically?
Benno: the current API of the `Project` trait only has one `project` function that assumes that the value is still accessible (through other projections for example) after it ran. We'd probably have to add a new `final_project` function that can assume that it's the last projection.
Xiang: maybe we can have a mir-opt to identify `project` locations where the base is last known to be live, if we have the library support like `final_project` as mentioned.
Benno: My problem here was the addition of such a `final_project` method, it makes the API pretty complicated as you now have to implement the two projections at the same time.
tmandry: I think `final_project` could be added later, as a provided method that defaults to calling `project`.
tmandry: I think to answer Josh's question though, maybe you can do the "inner" projections as in this RFC and then use an explicit `.map_final(|x| x.foo)` call or something? It wouldn't be very ergonomic.
Benno: :+1: on adding it later, we could have it be a separate trait that would permit the optimization, then only types that need it can implement it.
Josh: As long as it's possible to add it later, :+1: for not adding it in the initial version.
## Borrow checking of field projection and soundness
> Before `@[mut] base->field` is desugared, the compiler performs a borrow-check
Gary: Do we have any other precedent of desugaring after borrow checking (I guess only async functions)? How can we ensure soundness of this?
Xiang: *handwavingly* I have a hunch that we can track it like "place"s like any other move paths or move out index.
Benno: It doesn't have to happen before desugaring, if we can track the stuff through the desugaring that would work too. I just phrased it that way, because it's much clearer that way.
Xiang: ~~So encoding this information in some opaque type?~~ nvm I have to think through.
## Multiple-level of projection
> When the projection goes through multiple fields, so `@base->field1->field2`, then those fields are just appended in the `field_of!` invocation:
Boqun: I obviously don't have enough knowledge to know the cost of this desugaring, but this means for a:
```rust
struct Foo {
bar: Bar {
baz: i32
}
}
```
it's going to generate two `Field` types.
Benno: field types are only generated if you use `field_of!`, so you only pay for what you use
Boqun: Ok, but would desugaring to two `project()` calls also work?
Benno: you have to add parenthesis, `@(@foo->bar)->baz` does two projections and `@foo->bar->baz` only does one.
Boqun: Oh, I see. I asked this question mostly because I'm curious about the compile time impact. But as I said, I don't have the enough knowledge to make further comment here, but your "pay as you go" argument sounds reasonable to me at the moment.
## Unsafe `NonNull<T>` projections
> These projections need to be `unsafe`, because `NonNull` could wrap around the address space.
TC: Talk with me about this analysis. I'd interested to see a worked example of the safety comment someone would need to write here to satisfy the proof obligation, and I'm curious if there aren't some constraints we could add somewhere to make a `NonNull<T> -> NonNull<Field>` projection safe for the many cases where this kind of proof will be trivially satisfiable.
tmandry: I think it would be the same as for calling [pointer::offset](https://doc.rust-lang.org/stable/std/primitive.pointer.html#method.offset).
scottmcm: yes, it's always unsafe for the same reason that `&raw const (*p).foo` is unsafe. Plus it's unsafe for the same reason that `NonNull::wrapping_add` needs to be unsafe.
scottmcm: The usual way you can prove this is the same way that miri can: the allocation exists, so any movement you do inside that allocation (including the past-the-end) isn't going to wrap the address space. (The allocator, be that for local variables or `impl Allocator` is thus the root of the "can't wrap the address space" proof.)
## New kind of reference
Josh: This is something that has come up in various contexts for years. We've talked about the idea of a type being able to offer a pseudo-reference to a field but not actually "have" a materialized instance of that field. If you actually want an instance you'd have to copy/move/etc, but you can in some ways act as if you have one. For instance, as the simplest example, if you have an `Option<Struct>`, you can have a `&magic Option<Field>` (for some bikeshedded `magic`). You don't actually have an `Option<Field>` in memory anywhere, but you logically have one.
scottmcm: I don't know how to do this as a *type*, since different places might want to represent it in different ways. If it's generic or carrying metadata, then it's basically just passing `impl GetSetField` or `dyn GetSetField`?
## `Option` projections
Josh: This is potentially complicated due to the issue of materializing the existence of an `Option`, but it'd be nice to be able to project an `Option<&Struct>` to an `Option<&Field>`, as one part of other projections. (e.g. projecting through an Arc with an Option in it).
tmandry: +1, I would hope and expect that this design can handle that.
scottmcm: reminds me of field pointers in C++, as having something we can pass into the option to use to do the projection inside it.
Sign in with Wallet
Connect another wallet