---
title: "Design meeting 2024-01-31: Rust for Linux use cases"
tags: ["T-lang", "design-meeting", "minutes"]
date: 2024-01-31
discussion: https://rust-lang.zulipchat.com/#narrow/stream/410673-t-lang.2Fmeetings/topic/Design.20meeting.202024-01-31
url: https://hackmd.io/sceE6uE9S7W7tocc-wjORg
---

> # Summary
>
> We would like an opportunity to discuss what has been going well and what has been difficult with RfL. There are two recently relevant topics that would be good to discuss:
>
> * Bitfields have been somewhat of a pain point. Are there thoughts on official support? [Add bitfields support rfcs#3113](https://github.com/rust-lang/rfcs/pull/3113)?
> * Field projection, which improves pin ergonomics (likely also some of the bitfield pain points). [RFC: Field projection rfcs#3318](https://github.com/rust-lang/rfcs/pull/3318)
>
> It would be nice to just get the general feel of what the team thinks here, we do not need to plan for in-depth discussion.
>
> I think that a large portion of the meeting will likely just be introductions and figuring out what kind of communication we would like to have going forward.
>
> If time allows, we can go through the wanted features lists and spend 1-2 minutes per topic getting a pulse check (whether the Rust team would like / is open to / is against each item) so we know how best to proceed with workarounds vs. in-tree work.
>
> # Background reading
>
> * A list of new features for RfL [Rust wanted features Rust-for-Linux/linux#354](https://github.com/Rust-for-Linux/linux/issues/354)
> * A list of unstable features we use or would like to use [Rust unstable features needed for the kernel Rust-for-Linux/linux#2](https://github.com/Rust-for-Linux/linux/issues/2)
> * Some of the complications of safely accessing bitfields in extern structs [bitfields raw pointer accessors rust-bindgen#2674](https://github.com/rust-lang/rust-bindgen/issues/2674)

---

# Discussion

## Attendance

- People: TC, Trevor, Andreas, Boqun Feng, Wedson, Gary Guo, Alice Ryhl, Lukas Wirth, Josh, Santiago, Miguel, pnkfelix, Sid A, tmandry, scottmcm, eholk, nikomatsakis, CE, Benno Lossin

## Meeting roles

- Minutes, driver: TC

## Pulse check

Miguel: We'd like to get a pulse check on some of these.

## Key topics

Alice: Here are some key topics:

- Language features needed for custom `Arc`.
- Language features for bitfields.

gary guo: I think alloc, pin/field projections, const reference to static are the most important topics

miguel: What we need is our own `Arc`, we don't need the features of `Arc` per se. Doesn't mean that we have to stabilize what is in the stdlib. Another way to do it is some kind of "preview" support.

NM: When you said `Arc` features, are you talking about the special treatment that `Arc` gets for `dyn` upcasting and Self receivers?

Alice: dyn upcasting and self receivers, yes. (coerce_unsized, dispatch_from_dyn, receiver -- gary)

Alice: And to have a raw pointer to a static that ends up in const.

Josh: Confirming, you don't need a real pointer during const evaluation time, you just need a pointer that will be valid later duirng execution?

Alice: Correct.

Wedson: Needed to do things like vtable(?) operations. your operations structure needs a pointer to the module itself, in addition to the various functions, and the module is static data.

NM: You would like to allocate some global static space and embed a pointer to it in a const that you can access during runtime?

all: Yes.

NM: That makes sense. One thing we heard and makes sense to me is that for many kernel developers, this will be their first experience for Rust. We want that to be a good experience without too many paper cuts. How are you feeling about that, e.g. with respect to `Pin`?

Gary: Ergonomics and pin/projections are one of the top issues. We have lots of data structures that require pinning. We have a macro to enable doing this in safe Rust. You get less IDE support because of this and you have to use this macro everywhere when initializing data strucures.

https://github.com/Rust-for-Linux/pinned-init

NM: I'd like to get a tour of how you're using pinning.

Alice: The kernel needs intrusive linked lists everywhere. This is less important in userspace than it is in the kernel. Because we're wrapping C code, we have to do this a lot. So that introduces many macros.

Boqun: People don't always realize when their data structures are `!Unpin`.

Alice: The projection things also come in here.

Gary: This is all intertwined with issues of fallible allocation.

NM: How are these linked lists allocated?

Alice: They're often allocated early. An object can move between different lists. And it's an important property that adding an element to these lists is never fallible.

Wedson: There are contexts in which you can't allocate, so it can be important to do these early on.

NM: Are there ways to enforce that statically, and would you like that?

Wedson: Yes, we'd like that.

Alice: We have a linter about that, but it needs more work.

* https://github.com/rust-for-Linux/klint
* https://www.memorysafety.org/blog/gary-guo-klint-rust-tools/

Wedson: The rules are more complicated than what the linter does. If this could be done statically, we'd love that.

Alice: That the linked lists can be intrusive goes back to the issue of custom `Arc`.

NM: Would love to see links to that code.

Josh: Rust doesn't understand the ownership and borrowing here, but there will never be more than one reference, that's the idea?

Josh: The embedded references within a given type, since it can only be on one list, there can be only zero or one references?

Alice: We have some other mechanism to ensure that there's only one list `Arc`.

NM: It's not clonable so to speak, but you can create one from it that is.

Josh: To what degree is this a reusable type that we could offer in general?

Alice: There is no reason that you couldn't do this outside of the kernel.

Alice: Link to linked list: https://r.android.com/2849902

## Meta

NM: How can we keep our projects in better sync?

Josh: How can we keep this dialog going, and chew through this list of design issues.

NM: I'm concerned because these all seem like good items. But I'm not sure where the bandwidth would come from to drive this design work.

NM: Maybe there could be a lang team liaison.

Josh: If we are expecting to review designs brought to us, there's still a lot of value in smoketesting designs and starting those discussions early.

TC: It'd be good for the effort to go into the design work to add these things to the language rather than the work going into the workarounds.

NM: I'm volunteering some time to help consult on tactics etc. to help move this forward.

NM: On project goals, I wrote about this here:

https://smallcultfollowing.com/babysteps/blog/2023/11/28/project-goals/

Josh: I'm also happy to help with that, both at the meta-level (project goals/planning) and the object level (reviewing specific designs or iterating on designs)

Miguel: We will need help from the Rust side. What would we do, e.g., about the existing features that are unstable? If there is a way to speed things up, that would be good.

NM: There's probably some combination of low hanging fruit and some things that need more complex design. The perma-unstable ones are the most interesting.

TC: It would in particular be interesting to hear from RfL about the need for raw pointers as receivers, as that was something about which we had questions.

Josh: Here's that RFC:

https://github.com/rust-lang/rfcs/pull/3519

Josh: Anything marked permanently unstable should be treated as something that doesn't exist in Rust at all. Specialization is the classic example. We have no idea how to stabilize that in a sound way, so it'd need a redesign.

Miguel: We can use e.g. specialization in particular places where we know that it is sound.

Miguel: We don't need everything stabilized. What we need is stability even on unstable, that would be one solution. It'd be OK if we had e.g. a window of releases where unstable things work for at least some use cases for a window of time. We can promise that we would move off of the feature within a particular period.

Trevor: We use `RUSTC_BOOTSTRAP=1` right now.

Miguel: We'd like to use that, but have it work with a set of versions.

NM: From the Rust project's point of view, we'd like to get RfL on stable-stable. But what I hear you saying is that, if that's not feasible in the short term, it might be better for both of us to have some sort of preview state.

pnkfelix: This is similar to Niko's post:

https://smallcultfollowing.com/babysteps/blog/2023/09/18/stability-without-stressing-the-out/

Miguel: We would be using these in a constrained use case.

NM: It'd be better to use a set of features that we've both agreed upon, both projects.

Josh: Adding one thing, if there is a scenario today in which the board use case of a feature is not sound, but the kernel's specific use is sound, then that's a good argument for creating a new feature that is that subset. So this may help us find a path to stabilization.

NM: We should put RfL on crater.

Josh: Would take some care because it *is* using nightly features, but we should have some way to test and know if we've broken the kernel.

Josh: There's a spectrum of difficulty for stabilizing features. Having ongoing discussions would help with all of these. Spectrum from easiest to hardest:
- Seems to work as designed, "just" needs stabilizing
- Mostly works but needs some fixes before stabilizing
- Unsound as specified but a subset would work for the kernel and we might be able to stabilize that subset
- No obvious sound subset, needs redesign
- No design in the first place, needs design

Miguel: It would be helpful to spend time with people that understand where different issues fall on that spectrum.

## How much is lang?

TC: Let's talk about how much of this falls on the lang side.

Miguel: The biggest other thing is the `alloc` piece. The most flexible way to do what the kernel requires of us is to have our own alloc.

Miguel: There's also the `cfg` thing we need to disable some things in `core`.

Boqun: There are issues with `bindgen`. It would be good if people here could comment about that on the `bindgen` side.
(for example: https://github.com/rust-lang/rust-bindgen/issues/2674)

Josh: There are things we could do to make `bindgen` less necessary.

Josh: Portions of the `alloc` problem are in the lib space. However, there are ways that lang could help with that in terms of language features that could make that easier.

Josh: E.g., you need flags for allocation, and maybe a parameterized allocator. Instead, it's possible that what you need is the ability to use the "contexts and capabilities" mechanism. Then you could enter a context where allocation is required to be atomic. We could potentially solve that in the language using that feature.

https://tmandry.gitlab.io/blog/posts/2021-12-21-context-capabilities/

Alice: We would rather have the compilation fail than fail dynamically.

Boqun: There is ongoing discussion about how to make this just work.

Josh: However that debate ends up on the kernel side, having this feature would help with that argument. This would be a zero-overhead mechanism.

tmandry: When we're swapping out alloc, we're talking about changing the API as well as the implementation?

Wedson: Yes.

Gary: Contexts would not necessary work in this case, because you may want to call a function that may take away a capability.

tmandry: Is the allocator parameter to e.g. Vec sufficient for handling the other types?

Wedson: Allocation handling is a function of the context, not the type. You might have the *same* Vec, but in some contexts you need to do allocation for it using atomic alloc, in other cases you don't.

## Next steps

TC: We're seeing value in this discussion. NM, how do we keep this going?

NM: We should set up these narrower meetings we've discussed.

NM: It'd also be good to decide on some initial wins that we could pursue.

TC: What do we think about the first achievable things?

Alice: Arc (the bits needed for that).

Miguel: And keeping the conversation going.

Miguel: It'd also help to have a regular general meeting scheduled.

Josh: I'm happy to do it as often as the RfL people have availability. I could do twice a month.

NM: I'm happy to follow along as possible and find what the sustainable cadence is.

Josh: Let's start a meeting series about this.

all: +1.

tmandry: I'd like to get to a shared understanding of the use case and why it's important both for the kernel and for the Rust project. Perhaps the RfL team could present at a conference I'm helping organize in March.

Josh: Also want to get RfL feedback on the arbitrary self types v2 RFC.

(Some further discussion followed about next steps; TC will start a thread on #t-lang; Josh will create a LettuceMeet.)

(The meeting ended here, with much appreciation expressed by members of the Rust project for the work done by the RfL team to bring Rust into the kernel, and by the RfL team for the ongoing work done by members on the project on the Rust language.)