Rust Linux Kernel Module development

# Rust Linux Kernel Module development using Raspberry Pi 5 The goal of this tutorial is to load an out of tree kernel module written in Rust into an RPi kernel. For the module we will use the example from here <https://github.com/Rust-for-Linux/rust-out-of-tree-module> > This guide should work on most 64bit RPi's with support for Kernel `6.12`, check the RPi [arch](https://github.com/raspberrypi/linux/blob/rpi-6.12.y/Documentation/rust/arch-support.rst) for more details. > The point of this article is not to promote out of tree linux kernel modules. One should, if possible, put in efforts towards getting code into the mainline kernel. As per the [README.md](https://github.com/Rust-for-Linux/rust-out-of-tree-module/blob/main/README.md), we need 1. The *kernel that the module is built against needs to be Rust-enabled* (`CONFIG_RUST=y`). To verify this lets run ``` >grep CONFIG_RUST /boot/config-$(uname -r) CONFIG_RUSTC_VERSION=0 CONFIG_RUSTC_LLVM_VERSION=0 CONFIG_RUSTC_SUPPORTS_ARM64=y ``` so we don't see the `CONFIG_RUST=y`, which means the the crates (such as [libcore](https://doc.rust-lang.org/core/)) that will be used by the rust kernel module(s) are not compiled and linked into the kernel. We will need to fix this by building a new Linux Kernel for the Pi! But does the official RPi kernel support Rust ? A quick look into RPi arch doc says it does support `arm64` 🥳! <https://github.com/raspberrypi/linux/blob/rpi-6.12.y/Documentation/rust/arch-support.rst> > We do need to note that the doc also states that we need to use *LLVM for code generation* 2. To build the kernel module, *Kernel tree (KDIR) requires the Rust metadata (for eg. `rust/libcore.rmeta`)to be available*. This is only available when we build the kernel from source. Prebuilt kernels that comes shipped with images have these removed. ## Prepping the PI One can build the kernel natively in a Raspberry Pi or cross compile the kernel(much faster). The official raspberry pi foundation [document](https://www.raspberrypi.com/documentation/computers/linux_kernel.html) on linux kernel does provide some hints. Lets build the kernel natively on Pi, although really slow it does bring in some advantages. ## Toolchain setup ### Kernel build tools ``` sudo apt install bc bison flex libssl-dev libelf-dev make clang lld llvm ``` ### Rust deps #### Rust toolchain ``` curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh ``` > update paths as recommended by the installer #### Rust build tooling deps ``` rustup component add rust-src sudo apt install rust-clippy rustc rustfmt ``` ### Module Versioning packages (optional) This is required by [gendwarfksyms](https://docs.kernel.org/kbuild/gendwarfksyms.html) module which was introduced in kernel 6.14 <https://lwn.net/Articles/986892/> . This is only recommended if we want to build kernel >=`6.14` ``` sudo apt install libdwarf-dev libdw-dev ``` >`gendwarfksyms` (`CONFIG_GENDWARFKSYMS`) tool depends on `CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT` symbol to be set to `y`. `gendwarfksyms` needs the toolchain provided DWARF information. ### Toolchain checks Check if we are ready to build our new kernel with support for rust language. Lets clone the rust for linux repo and run some tests. In a separate folder lets try ``` git clone --depth=1 https://github.com/Rust-for-Linux/linux.git rust-for-linux cd rust-for-linux/ ``` #### Test ``` make LLVM=1 rustavailable ``` This must result in the following message. ``` Rust is available! ``` ## Cloning the Linux kernel Lets first try to know which kernel we are currently on. ``` > uname -r 6.12.47+rpt-rpi-2712 ```` So we have `6.12` as kernel version. Lets find something similar. > Same kernel version or the latest stable release version would be easiest path. Both happens to be `6.12` as of writing this doc. RPi officialy maintains their kernel source tree here <https://github.com/raspberrypi/linux> ``` >git ls-remote --branches https://github.com/raspberrypi/linux | grep 6.12 c2cbd9c611256e7b957f75c23d9f76d58a4893c1 refs/heads/rpi-4.7.y 7175dd6f062127fb973ef2ba03566c02b5b413bc refs/heads/rpi-4.9.y-rebase 5341e6a36944546b4edf717d5a116ebdfb997743 refs/heads/rpi-6.12.y ``` #### Clone `6.12` ``` git clone --depth=1 --branch rpi-6.12.y https://github.com/raspberrypi/linux ``` This may take some time. ## Configuring the kernel Inside the cloned folder Set Kernel name ``` KERNEL=kernel_2712 ``` Check if symbol was set, just to be sure. ``` echo $KERNEL ``` Generate default configs ``` make LLVM=1 bcm2712_defconfig ``` #### Setting custom configuration using menuconfig ``` make LLVM=1 menuconfig ``` > when in menuconfig, you can start search using `/`. This can then be used to find symbols and also see the `depends on` details of the symbols. Change the following configs 1. *Enable loadable module support* [Set to `y`] ----> * *Module versioning support* [Set to `n`] 2. *General setup* ----> * *Rust support* [Set to `y`] 3. *Kernel hacking* ----> * *Sample kernel code* [Set to `y`] ----> * *Rust samples* [Set to `y`] ----> * *Minimal* [Set to `y`] * *Printing macros* [Set to `y`]   4. *General setup* ----> * `(-v8-16k)` *Local version - append to kernel release* * Change to something that can be identified easily , say for eg. `-v8-16k-rusty-pi` Remember to save the configuration ! ## Building the kernel Building the kernel will take a loong time 1-2hrs in RPi 5. If you are connected to pi via ssh, the recommended practice may be to run the command in a [Tmux](https://github.com/tmux/tmux/wiki/Installing) or [Zellij](https://zellij.dev/documentation/installation.html) session. Inside the Tmux or Zellij session, run ``` make LLVM=1 -j6 Image.gz modules dtbs ``` Time for coffee ☕ and more coffee ! ## Installing the newly built kernel ### Load kernel modules ``` sudo make -j6 modules_install ``` ### Back-up the current one ``` sudo cp /boot/firmware/$KERNEL.img /boot/firmware/$KERNEL-backup.img ``` ### Copy the kernel image ``` sudo cp arch/arm64/boot/Image.gz /boot/firmware/$KERNEL-rusty-pi.img ``` ### Copy device tree ``` sudo cp arch/arm64/boot/dts/broadcom/*.dtb /boot/firmware/ sudo cp arch/arm64/boot/dts/overlays/*.dtb* /boot/firmware/overlays/ sudo cp arch/arm64/boot/dts/overlays/README /boot/firmware/overlays/ ``` ### Configure `config.txt` Edit/ Add `kernel` symbol in `/boot/firmware/config.txt` ``` ..... # Kernel image kernel=kernel_2712-rusty-pi.img # Run in 64-bit mode arm_64bit=1 .... ``` For more details, see [official docs](https://www.raspberrypi.com/documentation/computers/config_txt.html#kernel) from RPi. ### Reboot ``` sudo reboot ``` ### logs -> Rust says hello If UART was enabled and if you are connected via UART, you should see the following on screen while Pi boots. Alternatively you can also plugin your screen ``` [ 0.311832] rust_print: Emergency message (level 0) without args [ 0.317896] rust_print: Alert message (level 1) without args [ 0.323579] rust_print: Critical message (level 2) without args [ 0.329526] rust_print: Error message (level 3) without args [ 0.335216] rust_print: Emergency message (level 0) with args [ 0.340980] rust_print: Alert message (level 1) with args [ 0.346395] rust_print: Critical message (level 2) with args [ 0.352083] rust_print: Error message (level 3) with args ``` ### Sample Rust modules in kernel Check if modules are loaded ``` > modinfo rust_minimal name: rust_minimal filename: (builtin) author: Rust for Linux Contributors description: Rust minimal sample license: GPL file: samples/rust/rust_minimal > modinfo rust_print name: rust_print filename: (builtin) author: Rust for Linux Contributors description: Rust printing macros sample license: GPL file: samples/rust/rust_print > dmesg | grep -i rust_minimal [ 0.311983] rust_minimal: Rust minimal sample (init) [ 0.311985] rust_minimal: Am I built-in? true > dmesg | grep -i rust_print [ 0.311988] rust_print: Rust printing macros sample (init) [ 0.311989] rust_print: Emergency message (level 0) without args [ 0.318058] rust_print: Alert message (level 1) without args [ 0.323742] rust_print: Critical message (level 2) without args [ 0.329690] rust_print: Error message (level 3) without args [ 0.335371] rust_print: Warning message (level 4) without args [ 0.335372] rust_print: Notice message (level 5) without args [ 0.335373] rust_print: Info message (level 6) without args [ 0.335374] rust_print: A line that is continued without args [ 0.335376] rust_print: Emergency message (level 0) with args [ 0.341141] rust_print: Alert message (level 1) with args [ 0.346558] rust_print: Critical message (level 2) with args [ 0.352236] rust_print: Error message (level 3) with args [ 0.357653] rust_print: Warning message (level 4) with args [ 0.357654] rust_print: Notice message (level 5) with args [ 0.357655] rust_print: Info message (level 6) with args [ 0.357656] rust_print: A line that is continued with args [ 0.357658] rust_print: 1 [ 0.357660] rust_print: "hello, world" [ 0.357662] rust_print: [samples/rust/rust_print.rs:35:5] c = "hello, world" [ 0.357663] rust_print: "hello, world" ``` Logs show that modules are loaded and good to go ! ## Out of tree kernel module development We now have a compiled kernal source tree (`KDIR`) that has Rust features and also installed the same kernel in RPi. ### Building out of tree module #### Clone the template ``` git clone https://github.com/Rust-for-Linux/rust-out-of-tree-module git checkout 00b5a8ee2bf53532d115004d7636b61a54f49802 cd rust-out-of-tree-module ``` #### Quick fixes The example has some features which are not compatible with our rust library in `6.12` kernel. Hence this fix > The authors confirm that the template template meant to track mainline Change line 10, in `rust_out_of_tree.rs` ``` author: "Rust for Linux Contributors", ``` #### Build ``` make ``` #### To Load module ``` sudo insmod rust_out_of_tree.ko ``` #### To Remove module ``` sudo rmmod rust_out_of_tree ``` The logs can be viewed using `sudo dmesg` . Opening a tmux/Zellij session and having a pane open with `sudo dmesg -W` will show the latest logs in real time ## References Thanks to [Miguel Ojeda](https://ojeda.dev/) , Gary Guo and Benno Lossin for helping me understand the process Also thanks to the rust for linux community <https://rust-for-linux.zulipchat.com/> RPi Docs : https://www.raspberrypi.com/documentation/computers/linux_kernel.html