# Build Linux Kernel Image for Raspberry Pi from Source [Ubuntu Server]

###### tags: `linux2021 jserv 1-on-1`

# Raspberry Pi Ubuntu server

## [Steps](https://askubuntu.com/questions/1238261/customizing-the-kernel-arm64-using-ubuntu-20-04-lts-on-a-raspberry-pi-4)

* [Official debian build doc](https://wiki.ubuntu.com/Kernel/BuildYourOwnKernel)
* Since we are using ubuntu (which has the debian build system), the way to build for Desktop and raspberry pi is the same
* Install cross compiler (if you want to cross compile)
* `sudo apt install crossbuild-essential-arm64`
* if you just want the compiler, `sudo apt install gcc-aarch64-linux-gnu` should be enough
* you can also get the compiler [here](https://github.com/DLTcollab/toolchain-arm)
* install `git-lfs` (since the file is too large, you need to use [git lfs](https://www.atlassian.com/git/tutorials/git-lfs) to pull the files)
* extact by `tar -xvJf [filename]`
* add `export PATH=$PATH:/home/henry/toolchain-arm/gcc-arm-10.2-2020.11-x86_64-aarch64-none-linux-gnu/bin` to you `bashrc` file
* Export env variables

        export CC=aarch64-linux-gnu-gcc
        export $(dpkg-architecture -aarm64)
        export CROSS_COMPILE=aarch64-linux-gnu-

* Install build packages

    sudo apt update
    sudo apt build-dep linux linux-image-$(uname -r)   
    sudo apt install libncurses-dev linux-tools-common fakeroot

### Using good old `make`

* Get mainline source code `wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.14.tar.xz` and unzip `tar -xvJf linux-5.14.tar.xz`
* Patch the kernel
* (*for cacULE*)
* Follow the instructions down below
* During runtime, use `dmesg | grep -i "cacule cpu"` to check
* [Building from source](https://www.raspberrypi.org/documentation/computers/linux_kernel.html)
* `make distclean`
* patches stays, so if the patches are intended to be removed, use git
* `make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- defconfig`
* Make sure your `CROSS_COMPILE` name is set according to your compiler invocation prefix!
* `sed -i 's/CONFIG_LOCALVERSION_AUTO=y/CONFIG_LOCALVERSION_AUTO=n/' .config`
* In `.config` and search for `CONFIG_LOCALVERSION_AUTO=y` -> change it to `CONFIG_LOCALVERSION_AUTO=n`
* Disable adding `-gxxxx` string to the end of the uname. [Source](https://unix.stackexchange.com/questions/194129/linux-kernel-version-suffix-config-localversion)
* This will affect if the deb file can proper trigger the `/etc/kernel`'s post installation script to properly configure the boot loader. [Source](https://askubuntu.com/questions/1207467/raspberry-pi-4-custom-kernel-will-not-install-in-ubuntu-19-10)
* You can use `dpkg -x [deb package name]` to see the run scripts!
* `time make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- LOCALVERSION="-cfs-raspi" -j$(nproc) deb-pkg`
* Make sure your `LOCALVERSION` is set accordingly
* make sure your `CROSS_COMPILE` name is set according to your compiler invocation prefix!
* `LOCALVERSION` can only be lower case
* start with `-` is recommended, otherwise the name will follow directly behind the numeric version number with separation
* ending with `-raspi` is required for the auto installation to work (as mentioned above)
* Use `bindeb-pkg` to [build just binary package](https://patchwork.kernel.org/project/linux-kbuild/patch/[e-mail]/)
* Remove is done using `sudo apt autoremove --purge linux-image-5.14.0-cfs-raspi linux-headers-5.14.0-cfs-raspi`
* `sudo apt list --installed | grep "linux*"`
* Make sure a backup kernel is reinstalled tso the reboot will work
* `sudo apt install linux-image-5.4.0-1042-raspi`

#### All-in-one build command

make distclean
make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- defconfig
# run the cacULE script here is required
sed -i 's/CONFIG_LOCALVERSION_AUTO=y/CONFIG_LOCALVERSION_AUTO=n/' .config # to avoid the random string
time make ARCH=arm64 CROSS_COMPILE=aarch64-none-linux-gnu- LOCALVERSION="-cfs-raspi" -j$(nproc) deb-pkg # notice the lowercase restriction on LOCALVERSION (and the -raspi ending)

### For Raspberry Pi 5.4 kernel - using debian `debian/rules` tool

* Get the ubuntu linux for raspberry pi's kernel source code (kenrle version will be 5.4)

    mkdir ~/kbuild
    cd ~/kbuild
    git clone --depth=1 git://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/linux-raspi/+git/focal

* (*Only when you compile natively on raspberry pi*) uncomment the following 2 lines in `/etc/apt/sources.list` so you can fetch the source with `sudo apt-get source linux-image-unsigned-$(uname -r)`
* `deb-src http://ports.ubuntu.com/ubuntu-ports focal main restricted`
* `deb-src http://ports.ubuntu.com/ubuntu-ports focal-updates main restricted`
* Patch the kernel
* Follow the instructions down below
* Building (on rpi, from scratch: ~200min, creates 7 .deb files in ~/kbuild)
* Customizing uname
* (*doesn't work, unfortunately*) `fakeroot debian/rules editconfigs`
* [Update kernel name](https://stackoverflow.com/questions/28684811/how-to-change-version-string-of-the-kernel) -> `General Setup -> Local version - append to kernel release`
* If you check the build rule, it will tell you that this move will break the module generation script (you can try it yourself!)
* Update `debian.raspi/changelog` with things like `+CFS`

            In order to make your kernel "newer" than the stock Ubuntu kernel from which you are based you should add a local version modifier. Add something like "+test1" to the end of the first version number in the debian.master/changelog file, before building. This will help identify your kernel when running as it also appears in uname -a. Note that when a new Ubuntu kernel is released that will be newer than your kernel (which needs regenerating), so care is needed when upgrading. NOTE: do not attempt to use CONFIG_LOCALVERSION as this _will_ break the build.

* `time fakeroot debian/rules clean`
* `time fakeroot debian/rules binary-headers binary binary-perarch`
* cacULE patch will have changes in the config file, that you will need to respond to during compilation if would haven't run `fakeroot debian/rules editconfigs` before building from source
* Replacing the kernel

    # sudo apt install libunwind8 linux-tools-common libdw1
    sudo dpkg -i *.deb
    sync; sudo reboot

* Re-compile
* `time fakeroot debian/rules binary binary-perarch binary-headers`
* If `time fakeroot debian/rules clean` is run (e.g. kernel version changes)
* Use git to clean up
* `git clean -df`
* `git reset --hard HEAD`
* Use `rm` to clean up

            rm debian/stamps/stamp-build*
            rm -rf debian/linux-libc-dev/usr/include/aarch64-linux-gnu
            rm -rf debian/build/build-raspi/_____________________________________________dkms

## Changing kernel on Raspberry pi

* Since the installed version might be older than the current one, it might fail some how. So we have to [remove the current kernel and then reinstall the new one](https://askubuntu.com/questions/1316409/what-is-the-correct-way-to-select-an-older-installed-kernel-on-pi)

    # getting rid of auto-updated kernel

    # review installed kernels 
    apt list "linux-image*raspi"

    # get detail on an installed kernel
    apt show linux-image-5.4.0-1041-raspi

    # remove the "wrong" new version
    # NOTE YOU WILL GET A WARNING ABOUT REMOVING THE BOOT KERNEL (make sure you select NO lol, basically READ CAREFULLY on what the prmopt says)
    sudo apt autoremove --purge linux-image-5.4.0-1041-raspi 
    # DO NOT REBOOT

    # cd to location of my compiled kernel .deb files 
    # reference: kernel compile for ubuntu on pi
    # https://askubuntu.com/questions/1238261/customizing-the-kernel-arm64-using-ubuntu-20-04-lts-on-a-raspberry-pi-4
    cd ~/kbuild

    # install my previously compiled custom kernel as per instruction for installing a newly compiled or recompiled  kernel:
    sudo dpkg -i *.deb && sync

    sudo reboot

    # put my kernel on "hold" status to prevent auto-update
    echo "linux-image-5.4.0-1041-raspi hold" | sudo dpkg --set-selections

    # review dpkg status setting for linux-image:
    dpkg --get-selections | grep "linux-image"

### Issues

* `rmdir: failed to remove '/lib/modules/5.4.0-1042-raspi': Directory not empty`
* For some reason, it just can't, and **don't do it yourself, since it will break your device's ability to boot again**. See [this issue](https://answers.launchpad.net/ubuntu/+question/37009).

# Patch

* (*for cacULE*)
* `wget https://raw.githubusercontent.com/hamadmarri/cacule-cpu-scheduler/master/patches/CacULE/v5.14/cacule-5.14.patch`
* `wget https://raw.githubusercontent.com/hamadmarri/cacule-cpu-scheduler/master/scripts/apply_suggested_configs.sh`
* `patch -p1 < cacule-5.14.patch`
* ignore 1 level of depth
* `patch` and `git am` is might produce different results
* 3-way merge
* (*for muQSS*): 5.12 is the last version that muQSS has patch for
* patch
* several errors to resolve
* https://github.com/zen-kernel/zen-kernel/commit/59473b1177948ea41df9340c9010ab958008c880
* https://github.com/zen-kernel/zen-kernel/blob/5.13/master/kernel/sched/MuQSS.c
* http://ck-hack.blogspot.com/2019/11/linux-54-ck1-muqss-version-0196-for.html
* http://ck-hack.blogspot.com/2019/11/linux-54-ck1-muqss-version-0196-for.html

# For Raspbian

make distclean
KERNEL=kernel8
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- bcm2711_defconfig
time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j$(nproc) Image modules dtbs
sudo env PATH=$PATH make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- INSTALL_MOD_PATH=/media/home-tw/rootfs modules_install

sudo cp /media/home-tw/boot/$KERNEL.img /media/home-tw/boot/$KERNEL-backup.img
sudo cp arch/arm64/boot/Image /media/home-tw/boot/$KERNEL.img
sudo cp arch/arm64/boot/dts/broadcom/*.dtb /media/home-tw/boot/
sudo cp arch/arm64/boot/dts/overlays/*.dtb* /media/home-tw/boot/overlays/
sudo cp arch/arm64/boot/dts/overlays/README /media/home-tw/boot/overlays/

# Deprecated stuff

## Replacing the kernel

When the kernel is not properly ended with `-raspi`, the installation will need to pretty manual -> installation is fine, more like the removal will be pain in the ass

* install deb `sudo dpkg -i *.deb`
* `sudo flash-kernel 5.14.0-00577-g46f4945e2b39`
* ~~`cp arch/arm64/boot/dts/broadcom/*.dtb /boot/overlays`~~
* go to `/boot`, and manually change the soft links of `vmlinux` and `initrd.img`, also in the `firmware` folder, replace those to files
* [update `/boot`](https://wiki.ubuntu.com/ARM/RaspberryPi)