This tutorial is based on Ubuntu 20.04 and Xenomai 3.2.1.

### For Compiling and Installing on x86_64:

1. **[Prerequisites](#Prerequirements)**: Prepare the environment for compilation.
2. **[Xenomai Kernel](#Xenomai-Kernel)**: Download the Linux kernel and Xenomai source code, and configure the necessary options.
3. **[Compile and Install Kernel with Debian Packages](#Compile-And-Install-Kernel-with-debian-packages)**: Compile the Linux kernel and install it.
4. **[Xenomai Userspace](#Xenomai-Userspace)**: After installing and rebooting with the Xenomai kernel, install the Xenomai userspace library. This section also includes a sample Xenomai program to verify the installation.

> ℹ️ Note: If you want to install the kernel directly on the compiling machine, refer to the section on [#Compile-And-Install-Kernel-with-bzImage](#Compile-And-Install-Kernel-with-bzImage).

### For Compiling and Installing on ARM64 (Raspberry Pi 4):

1. **[Prerequisites](#Prerequirements)**: Prepare the environment for compilation.
2. **[Xenomai Kernel for Raspberry Pi 4 (Cross-Compile)](#Xenomai-Kernel-for-Raspberry-Pi-4-Cross-Compile)**: Download the Linux kernel and Xenomai source code, configure the options, and cross-compile the kernel for ARM64 on a x86_64 Linux system.
3. **[Install Kernel on Raspberry Pi](#Install-Kernel-On-Raspberry-Pi)**: After compiling the Linux kernel, transfer the Debian packages to the Raspberry Pi and install them.
4. **[Xenomai Userspace](#Xenomai-Userspace)**: After installing and rebooting with the Xenomai kernel, install the Xenomai userspace library. This section also includes a sample Xenomai program to verify the installation.

# Prerequirements

Tools for compiling xenomai kernel and xenomai userspace need to be installed first. You can use the commands below to install it.

sudo apt install -y make gcc libncurses5-dev flex bison libssl-dev libelf-dev dwarves zstd autoconf libtool curl dpkg-dev

# Xenomai Kernel

Download source code of Xenomai and Linux kernel

wget https://source.denx.de/Xenomai/linux-dovetail/-/archive/v5.15.166-dovetail1/linux-dovetail-v5.15.166-dovetail1.zip
wget https://source.denx.de/Xenomai/xenomai/-/archive/v3.2.1/xenomai-v3.2.1.tar.bz2

Extract the compressed source code files.

unzip linux-dovetail-v5.15.166-dovetail1.zip
tar xvf xenomai-v3.2.1.tar.bz2
export linux=$(pwd)/linux-dovetail-v5.15.166-dovetail1
export xenomai=$(pwd)/xenomai-v3.2.1

We need to run the `prepare-kernel` script for compiling the Linux kernel with Xenomai.

cd xenomai-v3.2.1/scripts
./prepare-kernel.sh --linux=$linux --arch=x86_64

We can configure the Linux kernel now.

cd $linux
make menuconfig

Setup the options

> Options that do not exist or cannot be changed are ignored.

* General setup
​   --> Local version - append to kernel release: -xenomai
    --> Timers subsystem
​       ---> High Resolution Timer Support [*]

​* Pocessor type and features
    ​ --> Processor family
        ​ ---> Core 2/newer Xeon
        (if "cat /proc/cpuinfo | grep family" returns 6, set as Generic otherwise)
     --> Multi-core scheduler support []


* Xenomai/cobalt
    ​ --> Sizes and static limits
        ​ ---> Number of registry slots  (512 --> 4096)
        ​ ---> Size of system heap (Kb)  (4096 --> 4096)
        ​ ---> Size of private heap (Kb) (256 --> 256)
        ​ ---> Size of shared heap (Kb)  (256 --> 256)
        ​ ---> Maximum number of POSIX timers per process (256 --> 512)
    ​ --> Drivers
        ​ ---> RTnet
            ​ ---> RTnet, TCP/IP socket interface (Enable)
                ​ ----> Drivers
                    ​ -----> New intel(R) PRO/1000 PCIe(Gigabit) [M]
                    ​ -----> Realtek 8169(Gigabit) [M]
                    ​ -----> Loopback [M]
                ​ ----> Add-Ons
                    ​ -----> Real-Time Capturing Support [M]

* Power management and ACPI options
    ​ --> CPU Frequency scaling
    ​   ---> CPU Frequency scaling []
    ​ --> ACPI (Advanced Configuration and Power Interface) Support
    ​   ---> Processor []
    ​ --> CPU Idle
        ​ ---> CPU idle PM support []

* Memory Management Options
    ​ ---> Transparent Hugepage Support []
    ​ ---> Allow for memory compaction []
    ​ ---> Contiguous Memory Allocation []
     ---> Page Migration []

​* Device Drivers
      --> Unisys visorbus driver []

Modify .config for preventing errors. We don't have the keys and we don't need the debug info.

./scripts/config --disable SYSTEM_TRUSTED_KEYS
./scripts/config --set-str SYSTEM_TRUSTED_KEYS ""
./scripts/config --disable SYSTEM_REVOCATION_KEYS
./scripts/config --set-str SYSTEM_REVOCATION_KEYS ""
./scripts/config --disable CONFIG_DEBUG_INFO_BTF
./scripts/config --set-str CONFIG_DEBUG_INFO_BTF "n"

Next, You have two options for compiling the kernel.

* [bzImage](#Compile-And-Install-Kernel-with-bzImage)
* [Debian Packages](#Compile-And-Install-Kernel-with-debian-packages)

## Compile And Install Kernel (with bzImage)

This will compiling and install on the host which is your running system.

make -j$(nproc) bzImage
make modules
make modules_install

## Compile And Install Kernel (with debian packages)

This will compiling for deb packages which you can send it to other or install the deb package on other host.

make -j$(nproc) bindeb-pkg

> **Compiling Options**
> * **-Zxz** change deb package compression type to xz.

> **Notice**
> After the compiling, the deb package files will in the parent folder which means on the previous directory `../`.

You can use the following commands for installing the Xenomai linux kernel.

sudo apt-get install ./linux-headers-5.10.191-xenomai_5.10.191-xenomai-1_amd64.deb
sudo apt-get install ./linux-image-5.10.191-xenomai-dbg_5.10.191-xenomai-1_amd64.deb
sudo apt-get install ./linux-image-5.10.191-xenomai_5.10.191-xenomai-1_amd64.deb
sudo apt-get install ./linux-libc-dev_5.10.191-xenomai-1_amd64.deb

We need to configure the grub for using our new linux kernel.

sudo nano /etc/default/grub

The content of `/etc/default/grub` should be look like below. We configure it to use the last selected kernel every boot.

GRUB_DEFAULT=saved
GRUB_SAVEDEFAULT=true
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX=""

After edited the file `/etc/default/grub`, you need the update the grub configuration.

sudo update-grub
sudo reboot now

After reboot you should see the grub menu, select `Advanced options for Ubuntu` and select the kernel which we have installed.

# Xenomai Kernel for Raspberry Pi 4 (Cross-Compile)

Download source code of Xenomai and Linux kernel

wget https://source.denx.de/Xenomai/linux-dovetail/-/archive/v5.15.166-dovetail1/linux-dovetail-v5.15.166-dovetail1.zip
wget https://source.denx.de/Xenomai/xenomai/-/archive/v3.2.1/xenomai-v3.2.1.tar.bz2

Extract the compressed source code files.

unzip linux-dovetail-v5.15.166-dovetail1.zip
tar xvf xenomai-v3.2.1.tar.bz2
export linux=$(pwd)/linux-dovetail-v5.15.166-dovetail1
export xenomai=$(pwd)/xenomai-v3.2.1

We need to run the `prepare-kernel` script for compiling the Linux kernel with Xenomai.

cd xenomai-v3.2.1/scripts
./prepare-kernel.sh --linux=$linux --arch=arm64

We can configure the Linux kernel now.

cd $linux
sudo apt install -y gcc-aarch64-linux-gnu
make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig
make menuconfig

Setup the options

> Options that do not exist or cannot be changed are ignored.

* General setup
​   --> Local version - append to kernel release: -xenomai
    --> Timers subsystem
​       ---> High Resolution Timer Support [*]

​* Pocessor type and features
    ​ --> Processor family
        ​ ---> Core 2/newer Xeon
        (if "cat /proc/cpuinfo | grep family" returns 6, set as Generic otherwise, raspberry pi returns nothing.)
     --> Multi-core scheduler support []


* Xenomai/cobalt
    ​ --> Sizes and static limits
        ​ ---> Number of registry slots  (512 --> 4096)
        ​ ---> Size of system heap (Kb)  (4096 --> 4096)
        ​ ---> Size of private heap (Kb) (256 --> 256)
        ​ ---> Size of shared heap (Kb)  (256 --> 256)
        ​ ---> Maximum number of POSIX timers per process (256 --> 512)
    ​ --> Drivers
        ​ ---> RTnet
            ​ ---> RTnet, TCP/IP socket interface (Enable)
                ​ ----> Drivers
                    ​ -----> New intel(R) PRO/1000 PCIe(Gigabit) [M]
                    ​ -----> Realtek 8169(Gigabit) [M]
                    ​ -----> Loopback [M]
                ​ ----> Add-Ons
                    ​ -----> Real-Time Capturing Support [M]

* Power management and ACPI options
    ​ --> CPU Frequency scaling
    ​   ---> CPU Frequency scaling []
    ​ --> ACPI (Advanced Configuration and Power Interface) Support
    ​   ---> Processor []
    ​ --> CPU Idle
        ​ ---> CPU idle PM support []

* Memory Management Options
    ​ ---> Transparent Hugepage Support []
    ​ ---> Allow for memory compaction []
    ​ ---> Contiguous Memory Allocation []
     ---> Page Migration []

​* Device Drivers
      --> Unisys visorbus driver []

Modify .config for preventing errors. We don't have the keys and we don't need the debug info.

./scripts/config --disable SYSTEM_TRUSTED_KEYS
./scripts/config --set-str SYSTEM_TRUSTED_KEYS ""
./scripts/config --disable SYSTEM_REVOCATION_KEYS
./scripts/config --set-str SYSTEM_REVOCATION_KEYS ""
./scripts/config --disable CONFIG_DEBUG_INFO_BTF
./scripts/config --set-str CONFIG_DEBUG_INFO_BTF "n"

Compile the linux kernel for deb packages.

make ARCH=arm64 KBUILD_DEBARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- CXXFLAGS="-march=armv8-a+crc -mtune=cortex-a72" CFLAGS="-march=armv8-a+crc -mtune=cortex-a72" -j$(nproc) bindeb-pkg

Next [Install-Kernel-On-Raspberry-Pi](#Install-Kernel-On-Raspberry-Pi)

## Install Kernel (On Raspberry Pi)

Copy your file to raspberry pi and install the packages by the following commands.

sudo apt install ./linux-headers-xxxx.deb
sudo apt install ./linux-image-xxxx.deb
sudo apt install ./linux-image-xxxx-dbg-xxxx.deb
sudo apt install ./linux-libc-dev_xxxx.deb

Use command `lsblk` to figure out the boot partition. From the results below boot partition is `/boot`.

pi@raspberrypi:~ $ lsblk
NAME        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
mmcblk0     179:0    0   58G  0 disk
├─mmcblk0p1 179:1    0  256M  0 part /boot
└─mmcblk0p2 179:2    0 57.7G  0 part /

Backup the old linux kernel in `/boot`.

mv /boot/kernel8.img /boot/kernel8.img.old

Use the compiled linux kernel with xenomai support.

mv /boot/vmlinuz-5.10.184-xeno /boot/kernel8.img

# Xenomai Userspace

> **⚠️ NOTICE**
> Steps below should be a root user to execute the commands
> **Tips: Command `sudo su -` can get into the root user**

Compile Xenomai Userspace and Install it on your system

wget https://source.denx.de/Xenomai/xenomai/-/archive/v3.2.1/xenomai-v3.2.1.tar.bz2
tar xvf xenomai-v3.2.1.tar.bz2 
cd xenomai-v3.2.1/
sudo apt install -y make automake libtool m4 autoconf fuse
./scripts/bootstrap
./configure
make
make install

> ℹ️ Use `./configure --enable-smp` on Raspberry Pi instead or it will not work.

> ℹ️ Use `./scripts/bootstrap --enable-dlopen-libs` for xenomai with dynamic library support.
> **REF:** https://github.com/LinuxCNC/linuxcnc/issues/1645

Configure Xenomai runtime environment for all user environment variable

! grep -q "/usr/xenomai/bin" /etc/bash.bashrc && echo "export PATH=/usr/xenomai/bin:\$PATH" | sudo tee -a /etc/bash.bashrc
echo "/usr/xenomai/lib" | sudo tee -a /etc/ld.so.conf.d/xenomai.conf
sudo ldconfig

Next we can run a xenomai example code

cd ~/
mkdir xeno_example
cd xeno_example

Use your favorite editor to copy and paste the code (xeno_example.c) below

nano xeno_example.c

**xeno_example.c**

#include <stdio.h>
#include <signal.h>
#include <unistd.h>

#include <alchemy/task.h>

RT_TASK hello_task;

// function to be executed by task
void helloWorld(void *arg)
{
  RT_TASK_INFO curtaskinfo;

  printf("Hello World!\n");

  // inquire current task
  rt_task_inquire(NULL,&curtaskinfo);

  // print task name
  printf("Task name : %s \n", curtaskinfo.name);
}

int main(int argc, char* argv[])
{
  char  str[10];

  printf("start task\n");
  sprintf(str,"hello");

  /* Create task
   * Arguments: &task,
   *            name,
   *            stack size (0=default),
   *            priority,
   *            mode (FPU, start suspended, ...)
   */
  rt_task_create(&hello_task, str, 0, 50, 0);

  /*  Start task
   * Arguments: &task,
   *            task function,
   *            function argument
   */
  rt_task_start(&hello_task, &helloWorld, 0);
}

Use your favorite editor to copy and paste the code (Makefile) below

nano Makefile

**Makefile**

XENO_CONFIG := /usr/xenomai/bin/xeno-config

CFLAGS := $(shell $(XENO_CONFIG) --posix --alchemy --cflags)
LDFLAGS := $(shell $(XENO_CONFIG) --posix --alchemy --ldflags)

CC := gcc   
EXECUTABLE := xeno_example

all: $(EXECUTABLE)

%: %.c
	$(CC) -o $@ $< $(CFLAGS) $(LDFLAGS)

clean: 
	rm -f $(EXECUTABLE)

Finally we can compile and run this example !

make
./xeno_example