Linux 核心設計課程專題: Real-Time Linux 研究

# Linux 核心設計課程專題: Real-Time Linux 研究 contributed by < `RainbowEye0486`, `fdfdd12345628` > ###### tags: `面試` 此文件中包含了如何在 rpi 上面編譯並且成功執行 Xenomai 4 (EVL)，以及 benchmark 後的成果。如果想要看 xenomai 3 的安裝與測試過程，請移駕至[Xenomai 3 — cobalt & mercury](https://hackmd.io/iuIcl6QkSjiWLq3RfNYd8Q) ## 硬體： Raspberry 4 我們採用直接在 rpi4 上面 compile 的方式(目前 cross compile 有點問題，在 rpi 4 上面直接 compile 大概需要花 1.5hr ，大概是筆電耗時兩倍)。我們使用的 rpi4 使用的 OS 是 [2021-05-07-raspios-buster-arm64-lite.zip](https://downloads.raspberrypi.org/raspios_lite_arm64/images/raspios_lite_arm64-2021-05-28/)，此 OS 的 kernel 版本是 5.10.17 。首先安裝 dependency ```shell sudo apt install git bc bison flex libssl-dev make libncurses-dev ``` 下載 linux-evl ```shell git clone https://git.xenomai.org/xenomai4/linux-evl.git ``` 進入 linux-evl ，並且設定環境變數。這裡選擇 5.10.19 是因為他最接近我們 OS 使用的 5.10.17 。 ```shell cd linux-evl git checkout v5.10.19-evl2-rebase export KERNEL=kernel8 export ARCH=arm64 ``` 載入預設 config ```shell make defconfig ``` 最後進行詳細 config ```shell make menuconfig ``` 將以下的 option 打開 - General setup - Local version (隨意設定，讓系統更新不會把你的 kernel 蓋掉) - Kernel .config support - Enable access to .config through /proc/config.gz (為了 evl 可以讀取 .config) - Kernel Features - EVL real-time core (本次主要目的) - Debug support - Timer frequency 1000Hz (高頻的 timer 意味更短的反應時間，但會增加系統執行成本) - CPU Power Managment - CPU Frequency scaling - Default CPUFreq gorvernor - performance (避免 cpu frequency 下降導致 delay 上升) 其他的 option 請根據自己需求設定。設定完後就可以開始 build ```shell make -j4 Image modules dtbs ``` 這裡大概需要等 80 分鐘。將 build 好的東西安裝起來 ```shell sudo make modules_install sudo cp arch/arm64/boot/dts/broadcom/*.dtb /boot/ sudo cp arch/arm64/boot/Image /boot/$KERNEL.img ``` 修改 /boot/config.txt ，確保系統開機進入我們的 kernel ``` # 加上這行 kernel=kernel8.img ``` 重新開機後，可以看到`dmesg`列出下面的訊息 ```shell $ dmesg | grep EVL [ 0.123482] IRQ pipeline: high-priority EVL stage added. [ 0.125676] EVL: core started [DEBUG] ``` 可以看到 EVL core 已經成功開啟。接著下載 libevl ```shell git clone https://git.xenomai.org/xenomai4/libevl.git ``` Compile 並且安裝，這裡會需要用到剛剛 linux-evl 的 header ，這裡請將 `UAPI` 這個參數自行修改到對應資料夾 ```shell cd libevl make -j4 UAPI=~/linux-evl-v5.10.19-evl2-rebase sudo make UAPI=~/linux-evl-v5.10.19-evl2-rebase install ``` 預設安裝位置為 /usr/evl ，安裝完後就可以測試 evl 了。因為執行 real-time 軟體需要使用 root 權限，這裡為了方便直接切換到 root 帳號來測試(這是壞習慣，不要模仿)。 ```shell $ sudo su # export PATH=$PATH:/usr/evl/bin # export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/evl/lib # evl test ``` 測試輸出如下 ```shell find: warning: you have specified the -maxdepth option after a non-option argument -type, but options are not positional (-maxdepth affects tests specified before it as well as those specified after it). Please specify options before other arguments. basic-xbuf: OK clock-timer-periodic: OK clone-fork-exec: OK detach-self: OK duplicate-element: OK element-visibility: OK fault: OK fpu-preload: OK fpu-stress: OK heap-torture: OK mapfd: OK monitor-deadlock: OK monitor-event: OK monitor-flags: OK monitor-pi: OK monitor-pi-deadlock: OK monitor-pp-dynamic: OK monitor-pp-lower: OK monitor-pp-nested: OK monitor-pp-pi: OK monitor-pp-raise: OK monitor-pp-tryenter: OK monitor-pp-weak: OK monitor-steal: OK monitor-wait-multiple: OK observable-hm: OK observable-inband: OK observable-master: OK observable-onchange: OK observable-oob: OK observable-race: OK observable-thread: OK poll-close: OK poll-flags: OK poll-nested: OK poll-observable-inband: OK poll-observable-oob: OK poll-sem: OK poll-xbuf: OK proxy-echo: OK proxy-eventfd: OK proxy-pipe: OK proxy-poll: OK sched-quota-accuracy.c:213: FAILED: evl_control_sched(44, &p, &q, test_cpu) (=Operation not supported) sched-quota-accuracy: no kernel support sched-tp-accuracy.c:225: FAILED: evl_control_sched(45, p, ((void *)0), test_cpu) (=Operation not supported) sched-tp-accuracy: no kernel support sem-close-unblock: OK sem-timedwait: OK sem-wait: OK simple-clone: OK stax-lock: OK stax-warn: OK thread-mode-bits: OK clock-timer-periodic.eshi: OK detach-self.eshi: OK heap-torture.eshi: OK monitor-event.eshi: OK monitor-flags.eshi: OK monitor-wait-multiple.eshi: OK poll-nested.eshi: OK poll-sem.eshi: OK proxy-eventfd.eshi: OK proxy-pipe.eshi: OK sem-timedwait.eshi: OK sem-wait.eshi: OK ``` 接著就可以測試 latency ```shell # latmus warming up on CPU0 (not isolated)... RTT| 00:00:01 (user, 1000 us period, priority 98, CPU0-noisol) RTH|----lat min|----lat avg|----lat max|-overrun|---msw|---lat best|--lat worst RTD| 3.241| 3.492| 7.030| 0| 0| 3.241| 7.030 RTD| 2.644| 3.246| 11.308| 0| 0| 2.644| 11.308 RTD| 2.593| 3.096| 19.392| 0| 0| 2.593| 19.392 RTD| 2.646| 3.092| 11.351| 0| 0| 2.593| 19.392 RTD| 2.657| 2.919| 9.874| 0| 0| 2.593| 19.392 RTD| 2.610| 2.893| 10.821| 0| 0| 2.593| 19.392 RTD| 2.614| 3.247| 9.806| 0| 0| 2.593| 19.392 RTD| 2.612| 2.907| 9.061| 0| 0| 2.593| 19.392 RTD| 2.652| 2.933| 9.240| 0| 0| 2.593| 19.392 ``` 測試後 latency 平均在 3ms ，最差不會超過 20ms :::warning 這個命令有誤！應該要用 `aarch64-linux-gnu-`! 詳見 [Kernel building](https://www.raspberrypi.org/documentation/linux/kernel/building.md): - `aarch64-linux-gnu-` 針對 Arm64 (Armv8-A) - `arm-linux-gnueabihf-` 則是 Armv7-A 架構 :notes: jserv ::: > 已經改成 64bit 建構方式了！ > [name=fdfdd12345628] ## 硬體：其他 rpi 或是其他硬體其他 rpi 編譯與安裝過程理論上差不多(可能要注意一下 `arm` 跟 `arm64` ，以及 `config.txt` 需要修改的地方，詳情請見 [rpi 官方說明](https://www.raspberrypi.org/documentation/linux/kernel/building.md))，但是其他硬體的安裝過程就需要看每個硬體了。 :::spoiler 棄用資訊 ## middleware 1. ros2/1 2. micro-ros 3. Xenomai ## linux kernel support 1. PREEMPT_RT ## RTOS kernel 1. freertos 2. Nuttx 3. Zephyr ## cloud 1. [GCP k8s auto-scaling](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler) ::: ## Reference 1. [FreeRTOS](https://zh.wikipedia.org/wiki/FreeRTOS) 2. [First micro-ROS Application on an RTOS](https://micro.ros.org//docs/tutorials/core/first_application_rtos/) 3. [Xenomai+現有rtos介紹](https://blog.csdn.net/pwl999/article/details/109412539) 4. [老師給的可能方向―ros2/realtime_support](https://github.com/ros2/realtime_support) 5. [Linux 核心設計: PREEMPT_RT 作為邁向硬即時作業系統的機制](https://hackmd.io/@sysprog/preempt-rt) 6. [The Real-Time Linux Kernel: A Survey on PREEMPT_RT](https://dl.acm.org/doi/fullHtml/10.1145/3297714) 7. [Xenomai](https://stackoverflow.com/questions/65163745/is-rtos-effective-for-cloud-edge-computing) 8. [快快樂樂設計嵌入式系統](http://wiki.csie.ncku.edu.tw/embedded/2012w3/HappyRTOS.pdf) 9. [Xenomai_成大資工 Wiki ](http://wiki.csie.ncku.edu.tw/embedded/xenomai) 10. [Xenomai (学习笔记)](https://blog.csdn.net/pwl999/article/details/109412539) 11. [aarch64 指令集介紹](http://wiki.csie.ncku.edu.tw/embedded/ARMv8) 12. [CyclicTest 安裝](https://wiki.ubuntu.com/UbuntuStudio/CyclicTest) 13. [patch](https://android.googlesource.com/kernel/common/+/android-3.18/Documentation/applying-patches.txt) 14. [xenomai3 install on raspberrypi 4](http://www.simplerobot.net/2019/12/xenomai-3-for-raspberry-pi-4.html) 15. [Raspberry Pi: Real Time System - Xenomai Patching Tutorial for Kernel 4.14.y](https://lemariva.com/blog/2018/07/raspberry-pi-xenomai-patching-tutorial-for-kernel-4-14-y?fbclid=IwAR0k-dXYQM9jnrjavWHf7z8ITnhqYIcrQPPrvRtlCg63q2_WAbgNokzmnbc) ###### tags: `linux2021`

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.