---
title: 'O-DU Installation Guide [Rel. F]'
disqus: hackmd
---
O-DU Installation Guide [Rel. F]
===
# 目錄
[TOC]
# Server Specifications
If you are first time to install O-DU Low in the new environment or new HDD, remember to adjust your OS Setting as below
Refer to : https://github.com/intel/FlexRAN#os-configuration
## server 1
# System installation settings
## Bios Configuration
### Advanced -> Power & Performance
```gherkin=
CPU Power and Performance Policy : Performance
```
### Advanced -> Power & Performance -> Uncore Power Management
```gherkin=
Uncore Frequency Scaling : Disabled
Performance P-limit : Disabled
```
### Advanced -> Power -> CPU P State Control
```gherkin=
Enhanced Intel SpeedStep (R) Tech : Enabled
Intel Configurable TDP : Enabled
Configurable TDP Level : Level 2
Intel(R) Turbo Boost Technology : Enabled
Energey Efficient Turbo : Disabled
```
### Advanced -> Power & Performance -> Hardware P States
```gherkin=
Hardware P-States : Disabled
```
### Advanced -> Power & Performance -> CPU C States Control
```gherkin=
Package C-State : C0/C1 state
C1E : Disabled
Processor C6 : Disabled
```
## OS configuration
First you should install CentOS 7 (7.5+) from CentOS official site, then install Real Time kernel and required tools to tune the OS real timer performance. CentOS 7.7 and RT kernel kernel-rt-3.10.0-1062.12.1.rt56.1042 have been validated for this release.
```gherkin=
yum install -y kernel-rt kernel-rt-devel kernel-rt-kvm rtctl rt-setup rt-tests tuna tuned-profiles-nfv tuned-profiles-nfv-host tuned-profiles-nfv-guest qemu-kvm-tools-ev
```
### root 權限設定
```gherkin=
1）進入超級用戶模式。也就是輸入 su - ,系統會讓你輸入超級用戶密碼，輸入密碼後就進入了超級用戶模式。
2）添加文件的寫權限。也就是輸入命令 chmod u+w /etc/sudoers。
3）編輯/etc/sudoers文件，也就是輸入命令 nano /etc/sudoers，找到這一行： root ALL=(ALL) ALL 在這下面添加 xxx ALL=(ALL) ALL (這裡的xxx是你的用戶名)
```
### Install RT Kernel
1. Download kernel-rt
* Here we need use https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
```gherkin=
uname -a
sudo wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
```
* 
2. Install other packages first
* Before install the kernel-rt, you need to install the following packages(must be in order)
tuna
```gherkin=
sudo yum install -y tuna
rpm -q tuna
```
* 
tuned
```gherkin=
sudo wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/tuned-2.9.0-1.el7fdp.noarch.rpm
sudo yum remove tuned.noarch -y
sudo rpm -ivh tuned-2.9.0-1.el7fdp.noarch.rpm
rpm -q tuned
```
* 
tuned-profiles-realtime
```gherkin=
sudo wget http://mirror.centos.org/centos/7/rt/x86_64/Packages/tuned-profiles-realtime-2.9.0-1.el7fdp.noarch.rpm
sudo rpm -ivh tuned-profiles-realtime-2.9.0-1.el7fdp.noarch.rpm
rpm -q tuned-profiles-realtime
```
* 
rt-setup
```gherkin=
sudo wget http://mirror.centos.org/centos/7/rt/x86_64/Packages/rt-setup-2.0-9.el7.x86_64.rpm
sudo rpm -ivh rt-setup-2.0-9.el7.x86_64.rpm
rpm -q rt-setup
```
* 
3. Install the kernel-rt
```gherkin=
sudo wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
sudo rpm -ivh kernel-rt-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
rpm -q kernel-rt
```
* 
4. Real-time Kernel Patching
* https://wiki.archlinux.org/title/Realtime_kernel_patchset
```gherkin=
sudo yum-config-manager --add-repo=http://linuxsoft.cern.ch/cern/centos/7/rt/CentOS-RT.repo
```
* Set **gpgcheck=1 to gpgcheck=0** for each entry in /etc/yum.repos.d/CentOS-RT.repo
* 路徑需要對，不然會找不到此資料夾
* 
```gherkin=
sudo nano /etc/yum.repos.d/CentOS-RT.repo
```
* 
* Install the yum group RealTime via
```gherkin=
sudo yum group install -y RealTime
```
* Install other packages
```gherkin=
sudo yum update -y
sudo yum install -y rt-tests
```
* Reboot server
```gherkin=
sudo reboot
```
* use uname -a to check the kernel is rt kernel
```gherkin=
uname -a
```
* 
* Centos 7 upgrade and switch kernel : 如需使用指定舊版本或是新版本時可切換
```gherkin=
uname -r #查看當前版本
cat /boot/grub2/grub.cfg |grep "menuentry " # 查看所有可用內核
rpm -qa |grep kernel-[0-9] # 查看全部內核包
grub2-set-default 'CentOS Linux (3.10.0-1062.12.1.rt56.1042.el7.x86_64) 7 (Core)' #設置默認啟動的內核
grub2-editenv list # 查看內核修改結果
yum remove kernel-3.10.0-1160.76.1.el7.x86_64 # 刪除指定的無用內核
sudo awk -F\' /^menuentry/{print\$2} /etc/grub2.cfg #列出目前安裝的內核
-------------------------------------------------------------
CentOS Linux (3.10.0-1160.80.1.el7.x86_64) 7 (Core)
CentOS Linux (3.10.0-1160.71.1.el7.x86_64) 7 (Core)
CentOS Linux (3.10.0-1062.12.1.rt56.1042.el7.x86_64) 7 (Core)
CentOS Linux (0-rescue-98d012eba8c7443aa602c226f7d2baed) 7 (Core)
---------------------------------------------------------------
sudo grub2-set-default 2 #根據上面列出的內核，它們從上到下編號。初始編號為0
sudo cat /boot/grub2/grubenv |grep saved #檢查設置是否配置
sudo grub2-mkconfig -o /boot/grub2/grub.cfg # 重建 GRUB2
```
* My build log
* https://1drv.ms/t/s!AtLp1AqbBT3_hQdhcQ4sVK9dkQ7q?e=V8Z7Za
### Install other packages
1. Install NFV packages
packager source : https://cbs.centos.org/koji/buildinfo?buildID=27696
qemu-kvm-tools-ev
```gherkin=
sudo wget http://ftp.ksu.edu.tw/FTP/CentOS/7/virt/x86_64/kvm-common/Packages/q/qemu-kvm-tools-ev-2.12.0-33.1.el7_7.4.x86_64.rpm
sudo rpm -ivh qemu-kvm-tools-ev-2.12.0-33.1.el7_7.4.x86_64.rpm
rpm -q qemu-kvm-tools-ev
```
* 
tuned-profiles-nfv-host
```gherkin=
sudo wget http://mirror.centos.org/centos/7/rt/x86_64/Packages/tuned-profiles-nfv-host-2.9.0-1.el7fdp.noarch.rpm
sudo rpm -ivh tuned-profiles-nfv-host-2.9.0-1.el7fdp.noarch.rpm
rpm -q tuned-profiles-nfv-host
```
* 
tuned-profiles-nfv-guest
```gherkin=
sudo wget http://mirror.centos.org/centos/7/rt/x86_64/Packages/tuned-profiles-nfv-guest-2.9.0-1.el7fdp.noarch.rpm
sudo rpm -ivh tuned-profiles-nfv-guest-2.9.0-1.el7fdp.noarch.rpm
rpm -q tuned-profiles-nfv-guest
```
* 
tuned-profiles-nfv
```gherkin=
sudo wget http://mirror.centos.org/centos/7/rt/x86_64/Packages/tuned-profiles-nfv-2.9.0-1.el7fdp.noarch.rpm
sudo rpm -ivh tuned-profiles-nfv-2.9.0-1.el7fdp.noarch.rpm
rpm -q tuned-profiles-nfv
```
* 
2. Install other RT packages
* Packages version Must be installed same as kernel-rt (3.10.0-1062.12.1.rt56.1042.el7.x86_64)
kernel-rt-devel
```gherkin=
sudo wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/kernel-rt-devel-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
sudo rpm -ivh kernel-rt-devel-3.10.0-1062.12.1.rt56.1042.el7.x86_64.rpm
rpm -q kernel-rt-devel
```
* 
rtctl
```gherkin=
sudo wget https://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/rtctl-1.13-2.el7.noarch.rpm
sudo rpm -ivh rtctl-1.13-2.el7.noarch.rpm
rpm -q rtctl
```
* 
* My build log
```gherkin=
```
### Tune the RT kernel performance
1. Edit /etc/tuned/realtime-virtual-host-variables.conf to add isolated_cores:
* Below give one example to tune the RT kernel performance for Intel Cascade Lake CPU 6248 Edit /etc/tuned/realtime-virtual-host-variables.conf to add isolated_cores=1-19:
```gherkin=
sudo nano /etc/tuned/realtime-virtual-host-variables.conf
# Examples: Here's the example of setting isolated_cores while having two CPUs and enable HT
# isolated_cores=1- <Equation 1*Equation 2 -1>
#
isolated_cores=1-19
```
2. Activate Real Time Profile
```gherkin=
tuned-adm profile realtime-virtual-host
```
3. Edit /etc/default/grub
* Append the following to the GRUB_CMDLINE_LINUX
```gherkin=
sudo nano /etc/default/grub
```
將下列文字內容複製到文件內
```gherkin=
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_CMDLINE_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_DEFAULT:+$GRUB_CMDLINE_LINUX_DEFAULT }\$tuned_params"
GRUB_INITRD_OVERLAY="${GRUB_INITRD_OVERLAY:+$GRUB_INITRD_OVERLAY }\$tuned_initrd"
GRUB_CMDLINE_LINUX="${GRUB_CMDLINE_LINUX_DEFAULT:+$GRUB_CMDLINE_LINUX_DEFAULT }\$tuned_params"
GRUB_INITRD_OVERLAY="${GRUB_INITRD_OVERLAY:+$GRUB_INITRD_OVERLAY }\$tuned_initrd"
```
4. Update the grub
* Apply the kernel-rt profile that we just modified to the grub
```gherkin=
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
```
* If you are using a UEFI based system
```gherkin=
sudo grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg
```
5. Reboot the server
```gherkin=
sudo reboot
```
6. Check the kernel parameter
* 來自Intel Flrexran的內核參數範例 https://github.com/intel/FlexRAN#os-configuration
```gherkin=
cat /proc/cmdline
```
* 
7. Set CPU Frequency Policy to Performance.
```gherkin=
sudo cpupower frequency-set -g performance
```
* 
8. Set CPU frequency using msr-tools
```gherkin=
git clone https://github.com/intel/msr-tools/
cd msr-tools/
git checkout msr-tools-1.3
make
cat <<EOF > turbo-2.5G.sh
#!/bin/sh
for i in {0..39}
do
#Set core 0-39 to 2.5GHz (0x1900). Please change according to your CPU model
./wrmsr -p \${i} 0x199 0x1900
done
#Set Uncore to Max
./wrmsr -p 0 0x620 0x1e1e
./wrmsr -p 39 0x620 0x1e1e
EOF
chmod 755 turbo-2.5G.sh
sh turbo-2.5G.sh
```
10. RT Test and Verify the test result
```gherkin=
sudo cyclictest -m -p95 -d0 -a 0-15 -t 16
```
# Build O-DO low
## Prerequisite
### Download odu/l2
```gherkin=
git clone -b f-release "https://gerrit.o-ran-sc.org/r/o-du/l2"
```
### Download odu/phy
```gherkin=
git clone "https://gerrit.o-ran-sc.org/r/o-du/phy"
```
### Download l1app
1. Download the Git-LFS
```gherkin=
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.rpm.sh | sudo bash
```
2. Install Git-LFS
```gherkin=
sudo yum install git-lfs -y
```
3. Set up Git LFS for your user account by running:
```gherkin=
git lfs install
```
* 
4. Download l1app
```gherkin=
git clone https://github.com/intel/FlexRAN.git
```
## O-DU Low Installation
### Download and Install oneAPI
1. Download and Install oneAPI
```gherkin=
#Download the oneAPI installer
wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18673/l_BaseKit_p_2022.2.0.262_offline.sh
#Install oneAPI
sudo sh ./l_BaseKit_p_2022.2.0.262_offline.sh
```
#### One Api 安裝步驟
1. 下一步
2. Recommended install 是預設安裝全部套件
3. custom install 是選擇要安裝什麼套件
4. 下一步
5. 選擇 skip Eclipse IDE
6. 下一步
7. Setup ICX(oneAPI) environment variable
```gherkin=
nano ~/.bashrc
# 將下列命令寫在文件變數中
source /opt/intel/oneapi/setvars.sh
export PATH=$PATH:/opt/intel/oneapi/compiler/2022.1.0/linux/bin
```
8. Check the version of ICX
```gherkin=
icx --version
```
### Install DPDK 20.11.3
1. Install python3 & pip3 (If you have installed them, you can skip this step)
```gherkin=
sudo yum install epel-release python36 python3-pip -y
```
2. Install meson & ninja (If you have installed them, you can skip this step)
```gherkin=
sudo pip3 install meson
sudo yum -y install ninja-build
```
3. Download DPDK
```gherkin=
wget http://static.dpdk.org/rel/dpdk-20.11.3.tar.xz
tar -xf dpdk-20.11.3.tar.xz
```
4. Set up the environment variable
```gherkin=
nano ~/.bashrc
export RTE_TARGET=x86_64-native-linuxapp-icx
export RTE_SDK=/home/osc/dpdk-stable-20.11.3
export WIRELESS_SDK_TOOLCHAIN=icx
export SDK_BUILD=build-avx512-icc
export PATH=$PATH:/opt/intel/oneapi/compiler/2022.1.0/linux/bin-llvm/
```
5. Modify the meson_options.txt
```gherkin=
cd dpdk-stable-20.11.3
nano meson_options.txt
```
:::success
In meson_options.txt
Before:
```gherkin=
option('machine', type: 'string', value: **'native'**,
description: 'set the target machine type')
```
After:
```gherkin=
option('machine', type: 'string', value: **'default'**,
description: 'set the target machine type')
```
:::
6. Edit Patch
* Edit drivers/net/i40e/i40e_ethdev.c
```gherkin=
none dpdk-stable-20.11.3/drivers/net/i40e/i40e_ethdev.c
```
* Change i40e_vsi_queues_bind_intr(main_vsi, I40E_ITR_INDEX_DEFAULT);
* into i40e_vsi_queues_bind_intr(main_vsi, I40E_ITR_INDEX_NONE)
* Add some lines in about line 2485
```gherkin=
i40e_aq_debug_write_global_register(hw,
0x0012A504,
0, NULL);
```
* Edit drivers/net/i40e/i40e_ethdev_vf.c
```gherkin=
none dpdk-stable-20.11.3/drivers/net/i40e/i40e_ethdev_vf.c
```
* Change map_info->vecmap[0].rxitr_idx = I40E_ITR_INDEX_DEFAULT;
* into map_info->vecmap[0].rxitr_idx = I40E_ITR_INDEX_NONE
* Edit drivers/net/ixgbe/ixgbe_ethdev.c
```gherkin=
nano dpdk-stable-20.11.3/drivers/net/ixgbe/ixgbe_ethdev.c
```
* Change hw->mac.type != ixgbe_mac_82598EB
* into hw->mac.type != ixgbe_mac_82598EB && hw->mac.type != ixgbe_mac_82599EB
7. Build DPDK
```gherkin=
meson build
cd build
ninja
ninja install
```
8. My build steps log
* https://1drv.ms/t/s!AtLp1AqbBT3_hQS85fC4Ssx99ge6?e=hM4Yhs
### Install Google Test
#### Background
:::info
* Google Test is a unit testing library for the C++ programming language, based on the xUnit architecture.
* Used by O-DU Low
:::
1. Download google test
https://github.com/google/googletest/releases/tag/release-1.7.0
```gherkin=
cd ~
wget https://github.com/google/googletest/archive/release-1.7.0.tar.gz
tar -xvf release-1.7.0.tar.gz
# change folder name
mv googletest-release-1.7.0 gtest-1.7.0
```
2. Build and installation
```gherkin=
#Install GCC from repository
sudo yum -y update
sudo yum -y install gcc
gcc --version #check version
sudo yum -y install gcc-c++
#modify variable
export GTEST_DIR=~/gtest-1.7.0
cd ${GTEST_DIR}
g++ -isystem ${GTEST_DIR}/include -I${GTEST_DIR} -pthread -c ${GTEST_DIR}/src/gtest-all.cc
```
* build libgtest.a
```gherkin=
ar -rv libgtest.a gtest-all.o
```
* cmake google-test & cmake is used to install google-test
```gherkin=
#install cmake 3.15.5
wget https://github.com/Kitware/CMake/releases/download/v3.15.5/cmake-3.15.5.tar.gz
tar -zxf cmake-3.15.5.tar.gz
cd cmake-3.15.5
./bootstrap --prefix=/usr --datadir=share/cmake --docdir=doc/cmake && make
sudo make install
cmake --version
#cmake is used to install google-test
cd ${GTEST_DIR}/build-aux
cmake ${GTEST_DIR}
```
```gherkin=
make
```
* build linked file
```gherkin=
cd ${GTEST_DIR}
ln -s build-aux/libgtest_main.a libgtest_main.a
```
#### Set the google test Path
* Add gtest path to bashrc so that linux can recognize installed google test
```gherkin=
sudo nano ~/.bashrc
export DIR_ROOT_GTEST=~/gtest-1.7.0
export GTEST_ROOT=/home/osc/gtest-1.7.0
# Apply it
source ~/.bashrc
```
### Set up the environment
:::info
Remember to modify the path according to your own directory, this environtment is used in all O-DU-PHY installation.
:::
1. Set the environment variable
```gherkin=
sudo nano phy/setupenv.sh
source phy/setupenv.sh
```
2. Append the command into .bashrc file
Whenever you start a session, it will run the all commands in .bashrc file automatically.
```gherkin=
# Edit the .bashrc file
nano ~/.bashrc
# Append the command below into the file
source phy/setupenv.sh
```
3. My build steps log
* https://1drv.ms/t/s!AtLp1AqbBT3_hQZacQjipwD7S5De?e=ypbnmf
### Install xRAN Library (FHI Library)
:::warning
環境路徑設定
* 記得根據自己的目錄修改路徑，所有O-DU-PHY安裝都會用到這個環境。
* 確保在 shell 文件中使用絕對路徑
* 確保所有變量都在你的setupenv.sh上（下面有一個例子）
**- Important parameter to double check**
| Parameter | Filled with | Example |
| ----------------------- | ------------------------------------ | ----------------------------- |
| DIR_ROOT_L1_BIN | your FlexRAN folder path | /home/osc/FlexRAN |
| DIR_ROOT_PHY | your phy folder path | /home/osc/phy |
| DIR_ROOT_DPDK | your dpdk installation folder path | /home/osc/dpdk-stable-20.11.3 |
| DIR_ROOT_GTEST | your google test installation folder | /home/osc/gtest-1.7.0 |
| WIRELESS_SDK_TARGET_ISA | avx512 | avx512 |
:::
設置環境變量
```gherkin=
## Edit the file and change the environtment variable to your own directory
sudo vim <Your_Directory>/phy/setupenv.sh
# Run the script
$ source <Your_Directory>/phy/setupenv.sh
```
將命令附加到.bashrc文件中
```gherkin=
nano vim ~/.bashrc
source <Your_Directory>/phy/setupenv.sh
```
install FHI Library
```gherkin=
# Because the `build.sh` run the command "pkgconf", but in CentOS, this application is called "pkg-config". Hence, we need to link them together.
sudo ln -s /usr/bin/pkg-config /usr/bin/pkgconf
#cannot execute
#cd $XRAN_DIR
#./build.sh xclean
# Build the FHI library
cd $XRAN_DIR/lib
make
# Compile sample-app.c
cd $XRAN_DIR/app
make
sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm -y
sudo yum install octave -y
sudo yum install -y expect
# Generate IQ Samples of Sample app
cd $XRAN_DIR/app
octave gen_test.m
```
* My build steps log
> https://1drv.ms/t/s!AtLp1AqbBT3_hQVVXzJOenxNPd9I?e=FdnlFw
* My IQ data log
> https://1drv.ms/t/s!AtLp1AqbBT3_hQPI_22e2p5xbsVo?e=SI4eIU
### Install WLS Library
#### Background
* Wls_lib 是一個無線服務庫，它支持實現 gNb 或 eNb 的應用程序使用的共享內存和緩衝區管理。
* 該庫使用 DPDK、libhugetlbfs 和 pthreads 通過從 DPDK 的角度共享相同的內存區域，在 L2 應用程序、API 轉換器模塊和 L1 應用程序之間提供更少的 memcpy 數據交換。
```gherkin=
# Install the essential package
sudo yum install libmlx5
# Change directory to wls_lib
cd phy/wls_lib
# Build WLS Library
./build.sh xclean #unable to execute
./build.sh #"Please define valid WIRELESS_SDK_TARGET_ISA environment variable ". Stop.
# Copy library to wls_lib
cd ../fapi_5g/include/
cp fapi_interface.h ../../wls_lib/
cp fapi_vendor_extension.h ../../wls_lib/
```
### FAPI 5G Install
#### Background
:::info
* ORAN FAPI 5G 轉換器模塊 (FAPI TM) 是一個獨立的應用程序，它使用 Small Cell Forum 定義的 5G FAPI 協議與 ODU-High 通信，並使用無線子系統接口使用 Intel L2-L1 API 與 ODU Low 通信用於處理接口所需的共享內存和緩衝區管理的庫 (WLS)。
* ORAN 5G FAPI TM 需要與 WLS 庫集成的數據平面設計套件 (DPDK)。
:::
#### Building and Installation FAPI
* Make sure RTE_SDK & DIR_WIRELESS_WLS variables are already set
#### Building & Installation
```gherkin=
# Build fapi_5g
cd <Your_DIR>/phy/fapi_5g/build
./build.sh xclean
./build.sh
```
:::warning
ISSUE:
* Error message nr5g_mac_phy_api.h file not found
* There is no nr5g_mac_phy_api.h in the folder
需修改makefile，待補
:::
### L1app & Testmac Testing (FAPI TM Testing)
```gherkin
# Run L1.sh on one console/ssh session
cd FlexRAN/l1/bin/nr5g/gnb/l1
# RUN L1.sh
sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./l1.sh -e
# Run FAPI on another console/ssh session
cd phy/fapi_5g/bin
# Run FAPI
sudo -E LD_LIBRARY_PATH=$LD_LIBRARY_PATH ./oran_5g_fapi --cfg=oran_5g_fapi.cfg
```
:::warning
:::
###### tags: `NTUST O-RAN`
Sign in with Wallet
Connect another wallet