[OAI gNB] Install support FHI7.2

:::warning # <center> [OAI gNB] Install support FHI7.2 </center> ::: :::success **🎯 Goals:** ::: :::info ⚓ **Installation environment:** - Branch: `use_msgq` - OS: Ubuntu 18.04.6 LTS (x86_64) - Stroge: 1TB - Memory: ::: :::info :bookmark: **Reference:** - [\[OAI gNB\] Installation Guide with E2 Interface](https://hackmd.io/@Yueh-Huan/BJBs4AFzs#tags-OAI "tags-OAI") - [**ORAN\_FHI7.2\_Tutorial**](https://gitlab.eurecom.fr/oai/openairinterface5g/-/blob/use_msgq/doc/ORAN_FHI7.2_Tutorial.md) - [Install the DPDK and Browse Sources](https://doc.dpdk.org/guides-18.08/linux_gsg/build_dpdk.html#:~:text=To%20install%20and%20make%20targets%2C%20use%20the%20make,the%20make%20command%20should%20be%3A%20make%20install%20T%3Di686-native-linuxapp-gcc) - [O-RAN FH Lib Introduction](https://docs.o-ran-sc.org/projects/o-ran-sc-o-du-phy/en/latest/Introduction_fh.html) ::: [toc] # Setup Environment ## 1.1 Download DPDK version 20.05.0 ``` wget http://fast.dpdk.org/rel/dpdk-20.05.tar.xz ``` ## 1.2 DPDK Compilation ``` tar -xvf dpdk-20.05.tar.xz cd dpdk-20.05 meson build cd build sudo ninja ``` :::success - Wait for about 3 minutes, and it will pause briefly during the second-to-last step. ::: ```shell sudo ninja install ``` ```shell cd .. make install T=x86_64-native-linuxapp-gcc ``` > To install and make targets, use the `make install T=<target>` command in the top-level DPDK directory. :::success - Wait for 10 min. and it will print a lot of LOGs. ::: ## 1.3 Ubuntu Install real timer kernel followed by updating boot arguments ```shell export isolcpus=0-2,8-17 nohz=on nohz_full=0-2,8-17 rcu_nocbs=0-2,8-17 rcu_nocb_poll nosoftlockup default_hugepagesz=1GB hugepagesz=1G hugepages=10 amd_iommu=on iommu=pt ## sudo su ## echo "isolcpus=0-2,8-17 nohz=on nohz_full=0-2,8-17 rcu_nocbs=0-2,8-17 rcu_nocb_poll nosoftlockup default_hugepagesz=1GB hugepagesz=1G hugepages=10 amd_iommu=on iommu=pt" > /proc/sys/kernel/cmdline ``` Isolated CPU 0-2 are used for DPDK/ORAN and CPU 8 for `ru_thread` in our example config ## 1.4 PTP configuration > Precision Time Protocol(PTP) You can refer to the [following o-ran link](https://docs.o-ran-sc.org/projects/o-ran-sc-o-du-phy/en/latest/PTP-configuration_fh.html) for PTP configuration. In our setup we used Fibrolan Falcon-RX for PTP grandmaster. ```shell git clone http://git.code.sf.net/p/linuxptp/code linuxptp cd linuxptp/ git checkout v2.0 make && make install ## ./ptp4l -i eno1 -m -H -2 -s -f configs/default.cfg -vvv ./ptp4l -i eno1 -m -H -2 -s -f configs/default.cfg ./phc2sys -w -m -s eno1 -R 8 -f configs/default.cfg ``` > eno1 is your <if_name> :::danger - For the second-to-last command. ```shell root@oaignb:/home/oaignb/ming/linuxptp# ./ptp4l -i eno1 -m -H -2 -s -f configs/default.cfg ptp4l[1234834.117]: selected /dev/ptp0 as PTP clock ptp4l[1234834.160]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE ptp4l[1234834.160]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE ptp4l[1234840.716]: selected local clock 94c691.fffe.a22a2c as best master ptp4l[1234847.652]: selected local clock 94c691.fffe.a22a2c as best master ``` - For the last command. ```shell root@oaignb:/home/oaignb/ming/linuxptp# ./phc2sys -w -m -s eno1 -R 8 -f configs/default.cfg phc2sys[1234936.443]: uds: sendto failed: No such file or directory phc2sys[1234937.444]: Waiting for ptp4l... phc2sys[1234937.444]: uds: sendto failed: No such file or directory phc2sys[1234938.445]: Waiting for ptp4l... phc2sys[1234938.445]: uds: sendto failed: No such file or directory phc2sys[1234939.446]: Waiting for ptp4l... phc2sys[1234939.446]: uds: sendto failed: No such file or directory phc2sys[1234940.448]: Waiting for ptp4l... phc2sys[1234940.448]: uds: sendto failed: No such file or directory ^Cphc2sys[1234941.016]: poll failed ``` > If I don't close the second-to-last command, I won't see the "No such file or directory" error. ::: # 2. Build OAI-FHI gNB ![](https://hackmd.io/_uploads/SJ5ySRqo2.png) ## 2.1 Install ICX :::info - [Install Intel C/C++ Compiler (ICX) 2022.1.0](https://hackmd.io/@nadhifmr/S15d9vkUi#tags-ORAN-ICC "tags-ORAN-ICC") ::: **OneAPI 2022.1.0** - Needed to use ICX instead of ICC - ICX is Intel nextgen compiler based on C/LLVM technology plus Intel proprietary optimizations and code generation ### 2.1.1 Download and Install oneAPI ``` ## Download the oneAPI installer wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18673/l_BaseKit_p_2022.2.0.262_offline.sh ## Install oneAPI sudo sh ./l_BaseKit_p_2022.2.0.262_offline.sh ## Follow the GUI to finish the installation ``` :::spoiler Finish screen ![](https://hackmd.io/_uploads/ByBSY39i2.png) ::: ### 2.1.2 Setup ICX(oneAPI) environment variable - Edit bashrc to add oneAPI installation files to PATH - PATH is used in Linux to tell where to search for executable files ```shell vim ~/.bashrc ## Append the commands below into .bashrc file source /opt/intel/oneapi/setvars.sh export PATH=$PATH:/opt/intel/oneapi/compiler/2022.1.0/linux/bin ``` > You should restart a session or open a new terminal after you set up the .bashrc file [color=pink] ### 2.1.3 Check the version of ICX ```shell icx --version ``` - After restart a session LOGs: ```shell root@oaignb:/home/oaignb/ming# icx --version Intel(R) oneAPI DPC++/C++ Compiler 2022.1.0 (2022.1.0.20220316) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /opt/intel/oneapi/compiler/2022.1.0/linux/bin-llvm ``` ## 2.2 Build ORAN Fronthaul Interface Library ### 2.2.1 Download ORAN FHI library ```shell git clone https://gerrit.o-ran-sc.org/r/o-du/phy.git cd phy git checkout oran_release_bronze_v1.1 ``` ### 2.2.2 Apply patches (available in `oai_folder/cmake_targets/tools/oran_fhi_integration_patches/`) ```shell cd /oai/cmake_targets/tools/oran_fhi_integration_patches git apply oran-fhi-1-compile-libxran-using-gcc-and-disable-avx512.patch git apply oran-fhi-2-return-correct-slot_id.patch git apply oran-fhi-3-disable-pkt-validate-at-process_mbuf.patch git apply oran-fhi-4-process_all_rx_ring.patch git apply oran-fhi-5-remove-not-used-dependencies.patch ``` ### 2.2.3 Set up the environment (change the path if you use different folders) - executing this step, please ensure that your `/root` path should have the following folders: ```shell linuxptp phy dpdk-20.05 ``` - **Run following command:** - export XRAN_LIB_DIR= `path-to` /phy/fhi_lib/lib/build - export XRAN_DIR=`path-to`/phy/fhi_lib - export RTE_SDK=`path-to`/dpdk-20.05 - export RTE_TARGET=`path-to`x86_64-native-linuxapp-gcc - export RTE_INCLUDE=`${RTE_SDK}`/`${RTE_TARGET}`/include  > When we built DPDK, we created the directory named `x86_64-native-linuxapp-gcc`. - My example ```shell export XRAN_LIB_DIR=/home/oaignb/ming/phy/fhi_lib/lib/build export XRAN_DIR=/home/oaignb/ming/phy/fhi_lib export RTE_SDK=/home/oaignb/ming/dpdk-20.05 export RTE_TARGET=/home/oaignb/ming/dpdk-20.05/x86_64-native-linuxapp-gcc export RTE_INCLUDE=/home/oaignb/ming/dpdk-20.05/x86_64-native-linuxapp-gcc/include ``` > Runtime Environment (RTE): DPDK運行時的環境設定 > The green parts have been changed by me. ### 2.2.4 Modify the Makefile ```shell root@oaignb:/home/oaignb/ming/phy/fhi_lib# nano lib/Makefile ``` #### 1. Change `icc` into `icx` ```shell=23 ############################################################## # Tools configuration ############################################################## CC := icx CPP := icx AS := as AR := ar LD := icx OBJDUMP := objdump ``` > change tag: `CC`, `CPP`, `LD` #### 2. Add parameter (`-qnextgen-diag`) ```shell=173 $(CC_OBJTARGETS) : @echo [CC] $(subst $(PROJECT_OBJ_DIR)/,,$@) @$(CC) -c $(CC_FLAGS_FULL) -o"$@" $(patsubst %.o,%.c,$(subst $(PROJECT_OBJ_DIR)/,,$@)) -qnextgen-diag $(CPP_OBJTARGETS) : @echo [CPP] $(subst $(PROJECT_OBJ_DIR)/,,$@) @$(CPP) -c $(CPP_FLAGS_FULL) -o"$@" $(patsubst %.o,%.cpp,$(subst $(PROJECT_OBJ_DIR)/,,$@)) -qnextgen-diag ``` > Add the parameter(`-qnextgen-diag`) at the end of line 175 and 179. ### 2.2.5 Compile Fronthaul Interface Library ```shell cd phy/fhi_lib/ ./build.sh ## ./build.sh xclean ``` :::danger ```shell ... -wr=<string> -ww=<string> [AR] build/libxran.a ar: build/obj//home/oaignb/ming/phy/fhi_lib/lib/ethernet/ethdi.o: No such file or directory Makefile:215: recipe for target 'build/libxran.a' failed make: *** [build/libxran.a] Error 1 "make $COMMAND_LINE MLOG=${MLOG} LIBXRANSO=${LIBXRANSO}" command exited with code 2. ``` > This is the latest encountered error (currently without a solution). ::: :::danger ```shell root@oaignb:/home/oaignb/ming/phy/fhi_lib# ./build.sh Number of commandline arguments: 0 MLOG folder is not set. Disable MLOG (MLOG_DIR=) Building xRAN Library LIBXRANSO = 0 MLOG = 0 ============================================================================================ Building ./build/libxran.a RTE_TARGET = /home/oaignb/ming/dpdk-20.05/x86_64-native-linuxapp-gcc ============================================================================================ [BUILD] lib : libxran [CC] /home/oaignb/ming/phy/fhi_lib/lib/ethernet/ethdi.o /bin/sh: 1: icc: not found Makefile:174: recipe for target 'build/obj//home/oaignb/ming/phy/fhi_lib/lib/ethernet/ethdi.o' failed make: *** [build/obj//home/oaignb/ming/phy/fhi_lib/lib/ethernet/ethdi.o] Error 127 "make $COMMAND_LINE MLOG=${MLOG} LIBXRANSO=${LIBXRANSO}" command exited with code 2. ``` > This is the error message before modifying the Makefile. ::: # 3. OAI gNB ## 3.1 Download Source Code ```shell git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git oai -b use_msgq ``` ## 3.2 Install Dependency ```shell ## sudo apt install intel-icc (still can't install) cd ./oai/cmake_targets ## Trying to test what is the correct command ./build_oai --gNB --ninja -t oran_fhlib_5g -I ./build_oai --eNB --gNB -c -C -w USRP --ninja ## Build Clean ## ./build_oai --clean ``` > Add, `-I` if you are building for the first time on server for installing external dependencies :::danger ```cmd root@oaignb:/home/oaignb/ming/oai/cmake_targets# ./build_oai --gNB --ninja -t oran_fhlib_5g -I websrv optional build not included in build-lib option as dependencies requirements not met Will compile gNB TP set to oran_fhlib_5g Will install external packages No radio head has been selected (HW set to None) RF HW set to None 2. Setting the OAI PATHS ... OPENAIR_DIR = /home/oaignb/ming/oai FreeDiameter prefix not found, install freeDiameter if EPC, HSS Installing packages Hit:1 http://tw.archive.ubuntu.com/ubuntu bionic InRelease Get:2 http://tw.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] Get:4 http://tw.archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB] Get:5 http://tw.archive.ubuntu.com/ubuntu bionic-updates/main amd64 DEP-11 Metadata [296 kB] Get:6 http://tw.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 DEP-11 Metadata [303 kB] Get:7 http://tw.archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 DEP-11 Metadata [2468 B] Get:8 http://tw.archive.ubuntu.com/ubuntu bionic-backports/main amd64 DEP-11 Metadata [8144 B] Get:9 http://tw.archive.ubuntu.com/ubuntu bionic-backports/universe amd64 DEP-11 Metadata [10.0 kB] Ign:10 http://linux.mellanox.com/public/repo/mlnx_ofed/latest/ubuntu18.04 bionic InRelease Ign:3 https://content.mellanox.com/ofed/MLNX_OFED-5.4-2.5.8.0/ubuntu20.04 focal InRelease Hit:12 http://ppa.launchpad.net/ettusresearch/uhd/ubuntu bionic InRelease Err:13 http://linux.mellanox.com/public/repo/mlnx_ofed/latest/ubuntu18.04 bionic Release 404 Not Found [IP: 168.62.212.37 80] Err:11 https://content.mellanox.com/ofed/MLNX_OFED-5.4-2.5.8.0/ubuntu20.04 focal Release 404 Not Found [IP: 107.178.241.102 443] Get:14 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] Get:15 http://security.ubuntu.com/ubuntu bionic-security/main amd64 DEP-11 Metadata [76.8 kB] Get:16 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 DEP-11 Metadata [62.4 kB] Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 DEP-11 Metadata [2464 B] Reading package lists... Done E: The repository 'http://linux.mellanox.com/public/repo/mlnx_ofed/latest/ubuntu18.04 bionic Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. E: The repository 'http://www.mellanox.com/downloads/ofed/MLNX_OFED-5.4-2.5.8.0/ubuntu20.04 focal Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details. build have failed ``` ::: # Additional <style> .title { color: #009933; font-weight:bold; } .highlight { color: #ff4d4d; font-weight:bold; border-bottom:2px red solid; padding-bottom:2px; } </style>