# Setup OAI 5G-NR SA with free5GC for Ubuntu 18.04

###### tags: `ORAN` `OAI`

:::info
[Update] 2022.04.17
* Add support for free5gc v3.1.0

[Update] 2022.02.26
* Fix steps of OAI 5G installtion for branch selection
* Add solution for failure of PDU Session Setup

[Update] 2022.02.22
* Change target branch from F1C_extensions_develop to develop
:::

## Environment

* OS: Ubuntu 18.04
* linux kernel version >= 5.4
* CPU: At least 2
* RAM: 16GB
* 5GC: free5gc stage 3 (v3.0.5/v3.1.0) in K8s
* gNB & UE: OAI with branch develop (short commitID: 454aae1d96)

## free5GC Installation

:::warning
In our lab, we choose to use K8s to deploy free5GC.
Actually, you can also follow [offical website](https://www.free5gc.org/installations/stage-3/) to setup free5GC without K8s.
:::

1. Install prerequisites

sudo apt-get install -y docker.io curl git
## update package repository
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
## install kubeadm, kubectl and kubelet in specific version
sudo apt-get install -y kubelet=1.19.0-00 kubeadm=1.19.0-00 kubectl=1.19.0-00
## mark these packages not need to be updated
sudo apt-mark hold kubelet kubectl kubeadm
## add kubectl auto-completion
echo "source <(kubectl completion bash)" >> ~/.bashrc
## This command is optional
## If you want to use docker without sudo
sudo usermod -aG docker <username>

2. Write configuration (named in cluster-config.yaml)

apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: v1.19.0
apiServer:
  extraArgs:
    feature-gates: "SCTPSupport=true"

3. Disable swap

## Disable swap in order for the kubelet to work properly
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

4. Deploy K8s Cluster

sudo kubeadm init --config cluster-config.yaml
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl taint nodes --all node-role.kubernetes.io/master-

5. Install Calico as CNI of this cluster

kubectl apply -f https://docs.projectcalico.org/v3.11/manifests/calico.yaml

6. Install Network File System (NFS) Server
* NFS serves as backend store for MongoDB

sudo apt install -y nfs-kernel-server
sudo mkdir -p /var/share/free5gc
echo "/var/share/free5gc *(rw,sync,no_root_squash)" | sudo tee -a /etc/exports
sudo exportfs -r
## check if directory is exported correctly
sudo showmount -e


7. Install gtp5g Kernel Module

sudo apt install -y build-essential
## for free5gc v3.0.5
git clone -b v0.2.1 https://github.com/free5gc/gtp5g.git
## for free5gc v3.1.0
git clone -b v0.5.3 https://github.com/free5gc/gtp5g.git
cd gtp5g
make
sudo make install
## check if gtp5g is loaded
lsmod | grep gtp5g


8. Download prepared repository

## for free5gc v3.0.5
git clone https://github.com/chih-hsi-chen/free5GC-stage3-on-k8s.git
## for free5gc v3.1.0
git clone -b v3.1.0 https://github.com/chih-hsi-chen/free5GC-stage3-on-k8s.git
## you will see following directory structure
## ├── Dockerfile // Pod Docker Image
## └── free5gc-nf-yml
##     ├── amf
##     ├── ausf
##     ├── mongo
##     ├── nrf
##     ├── nssf
##     ├── pcf
##     ├── smf
##     ├── udm
##     ├── udr
##     ├── upf
##     └── webui

9. Deploy pods in cluster

cd ~/free5GC-stage3-on-k8s
## for free5gc v3.0.5
sudo docker build -t free5gc-build:v3.0.5 .
## for free5gc v3.1.0
sudo docker build -t free5gc-build:v3.1.0 .
## order: mongo -> webui -> NRF -> NFs other than SMF -> SMF
cd free5gc-nf-yml/
kubectl apply -f mongo
kubectl apply -f webui
kubectl apply -f nrf
kubectl apply -f pcf
kubectl apply -f udm
kubectl apply -f udr
kubectl apply -f upf
kubectl apply -f amf
kubectl apply -f ausf
kubectl apply -f nssf
kubectl apply -f smf
## check if all pods are running
kubectl get pod -o wide

:::warning
If you encounter error during building docker image, like no connection to ubuntu package server, then you need to modify your DNS setting.

:::

10. Create subscriber (UE)
* Go to localhost:30300, username is admin and password is free5gc
* Add new subscriber (default one)
* Scroll down until you see submit button and just click it

## OAI 5G Installation
1. download and build

git clone https://gitlab.eurecom.fr/oai/openairinterface5g.git
cd openairinterface5g
git checkout develop
source oaienv
cd cmake_targets/
### Install dependencies packages for simulation mode
./build_oai -I -w SIMU
### Build gNB and nr-UE executable program for simulation mode
./build_oai --gNB --nrUE -w SIMU

2. set loopback ip address

sudo ip addr add 127.0.0.2 dev lo

## Setup configuration file for gNB and UE

### gNB (CU side)
* File location: openairinterface5g/targets/PROJECTS/GENERIC-NR-5GC/CONF/cu_gnb.conf

// Tracking area code, 0x0000 and 0xfffe are reserved values
    tracking_area_code  =  1;
    plmn_list = ({
			 mcc = 208;
			 mnc = 93;
			 mnc_length = 2;
			 snssaiList = (
			 {
				sst = 1;
				sd  = 0x010203; // 0 false, else true
			 },
			 {
				 sst = 1;
				 sd  = 0x112233; // 0 false, else true
			 }
			);
		});
    
    nr_cellid = 12345678L
    
    tr_s_preference = "f1";
    
    local_s_if_name = "lo";
    local_s_address = "127.0.0.1";
    remote_s_address = "127.0.0.2";
    local_s_portc   = 600;
    local_s_portd   = 2152;
    remote_s_portc  = 601;
    remote_s_portd  = 2152;
...
////////// AMF parameters:
    amf_ip_address      = ( { ipv4       = "192.168.2.2"; // AMF IP address
                              ipv6       = "192:168:30::17"; // not important
                              active     = "yes";
                              preference = "ipv4";
                            }
                          );


    NETWORK_INTERFACES :
    {
        GNB_INTERFACE_NAME_FOR_NG_AMF            = "enp0s3"; // any interface accessible to AMF
        GNB_IPV4_ADDRESS_FOR_NG_AMF              = "10.0.2.15/24"; // IP of above interface
        GNB_INTERFACE_NAME_FOR_NGU               = "enp0s3"; // any interface accessible to UPF
        GNB_IPV4_ADDRESS_FOR_NGU                 = "10.0.2.15/24"; // IP of above interface
        GNB_PORT_FOR_S1U                         = 2152; # Spec 2152
    };

:::warning
**Issue: Assertion failure in CU for non-existing band**
If you encounter the following assertion failure, you need to copy configuration of servingCellConfigCommon in DU to CU.
:::

### gNB (DU side)
* File location: openairinterface5g/targets/PROJECTS/GENERIC-NR-5GC/CONF/du_gnb.conf

// Tracking area code, 0x0000 and 0xfffe are reserved values
    tracking_area_code  =  1;
    plmn_list = ({
			 mcc = 208;
			 mnc = 93;
			 mnc_length = 2;
			 snssaiList = (
			 {
				sst = 1;
				sd  = 0x010203; // 0 false, else true
			 },
			 {
				sst = 1;
				sd  = 0x112233; // 0 false, else true
			 }
			);
		});
...
////////// Physical parameters:

    ssb_SubcarrierOffset  = 0;
    pdsch_AntennaPorts    = 1;
    pusch_AntennaPorts    = 1;
    min_rxtxtime          = 6;
    sib1_tda              = 1;
...
MACRLCs = (
  {
    num_cc           = 1;
    tr_s_preference  = "local_L1";
    tr_n_preference  = "f1";
    local_n_if_name = "lo";
    local_n_address = "127.0.0.2";
    remote_n_address = "127.0.0.1";
    local_n_portc   = 601;
    local_n_portd   = 2152;
    remote_n_portc  = 600;
    remote_n_portd  = 2152;

  }
...
# If you add this setting, you don't need to add RFSIMULATOR parameter during running CU
rfsimulator: {
    serveraddr = "server";
    serverport = "4043";
    options = (); #("saviq"); or/and "chanmod"
    modelname = "AWGN";
    IQfile = "/tmp/rfsimulator.iqs"
}

### UE
* File location: openairinterface5g/targets/PROJECTS/GENERIC-NR-5GC/CONF/ue.conf

uicc0 = {
imsi = "208930000000003";
key = "8baf473f2f8fd09487cccbd7097c6862";
opc= "8e27b6af0e692e750f32667a3b14605d";
dnn= "internet";
nssai_sst=1;
nssai_sd=0x010203;
}

## Execution (in order)

* All executable file is located at: openairinterface5g/cmake_targets/ran_build/build

### CU

cd openairinterface5g/cmake_targets/ran_build/build
## If you have set rfsimulator in DU configuration
sudo ./nr-softmodem --rfsim --sa -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/cu_gnb.conf
## Else
sudo RFSIMULATOR=server ./nr-softmodem --rfsim --sa -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/cu_gnb.conf

### DU

cd openairinterface5g/cmake_targets/ran_build/build
## If you have set rfsimulator in DU configuration
sudo ./nr-softmodem --rfsim --sa -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/du_gnb.conf
## Else
sudo RFSIMULATOR=server ./nr-softmodem --rfsim --sa -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/du_gnb.conf

### UE

cd openairinterface5g/cmake_targets/ran_build/build
sudo RFSIMULATOR=127.0.0.1 ./nr-uesoftmodem -r 106 --numerology 1 --band 78 -C 3619200000 --rfsim --sa --nokrnmod -O ../../../targets/PROJECTS/GENERIC-NR-5GC/CONF/ue.conf

## Test if the environment is successfully built
* ping the public network with oai interface, it should work

:::warning
**Issue 1: Assertion failure in CU for HARQ feedback time**
If you encounter the following assertion failure, you need to set the value of min_rxtxtime (at least 5) in DU.
Here is an example:
:::

:::warning
**Issue 2: UE cannot complete PDU Session Setup**
You need to check if gtp5g device in UPF is created well
* Fail
* Success

**Solution**
1. Terminate UPF (or whole free5GC)
2. re-install gtp5g module
3. Start UPF (or whole free5GC)
:::