---
# System prepended metadata

title: E2E Study Notes W1

---

{%hackmd ElOJTlJjTg-GQtxooVV5iw %}

# E2E Study Notes W1
> Johan ED Saputro


## GOALS
1. Bring DPDK features to PFCP
2. Bring DPDK features to gNB

## Milestone

![image](https://hackmd.io/_uploads/r1ccdAvv0.png)


### Environment preparation & Verification 

#### 5GC E2E Test

| Task    							| Status   |
| --------							| -------- |
| Install free5Gc & OAI             | <input type="checkbox" disabled="disabled" checked="checked"> |
| Try E2E between Core and RAn      | <input type="checkbox" disabled="disabled" checked="checked"> |

| Task     						 | Status   |
| -------- 						 | -------- |
| Test UE Sim                    | <input type="checkbox" disabled="disabled" checked="checked"> |
| Test using ping and iperf3     | <input type="checkbox" disabled="disabled" checked="checked"> |

#### DPDK

| Task     													 | Status   |
| -------- 													 | -------- |
| Install DPDK on UPF                                        | <input type="checkbox" disabled="disabled" checked="checked"> |
| Function Test of DPDK (ex: l2fwd, l3fwd, skeleton, ...)    | <input type="checkbox" disabled="disabled" checked="checked"> |


### Project Assignment
#### Lightweight 5GC 

| Task     			 | Status   |
| -------- 			 | -------- |
|     Study of NSSF  | <input type="checkbox" disabled="disabled"> |
|     Study of PCF   | <input type="checkbox" disabled="disabled"> |
|     Trace code based on the study in Free5G   | <input type="checkbox" disabled="disabled"> |
|     Think how to maintain the function of 5GC after NF removal  & Provide the proposal of improvement   | <input type="checkbox" disabled="disabled"> |
|     Implementation (Modification & Compilation & E2E test)    | <input type="checkbox" disabled="disabled"> |
|     Remove the NF & E2E test    | <input type="checkbox" disabled="disabled"> |
|     Test with multiple UEs   | <input type="checkbox" disabled="disabled"> |




### The Control Plane of DPDK 
#### The CP of DPDK
| Task     			 | Status   |
| -------- 			 | -------- |
| Study of DPDK in UP (CU-PDCP) | <input type="checkbox" disabled="disabled"> |
| Study of related works | <input type="checkbox" disabled="disabled"> |
| Study of CP protocol & DPDK control method | <input type="checkbox" disabled="disabled"> |
| Prepare environment (RIC &Xapp) | <input type="checkbox" disabled="disabled"> |
| Implementation of DPDK on CU-UP & DU-UP | <input type="checkbox" disabled="disabled"> |
| Proposal (Mobility solution) | <input type="checkbox" disabled="disabled"> |
| Proposal (QoS soluion.) | <input type="checkbox" disabled="disabled"> |
| Provide the architechture & call flow | <input type="checkbox" disabled="disabled"> |
| Implementation | <input type="checkbox" disabled="disabled"> |
| Test | <input type="checkbox" disabled="disabled"> |
| Final Report | <input type="checkbox" disabled="disabled"> |




## Installation
### Architecture
![5G E2E](https://hackmd.io/_uploads/H1u0_ikDC.png)

### Free5gc Installation

1. Install Golang
```
sudo rm -rf /usr/local/go
wget https://dl.google.com/go/go1.21.8.linux-amd64.tar.gz
sudo tar -C /usr/local -zxvf go1.21.8.linux-amd64.tar.gz
```

2. Install mongodb
```
sudo apt -y update
sudo apt -y install mongodb wget git
sudo systemctl start mongodb
```

3. Install Dependencies to build free5gc
```
sudo apt -y update
sudo apt -y install git gcc g++ cmake autoconf libtool pkg-config libmnl-dev libyaml-dev
```

4. Clone free5gc repository
```
cd ~
git clone --recursive -b v3.4.1 -j `nproc` https://github.com/free5gc/free5gc.git
cd free5gc
```

5. Build all NF
```
cd ~/free5gc
make
```

6. Install Web Console
	- Install Nodejs
	```
	curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - 
	sudo apt update
	sudo apt install -y nodejs
	corepack enable # setup yarn automatically
	```
	
	- Build webconsole
	```
	cd ~/free5gc
	make webconsole
	```

## UPF DPDK Installation
[Check my previous Installation](https://hackmd.io/8qw1hSxMSc29hWnOfC6lBg)


## Configuration
1. AMF
2. AUSF
3. NRF
4. NSSF
5. SMF
Add this line to SMF configuration, because by default, UPF to SMF is not encrypted

:::spoiler
```
  t3592:
    enable: true # true or false
    expireTime: 16s # default is 6 seconds
    maxRetryTimes: 3 # the max number of retransmission
  nrfUri: http://127.0.0.10:8000 # a valid URI of NRF
  nrfCertPem: cert/nrf.pem # NRF Certificate
  urrPeriod: 10 # default usage report period in seconds
  urrThreshold: 1000 # default usage report threshold in bytes
  requestedUnit: 1000
  ulcl: false
  nwInstFqdnEncoding: true
```
:::


## Testing
### UERANSIM
1. Configure UERANSIM

  - gNB (free5gc-gnb.yaml)
:::spoiler
```
mcc: '901'          # Mobile Country Code value
mnc: '70'           # Mobile Network Code value (2 or 3 digits)

nci: '0x000000010'  # NR Cell Identity (36-bit)
idLength: 32        # NR gNB ID length in bits [22...32]
tac: 1              # Tracking Area Code

linkIp: 10.99.1.69   # gNB's local IP address for Radio Link Simulation (Usually same with local IP)
ngapIp: 10.99.1.69   # gNB's local IP address for N2 Interface (Usually same with local IP)
gtpIp: 10.88.150.121    # gNB's local IP address for N3 Interface (Usually same with local IP)

# List of AMF address information
amfConfigs:
  - address: 10.99.1.230
    port: 38412

# List of supported S-NSSAIs by this gNB
slices:
  - sst: 0x1
    sd: 0x010203

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true
```
:::

   - UE
    
:::spoiler
```
# IMSI number of the UE. IMSI = [MCC|MNC|MSISDN] (In total 15 digits)
supi: 'imsi-901700000000001'
# Mobile Country Code value of HPLMN
mcc: '901'
# Mobile Network Code value of HPLMN (2 or 3 digits)
mnc: '70'
# SUCI Protection Scheme : 0 for Null-scheme, 1 for Profile A and 2 for Profile B
protectionScheme: 0
# Home Network Public Key for protecting with SUCI Profile A
homeNetworkPublicKey: '5a8d38864820197c3394b92613b20b91633cbd897119273bf8e4a6f4eec0a650'
# Home Network Public Key ID for protecting with SUCI Profile A
homeNetworkPublicKeyId: 1
# Routing Indicator
routingIndicator: '0000'

# Permanent subscription key
key: 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF'
# Operator code (OP or OPC) of the UE
op: 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF'
# This value specifies the OP type and it can be either 'OP' or 'OPC'
opType: 'OPC'
# Authentication Management Field (AMF) value
amf: '8000'
# IMEI number of the device. It is used if no SUPI is provided
imei: '356938035643803'
# IMEISV number of the device. It is used if no SUPI and IMEI is provided
imeiSv: '4370816125816151'

# List of gNB IP addresses for Radio Link Simulation
gnbSearchList:
  - 10.99.1.69

# UAC Access Identities Configuration
uacAic:
  mps: false
  mcs: false

# UAC Access Control Class
uacAcc:
  normalClass: 0
  class11: false
  class12: false
  class13: false
  class14: false
  class15: false

# Initial PDU sessions to be established
sessions:
  - type: 'IPv4'
    apn: 'internet'
    slice:
      sst: 0x01
      sd: 0x010203


# Supported integrity algorithms by this UE
integrity:
  IA1: true
  IA2: true
  IA3: true

# Supported encryption algorithms by this UE
ciphering:
  EA1: true
  EA2: true
  EA3: true

# Integrity protection maximum data rate for user plane
integrityMaxRate:
  uplink: 'full'
  downlink: 'full'
```
:::

2. Run gnb and UE
- gNB and UE
```
./nr-gnb -c ../config/free5gc-gnb.yaml
```

```
./nr-ue -c ../config/free5gc-ue.yaml
```
![image](https://hackmd.io/_uploads/rJEESrxDR.png)

- Wireshark
![image](https://hackmd.io/_uploads/BJBMBrlDA.png)

3. Testing
- Check IP
![image](https://hackmd.io/_uploads/ryp0ISxDR.png)

- Ping to google.com
![image](https://hackmd.io/_uploads/SJ0ZwBlw0.png)


### OpenAirInterface
:::info
For some reasons, integrating OAI and free5gc have some miscellenous problems. th 5GMM error while trying to have PDU session registration request.
:::



Here is the errors look like:
![image](https://hackmd.io/_uploads/HyF8K4WvC.png)

![image](https://hackmd.io/_uploads/rkavtVbDR.png)


To solve this, I have edit some code in AMF to bypassing the error. I don't know if this is good option or not. but the PDU is allocating IP and worked.

edit 

```
free5gc-v1/NFs/amf/internal/gmm/handler.go
```

![image](https://hackmd.io/_uploads/Hyl1oQWwR.png)

and this

![image](https://hackmd.io/_uploads/HJm6cX-vA.png)

1. Run gNB
```
./nr-softmodem -O /home/ubuntu/CONF/gnb.conf --gNBs.[0].min_rxtxtime 6 --rfsim --sa
```

![image](https://hackmd.io/_uploads/ByLOiXbwC.png)


2. Run nr-ue
```
./nr-uesoftmodem -r 106 --numerology 1 --band 78 -C 3619200000 --rfsim --sa --nokrnmod -O /home/ubuntu/CONF/nr-ue.conf
```

![image](https://hackmd.io/_uploads/SkSni7ZD0.png)

3. Check UPF
![image](https://hackmd.io/_uploads/HktEn7bvR.png)

This means, UPF was allocating IP for UE.

4. Check UE IP

Unfortunately, even IP was allocated by UPF, the UE couldn't get the IP. Still need to be troubleshoot.

![image](https://hackmd.io/_uploads/S1Ab67ZP0.png)



If you want to see the success story, check my previous note using open5Gs.

## Function Test of DPDK (ex: l2fwd, l3fwd, skeleton, …)

1. Mount huge for DPDK app
```
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
```

2. Load kernel module for dpdk
:::danger
For vfio sometimes you need to enable unsafe noiommu mode
:::

```
modprobe vfio
modprobe vfio-pci
echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
```

3. Bind interfaces to be used in this test
```
dpdk-devbind.py -s


Network devices using DPDK-compatible driver
============================================
0000:00:13.0 'Virtio network device 1000' drv=vfio-pci unused=
0000:00:14.0 'Virtio network device 1000' drv=vfio-pci unused=

Network devices using kernel driver
===================================
0000:00:12.0 'Virtio network device 1000' if=ens18 drv=virtio-pci unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

No 'ML' devices detected
========================
```

```
dpdk-devbind.py -b vfio-pci  0000:00:14.0
dpdk-devbind.py -b vfio-pci  0000:00:13.0
```

4. Build and run DPDK examples app

The examples app is located in ~/dpdk-stable-23.11.1/examples/

To build the example app, choose one of the examples and build like below:
```
cd ~/dpdk-stable-23.11.1/examples/l2fwd/
make -j$(nproc)
cd ./build
./l2fwd
```

![image](https://hackmd.io/_uploads/HkpW31SvC.png)

and then test the app

```
./l2fwd -n 2 -c f -- -q 1 -p 0x3
```

![image](https://hackmd.io/_uploads/BJmuPzBvA.png)




## Conslusion