{%hackmd ElOJTlJjTg-GQtxooVV5iw %}

# E2E Study Notes W1
> Johan ED Saputro

## GOALS
1. Bring DPDK features to PFCP
2. Bring DPDK features to gNB

## Milestone


### Environment preparation & Verification

#### 5GC E2E Test

| Task | Status |
| -------- | -------- |
| Install free5Gc & OAI | <input type="checkbox" disabled="disabled" checked="checked"> |
| Try E2E between Core and RAn | <input type="checkbox" disabled="disabled" checked="checked"> |

| Task | Status |
| -------- | -------- |
| Test UE Sim | <input type="checkbox" disabled="disabled" checked="checked"> |
| Test using ping and iperf3 | <input type="checkbox" disabled="disabled" checked="checked"> |

#### DPDK

| Task | Status |
| -------- | -------- |
| Install DPDK on UPF | <input type="checkbox" disabled="disabled" checked="checked"> |
| Function Test of DPDK (ex: l2fwd, l3fwd, skeleton, ...) | <input type="checkbox" disabled="disabled" checked="checked"> |

### Project Assignment
#### Lightweight 5GC

| Task | Status |
| -------- | -------- |
| Study of NSSF | <input type="checkbox" disabled="disabled"> |
| Study of PCF | <input type="checkbox" disabled="disabled"> |
| Trace code based on the study in Free5G | <input type="checkbox" disabled="disabled"> |
| Think how to maintain the function of 5GC after NF removal & Provide the proposal of improvement | <input type="checkbox" disabled="disabled"> |
| Implementation (Modification & Compilation & E2E test) | <input type="checkbox" disabled="disabled"> |
| Remove the NF & E2E test | <input type="checkbox" disabled="disabled"> |
| Test with multiple UEs | <input type="checkbox" disabled="disabled"> |

### The Control Plane of DPDK
#### The CP of DPDK
| Task | Status |
| -------- | -------- |
| Study of DPDK in UP (CU-PDCP) | <input type="checkbox" disabled="disabled"> |
| Study of related works | <input type="checkbox" disabled="disabled"> |
| Study of CP protocol & DPDK control method | <input type="checkbox" disabled="disabled"> |
| Prepare environment (RIC &Xapp) | <input type="checkbox" disabled="disabled"> |
| Implementation of DPDK on CU-UP & DU-UP | <input type="checkbox" disabled="disabled"> |
| Proposal (Mobility solution) | <input type="checkbox" disabled="disabled"> |
| Proposal (QoS soluion.) | <input type="checkbox" disabled="disabled"> |
| Provide the architechture & call flow | <input type="checkbox" disabled="disabled"> |
| Implementation | <input type="checkbox" disabled="disabled"> |
| Test | <input type="checkbox" disabled="disabled"> |
| Final Report | <input type="checkbox" disabled="disabled"> |

## Installation
### Architecture

### Free5gc Installation

1. Install Golang

sudo rm -rf /usr/local/go
wget https://dl.google.com/go/go1.21.8.linux-amd64.tar.gz
sudo tar -C /usr/local -zxvf go1.21.8.linux-amd64.tar.gz

2. Install mongodb

sudo apt -y update
sudo apt -y install mongodb wget git
sudo systemctl start mongodb

3. Install Dependencies to build free5gc

sudo apt -y update
sudo apt -y install git gcc g++ cmake autoconf libtool pkg-config libmnl-dev libyaml-dev

4. Clone free5gc repository

cd ~
git clone --recursive -b v3.4.1 -j `nproc` https://github.com/free5gc/free5gc.git
cd free5gc

5. Build all NF

cd ~/free5gc
make

6. Install Web Console
- Install Nodejs

	curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - 
	sudo apt update
	sudo apt install -y nodejs
	corepack enable # setup yarn automatically

- Build webconsole

	cd ~/free5gc
	make webconsole

## UPF DPDK Installation
[Check my previous Installation](https://hackmd.io/8qw1hSxMSc29hWnOfC6lBg)

## Configuration
1. AMF
2. AUSF
3. NRF
4. NSSF
5. SMF
Add this line to SMF configuration, because by default, UPF to SMF is not encrypted

:::spoiler

  t3592:
    enable: true # true or false
    expireTime: 16s # default is 6 seconds
    maxRetryTimes: 3 # the max number of retransmission
  nrfUri: http://127.0.0.10:8000 # a valid URI of NRF
  nrfCertPem: cert/nrf.pem # NRF Certificate
  urrPeriod: 10 # default usage report period in seconds
  urrThreshold: 1000 # default usage report threshold in bytes
  requestedUnit: 1000
  ulcl: false
  nwInstFqdnEncoding: true

:::

## Testing
### UERANSIM
1. Configure UERANSIM

- gNB (free5gc-gnb.yaml)
:::spoiler

mcc: '901'          # Mobile Country Code value
mnc: '70'           # Mobile Network Code value (2 or 3 digits)

nci: '0x000000010'  # NR Cell Identity (36-bit)
idLength: 32        # NR gNB ID length in bits [22...32]
tac: 1              # Tracking Area Code

linkIp: 10.99.1.69   # gNB's local IP address for Radio Link Simulation (Usually same with local IP)
ngapIp: 10.99.1.69   # gNB's local IP address for N2 Interface (Usually same with local IP)
gtpIp: 10.88.150.121    # gNB's local IP address for N3 Interface (Usually same with local IP)

# List of AMF address information
amfConfigs:
  - address: 10.99.1.230
    port: 38412

# List of supported S-NSSAIs by this gNB
slices:
  - sst: 0x1
    sd: 0x010203

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true

:::

- UE

:::spoiler

# IMSI number of the UE. IMSI = [MCC|MNC|MSISDN] (In total 15 digits)
supi: 'imsi-901700000000001'
# Mobile Country Code value of HPLMN
mcc: '901'
# Mobile Network Code value of HPLMN (2 or 3 digits)
mnc: '70'
# SUCI Protection Scheme : 0 for Null-scheme, 1 for Profile A and 2 for Profile B
protectionScheme: 0
# Home Network Public Key for protecting with SUCI Profile A
homeNetworkPublicKey: '5a8d38864820197c3394b92613b20b91633cbd897119273bf8e4a6f4eec0a650'
# Home Network Public Key ID for protecting with SUCI Profile A
homeNetworkPublicKeyId: 1
# Routing Indicator
routingIndicator: '0000'

# Permanent subscription key
key: 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF'
# Operator code (OP or OPC) of the UE
op: 'FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF'
# This value specifies the OP type and it can be either 'OP' or 'OPC'
opType: 'OPC'
# Authentication Management Field (AMF) value
amf: '8000'
# IMEI number of the device. It is used if no SUPI is provided
imei: '356938035643803'
# IMEISV number of the device. It is used if no SUPI and IMEI is provided
imeiSv: '4370816125816151'

# List of gNB IP addresses for Radio Link Simulation
gnbSearchList:
  - 10.99.1.69

# UAC Access Identities Configuration
uacAic:
  mps: false
  mcs: false

# UAC Access Control Class
uacAcc:
  normalClass: 0
  class11: false
  class12: false
  class13: false
  class14: false
  class15: false

# Initial PDU sessions to be established
sessions:
  - type: 'IPv4'
    apn: 'internet'
    slice:
      sst: 0x01
      sd: 0x010203


# Supported integrity algorithms by this UE
integrity:
  IA1: true
  IA2: true
  IA3: true

# Supported encryption algorithms by this UE
ciphering:
  EA1: true
  EA2: true
  EA3: true

# Integrity protection maximum data rate for user plane
integrityMaxRate:
  uplink: 'full'
  downlink: 'full'

:::

2. Run gnb and UE
- gNB and UE

./nr-gnb -c ../config/free5gc-gnb.yaml

./nr-ue -c ../config/free5gc-ue.yaml


- Wireshark

3. Testing
- Check IP

- Ping to google.com

### OpenAirInterface
:::info
For some reasons, integrating OAI and free5gc have some miscellenous problems. th 5GMM error while trying to have PDU session registration request.
:::

Here is the errors look like:


To solve this, I have edit some code in AMF to bypassing the error. I don't know if this is good option or not. but the PDU is allocating IP and worked.

edit

free5gc-v1/NFs/amf/internal/gmm/handler.go


and this


1. Run gNB

./nr-softmodem -O /home/ubuntu/CONF/gnb.conf --gNBs.[0].min_rxtxtime 6 --rfsim --sa


2. Run nr-ue

./nr-uesoftmodem -r 106 --numerology 1 --band 78 -C 3619200000 --rfsim --sa --nokrnmod -O /home/ubuntu/CONF/nr-ue.conf


3. Check UPF

This means, UPF was allocating IP for UE.

4. Check UE IP

Unfortunately, even IP was allocated by UPF, the UE couldn't get the IP. Still need to be troubleshoot.


If you want to see the success story, check my previous note using open5Gs.

## Function Test of DPDK (ex: l2fwd, l3fwd, skeleton, …)

1. Mount huge for DPDK app

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge

2. Load kernel module for dpdk
:::danger
For vfio sometimes you need to enable unsafe noiommu mode
:::

modprobe vfio
modprobe vfio-pci
echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode

3. Bind interfaces to be used in this test

dpdk-devbind.py -s


Network devices using DPDK-compatible driver
============================================
0000:00:13.0 'Virtio network device 1000' drv=vfio-pci unused=
0000:00:14.0 'Virtio network device 1000' drv=vfio-pci unused=

Network devices using kernel driver
===================================
0000:00:12.0 'Virtio network device 1000' if=ens18 drv=virtio-pci unused=vfio-pci *Active*

No 'Baseband' devices detected
==============================

No 'Crypto' devices detected
============================

No 'DMA' devices detected
=========================

No 'Eventdev' devices detected
==============================

No 'Mempool' devices detected
=============================

No 'Compress' devices detected
==============================

No 'Misc (rawdev)' devices detected
===================================

No 'Regex' devices detected
===========================

No 'ML' devices detected
========================

dpdk-devbind.py -b vfio-pci  0000:00:14.0
dpdk-devbind.py -b vfio-pci  0000:00:13.0

4. Build and run DPDK examples app

The examples app is located in ~/dpdk-stable-23.11.1/examples/

To build the example app, choose one of the examples and build like below:

cd ~/dpdk-stable-23.11.1/examples/l2fwd/
make -j$(nproc)
cd ./build
./l2fwd


and then test the app

./l2fwd -n 2 -c f -- -q 1 -p 0x3


## Conslusion