QCT 5GC Quick Start
===
###### tags: `Telecom` `QCT` `5G` `DPDK`
## N9 Case
:::info
Landslide接在EdgeN3和CoreN6
:::
### Network Connection
| VLAN | MAC | IX1 | NIC | PCI Addr | CNI |
| -------- | -------- | -------- |-------- |-------- | -------- |
| 493 | 3C:FD:FE:A9:E5:00 | 0/34 | enp28s0f0 | 1c:00.0 | CoreN1N2 |
| 493 | 3C:FD:FE:A9:EA:A9 | 0/33 | enp218s0f1 | da:00.1 | CoreN9 |
| 493 | 3C:FD:FE:A9:EB:48 | 0/97 | enp28s0f0 | 1c:00.0 | EdgeN3 |
| 496 | 3C:FD:FE:A9:E5:01 | 0/35 | enp28s0f1 | 1c:00.1 | CoreN6 |
| | 3C:FD:FE:A9:EB:49 | | enp28s0f1 | 1c:00.1 | EdgeN6 |
| | a8:1e:84:a2:26:0f | | enp61s0f0 | 3d:00.0 | CoreNn |
| | a8:1e:84:a2:26:14 | | enp61s0f0 | 3d:00.0 | EdgeNn |
### Trouble Shooting
#### Core Server
##### AMF Pod
1. Check VLAN setting correctly
```
vpp# ping 192.168.120.51 # Core N9
vpp# ping 192.168.120.109 # Edge N3
vpp# ping 192.168.130.109 # Edge N4
```
##### UPF Pod
1. Check rate setting
```
vpp# show hardware-interface
```
:::info
TwentyFiveGigabitEthernet: 符合網卡型號
Link speed: 符合線速
up: 狀態
:::
動態改變ip address和state的命令
```
# cat /etc/vpp.conf
```
2. Check VLAN setting correctly
```
vpp# ping 192.168.130.45 # SMF
```
3. Check SMF registration
如果SMF registration失敗，**必須在UPF執行後**，再重啟SMF
```
# helm delete upf --purge
# helm install upf
# helm delete smf
```
再去檢查UPF
```
vpp# show upf association
```
4. Check Landslide connection
#### Edge Server
##### UPF Pod
1. Check rate setting
2. Check VLAN setting correctly
```
vpp# ping 192.168.130.45 # SMF
vpp# ping 192.168.95.23 # breakout server
```
3. Check SMF registration
```
vpp# show upf association
Node: 192.168.130.45
Recovery Time Stamp: 1972/12/20 00:16:29:000
Sessions: 0
User Plane IP Resource Information:
IP: 192.168.120.109 NWI: epc TEID Range Indication: 2 TEID Range: 2
vpp# show upf pfd-list
app_id counts: 4
###app id: 1001, pfd counts: 3###
----FLAG: fd=1, url=2, dn=4, cp=0----
URL=url
...
```
4. Check Landslide connection
```
vpp# show upf session
Total session num = 1
```
#### Breakout Server
1. Check ICMP traffic from Edge N6 ping testing
```
tcpdump -ni enp134s0f0
05:28:28.831924 IP 192.168.95.109 > 192.168.95.23: ICMP echo request, id 36513, seq 1, length 76
05:28:28.831964 IP 192.168.95.23 > 192.168.95.109: ICMP echo reply, id 36513, seq 1, length 76
```
---
## Installation
要分別佈署Core Server和Edge Server
1. Prerequisite
* 管理者主機的設定
```
$ apt update
$ apt install python -y
$ apt install python-pip -y
$ pip install -r kubespray-5g/requirements.txt
$ ssh-keygen #產生~/.ssh/id_rsa和id_rsa.pub
$ ssh-copy-id root@10.103.11.70 #複製id_rsa.pub到欲登入主機的~/.ssh/authorized_keys
```
* 被操縱主機的設定
```
$ vi /etc/ssh/sshd_config
PermitRootLogin yes
$ service ssh restart
# configure kernel parameters at runtime
$ vi /etc/sysctl.conf
vm.nr_hugepages = 1024
$ sysctl -p
# kernel modules to load at boot time
$ vi /etc/modules
vfio-pci
$ reboot
```
* Breakout Server 的設定
```
# vi /etc/netplan/50-cloud-init.yaml
network:
ethernets:
enp134s0f0:
addresses:
- 192.168.95.23/24
gateway4: 192.158.95.1
nameservers:
addresses:
- 8.8.8.8
$ sudo netplan try
$ sudo netplan apply
$ ip route add 172.20.224.0/24 via 192.168.95.109
```
:::info
192.168.95.109 是 EdgeN6
172.20.224.100 是 UE 的固定 IP
:::
2. Configuration
```
$ vi kubespray-5g/inventory/5g_core/hosts.yml
$ vi charts-5g/dp/upf/values.yaml
$ vi kubespray-5g/inventory/5g_core/group_vars/k8s-cluster/k8s-net-multus.yml
$ unzip config.zip
Archive: config.zip
inflating: deployment.yaml-pcf
inflating: k8s-net-multus.yml-core
inflating: k8s-net-multus.yml-edge
inflating: values.yaml-amf
inflating: values.yaml-ausf
inflating: values.yaml-pcf
inflating: values.yaml-smf
inflating: values.yaml-udm
inflating: values.yaml-upf-core
inflating: values.yaml-upf-edge
```
3. Platform Installation
```
$ ansible-playbook -i kubespray-5g/inventory/5g_core/hosts.yml --become --become-user=root kubespray-5g/cluster.yml -vvv
$ kubectl get pod
```
:::info
-i specify inventory host path
-b, --become
set to yes to activate privilege escalation
--become-user BECOME_USER
run operations as this user (default is root)
-e, --extra-vars
set additional variables as key=value
:::
4. 5G Pod Installation
```
$ docker load --input 5gc-images.tar
$ docker images
$ helm install charts-5g/dp/upf --name upf
$ helm install charts-5g/cp/amf --name amf
$ helm install charts-5g/cp/smf --name smf
$ helm install charts-5g/cp/udm --name udm
$ helm install charts-5g/cp/ausf --name ausf
$ helm install charts-5g/cp/pcf --name pcf
$ kubectl get pod
```
5. Reset
```
$ ansible-playbook -i kubespray-5g/inventory/5g_core/hosts.yml kubespray-5g/reset.yml -b -v --private-key=~/.ssh/id_rsa -e kube_proxy_mode=iptable
```
6. Dashboard
* upf-agent: http://10.103.11.70:30000/vpp
* smf: http://10.103.11.70:30004/metrics
* prometheus: http://10.103.11.70:30003
* kibana: http://10.103.11.70:30001
* grafana: http://10.103.11.70:30002
account: admin
password: ```kubectl exec -it grafana-9f697df9c-q6zl6 -- env | grep PASS```
---
## Local Breakout
* Breakout Server的操作
```
# create app_2 rule 要ping得動
ping 172.20.224.100 # UE的固定IP
# delete app_2 rule 要ping不動
ping 172.20.224.100 # UE的固定IP
```
* Core Server的操作
```
$ kubectl exec -it pcf-pcf-6f78f7bc45-kbf8x -- /bin/bash
# /usr/local/bin/pcf -s -c /etc/pcf/config/pcf.conf -l /etc/pcf/config/asn.log.properties -i
# smpc pccrule create 466920100001101 5 app_2
# smpc pccrule delete 466920100001101 5 app_2
```
---
## Performance Tuning
提高pktgen收包率:
1. 關掉Turbo Boost和Hyper Threading
2. 降低rate，直到port 0 tx和port 1 rx幾乎一致
例如: ```Pktgen:/> set 0 rate 10```
3. 使用相同NUMA的CPU和NIC；增加CPU數目
```
# UPF的values.yaml
...
maincore: 0 # cpu編號
worker: 1 # cpu編號
rxqueue: 1
resources:
requests:
memory: 4096Mi
cpu: "2" # cpu數目
limits:
memory: 4096Mi
cpu: "2" # cpu數目
hugepages-2Mi: 256Mi
...
```
---
## Reference
### Switch Configuration
1. 將要用的enp設為```dhcp4: true```: [netplan](https://blog.toright.com/posts/6293/ubuntu-18-04-%E9%80%8F%E9%81%8E-netplan-%E8%A8%AD%E5%AE%9A%E7%B6%B2%E8%B7%AF%E5%8D%A1-ip.html)
2. 檢查**UP**狀態: ```ip link```
3. 設定VLAN
* 透過MAC取得switch interface 0/```#```
```
telnet 10.103.15.34
admin/(no password, press enter directly)
(Switch) #show mac-addr-table
```
* 增加VLAN
```
(Switch) #configure
(Switch) (Config)#interface 0/35
(Switch) (Interface 0/35)#switchport allowed vlan add 481
(Switch) (Interface 0/35)#switchport native vlan 481
```
* 移除VLAN
```
(Switch) #configure
(Switch) (Config)#interface 0/35
(Switch) (Interface 0/35)#no switchport native vlan
(Switch) (Interface 0/35)#switchport allowed vlan remove 481
```
4. 取得PCIe接腳: ```lshw -class network -businfo```
### Turbo Boost
若超過Intel(R) Xeon(R) Gold 6152 CPU @ **2.10GHz**，表示有開啟。
```
$ less /proc/cpuinfo | grep MHz
```
Sign in with Wallet
Connect another wallet