## Deployment with Free5GC, Kubernetes and Helm
Below is a step-by-step guide to deploy free5GC and UERANSIM
**Step 1**: Prepare a machine with the following requirements:
1. Kernel version 5.0.0-23-generic or newer. Or a rule of thumb: Ubuntu 18.04 LTS and later version will work well.
2. Install gtp5g kernel module from here: https://github.com/PrinzOwO/gtp5g
3. A Kubernetes version that support SCTP (from 1.20)
Install the required packages for user-plane
```
sudo apt -y install git gcc g++ cmake autoconf libtool pkg-config libmnl-dev libyaml-dev
```
**Step 2**: Install kubectl, microk8s, and helm
To install microk8s, please use the following commands:
```
sudo snap install microk8s --classic
sudo snap list
newgrp microk8s
sudo usermod -a -G microk8s <userid_using_microk8s>
sudo chown -f -R <userid_using_microk8s> ~/.kube
```
To install kubectl, please use the following commands:
```
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x kubectl
sudo mv kubectl /usr/local/bin/
```
To install helm, please use the following commands:
```
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
```
**Step 3**: Start a single-node cluster and enable Multus-CNI
ip forwarding is required for UDF, but Calico - default CNI requires additional configuration, so we will switch to Flannel for the sake of simplicity:
```
microk8s disable ha-cluster --force
```
Enable neccessary plugins by the following commands:
```
microk8s enable dns ingress dashboard storage community helm3
microk8s enable multus
```
**Step 4**: Deploy the Helm chart
Using the following commands to add helm chart repository:
```
microk8s kubectl create namespace free5gc
microk8s helm3 repo add towards5gs 'https://raw.githubusercontent.com/haidinhtuan/free5gc-kubernetes/main/repo/'
microk8s helm3 repo update
```
The free5GC deployment on Kubernetes requires some additional network configuration, therefore we need to do the following:
Check the available network interfaces by the command:
```
ip address
```
We interest in the actual network interface that connect to the data network, in the following scenario, it should be the ens18 interface:
```
dissident2@dissident2:~$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether e6:5f:ff:00:76:c3 brd ff:ff:ff:ff:ff:ff
altname enp0s18
inet 130.149.223.200/26 brd 130.149.223.255 scope global ens18
valid_lft forever preferred_lft forever
inet6 fe80::e45f:ffff:fe00:76c3/64 scope link
valid_lft forever preferred_lft forever
```
Enable the promiscous mode on this interface by the command:
```
sudo ip link set ens18 promisc on
```
Deploy the Helm chart for free5GC's core components by adapting the following parameter to fit the physical interface (ens18 in this case): `global.n2network.masterIf`, `global.n3network.masterIf`, `global.n4network.masterIf`, `global.n6network.masterIf`, `global.n9network.masterIf`, `global.n6network.subnetIP`, `global.n6network.gatewayIP`, `global.n6network.cidr`, `free5gc-upf.upf.n6if.ipAddress`.
Below is one example:
```
microk8s helm -n free5gc install free5gc-core towards5gs/free5gc --set global.n2network.masterIf=ens18,global.n3network.masterIf=ens18,global.n4network.masterIf=ens18,global.n6network.masterIf=ens18,global.n9network.masterIf=ens18,global.n6network.subnetIP=130.149.223.192,global.n6network.gatewayIP=130.149.223.193,global.n6network.cidr=26,free5gc-upf.upf.n6if.ipAddress=130.149.223.205,global.n2network.type=macvlan,global.n3network.type=macvlan,global.n4network.type=macvlan,global.n6network.type=macvlan,global.n9network.type=macvlan
```
If the configuration is correct, all components of free5GC should be running as below:
```
dissident2@dissident2:~$ microk8s kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
free5gc free5gc-core-free5gc-amf-amf-589656c587-k2r88 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-ausf-ausf-5f4c977f8b-85jws 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-dbpython-dbpython-649c48b655-6bl4d 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-nrf-nrf-cf94f8654-ghlr8 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-nssf-nssf-5b775bd7c5-wl5t2 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-pcf-pcf-8ff9774d-rlpc7 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-smf-smf-64bfc4fcf5-q4jb9 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-udm-udm-654cd4755d-kk8t9 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-udr-udr-57797d9479-4vdhs 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-upf-upf-79675d49f6-d29qv 1/1 Running 0 3m50s
free5gc free5gc-core-free5gc-webui-webui-68f97fbdf-8j4jb 1/1 Running 0 3m50s
free5gc mongodb-0 1/1 Running 0 3m50s
kube-system calico-kube-controllers-7bdbfc669-sz9tr 1/1 Running 1 (5m23s ago) 5m36s
kube-system calico-node-dkg7z 1/1 Running 0 5m36s
kube-system coredns-787d4945fb-6cmzt 1/1 Running 1 (5m21s ago) 5m36s
kube-system etcd-minikube 1/1 Running 0 5m49s
kube-system kube-apiserver-minikube 1/1 Running 0 5m48s
kube-system kube-controller-manager-minikube 1/1 Running 0 5m48s
kube-system kube-multus-ds-l99l2 1/1 Running 0 4m21s
kube-system kube-proxy-7gltn 1/1 Running 0 5m36s
kube-system kube-scheduler-minikube 1/1 Running 0 5m49s
kube-system storage-provisioner 1/1 Running 1 (5m5s ago) 5m47s
```
Run the following commands to enable connection to the Web UI:
```
microk8s kubectl get svc -n free5gc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
webui-service NodePort 10.98.157.168 <none> 5000:30500/TCP 4m43s
# kubernets port forward to access the NodePort service
microk8s kubectl port-forward --namespace free5gc svc/webui-service 5000:5000
# do ssh port forwarding to access the web-ui service from local machine
# now web ui can access in your local machine localhost:5000
# run this from a different terminal window
ssh -L localhost:5000:localhost:5000 dissident2@130.149.223.200
```
Add new subscriber via the Web UI with the credential admin/free5gc. Click on Subscriber on the left panel, then click on the button New Subscriber on the top right. A new pop up window appears, scroll all the way to the bottom and click Submit.
Deploy the Helm chart for free5GC's UERANSIM with the following configuration:
```
microk8s helm3 -n free5gc install free5gc-ueransim towards5gs/ueransim --set global.n2network.masterIf=ens18,global.n3network.masterIf=ens18,global.n2network.type=macvlan,global.n3network.type=macvlan
```
The gNodeB and the UE pod should be up and running at this point:
```
dissident2@dissident2:~$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
free5gc free5gc-core-free5gc-amf-amf-589656c587-k2r88 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-ausf-ausf-5f4c977f8b-85jws 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-dbpython-dbpython-649c48b655-6bl4d 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-nrf-nrf-cf94f8654-ghlr8 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-nssf-nssf-5b775bd7c5-wl5t2 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-pcf-pcf-8ff9774d-rlpc7 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-smf-smf-64bfc4fcf5-q4jb9 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-udm-udm-654cd4755d-kk8t9 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-udr-udr-57797d9479-4vdhs 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-upf-upf-79675d49f6-d29qv 1/1 Running 0 7m56s
free5gc free5gc-core-free5gc-webui-webui-68f97fbdf-8j4jb 1/1 Running 0 7m56s
free5gc free5gc-ueransim-gnb-8445d69f4f-mlf9b 1/1 Running 0 17s
free5gc free5gc-ueransim-ue-7dcbc6d4c6-czhmg 1/1 Running 0 17s
free5gc mongodb-0 1/1 Running 0 7m56s
kube-system calico-kube-controllers-7bdbfc669-sz9tr 1/1 Running 1 (9m29s ago) 9m42s
kube-system calico-node-dkg7z 1/1 Running 0 9m42s
kube-system coredns-787d4945fb-6cmzt 1/1 Running 1 (9m27s ago) 9m42s
kube-system etcd-minikube 1/1 Running 0 9m55s
kube-system kube-apiserver-minikube 1/1 Running 0 9m54s
kube-system kube-controller-manager-minikube 1/1 Running 0 9m54s
kube-system kube-multus-ds-l99l2 1/1 Running 0 8m27s
kube-system kube-proxy-7gltn 1/1 Running 0 9m42s
kube-system kube-scheduler-minikube 1/1 Running 0 9m55s
kube-system storage-provisioner 1/1 Running 1 (9m11s ago) 9m53s
```
Check the UE's bash for uesimtun0 interface
```
dissident2@dissident2:~$ kubectl -n free5gc exec -it free5gc-ueransim-ue-7dcbc6d4c6-s7992 -- bash
root@free5gc-ueransim-ue-7dcbc6d4c6-s7992:/ueransim/build# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000
link/ipip 0.0.0.0 brd 0.0.0.0
4: eth0@if27: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1480 qdisc noqueue state UP group default
link/ether da:d6:04:61:29:82 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.244.120.86/32 scope global eth0
valid_lft forever preferred_lft forever
5: uesimtun0: <POINTOPOINT,PROMISC,NOTRAILERS,UP,LOWER_UP> mtu 1400 qdisc fq_codel state UNKNOWN group default qlen 500
link/none
inet 10.1.0.1/32 scope global uesimtun0
valid_lft forever preferred_lft forever
root@free5gc-ueransim-ue-7dcbc6d4c6-s7992:/ueransim/build#
```
Check the net.ipv4.ip_forward at the UPF:
```
kubectl -n free5gc exec -it free5gc-core-free5gc-upf-upf-79675d49f6-d29qv -- cat /proc/sys/net/ipv4/ip_forward
```
Capture HTTP traffic:
```
tcpdump -i any -s 0 -A 'tcp port 80 or tcp port 8080' - w amf_trace.pcap
```
The returned value should be 1.
Check logs on the UE to verify the Ue registration:
```
UERANSIM v3.2.6
[2023-05-23 12:23:12.505] [nas] [info] UE switches to state [MM-DEREGISTERED/PLMN-SEARCH]
[2023-05-23 12:23:14.707] [nas] [error] PLMN selection failure, no cells in coverage
[2023-05-23 12:23:16.909] [nas] [error] PLMN selection failure, no cells in coverage
[2023-05-23 12:23:17.507] [rrc] [warning] Acceptable cell selection failed, no cell is in coverage
[2023-05-23 12:23:17.507] [rrc] [error] Cell selection failure, no suitable or acceptable cell found
[2023-05-23 12:23:18.010] [nas] [info] UE switches to state [MM-DEREGISTERED/NO-CELL-AVAILABLE]
[2023-05-23 12:23:22.518] [rrc] [debug] New signal detected for cell[1], total [1] cells in coverage
[2023-05-23 12:23:22.518] [nas] [info] Selected plmn[208/93]
[2023-05-23 12:23:22.518] [rrc] [info] Selected cell plmn[208/93] tac[1] category[SUITABLE]
[2023-05-23 12:23:22.518] [nas] [info] UE switches to state [MM-DEREGISTERED/PS]
[2023-05-23 12:23:22.518] [nas] [info] UE switches to state [MM-DEREGISTERED/NORMAL-SERVICE]
[2023-05-23 12:23:22.518] [nas] [debug] Initial registration required due to [MM-DEREG-NORMAL-SERVICE]
[2023-05-23 12:23:22.518] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2023-05-23 12:23:22.518] [nas] [debug] Sending Initial Registration
[2023-05-23 12:23:22.518] [nas] [info] UE switches to state [MM-REGISTER-INITIATED]
[2023-05-23 12:23:22.518] [rrc] [debug] Sending RRC Setup Request
[2023-05-23 12:23:22.519] [rrc] [info] RRC connection established
[2023-05-23 12:23:22.519] [rrc] [info] UE switches to state [RRC-CONNECTED]
[2023-05-23 12:23:22.519] [nas] [info] UE switches to state [CM-CONNECTED]
[2023-05-23 12:23:22.550] [nas] [debug] Authentication Request received
[2023-05-23 12:23:22.550] [nas] [debug] Sending Authentication Failure due to SQN out of range
[2023-05-23 12:23:22.563] [nas] [debug] Authentication Request received
[2023-05-23 12:23:22.578] [nas] [debug] Security Mode Command received
[2023-05-23 12:23:22.578] [nas] [debug] Selected integrity[2] ciphering[0]
[2023-05-23 12:23:22.628] [nas] [debug] Registration accept received
[2023-05-23 12:23:22.628] [nas] [info] UE switches to state [MM-REGISTERED/NORMAL-SERVICE]
[2023-05-23 12:23:22.628] [nas] [debug] Sending Registration Complete
[2023-05-23 12:23:22.628] [nas] [info] Initial Registration is successful
[2023-05-23 12:23:22.628] [nas] [debug] Sending PDU Session Establishment Request
[2023-05-23 12:23:22.632] [nas] [debug] UAC access attempt is allowed for identity[0], category[MO_sig]
[2023-05-23 12:23:22.876] [nas] [debug] PDU Session Establishment Accept received
[2023-05-23 12:23:22.876] [nas] [info] PDU Session establishment is successful PSI[1]
[2023-05-23 12:23:22.887] [app] [info] Connection setup for PDU session[1] is successful, TUN interface[uesimtun0, 10.1.0.1] is up.
```
List of IP Address:
| Free5GC Component | IP Address |
|------------------|-----------------|
| AMF | 10.1.71.249 |
| SMF | 10.1.71.246 |
| UPF | 10.1.71.243 |
| NRF | 10.1.71.251 |
| UDR | 10.1.71.250 |
| AUSF | 10.1.71.254 |
| PCF | 10.1.71.244 |
| UDM | 10.1.71.253 |
| NSSF | 10.1.71.252 |
LUA error:
Expert Info (Error/Undecoded): Lua Error: ...sers\haidi\AppData\Roaming\Wireshark\plugins\openapi.lua:98: table index is nil
Use this command to verify the connection from UE to data network:
``kubectl -n free5gc exec -it free5gc-v2-ueransim-ue-6c7b7b4c54-qsxc7 -- ping -I uesimtun0 www.google.com``
Useful commands to update the Helm chart:
```
helm lint ./charts/free5gc/charts/*
helm lint ./charts/*
helm package --destination ./repo/ ./charts/*
helm repo index --url https://haidinhtuan.github.io/free5gc-kubernetes/ .
git add .
git commit -m "update Helm chart"
git push
```
**Useful links:**
original helm implementation - https://github.com/Orange-OpenSource/towards5gs-helm
ueransim helm config & documentation - https://github.com/Orange-OpenSource/towards5gs-helm/tree/main/charts/ueransim
problems with the interface not being created - https://github.com/Orange-OpenSource/towards5gs-helm/issues/46
*Note:* Right now, in my dissident machine, I have all the pods running correctly (free5gc and ueransim) but the
interface on the ue is not created. The configuration of the helms is in this repository:
https://github.com/oanabutaru98/towards5gs
*Note:* Common problems:
* Multus pod stop working, or stuck in CrashLoopBackOff: update the git folder using: git pull && docker system prune
* Cannot connect to VM after flushing iptables rules: sudo iptables -P INPUT ACCEPT
Sign in with Wallet
Connect another wallet