VNF Documentation

As a VNF 5G Implementation Division on Kubernetes, I design and deploy high-performance OpenAirInterface (OAI) 5G use cases. I optimize network performance, ensure security integration, and collaborate on defining use cases. My focus is on automation for efficient seamless deployment 5G integration in Kubernetes environments.

OpenNetra: Description

netra dash

Open Netra, standing for Open Network Training, is a comprehensive solution designed to facilitate the training, simulation, and management of 5G networks. The primary functionalities of Open Netra include E2E (End-to-End) 5G dashboard simulation and RAN (Radio Access Network) monitoring & configuration. The entire infrastructure is built on a foundation of virtualized network functions, specifically implemented as OpenAirInterface containers, orchestrated and managed by Kubernetes.

Pre-Requisite

  1. Production Ready Kubernetes Cluster

    Kubernetes Installation Detail

  2. (Optional) Multus CNI if using multiple interfaces for NFs

    Multus CNI Quick Start

  3. Kube-Sniff & Wireshark

    • Setup Krew (kubectl package-manager)

      Make sure git are installed

      ​​​​​​​​sudo apt install git 
      

      Run this command to download and install krew

      ​​​​​​​​(
      ​​​​​​​​  set -x; cd "$(mktemp -d)" &&
      ​​​​​​​​  OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
      ​​​​​​​​  ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
      ​​​​​​​​  KREW="krew-${OS}_${ARCH}" &&
      ​​​​​​​​  curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
      ​​​​​​​​  tar zxvf "${KREW}.tar.gz" &&
      ​​​​​​​​  ./"${KREW}" install krew
      ​​​​​​​​)
      

      Add the $HOME/.krew/bin directory to your PATH environment variable.

      ​​​​​​​​export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
      
    • Install Kubectl-Sniff

      Install it from krew package manager

      ​​​​​​​​kubectl krew install sniff
      
    • Install Wireshark

      Setup wireshark from os repository

      ​​​​​​​​sudo apt install wireshark -y
      

      Configure the permission problem

      ​​​​​​​​sudo gpasswd -a $USER wireshark
      ​​​​​​​​sudo adduser $USER wireshark
      ​​​​​​​​sudo setcap 'CAP_NET_RAW+eip CAP_NET_ADMIN+eip' /usr/bin/dumpcap
      
    • How to use ksniff

      If used in Wireshark GUI

      image

      ​​​​​​​​kubectl sniff oai-nr-ue-2-786b494559-jtz97 -n oai
      

      If used in Terminal

      ​​​​​​​​kubectl sniff oai-nr-ue-2-786b494559-jtz97 -n oai -f "tcp" -o - | tshark -r -
      
      Example output:
      ​​​​​​​​INFO[0000] using tcpdump path at: '/home/rafli/.krew/store/sniff/v1.6.2/static-tcpdump'
      ​​​​​​​​INFO[0000] no container specified, taking first container we found in pod.
      ​​​​​​​​INFO[0000] selected container: 'nr-ue'
      ​​​​​​​​INFO[0000] sniffing method: upload static tcpdump
      ​​​​​​​​INFO[0000] sniffing on pod: 'oai-nr-ue-2-786b494559-jtz97' [namespace: 'oai', container: 'nr-ue', filter: 'tcp', interface: 'any']
      ​​​​​​​​INFO[0000] uploading static tcpdump binary from: '/home/rafli/.krew/store/sniff/v1.6.2/static-tcpdump' to: '/tmp/static-tcpdump'
      ​​​​​​​​INFO[0000] uploading file: '/home/rafli/.krew/store/sniff/v1.6.2/static-tcpdump' to '/tmp/static-tcpdump' on container: 'nr-ue'
      ​​​​​​​​INFO[0000] executing command: '[/bin/sh -c test -f /tmp/static-tcpdump]' on container: 'nr-ue', pod: 'oai-nr-ue-2-786b494559-jtz97', namespace: 'oai'
      ​​​​​​​​INFO[0000] command: '[/bin/sh -c test -f /tmp/static-tcpdump]' executing successfully exitCode: '0', stdErr :''
      ​​​​​​​​INFO[0000] file found: ''
      ​​​​​​​​INFO[0000] file was already found on remote pod
      ​​​​​​​​INFO[0000] tcpdump uploaded successfully
      ​​​​​​​​INFO[0000] output file option specified, storing output in: '-'
      ​​​​​​​​INFO[0000] start sniffing on remote container
      ​​​​​​​​INFO[0000] executing command: '[/tmp/static-tcpdump -i any -U -w - tcp]' on container: 'nr-ue', pod: 'oai-nr-ue-2-786b494559-jtz97', namespace: 'oai'
      ​​​​​​​​    1   0.000000 10.233.75.62 → 10.233.75.54 TCP 100 44242 → 4043 [PSH, ACK] Seq=1 Ack=1 Win=24091 Len=32 TSval=2131950554 TSecr=1253996121
      ​​​​​​​​    2   0.000066 10.233.75.62 → 10.233.75.54 TCP 64376 44242 → 4043 [PSH, ACK] Seq=33 Ack=1 Win=24091 Len=64308 TSval=2131950554 TSecr=1253996121
      ​​​​​​​​    3   0.000100 10.233.75.54 → 10.233.75.62 TCP 68 4043 → 44242 [ACK] Seq=1 Ack=64341 Win=16287 Len=0 TSval=1253996122 TSecr=2131950554
      

1. CNF Infra Setup 💻

helm-chart

1.1 Setup OAI Core ☁️

  • Deployment Plan

    Core Multi Interface

    Purpose Interface IP Address Node
    AMF - N2 ens33 172.21.6.94/22 3
    UPF - N3 ens33 172.21.8.95/22 3
    UPF - N4 ens33 192.168.24.2/24 3
    UPF - N6 ens33 192.168.22.2/24 3
    SMF - N6 ens33 192.168.24.3/24 3
  • Helm Step Installation

    I installed the basic core network setup. Because it support to utilize external database for network function.

    • Clone Core 5G Repository

      ​​​​​​​​git clone https://github.com/adaptivenetworklab/AN-OPEN-NETRA-VNF
      
    • Create Namespace named core-network

      ​​​​​​​​kubectl create ns core-network
      
    • Helm Custom Chart For Core Network Deployment

      ​​​​​​​​# Core Network Chart
      ​​​​​​​​cd AN-OPEN-NETRA-VNF/oai-5g-core/
      ​​​​​​​​mysql  oai-5g-advance  oai-5g-basic  oai-5g-mini  oai-amf  oai-ausf  oai-nrf  oai-nssf  oai-smf  oai-traffic-server  oai-udm  oai-udr  oai-upf
      ​​​​​​​​helm dependency update
      ​​​​​​​​
      ​​​​​​​​# RAN Chart
      ​​​​​​​​ls AN-OPEN-NETRA-VNF/user_n/oai-e2e/
      ​​​​​​​​oai-cu  oai-du  oai-nr-ue
      ​​​​​​​​helm dependency update
      

      Starting version 2.0.0 of OAI 5G Core network functions their configuration will be in config.yaml and all infrastructure related information including image definition will be in values.yaml.

      Helm chart of every network function looks similar and has the below structure. Only the chart of mysql database and NRF is different.

    • Configuring Core Network

      Network function discovers each-other using NRF and instead of using the ip-address of network functions we rely on using their FQDN, Kubernetes service concept. To communicate with each other whether we deploy them in reference point architecture or service based architecture.

      For example: AMF registers with NRF using NRF FQDN (oai-nrf.core-network.svc.cluster.local). This way we can get rid of any static ip-address configuration.

      • Configure Multiple Interfaces

        • The network functions will use different virtual ethernet (veth) interfaces to bind their different logical interface. Example AMF communicates with gNB using N2 and with SMF and NRF using Namf, the Service Base Interface (SBI).

        • This type of configuration is also used when gNB is outside of the cluster or UPF is outside of the cluster.

        • To make the above seperation we are using multus to provide multiple ethernet interfaces to network functions which have multiple communication interfaces.

        • Only AMF, SMF and UPF have the possiblity to use multus. Other network functions can also use multus but then it needs to be configured.

        • To configure multus for AMF, SMF or UPF, in values.yaml of each network function edit the multus section.

          ​​​​​​​​​​​​​​​​## Example from oai-5g-basic/values.yaml
          ​​​​​​​​​​​​​​​​multus:
          ​​​​​​​​​​​​​​​​  ## If you don't want to add a default route in your pod then leave this field empty
          ​​​​​​​​​​​​​​​​  defaultGateway: ""
          ​​​​​​​​​​​​​​​​  n2Interface:
          ​​​​​​​​​​​​​​​​    create: true
          ​​​​​​​​​​​​​​​​    Ipadd: "172.21.6.94"
          ​​​​​​​​​​​​​​​​    Netmask: "22"
          ​​​​​​​​​​​​​​​​    ## If you do not have a gateway leave the field empty
          ​​​​​​​​​​​​​​​​    Gateway:
          ​​​​​​​​​​​​​​​​    ## If you do not want to add any routes in your pod then leave this field empty
          ​​​​​​​​​​​​​​​​    routes: 
          ​​​​​​​​​​​​​​​​    hostInterface: "ens33" # Interface of the host machine on which this pod will be scheduled
          
      • Configure Helm Chart Parameters

        In the config.yaml of oai-5g-basic helm charts you will see the configurable parameters for all the network functions check, the PLMN, DNN and subscriber information in mysql database

        For basic and advance deployment check the database oai_db-basic.sql

        A new subscriber entry can be added directly in the sql file or it can be added once the core network is already deployed.

        To add the entry before deploying the core network, make sure you have all the required subscriber information IMSI(ueid/supi), Key(encPermanentKey), OPC(encOpcKey), PLMN, NSSAI(SST, SD), DNN

        ​​​​​​​​​​​​vim AN-OPEN-NETRA-VNF/oai-5g-core/mysql/initialization/oai_db-basic.sql
        
        ​​​​​​​​​​​​# Add a new or edit existing entries after AuthenticationSubscription table
        ​​​​​​​​​​​​INSERT INTO `AuthenticationSubscription` (`ueid`, `authenticationMethod`, `encPermanentKey`, `protectionParameterId`, `sequenceNumber`, `authenticationManagementField`, `algorithmId`, `encOpcKey`, `encTopcKey`, `vectorGenerationInHss`, `n5gcAuthMethod`, `rgAuthenticationInd`, `supi`) VALUES
        ​​​​​​​​​​​​('208990100001101', '5G_AKA', 'a92d71ed92f43ebae1720f34fd2e6966', 'a92d71ed92f43ebae1720f34fd2e6966', '{\"sqn\": \"000000000020\", \"sqnScheme\": \"NON_TIME_BASED\", \"lastIndexes\": {\"ausf\": 0}}', '8000', 'milenage', 'C42449363BBAD02B66D16BC975D77CC1', NULL, NULL, NULL, NULL, '208990100001101'),
        ​​​​​​​​​​​​
        ​​​​​​​​​​​​# Add the PDN/DNN information after SessionManagementSubscriptionData table
        ​​​​​​​​​​​​# To assign a static ip-address use the below entry
        ​​​​​​​​​​​​INSERT INTO `SessionManagementSubscriptionData` (`ueid`, `servingPlmnid`, `singleNssai`, `dnnConfigurations`, `internalGroupIds`, `sharedVnGroupDataIds`, `sharedDnnConfigurationsId`, `odbPacketServices`, `traceData`, `sharedTraceDataId`, `expectedUeBehavioursList`, `suggestedPacketNumDlList`, `3gppChargingCharacteristics`) VALUES
        ​​​​​​​​​​​​('208990100001101', '20899', '{\"sd\": \"16777204\", \"sst\": 1}', '{\"ims\": {\"sscModes\": {\"defaultSscMode\": \"SSC_MODE_1\"}, \"sessionAmbr\": {\"uplink\": \"1000Mbps\", \"downlink\": \"1000Mbps\"}, \"5gQosProfile\": {\"5qi\": 2, \"arp\": {\"preemptCap\": \"NOT_PREEMPT\", \"preemptVuln\": \"PREEMPTABLE\", \"priorityLevel\": 15}, \"priorityLevel\": 1}, \"pduSessionTypes\": {\"defaultSessionType\": \"IPV4V6\"}}, \"oai\": {\"sscModes\": {\"defaultSscMode\": \"SSC_MODE_1\"}, \"sessionAmbr\": {\"uplink\": \"1000Mbps\", \"downlink\": \"1000Mbps\"}, \"5gQosProfile\": {\"5qi\": 1, \"arp\": {\"preemptCap\": \"NOT_PREEMPT\", \"preemptVuln\": \"PREEMPTABLE\", \"priorityLevel\": 15}, \"priorityLevel\": 1}, \"pduSessionTypes\": {\"defaultSessionType\": \"IPV4\"}, \"staticIpAddress\": [{\"ipv4Addr\": \"12.1.1.100\"}]}}', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL);
        

        In the config file smf.use_local_subscription_info should be yes to use the user DNN subscription information from the database. Else it will be used as defined in the configuration file.

      Clone and see me for detail configuration

    • Helm install with related chart

      Helm charts have an order of deployment for the proper configuration of core network.

      Once the configuration is finished the charts can be deployed with a user who has the rights to

      1. Create RBAC (Optional only if Openshift is used)
      2. Run pod (only UPF needs that) with privileged and anyuid scc (optional only required if you have scc configure in your cluster)
      3. Create multus binds (optional only if multus is used)

      Install with this helm command:

      ​​​​​​​​cd AN-OPEN-NETRA-VNF/oai-5g-core/
      ​​​​​​​​helm install basic oai-5g-basic/ --namespace core-network
      
      The output is similar to:
      ​​​​​​​​NAME: basic
      ​​​​​​​​LAST DEPLOYED: Tue Dec 12 10:04:40 2023
      ​​​​​​​​NAMESPACE: default
      ​​​​​​​​STATUS: deployed
      ​​​​​​​​REVISION: 1
      ​​​​​​​​TEST SUITE: None
      

      This command can take around 3-5 mins depending on your network speed and cluster configuration (computational capacity). You can use the wait command to see if the core network functions are running or not

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/instance=basic --timeout=3m -n core-network
      

      Check the logs smf and upf to see that the PFCP session is properly configured,

      ​​​​​​​​kubectl logs -l app.kubernetes.io/name=oai-smf -n core-network | grep 'handle_receive(16 bytes)' | wc -l
      ​​​​​​​​kubectl logs -l app.kubernetes.io/name=oai-upf -n core-network | grep 'handle_receive(16 bytes)' | wc -l
      

      If the value is more than 1 for both then it will verify that smf and upf have successfully registered to nrf and there is a PFCP session.

    • Wait until all the pods are up and running (k9s)

    image

    • Component topology on 5G Core Network (KubeView)

    image

  • Configuring PhpMyAdmin for Subscriber Management

    • Deploy the secret first

      ​​​​​​​​kubectl apply -f phypmyadmin/secret.yaml
      
    • Deploy the rest of the manifest

      ​​​​​​​​kubectl apply -f phypmyadmin/
      
    • PMA Dashboard
      image

      Setup your custom credential in the core manifest configuration file.

1.2 Setup OAI gNB-UE 🗼

  • Deployment Plan: F1 Split RAN

    CU Interface

    Purpose Interface IP Address Port
    N1 & N2 ens33 172.21.6.90/22 38472
    N3 ens33 172.21.8.91/22 2152
    F1 ens33 192.168.1.5/27 –-

    DU & RU Interface

    Purpose Interface IP Address Port
    F1 ens33 192.168.1.6/27 38472

    UE Interface

    Purpose Interface IP Address Port
    Net1 ens33 192.168.1.7/24 38472
  • Configmap CU File Templete

    ​​​​ cu.conf: | ​​​​ Active_gNBs = ( "{{ .Values.config.cuName}}"); ​​​​ # Asn1_verbosity, choice in: none, info, annoying ​​​​ Asn1_verbosity = "none"; ​​​​ Num_Threads_PUSCH = 8; ​​​​ gNBs = ​​​​ ( ​​​​ { ​​​​ ////////// Identification parameters: ​​​​ gNB_ID = 0xe04; ////////// Kalau bang nino: 0xe01 ​​​​ # cell_type = "CELL_MACRO_GNB"; ​​​​ gNB_name = "{{ .Values.config.cuName}}"; ​​​​ // Tracking area code, 0x0000 and 0xfffe are reserved values ​​​​ tracking_area_code = {{ .Values.config.tac}}; ​​​​ plmn_list = ({ mcc = {{ .Values.config.mcc}}; mnc = {{ .Values.config.mnc}}; mnc_length = 2; snssaiList = ({ sst = {{ .Values.config.sst}}; }) }); ​​​​ nr_cellid = 12345678L; ​​​​ tr_s_preference = "f1"; ​​​​ local_s_if_name = "{{ .Values.config.f1IfName}}"; ​​​​ local_s_address = "{{ .Values.multus.f1Interface.IPadd}}"; ​​​​ remote_s_address = "127.0.0.1"; ​​​​ local_s_portc = 501; ​​​​ local_s_portd = {{ .Values.config.f1cuPort}}; ​​​​ remote_s_portc = 500; ​​​​ remote_s_portd = {{ .Values.config.f1duPort}}; ​​​​ # ------- SCTP definitions ​​​​ SCTP : ​​​​ { ​​​​ # Number of streams to use in input/output ​​​​ SCTP_INSTREAMS = 2; ​​​​ SCTP_OUTSTREAMS = 2; ​​​​ }; ​​​​ ////////// AMF parameters: ​​​​ amf_ip_address = ( { ipv4 = "{{ .Values.config.amfhost}}"; ​​​​ ipv6 = "192:168:30::17"; ​​​​ active = "yes"; ​​​​ preference = "ipv4"; ​​​​ } ​​​​ ); ​​​​ NETWORK_INTERFACES : ​​​​ { ​​​​ GNB_INTERFACE_NAME_FOR_NG_AMF = "{{ .Values.config.n2IfName}}"; ​​​​ GNB_IPV4_ADDRESS_FOR_NG_AMF = "{{ .Values.multus.n2Interface.IPadd}}"; ​​​​ GNB_INTERFACE_NAME_FOR_NGU = "{{ .Values.config.n3IfName}}"; ​​​​ GNB_IPV4_ADDRESS_FOR_NGU = "{{ .Values.multus.n3Interface.IPadd}}"; ​​​​ GNB_PORT_FOR_S1U = 2152; # Spec 2152 ​​​​ }; ​​​​ } ​​​​ ); ​​​​ ... ​​​​
  • Conf DU File

    ​​​​ du.conf: | ​​​​ Active_gNBs = ( "{{ .Values.config.duName}}"); ​​​​ # Asn1_verbosity, choice in: none, info, annoying ​​​​ Asn1_verbosity = "none"; ​​​​ gNBs = ​​​​ ( ​​​​ { ​​​​ ////////// Identification parameters: ​​​​ gNB_ID = 0xe05; ////////// Yang bang Nino: 0xe02 ​​​​ gNB_DU_ID = 0xe06; ​​​​ # cell_type = "CELL_MACRO_GNB"; ​​​​ gNB_name = "{{ .Values.config.duName}}"; ​​​​ // Tracking area code, 0x0000 and 0xfffe are reserved values ​​​​ tracking_area_code = {{ .Values.config.tac}} ; ​​​​ plmn_list = ({ mcc = {{ .Values.config.mcc}}; mnc = {{ .Values.config.mnc}}; mnc_length = 2; snssaiList = ({ sst = {{ .Values.config.sst}}; }) }); ​​​​ nr_cellid = 12345678L; ​​​​ ////////// Physical parameters: ​​​​ ​​​​ ... ​​​​ ... ​​​​ ... ​​​​ ​​​​ ////////// Physical parameters: ​​​​ # ------- SCTP definitions ​​​​ SCTP : ​​​​ { ​​​​ # Number of streams to use in input/output ​​​​ SCTP_INSTREAMS = 2; ​​​​ SCTP_OUTSTREAMS = 2; ​​​​ }; ​​​​ } ​​​​ ); ​​​​ MACRLCs = ( ​​​​ { ​​​​ num_cc = 1; ​​​​ tr_s_preference = "local_L1"; ​​​​ tr_n_preference = "f1"; ​​​​ local_n_if_name = "{{ .Values.config.f1IfName}}"; ​​​​ local_n_address = "{{ .Values.multus.f1Interface.IPadd}}"; ​​​​ remote_n_address = "{{ .Values.config.cuHost}}"; ​​​​ local_n_portc = 500; ​​​​ local_n_portd = {{ .Values.config.f1duPort}}; ​​​​ remote_n_portc = 501; ​​​​ remote_n_portd = {{ .Values.config.f1cuPort}}; ​​​​ pusch_TargetSNRx10 = 200; ​​​​ pucch_TargetSNRx10 = 200; ​​​​ } ​​​​ ); ​​​​ L1s = ( ​​​​ { ​​​​ num_cc = 1; ​​​​ tr_n_preference = "local_mac"; ​​​​ prach_dtx_threshold = 200; ​​​​ pucch0_dtx_threshold = 150; ​​​​ ofdm_offset_divisor = 8; #set this to UINT_MAX for offset 0 ​​​​ } ​​​​ ); ​​​​ RUs = ( ​​​​ { ​​​​ local_rf = "yes" ​​​​ nb_tx = 1 ​​​​ nb_rx = 1 ​​​​ att_tx = 0 ​​​​ att_rx = 0; ​​​​ bands = [78]; ​​​​ max_pdschReferenceSignalPower = -27; ​​​​ max_rxgain = 114; ​​​​ eNB_instances = [0]; ​​​​ #beamforming 1x4 matrix: ​​​​ bf_weights = [0x00007fff, 0x0000, 0x0000, 0x0000]; ​​​​ clock_src = "internal"; ​​​​ } ​​​​ ); ​​​​ THREAD_STRUCT = ( ​​​​ { ​​​​ #three config for level of parallelism "PARALLEL_SINGLE_THREAD", "PARALLEL_RU_L1_SPLIT", or "PARALLEL_RU_L1_TRX_SPLIT" ​​​​ parallel_config = "PARALLEL_SINGLE_THREAD"; ​​​​ #two option for worker "WORKER_DISABLE" or "WORKER_ENABLE" ​​​​ worker_config = "WORKER_ENABLE"; ​​​​ } ​​​​ ); ​​​​ rfsimulator: { ​​​​ serveraddr = "server"; ​​​​ serverport = 4043; ​​​​ options = (); #("saviq"); or/and "chanmod" ​​​​ modelname = "AWGN"; ​​​​ IQfile = "/tmp/rfsimulator.iqs" ​​​​ } ​​​​ log_config: { ​​​​ ... ​​​​ ... ​​​​ ... ​​​​ };
  • Helm Step Installation

    • Helm Configuration of OAI-CU and OAI-DU

      Very Important : To access internet in NR-UE the N6/SGI interface of UPF should be able access the internet.

      OAI-CU requires the ip-address or service name of AMF. In case in AMF multus is used and N1/N2 interface is bind to multus interface, then please provide AMF ip-address.

      Clone & see me for the detail CU/DU configuration

    • Deploy OAI-CU

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-cu-user-n oai-cu/ --namespace user-n
      

      Check if the oai-cu is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-cu-user-n --namespace user-n --timeout=3m
      
    • Deploy OAI-DU (RFSimulator)

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-du-user-n oai-du/ --namespace user-n
      

      Check if the oai-du is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-du-user-n --namespace user-n --timeout=3m
      
    • Validation CU > DU Connection

      To check if oai-du is connected to oai-cu, read the logs of cu and check F1 setup procedure was correct or not

      ​​​​​​​​kubectl logs --namespace user-n $(kubectl get pods --namespace user-n | grep oai-cu | awk '{print $1}') | grep 'Received F1 Setup Request'
      
      The output is similar to:
      ​​​​​​​​838801.974035 [NR_RRC] I Received F1 Setup Request from gNB_DU 3584 (oai-du-rfsim) on assoc_id 189
      
    • Validation CU > AMF Connection

      To check if oai-cu is connected to amf, read the logs of amf and check N2 setup procedure was correct or not,

      ​​​​​​​​kubectl logs --namespace core-network $(kubectl get pods --namespace core-network | grep oai-amf| awk '{print $1}') | grep 'Connected'
      
      The output is similar to:
      ​​​​​​​​Defaulted container "amf" out of: amf, init (init)
      ​​​​​​​​[2023-12-12 10:49:54.879] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:34.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:54.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:51:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      
    • Deploy OAI-NR-UE RFSimulator

      Configure the UE Chart

      Clone & see me for the detail OAI-NR-UE configuration

      Do helm install to start the deployment

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install nr-ue oai-nr-ue-user-n/ --namespace oai-gnb-ue
      

      Check if the oai-nr-ue is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-nr-ue-user-n --timeout=3m --namespace user-n
      
    • Validation UE got IP Address

      Now you are start reading the logs of amf, smf and other network function to understand the message flow. Once the pdu session establishment procedure is finished you will receive an ip-address. You can start performing some testing.

      check if the UE received an ip-address

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ifconfig oaitun_ue1 | grep -E '(^|\s)inet($|\s)' | awk '{print $2}'
      
      The output is similar to:
      ​​​​​​​​12.1.1.100
      
    • Performing E2E traffic testing

      Inside the nr-ue pod there is an extra tcdump container which can be use to perform traffic testing via iperf3 or

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ping -I oaitun_ue1 -c4 google.fr
      ​​​​​​​​PING google.fr (216.58.213.67) from 12.1.1.100 oaitun_ue1: 56(84) bytes of data.
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=1 ttl=117 time=27.0 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=2 ttl=117 time=22.3 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=3 ttl=117 time=24.1 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=4 ttl=117 time=22.7 ms
      
      ​​​​​​​​--- google.fr ping statistics ---
      ​​​​​​​​4 packets transmitted, 4 received, 0% packet loss, time 3003ms
      ​​​​​​​​rtt min/avg/max/mdev = 22.375/24.072/27.031/1.833 ms
      
      ​​​​​​​​## incase above doesn't work try with 8.8.8.8 instead of dns. If that works then probably you have't configure dns properly in SMF. 
      

2. Validation & Testing

2.1 Control Plane

This part is where I validate the end-to-end control plane connection between the OAI RAN components.

  • Testing Backhaul (NGAP)

    • SCTP Socket

      berhasil1

      Validate the port on CU pod, using netstat -Spn

    • Wireshark

      image

  • Testing Midhaul (F1AP)

    • SCTP Socket

      image

    • Wireshark

      image

  • Testing Fronthaul
    This build use simulator mode of the oai/openairinterface5g so one way to make sure that the fronthaul setup is correct is to check the port 4043 is on the LISTEN state and will be used by UE to connect into the gNB.

    berhasil2

    Parameter configured in gNB.du.conf

    ​​​​#... ​​​​rfsimulator: { ​​​​serveraddr = "server"; ​​​​ serverport = "4043"; ​​​​ options = (); #("saviq"); or/and "chanmod" ​​​​ modelname = "AWGN"; ​​​​ IQfile = "/tmp/rfsimulator.iqs" ​​​​} ​​​​#...
  • Testing Air Interface (Simulator)

    • nr-softmodem

      image alt

    • TCP Socket

      image

      The air interface connection between UE and RFsim is emulated through tcp socket, that's why you will have an established tcp connection between the nr-softmodem and nr-uesoftmodem if your RFsim is connected properly with the UEsim.

    • UE Attach Request & Response

      image alt

2.2 Data Plane

An end-to-end connection of 5G setup should shows a successfull PDU and GTP tunnel traffic.

  • RFSimulator
    image alt

  • PDU Session
    image

    A success attach procedure should shows that the core network successfully finished the UEContextSetupRequest from UE and give UE dataplane IP.

  • GTP Tunnel

    • Do routing in UE and UPF in the first place

      • UPF Routing Configuration

        image_2024-03-20_14-02-37

      • UE Routing Configuration

        image_2024-03-20_14-02-59

    • Ping test from UE simulator to UPF node through GTP Tunnel.
      image alt

    • Captured gtp traffic In Wireshark.
      image alt

2.3 Mon & Log

Monitoring CU - nrRRC_stats.log | Unattached UE

image

Monitoring CU - nrRRC_stats.log | Attached UE

image

Monitoring DU - nrL1_stats.log

image

Monitoring DU - nrMAC_stats.log

image

Monitoring UE - nrL1_UE_stats-0.log

image

Example UE Graph - nrL1_UE_stats-0.log

WhatsApp Image 2024-01-23 at 13.34.37_17132ecf

3. Use Cases 📳

3.1 Effective Resource Planning

Effective Resource Planning Usage for Users (NS Limit) w/ @arigints

  • Core Network

    ​​​​ari@node3:~$ k top pods -n core-network
    ​​​​NAME                                     CPU(cores)   MEMORY(bytes)
    ​​​​basic-mysql-569dbf6959-jcbtx             5m           193Mi
    ​​​​oai-amf-d8ff8dd4c-fb5qh                  19m          26Mi
    ​​​​oai-ausf-8568fbdbc4-44pmx                30m          19Mi
    ​​​​oai-nrf-5754745f69-ddbdw                 36m          18Mi
    ​​​​oai-smf-7897876bbc-27rnl                 11m          55Mi
    ​​​​oai-udm-fd4cfc587-gwwbp                  33m          20Mi
    ​​​​oai-udr-545ff7fb6d-jpl4w                 32m          24Mi
    ​​​​oai-upf-d66f7b659-5vsmd                  10m          196Mi
    ​​​​phpmyadmin-deployment-658867877d-5mh8x   1m           68Mi
    
  • Single

    ​​​ ari@node3:~$ k top pods -n user-n
    ​​​ NAME                         CPU(cores)   MEMORY(bytes)
    ​​​ oai-cu-6b96c97dd7-987st      1m           116Mi
    ​​​ oai-du-795877db8b-kpggd      1577m        1303Mi
    ​​​ oai-nr-ue-76c9866b57-xf668   1215m        533Mi
    
  • Multi DU

    ​​​ ari@node3:~$ k top pods -n user-n
    ​​​ NAME                               CPU(cores)   MEMORY(bytes)
    ​​​ oai-cu-mdu-cf7c86df-q7q7m          3m           116Mi
    ​​​ oai-du-mdu-1-5f4b59779f-vvzrg      1597m        1303Mi
    ​​​ oai-du-mdu-2-979d59d77-92zl5       1591m        1303Mi
    ​​​ oai-nr-ue-mdu-1-679cd477b6-7xwtb   966m         530Mi
    ​​​ oai-nr-ue-mdu-2-b77bf7d9b-nwkwd    952m         530Mi
    
  • Multi UE

    ​​​ ari@node3:~$ k top pods -n user-n
    ​​​ NAME                               CPU(cores)   MEMORY(bytes)
    ​​​ oai-cu-mue-7756669d95-dlql7        14m          116Mi
    ​​​ oai-du-mue-976b4cf78-bp8g5         1365m        1309Mi
    ​​​ oai-nr-ue-mue-1-6779c8cb54-vpc5t   646m         530Mi
    ​​​ oai-nr-ue-mue-2-69878cc8b-fzmt8    672m         529Mi
    
  • Raw Calculation

    • Core Network CPU and Memory Utilization

      First, we'll sum up the CPU and memory usage for all the components in the Core Network:

      ​​​​​​​​​​  Total CPU (m): 5+19+30+36+11+33+32+10+1=177m
      ​​​​​​​​​​  Total Memory (Mi): 193+26+19+18+55+20+24+196+68=619Mi
      

      Next, we'll calculate the percentage of CPU and memory each component uses relative to these totals.

    • Single CPU and Memory Utilization

      For the Single category:

      ​​​​​​​​​​  Total CPU (m): 1+1577+1215=2793m
      ​​​​​​​​​​  Total Memory (Mi): 116+1303+533=1952Mi
      
    • Multi DU CPU and Memory Utilization

      For the Multi DU category:

      ​​​​​​​​​​  Total CPU (m): 3+1597+1591+966+952=5109m
      ​​​​​​​​​​  Total Memory (Mi): 116+1303+1303+530+530=3782Mi
      
    • Multi UE CPU and Memory Utilization

      For the Multi UE category:

      ​​​​​​​​​​  Total CPU (m): 14+1365+646+672=2697m
      ​​​​​​​​​​  Total Memory (Mi): 116+1309+530+529=2484Mi
      
  • Planning for Reducing Resource
    image

3.2 Config-able RAN Parameter

image

  • CU Configuration

    ​​​​config:
    ​​​​  mountConfig: true         #If config file is mounted then please edit mount.conf in configmap.yaml properly 
    ​​​​  timeZone: "Asia/Jakarta"
    ​​​​  useAdditionalOptions: "--sa --log_config.global_log_options level,nocolor,time"
    ​​​​  # If mounting the configuration file then below parameters are not used
    ​​​​  cuName: "oai-cu"
    ​​​​  cuId: "0xe04"
    ​​​​  cellId: "12345678L"
    ​​​​  mcc: "208"   # check the information with AMF, SMF, UPF
    ​​​​  mnc: "99"    # check the information with AMF, SMF, UPF
    ​​​​  tac: "0xa001"     # check the information with AMF
    ​​​​  sst: "1"  #currently only 4 standard values are allowed 1,2,3,4
    ​​​​  amfhost: "172.21.6.94"  # amf ip-address or service-name oai-amf-svc or 172.21.6.94
    ​​​​  n2IfName: "n2"    # if multus.n2Interface.create is true then use n2
    ​​​​  n3IfName: "n3"   #if multus.n3Interface.create is true then use n3 or you can only use 1 interface n2 or eth0 
    ​​​​  f1IfName: "f1"   #if multus.f1Interface.create is true then use multus.f1Interface.Ipadd
    ​​​​  f1cuPort: "2152"   #2153 if using same interface for f1 and n3 else standard port 2152 should be use if f1 and n3 interface are different
    ​​​​  f1duPort: "2152"   #2153 if using same interface for f1 and n3 else standard port 2152 should be use if f1 and n3 interface are different
    
  • DU Configuration

    ​​​​config:
    ​​​​  mountConfig: true          #If config file is mounted then please edit mount.conf in templates/configmap.yaml properly 
    ​​​​  timeZone: "Asia/Jakarta"
    ​​​​  useAdditionalOptions: "--sa --rfsim -r 106 --numerology 1 -C 3619200000 --nokrnmod --log_config.global_log_options level,nocolor,time"
    ​​​​  gnbId: "0xe05"
    ​​​​  duName: "oai-du-rfsim"
    ​​​​  duId: "0xe06"
    ​​​​  cellId: "12345678L"
    ​​​​  mcc: "208"   # check the information with AMF, SMF, UPF
    ​​​​  mnc: "99"    # check the information with AMF, SMF, UPF
    ​​​​  tac: "0xa001"     # check the information with AMF
    ​​​​  sst: "1"  #currently only 4 standard values are allowed 1,2,3,4
    ​​​​  usrp: rfsim   #allowed values rfsim, b2xx, n3xx or x3xx
    ​​​​  f1IfName: "f1"   #if multus.f1Interface.create is true then use f1
    ​​​​  cuHost: "192.168.1.1" ## Ip-address or hostname
    ​​​​  f1cuPort: "2152"   #2153 if using same interface for f1 and n3 else standard port 2152 should be use if f1 and n3 interface are different
    ​​​​  f1duPort: "2152"   #2153 if using same interface for f1 and n3 else standard port 2152 should be use if f1 and n3 interface are different   
    
  • RU Configuration

    ​​​​////////// Physical parameters:
    
    ​​​​    min_rxtxtime                                              = 6;
    ​​​​    force_256qam_off = 1;
    
    ​​​​    servingCellConfigCommon = (
    ​​​​    {
    ​​​​  #spCellConfigCommon
    
    ​​​​      physCellId                                                    = 0;
    
    ​​​​#  downlinkConfigCommon
    ​​​​    #frequencyInfoDL
    ​​​​      # this is 3600 MHz + 43 PRBs@30kHz SCS (same as initial BWP)
    ​​​​      absoluteFrequencySSB                                          = 641280;
    ​​​​      dl_frequencyBand                                                 = 78;
    ​​​​      # this is 3600 MHz
    ​​​​      dl_absoluteFrequencyPointA                                       = 640008;
    ​​​​      #scs-SpecificCarrierList
    ​​​​        dl_offstToCarrier                                              = 0;
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​        dl_subcarrierSpacing                                           = 1;
    ​​​​        dl_carrierBandwidth                                            = 106;
    ​​​​      #initialDownlinkBWP
    ​​​​      #genericParameters
    ​​​​        # this is RBstart=27,L=48 (275*(L-1))+RBstart
    ​​​​        initialDLBWPlocationAndBandwidth                               = 28875; # 6366 12925 12956 28875 12952
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​        initialDLBWPsubcarrierSpacing                                           = 1;
    ​​​​      #pdcch-ConfigCommon
    ​​​​        initialDLBWPcontrolResourceSetZero                              = 12;
    ​​​​        initialDLBWPsearchSpaceZero                                             = 0;
    
    ​​​​  #uplinkConfigCommon 
    ​​​​      #frequencyInfoUL
    ​​​​      ul_frequencyBand                                                 = 78;
    ​​​​      #scs-SpecificCarrierList
    ​​​​      ul_offstToCarrier                                              = 0;
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​      ul_subcarrierSpacing                                           = 1;
    ​​​​      ul_carrierBandwidth                                            = 106;
    ​​​​      pMax                                                          = 20;
    ​​​​      #initialUplinkBWP
    ​​​​      #genericParameters
    ​​​​        initialULBWPlocationAndBandwidth                            = 28875;
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​        initialULBWPsubcarrierSpacing                                           = 1;
    ​​​​      #rach-ConfigCommon
    ​​​​        #rach-ConfigGeneric
    ​​​​          prach_ConfigurationIndex                                  = 98;
    ​​​​#prach_msg1_FDM
    ​​​​#0 = one, 1=two, 2=four, 3=eight
    ​​​​          prach_msg1_FDM                                            = 0;
    ​​​​          prach_msg1_FrequencyStart                                 = 0;
    ​​​​          zeroCorrelationZoneConfig                                 = 13;
    ​​​​          preambleReceivedTargetPower                               = -96;
    ​​​​#preamblTransMax (0...10) = (3,4,5,6,7,8,10,20,50,100,200)
    ​​​​          preambleTransMax                                          = 6;
    ​​​​#powerRampingStep
    ​​​​# 0=dB0,1=dB2,2=dB4,3=dB6
    ​​​​        powerRampingStep                                            = 1;
    ​​​​#ra_ReponseWindow
    ​​​​#1,2,4,8,10,20,40,80
    ​​​​        ra_ResponseWindow                                           = 4;
    ​​​​#ssb_perRACH_OccasionAndCB_PreamblesPerSSB_PR
    ​​​​#1=oneeighth,2=onefourth,3=half,4=one,5=two,6=four,7=eight,8=sixteen
    ​​​​        ssb_perRACH_OccasionAndCB_PreamblesPerSSB_PR                = 4;
    ​​​​#oneHalf (0..15) 4,8,12,16,...60,64
    ​​​​        ssb_perRACH_OccasionAndCB_PreamblesPerSSB                   = 14;
    ​​​​#ra_ContentionResolutionTimer
    ​​​​#(0..7) 8,16,24,32,40,48,56,64
    ​​​​        ra_ContentionResolutionTimer                                = 7;
    ​​​​        rsrp_ThresholdSSB                                           = 19;
    ​​​​#prach-RootSequenceIndex_PR
    ​​​​#1 = 839, 2 = 139
    ​​​​        prach_RootSequenceIndex_PR                                  = 2;
    ​​​​        prach_RootSequenceIndex                                     = 1;
    ​​​​        # SCS for msg1, can only be 15 for 30 kHz < 6 GHz, takes precendence over the one derived from prach-ConfigIndex
    ​​​​        #  
    ​​​​        msg1_SubcarrierSpacing                                      = 1,
    ​​​​# restrictedSetConfig
    ​​​​# 0=unrestricted, 1=restricted type A, 2=restricted type B
    ​​​​        restrictedSetConfig                                         = 0,
    
    ​​​​        msg3_DeltaPreamble                                          = 1;
    ​​​​        p0_NominalWithGrant                                         =-90;
    
    ​​​​# pucch-ConfigCommon setup :
    ​​​​# pucchGroupHopping
    ​​​​# 0 = neither, 1= group hopping, 2=sequence hopping
    ​​​​        pucchGroupHopping                                           = 0;
    ​​​​        hoppingId                                                   = 40;
    ​​​​        p0_nominal                                                  = -90;
    ​​​​# ssb_PositionsInBurs_BitmapPR
    ​​​​# 1=short, 2=medium, 3=long
    ​​​​      ssb_PositionsInBurst_PR                                       = 2;
    ​​​​      ssb_PositionsInBurst_Bitmap                                   = 1;
    
    ​​​​# ssb_periodicityServingCell
    ​​​​# 0 = ms5, 1=ms10, 2=ms20, 3=ms40, 4=ms80, 5=ms160, 6=spare2, 7=spare1 
    ​​​​      ssb_periodicityServingCell                                    = 2;
    
    ​​​​# dmrs_TypeA_position
    ​​​​# 0 = pos2, 1 = pos3
    ​​​​      dmrs_TypeA_Position                                           = 0;
    
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​      subcarrierSpacing                                             = 1;
    
    
    ​​​​  #tdd-UL-DL-ConfigurationCommon
    ​​​​# subcarrierSpacing
    ​​​​# 0=kHz15, 1=kHz30, 2=kHz60, 3=kHz120  
    ​​​​      referenceSubcarrierSpacing                                    = 1;
    ​​​​      # pattern1 
    ​​​​      # dl_UL_TransmissionPeriodicity
    ​​​​      # 0=ms0p5, 1=ms0p625, 2=ms1, 3=ms1p25, 4=ms2, 5=ms2p5, 6=ms5, 7=ms10
    ​​​​      dl_UL_TransmissionPeriodicity                                 = 6;
    ​​​​      nrofDownlinkSlots                                             = 7;
    ​​​​      nrofDownlinkSymbols                                           = 6;
    ​​​​      nrofUplinkSlots                                               = 2;
    ​​​​      nrofUplinkSymbols                                             = 4;
    
    ​​​​      ssPBCH_BlockPower                                             = -25;
    ​​​​      }
    
    ​​​​  );
    
  • UE Configuration

    ​​​​config:
    ​​​​  timeZone: "Asia/Jakarta"
    ​​​​  rfSimServer: "192.168.1.2"    # ip-address of rfsim or service name oai-gnb or oai-du
    ​​​​  fullImsi: "208990100001101"       # make sure all the below entries are present in the subscriber database
    ​​​​  fullKey: "a92d71ed92f43ebae1720f34fd2e6966" 
    ​​​​  opc: "C42449363BBAD02B66D16BC975D77CC1"
    ​​​​  dnn: "oai"
    ​​​​  sst: "1"                     # configure according to gnb and amf, smf and upf 
    ​​​​  sd: "FFFFF4"
    ​​​​  usrp: "rfsim"            # allowed rfsim, b2xx, n3xx, x3xx
    ​​​​  useAdditionalOptions: "--sa --rfsim -r 106 --numerology 1 -C 3619200000 --nokrnmod --log_config.global_log_options level,nocolor,time"
    

3.3 IP Configuration Planning

  • System Topologi

    Capstone Related-IP Planning.drawio(1)(1)

    Each user has 3 stages. First they have to configure the Single DU & UE, while the other stages are down (Replicaset equal to 0). Then after all the parameters on stages 1 complete, users can go to the next stages to complete the training.

  • Subnet IP for User N

    • Multi DU

      Purpose Interface IP Address Port
      DU 1 F1 172.21.6.90/22 38472
      DU 2 F1 172.21.8.91/22 2152
      UE 1 Net1 192.168.1.5/27 –-
      UE 2 Net1 192.168.1.5/27 –-
    • Multi UE

      Purpose Interface IP Address Port
      DU 1 F1 172.21.6.90/22 38472
      UE 1 Net1 192.168.1.5/27 –-
      UE 2 Net1 192.168.1.5/27 –-
  • Restriction IP to Access

    Capstone Related-Bridge

    And using /28 block IP per users. Or using VLAN but still TBD.

3.4 Multiple DU & UE 📱

image

  • Helm Step Installation

    • Helm Configuration of OAI-CU and OAI-DU

      Very Important : To access internet in NR-UE the N6/SGI interface of UPF should be able access the internet.

      OAI-CU requires the ip-address or service name of AMF. In case in AMF multus is used and N1/N2 interface is bind to multus interface, then please provide AMF ip-address.

      Clone & see me for the detail CU/DU configuration

    • Deploy OAI-CU

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-cu-user-n oai-cu/ --namespace user-n
      

      Check if the oai-cu is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-cu-user-n --namespace user-n --timeout=3m
      
    • Deploy OAI-DU (RFSimulator)

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-du-user-n oai-du/ --namespace user-n
      

      Check if the oai-du is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-du-user-n --namespace user-n --timeout=3m
      
    • Validation CU > DU Connection

      To check if oai-du is connected to oai-cu, read the logs of cu and check F1 setup procedure was correct or not

      ​​​​​​​​kubectl logs --namespace user-n $(kubectl get pods --namespace user-n | grep oai-cu | awk '{print $1}') | grep 'Received F1 Setup Request'
      
      The output is similar to:
      ​​​​​​​​838801.974035 [NR_RRC] I Received F1 Setup Request from gNB_DU 3584 (oai-du-rfsim) on assoc_id 189
      
    • Validation CU > AMF Connection

      To check if oai-cu is connected to amf, read the logs of amf and check N2 setup procedure was correct or not,

      ​​​​​​​​kubectl logs --namespace core-network $(kubectl get pods --namespace core-network | grep oai-amf| awk '{print $1}') | grep 'Connected'
      
      The output is similar to:
      ​​​​​​​​Defaulted container "amf" out of: amf, init (init)
      ​​​​​​​​[2023-12-12 10:49:54.879] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:34.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:54.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:51:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      
    • Deploy OAI-NR-UE RFSimulator

      Configure the UE Chart

      Clone & see me for the detail OAI-NR-UE configuration

      Do helm install to start the deployment

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install nr-ue oai-nr-ue-user-n/ --namespace oai-gnb-ue
      

      Check if the oai-nr-ue is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-nr-ue-user-n --timeout=3m --namespace user-n
      
    • Validation UE got IP Address

      Now you are start reading the logs of amf, smf and other network function to understand the message flow. Once the pdu session establishment procedure is finished you will receive an ip-address. You can start performing some testing.

      check if the UE received an ip-address

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ifconfig oaitun_ue1 | grep -E '(^|\s)inet($|\s)' | awk '{print $2}'
      
      The output is similar to:
      ​​​​​​​​12.1.1.100
      
    • Performing E2E traffic testing

      Inside the nr-ue pod there is an extra tcdump container which can be use to perform traffic testing via iperf3 or

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ping -I oaitun_ue1 -c4 google.fr
      ​​​​​​​​PING google.fr (216.58.213.67) from 12.1.1.100 oaitun_ue1: 56(84) bytes of data.
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=1 ttl=117 time=27.0 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=2 ttl=117 time=22.3 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=3 ttl=117 time=24.1 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=4 ttl=117 time=22.7 ms
      
      ​​​​​​​​--- google.fr ping statistics ---
      ​​​​​​​​4 packets transmitted, 4 received, 0% packet loss, time 3003ms
      ​​​​​​​​rtt min/avg/max/mdev = 22.375/24.072/27.031/1.833 ms
      
      ​​​​​​​​## incase above doesn't work try with 8.8.8.8 instead of dns. If that works then probably you have't configure dns properly in SMF. 
      

3.5 Multiple UE - Single DU 💾

image

  • Helm Step Installation

    • Helm Configuration of OAI-CU and OAI-DU

      Very Important : To access internet in NR-UE the N6/SGI interface of UPF should be able access the internet.

      OAI-CU requires the ip-address or service name of AMF. In case in AMF multus is used and N1/N2 interface is bind to multus interface, then please provide AMF ip-address.

      Clone & see me for the detail CU/DU configuration

    • Deploy OAI-CU

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-cu-user-n oai-cu/ --namespace user-n
      

      Check if the oai-cu is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-cu-user-n --namespace user-n --timeout=3m
      
    • Deploy OAI-DU (RFSimulator)

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install oai-du-user-n oai-du/ --namespace user-n
      

      Check if the oai-du is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-du-user-n --namespace user-n --timeout=3m
      
    • Validation CU > DU Connection

      To check if oai-du is connected to oai-cu, read the logs of cu and check F1 setup procedure was correct or not

      ​​​​​​​​kubectl logs --namespace user-n $(kubectl get pods --namespace user-n | grep oai-cu | awk '{print $1}') | grep 'Received F1 Setup Request'
      
      The output is similar to:
      ​​​​​​​​838801.974035 [NR_RRC] I Received F1 Setup Request from gNB_DU 3584 (oai-du-rfsim) on assoc_id 189
      
    • Validation CU > AMF Connection

      To check if oai-cu is connected to amf, read the logs of amf and check N2 setup procedure was correct or not,

      ​​​​​​​​kubectl logs --namespace core-network $(kubectl get pods --namespace core-network | grep oai-amf| awk '{print $1}') | grep 'Connected'
      
      The output is similar to:
      ​​​​​​​​Defaulted container "amf" out of: amf, init (init)
      ​​​​​​​​[2023-12-12 10:49:54.879] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:34.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:50:54.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      ​​​​​​​​[2023-12-12 10:51:14.880] [amf_app] [info] |      1      |    Connected     |        0xe000        |       oai-cu           |               001, 01              | 
      
    • Deploy OAI-NR-UE RFSimulator

      Configure the UE Chart

      Clone & see me for the detail OAI-NR-UE configuration

      Do helm install to start the deployment

      ​​​​​​​​cd user_n/oai-e2e/
      ​​​​​​​​helm install nr-ue oai-nr-ue-user-n/ --namespace oai-gnb-ue
      

      Check if the oai-nr-ue is started

      ​​​​​​​​kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=oai-nr-ue-user-n --timeout=3m --namespace user-n
      
    • Validation UE got IP Address

      Now you are start reading the logs of amf, smf and other network function to understand the message flow. Once the pdu session establishment procedure is finished you will receive an ip-address. You can start performing some testing.

      check if the UE received an ip-address

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ifconfig oaitun_ue1 | grep -E '(^|\s)inet($|\s)' | awk '{print $2}'
      
      The output is similar to:
      ​​​​​​​​12.1.1.100
      
    • Performing E2E traffic testing

      Inside the nr-ue pod there is an extra tcdump container which can be use to perform traffic testing via iperf3 or

      ​​​​​​​​kubectl exec -it -n user-n -c nr-ue $(kubectl get pods -n user-n | grep oai-nr-ue-user-n | awk '{print $1}') -- ping -I oaitun_ue1 -c4 google.fr
      ​​​​​​​​PING google.fr (216.58.213.67) from 12.1.1.100 oaitun_ue1: 56(84) bytes of data.
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=1 ttl=117 time=27.0 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=2 ttl=117 time=22.3 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=3 ttl=117 time=24.1 ms
      ​​​​​​​​64 bytes from par21s18-in-f3.1e100.net (216.58.213.67): icmp_seq=4 ttl=117 time=22.7 ms
      
      ​​​​​​​​--- google.fr ping statistics ---
      ​​​​​​​​4 packets transmitted, 4 received, 0% packet loss, time 3003ms
      ​​​​​​​​rtt min/avg/max/mdev = 22.375/24.072/27.031/1.833 ms
      
      ​​​​​​​​## incase above doesn't work try with 8.8.8.8 instead of dns. If that works then probably you have't configure dns properly in SMF. 
      

Reference

  1. Core Network Configuration: https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed/-/blob/v2.0.1/docs/CONFIGURATION.md?ref_type=tags
  2. RAN Configuration: https://gitlab.eurecom.fr/oai/cn5g/oai-cn5g-fed/-/blob/v2.0.1/docs/DEPLOY_SA5G_HC.md?ref_type=tags#71-deploy-oai-cu-cp-oai-cu-up-and-oai-du