---
# System prepended metadata

title: ORANSC Deployment

---

# ORANSC deployment
###### tags: `ric`, `oransc`, `osc`

## Tutorial for installing RIC
https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-ric-dep/en/latest/installation-guides.html#installing-near-realtime-ric-in-ric-cluster
Use release H, ubuntu 20.04

## 1. Topology:
Master node: `ssh master@172.27.2.51`

## 2. Undeploy RIC platform


One approach is to reset the whole k8s cluster: https://saturncloud.io/blog/resetting-your-kubernetes-cluster-a-guide/

Or just undeploy RIC platform:
```bash=
sudo su
cd /root/dep/bin
./undeploy-ric-platform
```
## 3. Deploy RIC platform
https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-ric-dep/en/latest/installation-guides.html#installing-near-realtime-ric-in-ric-cluster

```===bash
To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.27.2.53:6443 --token hxicyu.r07zaq17423q0tve \
    --discovery-token-ca-cert-hash sha256:56e0195ca885a8ef9d4f460f59139c1c31a1aee4183b5637aa464014ac032ab6
+ cd /root
+ rm -rf .kube
+ mkdir -p .kube
+ cp -i /etc/kubernetes/admin.conf /root/.kube/config
+ chown root:root /root/.kube/config
+ export KUBECONFIG=/root/.kube/config
+ KUBECONFIG=/root/.kube/config
+ echo KUBECONFIG=/root/.kube/config
+ kubectl get pods --all-namespaces
No resources found
+ kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/v0.18.1/Documentation/kube-flannel.yml

```

## 4. To expose External IP
```

kubectl patch svc service-ricplt-e2term-sctp-alpha -n ricplt -p '{"spec":{"externalIPs":["10.5.0.3"]}}'
```


## 5. Usefull commands 
```bas=
kubectl get pod -A -o wide
kubectl get svc -A -o wide
kubectl get deployment -A -o wide

# Get IP of E2 Manager HTTP
kubectl get -n ricplt service/service-ricplt-e2mgr-http -o jsonpath='{.spec.clusterIP}' 
# Get IP of Submgr
kubectl get -n ricplt service/service-ricplt-submgr-http -o jsonpath='{.spec.clusterIP}'
# Get IP of appmgr
kubectl get -n ricplt service/service-ricplt-appmgr-http -o jsonpath='{.spec.clusterIP}'

# Get E2Termination logs
kubectl logs -f -n ricplt deployment/deployment-ricplt-e2term-alpha --tail=100
# Get submgr logs
kubectl logs -f -n ricplt deployment/deployment-ricplt-submgr --tail=100
# Get appmgr logs
kubectl logs -f -n ricplt deployment/deployment-ricplt-appmgr --tail=100
# Get e2mgr logs
kubectl logs -f -n ricplt deployment/deployment-ricplt-e2mgr --tail=100
# Get rtmgr logs
kubectl logs -f -n ricplt deployment/deployment-ricplt-rtmgr --tail=100
# Get kpimon-go logs
kubectl logs -f -n ricxapp deployment/ricxapp-kpimon-go --tail=100
```
## 6. Useful commands with appmgr
Ref: https://github.com/o-ran-sc/ric-plt-appmgr
```bas=
export APPMGR_IP=$(kubectl get -n ricplt service/service-ricplt-appmgr-http -o jsonpath='{.spec.clusterIP}')

# Query status of all xApps
curl -H "Content-Type: application/json" http://$APPMGR_IP:8080/ric/v1/xapps
# Query status of kpimon-go
curl -H "Content-Type: application/json" http://$APPMGR_IP:8080/ric/v1/xapps/xappkpimon-go
# List all subscriptions
curl -H "Content-Type: application/json" http://$APPMGR_IP:8080/ric/v1/subscriptions
# Get a specific subscription by Id
curl -H "Content-Type: application/json" http://$APPMGR_IP:8080/ric/v1/subscriptions/{ID}
```


## 7. Deploy kpimon-go-xapp

### 7.1 Before install xApp, check following conditions
All pods are ready except for tiller.
```bas=

root@osc-single:/home/fcp/ric-dep/bin# kubectl get pod -A
NAMESPACE     NAME                                                         READY   STATUS      RESTARTS   AGE
kube-system   coredns-5644d7b6d9-bf2xf                                     1/1     Running     0          2d18h
kube-system   coredns-5644d7b6d9-qqfps                                     1/1     Running     0          2d18h
kube-system   etcd-osc-single                                              1/1     Running     0          2d18h
kube-system   kube-apiserver-osc-single                                    1/1     Running     7          2d18h
kube-system   kube-controller-manager-osc-single                           1/1     Running     6          2d18h
kube-system   kube-flannel-ds-ndqqb                                        1/1     Running     0          2d18h
kube-system   kube-proxy-29lfb                                             1/1     Running     0          2d18h
kube-system   kube-scheduler-osc-single                                    1/1     Running     5          2d18h
ricinfra      deployment-tiller-ricxapp-68f777c4d4-qhv9f                   1/1     Running     0          4m19s
ricinfra      tiller-secret-generator-c2z42                                0/1     Completed   0          4m19s
ricplt        deployment-ricplt-a1mediator-56fdfd447d-c6hsn                1/1     Running     0          3m2s
ricplt        deployment-ricplt-alarmmanager-58b75c6bc9-c6mtv              1/1     Running     0          2m10s
ricplt        deployment-ricplt-appmgr-77986c9cbb-gdm7c                    1/1     Running     0          3m53s
ricplt        deployment-ricplt-e2mgr-577c8958d9-twgpj                     1/1     Running     2          3m28s
ricplt        deployment-ricplt-e2term-alpha-75d76b7f58-5nj6q              1/1     Running     0          3m15s
ricplt        deployment-ricplt-o1mediator-5c5cf8689c-hvr25                1/1     Running     0          2m23s
ricplt        deployment-ricplt-rtmgr-7454cfb4df-khnf2                     1/1     Running     2          3m40s
ricplt        deployment-ricplt-submgr-78489f94f-l9sf9                     1/1     Running     0          2m49s
ricplt        deployment-ricplt-vespamgr-84f7d87dfb-cbvph                  1/1     Running     0          2m36s
ricplt        r4-infrastructure-kong-7995f4679b-vjxfr                      2/2     Running     2          4m19s
ricplt        r4-infrastructure-prometheus-alertmanager-5798b78f48-mdtjt   2/2     Running     0          4m19s
ricplt        r4-infrastructure-prometheus-server-c8ddcfdf5-d6mlr          1/1     Running     0          4m19s
ricplt        statefulset-ricplt-dbaas-server-0                            1/1     Running     0          4m6s

```
Routing Manager updates all routes correctly (a1 currently not correct but we are not using now)
```bash!
{"ts":1695700003752,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"ep: &{10.107.232.231:38000 E2TERMINST platform 10.107.232.231 38000 [] [] [] <nil> false false 4}"}
{"ts":1695700003752,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"rpe.generatePlatformRoutes invoked"}
{"ts":1695700003752,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Endpoint: SUBMAN, xAppType: platform"}
{"ts":1695700003752,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Endpoint: E2MAN, xAppType: platform"}
{"ts":1695700003752,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Endpoint: A1MEDIATOR, xAppType: platform"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Endpoint: E2TERMINST, xAppType: platform"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"rmr.GeneratePolicies returns: [newrt|start\n mse|12010,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12020,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12011|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12021|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12012|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12022|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12023|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|10060,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10070,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10071,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10360,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10081,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10082,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10371,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10372,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|12001|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1100|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10061|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10062|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10071|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10361|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10362|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10370|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12007|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10020|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1102|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|20012|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|20011|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|12030|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1090,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|1101,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12002,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12003,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12031,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12032,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12072,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12005,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n newrt|end\n meid_map|start\n meid_map|end|0\n]"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"rmr.GeneratePolicies returns: &{[] map[10.107.232.231:38000:{E2TERMINST 10.107.232.231:38000 []}] [] [{SUBMAN service-ricplt-submgr-rmr.ricplt 4560} {E2MAN service-ricplt-e2mgr-rmr.ricplt 3801} {A1MEDIATOR service-ricplt-a1mediator-rmr.ricplt 4562}]}"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Invoked: sbi.DistributeAll"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"args: [newrt|start\n mse|12010,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12020,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12011|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12021|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12012|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12022|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12023|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|10060,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10070,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10071,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10360,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10081,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10082,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10371,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10372,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|12001|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1100|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10061|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10062|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10071|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10361|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10362|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10370|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12007|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10020|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1102|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|20012|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|20011|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|12030|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1090,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|1101,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12002,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12003,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12031,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12032,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12072,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12005,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n newrt|end\n meid_map|start\n meid_map|end|0\n]"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Push policy to endpoint: 10.107.232.231:38000"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Invoked send_data to endpoint: 10.107.232.231:38000 call_id: 49"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Push policy to endpoint: service-ricplt-e2mgr-rmr.ricplt:3801"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Invoked send_data to endpoint: service-ricplt-e2mgr-rmr.ricplt:3801 call_id: 49"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Push policy to endpoint: service-ricplt-a1mediator-rmr.ricplt:4562"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Invoked send_data to endpoint: service-ricplt-a1mediator-rmr.ricplt:4562 call_id: 49"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Push policy to endpoint: service-ricplt-submgr-rmr.ricplt:4560"}
{"ts":1695700003753,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Invoked send_data to endpoint: service-ricplt-submgr-rmr.ricplt:4560 call_id: 49"}
{"ts":1695700003754,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Update Routes to Endpoint: 10.107.232.231:38000 successful, call_id: 49, Payload length: 2508, Route Update Status: OK(# of Entries:48"}
{"ts":1695700003754,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Update Routes to Endpoint: service-ricplt-e2mgr-rmr.ricplt:3801 successful, call_id: 49, Payload length: 2508, Route Update Status: OK(# of Entries:48"}
{"ts":1695700003757,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Update Routes to Endpoint: service-ricplt-submgr-rmr.ricplt:4560 successful, call_id: 49, Payload length: 2508, Route Update Status: OK(# of Entries:48"}
{"ts":1695700003757,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T03:46:43"},"msg":"Update Routes to Endpoint: service-ricplt-a1mediator-rmr.ricplt:4562 successful, call_id: 49, Payload length: 2508, Route Update Status: OK(# of Entries:48"}
```



* Install dms_cli
```bash=
#Git clone appmgr
git clone "https://gerrit.o-ran-sc.org/r/ric-plt/appmgr"

#Change dir to xapp_onboarder
cd appmgr/xapp_orchestrater/dev/xapp_onboarder

#If pip3 is not installed, install using the following command
yum install python3-pip

#In case dms_cli binary is already installed, it can be uninstalled using following command
pip3 uninstall xapp_onboarder

#Install xapp_onboarder using following command
pip3 install ./
```


* Install  and Deploy kpimon-go
Repo: https://github.com/o-ran-sc/ric-app-kpimon-go/blob/master/deploy/config.json
Ref: https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-ric-dep/en/latest/installation-guides.html#installing-near-realtime-ric-in-ric-cluster
```bas=
# Run only one time
mkdir /home/root/xapp_charts
docker run --rm -u 0 -it -d -p 8090:8080 -e DEBUG=1 -e STORAGE=local -e STORAGE_LOCAL_ROOTDIR=/charts -v /home/root/xapp_charts:/charts chartmuseum/chartmuseum:latest
export CHART_REPO_URL=http://0.0.0.0:8090
```
If you have run the first command in other terminal, just export the environment variable in the current terminal for `dms_cli` command.

### 7.2 Rebuild image for kpimon-go service
```bash=
git clone https://github.com/o-ran-sc/ric-app-kpimon-go.git
```
Add "name"="kpimon-go" to config-file.yaml, then rebuild image
```bash=

# Build image with the image name as in deploy/config.json, for ex:
docker build . -t nexus3.o-ran-sc.org:10004/o-ran-sc/ric-app-kpimon-go:1.0.1

# Onboard
cd ric-app-kpimon-go/deploy
dms_cli onboard --config_file_path=config.json --shcema_file_path=schema.json
# Deploy
dms_cli install --xapp_chart_name=kpimon-go --version=2.0.1 --namespace=ricxapp
# To undeploy later
# dms_cli uninstall --xapp_chart_name=kpimon-go --version=2.0.1 --namespace=ricxapp
```
* Register KPIMON with APPMGR
Create `register.json` as below file. 
```bas=
{
  "appName": "kpimon-go",
  "appVersion": "2.0.1",
  "configPath": "",
  "appInstanceName": "kpimon-go",
  "httpEndpoint": "<IP of KPIMON-http service>:8080",
  "rmrEndpoint": "<IP of KPIMON-rmr service>:4560",
  "config": "{\"xapp_name\":\"kpimon-go\",\"version\":\"2.0.1\",\"containers\":[{\"name\":\"kpimon-go\",\"image\":{\"registry\":\"nexus3.o-ran-sc.org:10004\",\"name\":\"o-ran-sc/ric-app-kpimon-go\",\"tag\":\"1.0.1\"}}],\"messaging\":{\"ports\":[{\"name\":\"http\",\"container\":\"kpimon-go\",\"port\":8080,\"description\":\"http service\"},{\"name\":\"rmr-data\",\"container\":\"kpimon-go\",\"port\":4560,\"rxMessages\":[\"RIC_SUB_RESP\",\"RIC_INDICATION\"],\"txMessages\":[\"RIC_SUB_REQ\"],\"policies\":[],\"description\":\"rmr receive data port for xappkpimon\"},{\"name\":\"rmr-route\",\"container\":\"kpimon-go\",\"port\":4561,\"description\":\"rmr route port for xappkpimon\"}]},\"rmr\":{\"protPort\":\"tcp:4560\",\"maxSize\":2072,\"numWorkers\":1,\"rxMessages\":[\"RIC_SUB_RESP\",\"RIC_INDICATION\"],\"txMessages\":[\"RIC_SUB_REQ\"],\"policies\":[]}}"
}
```
Note to change `<IP of KPIMON-http service>` and `<IP of KPIMON-rmr service>` by your deployed IPs (via `kubectl get service -n ricxapp`). For example:
```==bash===
ricxapp       service-ricxapp-kpimon-go-http              ClusterIP   10.106.125.252   <none>        8080/TCP                          34s
ricxapp       service-ricxapp-kpimon-go-rmr               ClusterIP   10.97.221.160    <none>        4560/TCP,4561/TCP                 34s
```

```
export APPMGR_IP=$(kubectl get -n ricplt service/service-ricplt-appmgr-http -o jsonpath='{.spec.clusterIP}')
curl -v -X POST "http://$APPMGR_IP:8080/ric/v1/register" -H "accept: application/json" -H "Content-Type: application/json" -d "@register.json"
```

* Expected: KPIMON xapp always
```bas=
kubectl logs -n ricxapp services/service-ricxapp-kpimon-go-rmr
.....
{"ts":1695477847088,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-23T14:04:07"},"msg":"Application='' is not ready yet, waiting ..."}
```

Reason: KPIMON does not register to appmgr properly (Reference: https://wiki.o-ran-sc.org/display/SIM/E2+Simulator). 
To fix:
```bash=
export KPIMON_GO_HTTP_IP=$(kubectl get -n ricxapp service/service-ricxapp-kpimon-go-http -o jsonpath='{.spec.clusterIP}')
curl -X GET http://$KPIMON_GO_HTTP_IP:8080/ric/v1/health/ready
```
* Expected logs:
KPIMON-GO log
```bas=
2023/09/26 09:52:13 Indication Header format = 1
 parsing for cell metrics

 No of cell = 1

 No of cell metrics = 11
2023/09/26 09:52:13 no_of_cell_metrics != v.NumField()
2023/09/26 09:52:13 RIC Indication message from {gnb_001_001_123456} received
2023/09/26 09:52:13 Indication Header format = 1
2023/09/26 09:52:13 no_of_ue_metrics != v.NumField()
 parsing for UE metrics
 No of ue metrics = 12

 No of ue= 1
{"ts":1695721934939,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T09:52:14"},"msg":"Message received: name=RIC_INDICATION meid=gnb_001_001_123456 subId=4 txid=   370565267051 len=119"}
{"ts":1695721934939,"crit":"INFO","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T09:52:14"},"msg":"Message received: name=RIC_INDICATION meid=gnb_001_001_123456 subId=4 txid=   370565267052 len=529"}
{"ts":1695721934939,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T09:52:14"},"msg":"Received message type: 12050"}
{"ts":1695721934939,"crit":"DEBUG","id":"kpimon","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T09:52:14"},"msg":"Received message type: 12050"}
 parsing for cell metrics

 No of cell = 1

 No of cell metrics = 11
2023/09/26 09:52:14 RIC Indication message from {gnb_001_001_123456} received
2023/09/26 09:52:14 Indication Header format = 1
2023/09/26 09:52:14 no_of_cell_metrics != v.NumField()
2023/09/26 09:52:14 RIC Indication message from {gnb_001_001_123456} received
2023/09/26 09:52:14 Indication Header format = 1
 parsing for UE metrics
 No of ue metrics = 12

 No of ue= 1

```
Subcription Manager (No error after line 22)
```bas=
1695720020345 1/RMR [INFO] sends: ts=1695720020 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricplt-e2mgr-rmr.ricplt:3801 open=0 succ=0 fail=0 (hard=0 soft=0)
1695720020345 1/RMR [INFO] sends: ts=1695720020 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricplt-a1mediator-rmr.ricplt:4562 open=0 succ=0 fail=0 (hard=0 soft=0)
CRESTSubscriptionRequest
  SubscriptionID = ''
  ClientEndpoint.Host = service-ricxapp-kpimon-go-http.ricxapp
  ClientEndpoint.HTTPPort = 8080
  ClientEndpoint.RMRPort = 4560
  Meid = gnb_001_001_123456
  E2SubscriptionDirectives = nil
  RANFunctionID = 2
  SubscriptionDetail.XappEventInstanceID = 1
  SubscriptionDetail.EventTriggers = [8 39 15]
  SubscriptionDetail.ActionToBeSetup.ActionID = 1
  SubscriptionDetail.ActionToBeSetup.ActionType = report
  SubscriptionDetail.ActionToBeSetup.ActionDefinition = [0 1 1 8 0 22 0 160 68 82 66 46 85 69 84 104 112 68 108 1 32 0 0 0 160 68 82 66 46 85 69 84 104 112 85 108 1 32 0 0 0 176 80 69 69 46 65 118 103 80 111 119 101 114 1 32 0 0 0 144 80 69 69 46 69 110 101 114 103 121 1 32 0 0 1 144 81 111 115 70 108 111 119 46 84 111 116 80 100 99 112 80 100 117 86 111 108 117 109 101 68 108 1 32 0 0 1 144 81 111 115 70 108 111 119 46 84 111 116 80 100 99 112 80 100 117 86 111 108 117 109 101 85 108 1 32 0 0 0 160 82 82 67 46 67 111 110 110 77 97 120 1 32 0 0 0 176 82 82 67 46 67 111 110 110 77 101 97 110 1 32 0 0 0 208 82 82 85 46 80 114 98 65 118 97 105 108 68 108 1 32 0 0 0 208 82 82 85 46 80 114 98 65 118 97 105 108 85 108 1 32 0 0 0 176 82 82 85 46 80 114 98 84 111 116 68 108 1 32 0 0 0 176 82 82 85 46 80 114 98 84 111 116 85 108 1 32 0 0 0 192 82 82 85 46 80 114 98 85 115 101 100 68 108 1 32 0 0 0 192 82 82 85 46 80 114 98 85 115 101 100 85 108 1 32 0 0 0 160 86 105 97 118 105 46 71 101 111 46 120 1 32 0 0 0 160 86 105 97 118 105 46 71 101 111 46 121 1 32 0 0 0 160 86 105 97 118 105 46 71 101 111 46 122 1 32 0 0 0 192 86 105 97 118 105 46 71 110 98 68 117 73 100 1 32 0 0 0 160 86 105 97 118 105 46 78 114 67 103 105 1 32 0 0 0 160 86 105 97 118 105 46 78 114 80 99 105 1 32 0 0 1 96 86 105 97 118 105 46 82 97 100 105 111 46 97 110 116 101 110 110 97 84 121 112 101 1 32 0 0 1 32 86 105 97 118 105 46 82 97 100 105 111 46 97 122 105 109 117 116 104 1 32 0 0 1 0 86 105 97 118 105 46 82 97 100 105 111 46 112 111 119 101 114 1 32 0 0 64 39 15 0 0 241 16 18 52 86 0 16]
  SubscriptionDetail.ActionToBeSetup.SubsequentAction.SubsequentActionType = continue
  SubscriptionDetail.ActionToBeSetup..SubsequentAction.TimeToWait = w10ms
  SubscriptionDetail.ActionToBeSetup.ActionID = 2
  SubscriptionDetail.ActionToBeSetup.ActionType = report
  SubscriptionDetail.ActionToBeSetup.ActionDefinition = [0 1 3 64 0 29 0 160 68 82 66 46 85 69 67 113 105 68 108 0 0 16 0 0 0 80 68 82 66 46 85 69 67 113 105 85 108 0 0 16 0 0 0 80 68 82 66 46 85 69 84 104 112 68 108 0 0 16 0 0 0 80 68 82 66 46 85 69 84 104 112 85 108 0 0 16 0 0 0 200 81 111 115 70 108 111 119 46 84 111 116 80 100 99 112 80 100 117 86 111 108 117 109 101 68 108 0 0 16 0 0 0 96 82 82 85 46 80 114 98 85 115 101 100 68 108 0 0 16 0 0 0 96 82 82 85 46 80 114 98 85 115 101 100 85 108 0 0 16 0 0 0 80 84 66 46 84 111 116 78 98 114 68 108 0 0 16 0 0 0 80 84 66 46 84 111 116 78 98 114 85 108 0 0 16 0 0 0 96 86 105 97 118 105 46 67 101 108 108 46 105 100 0 0 16 0 0 0 80 86 105 97 118 105 46 71 101 111 46 120 0 0 16 0 0 0 80 86 105 97 118 105 46 71 101 111 46 121 0 0 16 0 0 0 80 86 105 97 118 105 46 71 101 111 46 122 0 0 16 0 0 0 96 86 105 97 118 105 46 81 111 83 46 53 113 105 0 0 16 0 0 0 120 86 105 97 118 105 46 81 111 83 46 67 101 108 108 73 100 0 0 16 0 0 0 112 86 105 97 118 105 46 81 111 83 46 68 114 98 73 100 0 0 16 0 0 0 104 86 105 97 118 105 46 81 111 83 46 71 102 98 114 0 0 16 0 0 0 104 86 105 97 118 105 46 83 108 105 99 101 46 105 100 0 0 16 0 0 0 112 86 105 97 118 105 46 85 69 46 66 101 97 109 73 100 0 0 16 0 0 0 128 86 105 97 118 105 46 85 69 46 70 114 97 109 101 67 110 116 0 0 16 0 0 0 112 86 105 97 118 105 46 85 69 46 82 115 83 105 110 114 0 0 16 0 0 0 96 86 105 97 118 105 46 85 69 46 82 115 114 112 0 0 16 0 0 0 96 86 105 97 118 105 46 85 69 46 82 115 114 113 0 0 16 0 0 0 112 86 105 97 118 105 46 85 69 46 84 99 82 110 116 105 0 0 16 0 0 0 136 86 105 97 118 105 46 85 69 46 97 110 111 109 97 108 105 101 115 0 0 16 0 0 0 80 86 105 97 118 105 46 85 69 46 105 100 0 0 16 0 0 0 184 86 105 97 118 105 46 85 69 46 115 101 114 118 105 110 103 68 105 115 116 97 110 99 101 0 0 16 0 0 0 104 86 105 97 118 105 46 85 69 46 115 112 101 101 100 0 0 16 0 0 0 208 86 105 97 118 105 46 85 69 46 116 97 114 103 101 116 84 104 114 111 117 103 104 112 117 116 68 108 0 0 16 0 0 0 208 86 105 97 118 105 46 85 69 46 116 97 114 103 101 116 84 104 114 111 117 103 104 112 117 116 85 108 0 0 16 0 0 32 39 15]
  SubscriptionDetail.ActionToBeSetup.SubsequentAction.SubsequentActionType = continue
  SubscriptionDetail.ActionToBeSetup..SubsequentAction.TimeToWait = w10ms
1695720321556 1/RMR [INFO] sends: ts=1695720321 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricxapp-kpimon-go-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)
1695720321556 1/RMR [INFO] sends: ts=1695720321 src=service-ricplt-submgr-rmr.ricplt:4560 target=10.107.232.231:38000 open=1 succ=3 fail=0 (hard=0 soft=0)
1695720321556 1/RMR [INFO] sends: ts=1695720321 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricplt-e2mgr-rmr.ricplt:3801 open=0 succ=0 fail=0 (hard=0 soft=0)
1695720321556 1/RMR [INFO] sends: ts=1695720321 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricplt-a1mediator-rmr.ricplt:4562 open=0 succ=0 fail=0 (hard=0 soft=0)
1695720622363 1/RMR [INFO] sends: ts=1695720622 src=service-ricplt-submgr-rmr.ricplt:4560 target=service-ricxapp-kpimon-go-rmr.ricxapp:4560 open=0 succ=0 fail=0 (hard=0 soft=0)

```
Routing Manager (No error for kpimon-go-rmr endpoint)
```bash=
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"name: E2TERMINST"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"ep: &{10.107.232.231:38000 E2TERMINST platform 10.107.232.231 38000 [] [] [] <nil> false false 4}"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rpe.generatePlatformRoutes invoked"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: kpimon-go, xAppType: kpimon-go"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rpe.generateXappRoutes invoked"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: kpimon-go, xAppType: kpimon-go"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rpe.addSubscriptionRoutes invoked"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Subscription: {4 service-ricxapp-kpimon-go-rmr.ricxapp 4560}"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"xApp UUID: service-ricxapp-kpimon-go-rmr.ricxapp:4560"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"name: service-ricxapp-kpimon-go-rmr.ricxapp:4560"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"ep: &{service-ricxapp-kpimon-go-rmr.ricxapp:4560 kpimon-go kpimon-go service-ricxapp-kpimon-go-rmr.ricxapp 4560 [RIC_SUB_REQ] [RIC_SUB_RESP RIC_INDICATION] [] <nil> false false 5}"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"xApp UUID is matched for selected xApp.UUID: service-ricxapp-kpimon-go-rmr.ricxapp:4560 and xApp.Name: kpimon-go"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rpe.generateXappToXappRoutes invoked"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"RecvxAppEp.RxMessages Endpoint: kpimon-go, xAppType: kpimon-go and rxmsg: RIC_SUB_RESP "}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"RecvxAppEp.RxMessages Endpoint: kpimon-go, xAppType: kpimon-go and rxmsg: RIC_INDICATION "}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: SUBMAN, xAppType: platform"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: E2MAN, xAppType: platform"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: A1MEDIATOR, xAppType: platform"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Endpoint: E2TERMINST, xAppType: platform"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rmr.GeneratePolicies returns: [newrt|start\n mse|12010,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12020,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12011|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12021|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12012|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12022|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12023|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|10060,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10070,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10071,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10360,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10081,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10082,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10371,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10372,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|12001|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1100|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10061|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10062|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10071|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10361|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10362|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10370|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12007|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10020|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1102|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|20012|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|20011|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|12030|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1090,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|1101,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12002,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12003,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12031,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12032,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12072,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12005,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12010,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12020,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12040,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|%meid\n mse|12041|-1|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12042|-1|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12011,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12012,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12021,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12022,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12050|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12041|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12042|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n newrt|end\n meid_map|start\n mme_ar|10.107.232.231:38000|gnb_001_001_123456\n meid_map|end|1\n]"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"rmr.GeneratePolicies returns: &{[{kpimon-go deployed 2.0.1 [{kpimon-go deployed service-ricxapp-kpimon-go-rmr.ricxapp 4560 [RIC_SUB_REQ] [RIC_SUB_RESP RIC_INDICATION] []}]}] map[10.107.232.231:38000:{E2TERMINST 10.107.232.231:38000 [gnb_001_001_123456]}] [mme_ar|10.107.232.231:38000|gnb_001_001_123456 mme_del|gnb_001_001_123456 mme_ar|10.107.232.231:38000|gnb_001_001_123456 mme_del|gnb_001_001_123456 mme_ar|10.107.232.231:38000|gnb_001_001_123456] [{SUBMAN service-ricplt-submgr-rmr.ricplt 4560} {E2MAN service-ricplt-e2mgr-rmr.ricplt 3801} {A1MEDIATOR service-ricplt-a1mediator-rmr.ricplt 4562}]}"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked: sbi.DistributeAll"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"args: [newrt|start\n mse|12010,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12020,service-ricplt-submgr-rmr.ricplt:4560|-1|%meid\n mse|12011|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12021|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12012|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12022|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12023|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|10060,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10070,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10071,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10360,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10081,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10082,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10371,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|10372,service-ricplt-e2mgr-rmr.ricplt:3801|-1|%meid\n mse|12001|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1100|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10061|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10062|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10071|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10361|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10362|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10370|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12007|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10080|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|10020|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1102|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|20012|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|20011|-1|service-ricplt-a1mediator-rmr.ricplt:4562\n mse|12030|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12070|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|1090,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|1101,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12002,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12003,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12031,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12032,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12072,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12004|-1|service-ricplt-e2mgr-rmr.ricplt:3801\n mse|12005,service-ricplt-e2mgr-rmr.ricplt:3801|-1|10.107.232.231:38000\n mse|12010,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12020,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|service-ricplt-submgr-rmr.ricplt:4560\n mse|12040,service-ricxapp-kpimon-go-rmr.ricxapp:4560|-1|%meid\n mse|12041|-1|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12042|-1|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12011,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12012,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12021,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12022,service-ricplt-submgr-rmr.ricplt:4560|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12050|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12041|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n mse|12042|4|service-ricxapp-kpimon-go-rmr.ricxapp:4560\n newrt|end\n meid_map|start\n mme_ar|10.107.232.231:38000|gnb_001_001_123456\n meid_map|end|1\n]"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Push policy to endpoint: service-ricxapp-kpimon-go-rmr.ricxapp:4560"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked send_data to endpoint: service-ricxapp-kpimon-go-rmr.ricxapp:4560 call_id: 23"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Push policy to endpoint: service-ricplt-e2mgr-rmr.ricplt:3801"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked send_data to endpoint: service-ricplt-e2mgr-rmr.ricplt:3801 call_id: 23"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Push policy to endpoint: 10.107.232.231:38000"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked send_data to endpoint: 10.107.232.231:38000 call_id: 23"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Push policy to endpoint: service-ricplt-a1mediator-rmr.ricplt:4562"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked send_data to endpoint: service-ricplt-a1mediator-rmr.ricplt:4562 call_id: 23"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Push policy to endpoint: service-ricplt-submgr-rmr.ricplt:4560"}
{"ts":1695723274444,"crit":"DEBUG","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Invoked send_data to endpoint: service-ricplt-submgr-rmr.ricplt:4560 call_id: 23"}
{"ts":1695723274445,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Update Routes to Endpoint: service-ricxapp-kpimon-go-rmr.ricxapp:4560 successful, call_id: 23, Payload length: 3454, Route Update Status: OK(# of Entries:61"}
{"ts":1695723274447,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Update Routes to Endpoint: service-ricplt-submgr-rmr.ricplt:4560 successful, call_id: 23, Payload length: 3454, Route Update Status: OK(# of Entries:61"}
{"ts":1695723274447,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Update Routes to Endpoint: service-ricplt-e2mgr-rmr.ricplt:3801 successful, call_id: 23, Payload length: 3454, Route Update Status: OK(# of Entries:61"}
{"ts":1695723274447,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Update Routes to Endpoint: 10.107.232.231:38000 successful, call_id: 23, Payload length: 3454, Route Update Status: OK(# of Entries:61"}
{"ts":1695723274447,"crit":"INFO","id":"rtmgr","mdc":{"CONTAINER_NAME":"","HOST_NAME":"","PID":"1","POD_NAME":"","SERVICE_NAME":"","SYSTEM_NAME":"","time":"2023-09-26T10:14:34"},"msg":"Update Routes to Endpoint: service-ricplt-a1mediator-rmr.ricplt:4562 successful, call_id: 23, Payload length: 3454, Route Update Status: OK(# of Entries:61"}

```

* To restart kpimon (IP of pod/service will not change)
```bash=
kubectl rollout restart deploy -n ricxapp ricxapp-kpimon-go
```

* To get list of RAN in e2mgr
```bash!
export E2MGR_IP=$(kubectl get -n ricplt service/service-ricplt-e2mgr-http -o jsonpath='{.spec.clusterIP}')

# Query status of all RAN
curl -H "Content-Type: application/json" http://$E2MGR_IP:3800/v1/e2t/list
```

## 8. Deploy xApp rc:
Steps to deploy the xapp using `dms_cli` tool

* 1)Create a local helm repository with a port other than 8080 on host (**Optional if the first time use local chart**)
        `$docker run --rm -u 0 -it -d -p 8090:8080 -e DEBUG=1 -e STORAGE=local -e STORAGE_LOCAL_ROOTDIR=/charts -v $(pwd)/charts:/charts chartmuseum/chartmuseum:latest`

* 2)Set CHART_REPO_URL env variable
        `$export CHART_REPO_URL=http://0.0.0.0:8090`

* 3)Check dms_cli health is active(true) or not
```==bash==
$dms_cli health
True
```

* 4)Onboard the rc xapp using the below command :
Make sure that you have the xapp descriptor config file and the schema file at your local file system
```==bash=
root@fcp-vm2:/home/fcp/xApp/rc# ls xapp-descriptor/
config.json  schema.json
$dms_cli onboard --config_file_path=config.json --shcema_file_path=schema.json
```
After this command: xapp should be created
        Eg:
```==bash===
$dms_cli onboard --config_file_path=config-file.json --shcema_file_path=schema.json
{
"status": "Created"
}
```

5)List the helm charts from helm repository (Optional)
        `$curl -X GET http://localhost:8090/api/charts | jq .`
        Eg:
```===bash===
$curl -X GET http://$(hostname):8090/api/charts | jq .
{
 "rc": [
    {
      "name": "rc",
      "version": "1.0.0",
      "description": "Standard xApp Helm Chart",
      "apiVersion": "v1",
      "appVersion": "1.0",
      "urls": [
        "charts/rc-1.0.0.tgz"
      ],
      "created": "2022-04-20T05:26:37.891030646Z",
      "digest": "3636f4fba220aab5739d737eb3cd3d74b5012b029bde95f114b05f9eaa786b1f"
    }
  ]
}
```
6) Install the xApp
        `$dms_cli install --xapp_chart_name=XAPP_CHART_NAME --version=VERSION --namespace=NAMESPACE`
        Eg:
```==bash===
 $dms_cli install --xapp_chart_name=rc --version=1.0.0 --namespace=ricxapp
        status: OK
```
after this step the xapp should be in running state

```==bash==
root@fcp-vm2:/home/fcp/xApp/rc# kubectl get pods -n ricxapp
NAME                                 READY   STATUS    RESTARTS   AGE
ricxapp-kpimon-go-6b558d68b6-gxdz5   1/1     Running   0          22h
ricxapp-rc-7894f4fdcc-kxl9z          1/1     Running   0          121
root@fcp-vm2:/home/fcp/xApp/rc# kubectl get service -n ricxapp
NAME                             TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
aux-entry                        ClusterIP   10.105.32.236    <none>        80/TCP,443/TCP      4d23h
service-ricxapp-kpimon-go-http   ClusterIP   10.106.125.252   <none>        8080/TCP            22h
service-ricxapp-kpimon-go-rmr    ClusterIP   10.97.221.160    <none>        4560/TCP,4561/TCP   22h
service-ricxapp-rc-grpc-server   ClusterIP   10.100.238.200   <none>        7777/TCP            122m
service-ricxapp-rc-http          ClusterIP   10.102.128.17    <none>        8080/TCP            122m
service-ricxapp-rc-rmr           ClusterIP   10.107.80.110    <none>        4560/TCP,4561/TCP   122m

```

7) restart rtMgr so rmr is ready (**optional**)

## 9. Test with VIAVI RIC Test:

### 9.1 Reconnect VIAVI RIC Test to OSC RIC:
* When reconnect the VIAVI RIC Test to OSC RIC, the routing manager and subscription mananger of OSC RIC can be confused. We should restart the two services before reconnecting RIC Test.
```===bash===
kubectl rollout restart deploy -n ricplt deployment-ricplt-rtmgr
kubectl rollout restart deploy -n ricplt deployment-ricplt-submgr
```
* subsequently, we also need to restart and re-register the KPI-Mon:
```===bash
kubectl rollout restart deploy -n ricxapp ricxapp-kpimon-go
```
Then do above [steps](#72-Rebuild-image-for-kpimon-go-service)
### 9.2 RAN Control RC:
* Use the `grpcurl` to send `service-ricxapp-rc-grpc-server` by using script in `unitTest` folder:
```===bash===
root@fcp-vm2:/home/fcp/xApp/rc/unitTest# vi grpcClient_viavi.sh

```
