Introduction to E2 Termination

--- title: Introduction to E2 Termination image: https://i.imgur.com/NTydYzR.png --- :::warning # <center><i class="fa fa-book"></i> 【G Release】<br> Introduction to E2T (source code)</center> ::: ###### tags: `study` `E2` `E2 Termination` `source code` >name: ric-plt/e2 >version: 6.0.1 >tree: 264f9e013884dff74fd2b835db38acf17226f34b :::success **🎯 Goals:** - [x] - <a href="">A brief introduction to E2T by source code</a> - [ ] receiveDataFromSctp - [ ] - <a href="">Summary </a> ::: :::info :bookmark: **Reference:** - [Introduction to E2 Termination](/@MingHung/E2termination) - [Source code](https://gerrit.o-ran-sc.org/r/gitweb?p=ric-plt/e2.git;a=summary) ![](https://i.imgur.com/0L2Bx6q.png) ::: [toc] # Module 1: Source code work Plot the **relationship** of all subroutines: ``` mermaid graph LR; Main-->init_log-->dynamic_log_level_change-->read_env_param; dynamic_log_level_change-->enable_log_change_notify; Main-->approx_CPU_MHz; Main-->buildConfiguration-->openConfigFile; Main-->startPrometheus-->getinterfaceip; startPrometheus-->buildE2TPrometheusCounters; Main-->getRmrContext; Main-->buildInotify; Main-->buildListeningPort; Main-->listener-->More_details_as_shown_below; Main-->handleTermInit-->sendTermInit-->translateRmrErrorMessages; ``` :::spoiler **Introduce** ![](https://i.imgur.com/hKtBFLN.png) - [startPrometheus](#15-startPrometheus-Line-590) - [listener](#110-listener-Line-764) - [handleTermInit](#111-handleTermInit-Line-648) ::: ``` mermaid graph LR; listener-->handlepoll_error-->sendRequestToXapp-->sendRmrMessage-->buildJsonMessage; handlepoll_error-->cleanHashEntry; listener-->handleEinprogressMessages-->sendSctpMsg-->fetchStreamId; sendSctpMsg-->cleanHashEntry; sendSctpMsg-->buildJsonMessage; listener-->setSocketNoBlocking; listener-->addToEpoll-->cleanHashEntry; listener-->receiveXappMessages-->PER_FromXML; receiveXappMessages-->sendDirectionalSctpMsg-->getRequestMetaData; sendDirectionalSctpMsg-->sendMessagetoCu-->sendSctpMsg; receiveXappMessages-->sendRequestToXapp; receiveXappMessages-->buildJsonMessage; listener-->handleConfigChange-->openConfigFile; listener-->receiveDataFromSctp-->asnInitiatingRequest-->buildPrometheusList; asnInitiatingRequest-->buildAndSendSetupRequest; asnInitiatingRequest-->sendRequestToXapp; asnInitiatingRequest-->XML_From_PER; asnInitiatingRequest-->sendRmrMessage; asnInitiatingRequest-->buildJsonMessage; buildAndSendSetupRequest-->translateRmrErrorMessages; buildAndSendSetupRequest-->buildJsonMessage; receiveDataFromSctp-->asnSuccessfulMsg-->sendRequestToXapp; asnSuccessfulMsg-->sendRmrMessage; asnSuccessfulMsg-->buildJsonMessage; receiveDataFromSctp-->asnUnSuccsesfulMsg-->sendRmrMessage; asnUnSuccsesfulMsg-->sendRequestToXapp; asnUnSuccsesfulMsg-->buildJsonMessage; receiveDataFromSctp-->sendRequestToXapp; ``` ``` mermaid graph LR; ``` ## 1.1 Main code (Line: 547) - The **starting** of the main work - Define the struct: | sctpParams | |:------------ | | device | | generator | | distribution | ## 1.2 init log (MDC LOG) (Line: 561) ## 1.3 Initializing signal (Line: 562-573) ## 1.4 buildConfiguration (Line: 581) - Function code (Line: 347) 1. Check path exist and open it! (sctpParams.**configFilePath**/sctpParams.**configFileName**) 2. Copy the name to temp file as well 3. Get some StringValue Config 4. Running parameters for instance with sctpParams.**ka_message** | Config Name | nano | volume | local-ip | sctp-port | external-fqdn | pod_name | trace | | ----------- | -------------- | ------ | -------- |:--------- |:------------- | -------- | ------------------ | | Value Info | (int) RMR port | String | String | Int | String | String | Judging start/stop | ## 1.5 start[Prometheus](#Prometheus--Monitoring-data-sources) (Line: 590) 1. Get POD_NAME (Line: 515) 2. Set metric to E2TBeta or E2TAlpha 3. **Get eth0 interface IP** ++by getinterfaceip() (Line: 63)++ `=> Get hostname and then get IP` 4. **Set parameter :** ```c= sctpParams.prometheusFamily = &BuildCounter() .Name(metric.c_str()) .Help("E2T instance metrics") .Labels({{"POD_NAME", sctpParams.podName}}) .Register(*sctpParams.prometheusRegistry); ``` 5. **buildE2TPrometheusCounters**(sctpParams) (Line: 1774) - Build E2T instance level metrics - **For example code**: (Add counters) ```c=1775 sctpParams.e2tCounters[IN_INITI][MSG_COUNTER][(ProcedureCode_id_E2setup)] = &sctpParams.prometheusFamily->Add({{"counter", "SetupRequestMsgs"}}); ``` - **[MSG_COUNTER]** - [Google Sheet can set filter](https://docs.google.com/spreadsheets/d/1n4MYvd50n0iiSqNBFY1swQJk8Y56vFGePyY0prmZ6Bo/edit?usp=sharing) | e2tCounters Type | Name ⬇ | |:---------------- |:------------------------------ | | IN_INITI | SetupRequestMsgs | | OUT_SUCC | SetupResponseMsgs | | OUT_UN_SUCC | SetupRequestFailureMsgs | | IN_INITI | E2NodeConfigUpdateMsgs | | OUT_SUCC | E2NodeConfigUpdateResponseMsgs | | OUT_UN_SUCC | E2NodeConfigUpdateFailureMsgs | | IN_INITI | ErrorIndicationMsgs | | IN_INITI | ResetRequestMsgs | | OUT_SUCC | ResetAckMsgs | | IN_INITI | RICServiceUpdateMsgs | | OUT_SUCC | RICServiceUpdateRespMsgs | | OUT_UN_SUCC | RICServiceUpdateFailureMsgs | | OUT_INITI | RICControlMsgs | | IN_SUCC | RICControlAckMsgs | | IN_UN_SUCC | RICControlFailureMsgs | | OUT_INITI | RICSubscriptionMsgs | | IN_SUCC | RICSubscriptionAckMsgs | | IN_UN_SUCC | RICSubscriptionFailureMsgs | | OUT_INITI | RICSubscriptionDeleteMsgs | | IN_SUCC | RICSubscriptionDeleteAckMsgs | | IN_UN_SUCC | RICSubscriptionDeleteFailMsgs | | IN_INITI | RICIndicationMsgs | | OUT_INITI | RICServiceQueryMsgs | ![](https://i.imgur.com/8mOrdfh.png) ## 1.6 start epoll (Line: 593) - I/O multiplexing using epoll - The core concept of epoll is epoll instance - Think of it as a container containing two concatenated lists : - a linked list of file descriptors to listen for - Prepared file descriptor link list ## 1.7 getRmrContext (Line: 2500) - where to call: ```c= getRmrContext(sctpParams); // (Line: 599) ``` ### 1.7.1 RMR INIT ### 1.7.2 RMR Running 1. After RMR INIT wait for RMR_Ready 2. RMR running 3. Get the RMR FD for the epoll 4. Add RMR FD to epoll 5. Add listening RMR descriptor FD to epoll > FD: File descriptor [color=pink] ## 1.8 buildInotify (Line: 729) - where to call: ```c=605 if (buildInotify(sctpParams) == -1) // (Line: 605) ``` @return `-1 failed` `0 success` ### 1.8.1 **INIT** and **Add** the FD and WD ```c= /*FD*/ sctpParams.inotifyFD = inotify_init1(IN_NONBLOCK); /*WD*/ sctpParams.inotifyWD = inotify_add_watch(sctpParams.inotifyFD, (const char *)sctpParams.configFilePath.c_str(), (unsigned)IN_OPEN | (unsigned)IN_CLOSE_WRITE | (unsigned)IN_CLOSE_NOWRITE); ``` ### 1.8.2 Add listening RMR FD to epoll ```c= epoll_ctl(sctpParams.epoll_fd, EPOLL_CTL_ADD, sctpParams.inotifyFD, &event) ``` ## 1.9 buildListeningPort (Line: 278) - where to call: ```c=612 buildListeningPort(sctpParams) != 0 // (Line: 612) ``` ### 1.9.1 Add FD ```c= sctpParams.listenFD = socket(AF_INET6, SOCK_STREAM, IPPROTO_SCTP); ``` ### 1.9.2 INIT Msg ```c= initmsg.sinit_num_ostreams = 2; initmsg.sinit_max_instreams = 2; initmsg.sinit_max_attempts = 4; ``` | streams num | max in streams | max attempts | |:-----------:|:--------------:|:------------:| | 2 | 2 | 4 | ### 1.9.3 set Server Address ```c= serverAddress.sin6_family = AF_INET6; serverAddress.sin6_addr = in6addr_any; serverAddress.sin6_port = htons(sctpParams.sctpPort); ``` ### 1.9.4 Add listening port to epoll ```c= epoll_ctl(sctpParams.epoll_fd, EPOLL_CTL_ADD, sctpParams.listenFD, &event) ``` ## 1.10 listener (Line: 764) - where to call: ```c=626 threads[i] = std::thread(listener, &sctpParams); // (Line: 626) ``` :::info 1. **Get** some params 2. **Create** and init RMR 3. **Start EPOLL Wait.** 4. **Handle every epoll event** ::: 1. **Get** some params | pod_name | container_name | service_name | host_name | system_name | pid | | -------- | -------------- |:------------ | --------- | ----------- | --- | 2. **Create** and init RMR - Set Message by rmrMessageBuffer | Message | | ---------- | | recive | | send | | keep alive | 3. **Start EPOLL Wait.** ```c=828 auto numOfEvents = epoll_wait(params->epoll_fd, events, MAXEVENTS, params->epollTimeOut); ``` 4. **Handle every epoll event** - **epoll out [EPOLLOUT]** ```c=865 handleEinprogressMessages(events[i], message, rmrMessageBuffer, params); ``` <br> - **Different type** in events[i].data.fd | listenFD | rmrListenFd | inotifyFD | Else | |:--------------:|:---------------------:|:----------------------:|:---------------------:| | New connection | Got message from XAPP | Got event from inotify | fd waiting to be read | | Line:886 | Line:938 | Line:950 | Line:953 | - **[listenFD]:** When new connection is requested from RAN start build connection - **[rmrListenFd]:** new RMR messag, use by: ```c=945 receiveXappMessages(params->sctpMap, rmrMessageBuffer, message.message.time) ``` - **[inotifyFD]:** configuration update, use by: ```c=952 handleConfigChange(params); ``` - **[Else]:** We RMR_ERR_RETRY have data on the fd waiting to be read. **Read and display it.** We must read whatever data is available completely, as we are running **in edge-triggered mode** and won't **get a notification again** for the same data ```c=961 receiveDataFromSctp(&events[i], params->sctpMap, num_of_SCTP_messages, rmrMessageBuffer, message.message.time); ``` ### 1.10.1 receiveXappMessages (Line: 2615) :::info **1. Get recive Message type 2. Increment 3. sendDirectionalSctpMsg** ::: 1. **Get** the rmrMessageBuffer.rcvMessage->mtype: - If type is not the following three items, it means **failed to send message no CU entry.** | RIC_SCTP_CLEAR_ALL | E2_TERM_KEEP_ALIVE_REQ | RIC_HEALTH_CHECK_REQ | | -------- | -------- | -------- | 2. **Increment** corresponds to Counters - For example with code - Update E2T instance level metrics in **[OUT_SUCC] and [ProcedureCode_id_E2setup]**: ```c= message.peerInfo->sctpParams->e2tCounters[OUT_SUCC][MSG_COUNTER][ProcedureCode_id_E2setup]->Increment(); ``` **Different type** in rmrMessageBuffer.rcvMessage->mtype case: | mtype case | Index_0 | Index_2 | | ---------------------------------------- | -------------------------------- | --------------------------------------------------------- | | RIC_E2_SETUP_RESP (12002) | [OUT_SUCC] | [ProcedureCode_id_E2setup] | | RIC_E2_SETUP_FAILURE (12003) | [OUT_UN_SUCC] | [ProcedureCode_id_E2setup] | | RIC_E2NODE_CONFIG_UPDATE_ACK (12071) | [OUT_SUCC] | [ProcedureCode_id_E2nodeConfigurationUpdate] | | RIC_E2NODE_CONFIG_UPDATE_FAILURE (12072) | [OUT_UN_SUCC] | [ProcedureCode_id_E2nodeConfigurationUpdate] | | RIC_ERROR_INDICATION (12007) | [OUT_INITI] | [ProcedureCode_id_ErrorIndication] | | RIC_SUB_REQ (12010) | [OUT_INITI] | [ProcedureCode_id_RICsubscription] | | RIC_SUB_DEL_REQ (12020) | [OUT_INITI] | [ProcedureCode_id_RICsubscriptionDelete] | | RIC_CONTROL_REQ (12040) | [OUT_INITI] | [ProcedureCode_id_RICcontrol] | | RIC_SERVICE_QUERY | [OUT_INITI] | [ProcedureCode_id_RICserviceQuery] | | RIC_SERVICE_UPDATE_ACK (12031) | [OUT_SUCC] | [ProcedureCode_id_RICserviceUpdate] | | RIC_SERVICE_UPDATE_FAILURE (12032) | [OUT_UN_SUCC] | [ProcedureCode_id_RICserviceUpdate] | | RIC_E2_RESET_REQ (12004) | [OUT_INITI] | [ProcedureCode_id_Reset] | | RIC_E2_RESET_RESP (12005) | [OUT_SUCC] | [ProcedureCode_id_Reset] | | RIC_SCTP_CLEAR_ALL (1090) | close(peerInfo->fileDescriptor); | loop on all keys and close socket and then erase all map. | | E2_TERM_KEEP_ALIVE_REQ (1101) | send message back | mtype = E2_TERM_KEEP_ALIVE_RESP | | RIC_HEALTH_CHECK_REQ (100) | send message back | mtype = RIC_HEALTH_CHECK_RESP | > REQ: Requirement [color=pink] - [Google Sheet can set filter](https://docs.google.com/spreadsheets/d/1n4MYvd50n0iiSqNBFY1swQJk8Y56vFGePyY0prmZ6Bo/edit?usp=sharing) 3. sendDirectionalSctpMsg() ### 1.10.2 receiveDataFromSctp (Line: 1359) need to write detail with ... 1. Get the identity of the interface 2. Read the buffer directly to rmr payload  ## 1.11 handleTermInit (Line: 648) :::info **loop over** term_init ==until first message from xApp== ::: :::info **RIC_E2NODE_CONFIG_UPDATE_FAILURE:** ::: - where to call: ```c= handleTermInit(sctpParams); // (Line: 639) ``` ### 1.11.1 sendTermInit() (Line: 671) - Send to e2 manager init of e2 term - where to call: ```c= sendTermInit(sctpParams); // (Line: 649) ``` 1. **Define msg**: ```c= rmr_mbuf_t *msg = rmr_alloc_msg(sctpParams.rmrCtx, sctpParams.ka_message_length); ``` 2. **Set msg** content: ```c= msg->mtype = E2_TERM_INIT; msg->state = 0; ``` | mtype | state | | ------------ | ----- | | E2_TERM_INIT | 0 | 3. **Type conversion** **[bytes]** to **[payload]** and **[xact]** 4. **Send Message** by `rmr_send_msg(sctpParams.rmrCtx, msg)` ### 1.11.2 Until first message from xApp - load num of XAPP messages ```c= auto xappMessages = num_of_XAPP_messages.load(std::memory_order_acquire); // (Line: 655) ``` - **Got a message** from some application, **stop sending** E2_TERM_INIT or - After usleep(100000) run 1000 times, return **GOT No messages from any xApp**, then send Term INIT again. ## END  # Additional :::spoiler more ## Introduction to MDC MDC ( Mapped Diagnostic Contexts ), which is a thread-safe container for storing diagnostic logs. One of the goals of Logback's design is to audit and debug decentralized application systems. In today's decentralized systems, many requests need to be processed at the same time. How to distinguish the log output from which request? We can generate a logger for each request, but this will waste a lot of resources, and this method will consume server resources as the number of requests increases, so this method is not recommended. A more lightweight implementation is to use the MDC mechanism to put the unique identifier of the request in the MDC container, such as sessionId, before processing the request. of. And clear the MDC container after request processing is complete.  ## `auto` in C Language Reference: [stackoverflow](https://stackoverflow.com/questions/2192547/where-is-the-c-auto-keyword-used) > defines a local variable as having a local lifetime - **The `auto` keyword is useless in the C language.** It is there because before the C language there existed a B language in which that keyword was necessary for declaring local variables. (B was developed into NB, which became C). Here is the reference manual for B. ## Prometheus : Monitoring data sources - [Back to 1.5](#15-startPrometheus-Line-590) > Reference: [IT Help](https://ithelp.ithome.com.tw/articles/10222705) [color=lightblue] > [Nokia PDF](https://wiki.o-ran-sc.org/download/attachments/10715420/Near_RT_RIC_for_ONS.pdf)[color=blue] ![](https://i.imgur.com/VqjRUj5.png) ---- :::  <style> /* 僅指定連結的圖片去背且靠右模擬置中 */ img[src^="https"]{ /* position: relative; left: 17%; */ background-color: #fff0; } .title { color: #009933; font-weight:bold; } .highlight { color: #ff4d4d; font-weight:bold; border-bottom:2px red solid; padding-bottom:2px; } </style>