Daily Note 24/08/2020

# Daily Note 24/08/2020 ###### tags: `Daily Notes` , `Acumos` ## Name : Christofel Rio Goenawan ## University : Bandung Institute of Technology (ITB) --- ## Schedule: 1. Study Detailed Works of Acumos AI Boreas Release's Interfaces and APIs from Documentation. 2. Study Detailed Works of Acumos AI's Federation APIs from Documentation. 3. Try to Solve Connection Issue in Nexus Installation for Acumos AI ## Outcome : 1. Explained Detailed Works of Acumos AI Boreas Release's Interfaces and APIs from Documentation. 2. Explained Detailed Works of Acumos AI's Federation APIs from Documentation. 3. Try to re- install Docker and Kubernetes but still got some issues. ## Further Plan : - Continue to deploy Acumos AIO in NTUST server - Study more detailed about AIO Installation in O- RAN --- ## Daily Log ### 1.Study Detailed Works of Acumos AI Boreas Release's Interfaces and APIs from Documentation. (9.00) - Study more detail explanation in [Documentation](https://docs.acumos.org/en/boreas/) and other sources. ### 2.Study Detailed Works of Acumos AI's Federation APIs from Documentation.(12.00) - Study more detail explanation in [Documentation](https://docs.acumos.org/en/boreas/submodules/federation/docs/developer-guide.html) and other sources. ### 3. Try to Solve Connection Issue in Nexus Installation for Acumos AI.(15.00) - Continue to try to deploy Acumos AIO in NTUST server using Prep- Deploy Process based one previous [study notes](https://hackmd.io/@christofel04/TEEP_Daily_Notes_10_7_2020). - Try to solve Connection Issue in Nexus Installation for Acumos AI based one previous [study notes](https://hackmd.io/@christofel04/TEEP_Daily_Notes_18_8_2020). --- ## Report ### 1. Acumos AI Boreas Release Interfaces and APIs >- In this note Writer use [Documentation](https://docs.acumos.org/en/boreas/) as reference. >- In this note Writer Study Acumos AI Boreas Release Interfaces and APIs in general view. The Detailed Study can be seen in next notes. From documentation Acumos AI Boreas Release Interfaces and APIs can be divided into 2 kind as below. 1. **External Interfaces and APIs** 2. **Internal Interfaces and APIs** --- #### 1. Acumos AI Boreas Release External Interfaces and APIs From documentation there are 7 main external Interfaces and APIs as below. ##### a. Toolkit Onboarding - The various clients used to **onboard models call the APIs in the Onboarding service**. The simple Onboarding diagram can be seen as below. ![](https://i.imgur.com/l9ltV1X.png) - There are 3 types of onboarding in Acumos AI Boreas Releases as below. 1. **Onboarding for Model Developed in R , Python and Java** 2. **Onboarding for Model Developed in Interchanged Format** 3. **Onboarding for Dockerized Model** :::info **The detailed study of Acumos AI Boreas Releases Onboarding can be seen [here](https://hackmd.io/@christofel04/TEEP_Daily_Notes_25_8_2020).** ::: --- ##### b. Web APIs - The portal Web API (E2) are the **interface for the users to upload their models to the platform**. - It **provides means to share AI microservices along with information on how they perform**. - The diagram can be seen as below. ![](https://i.imgur.com/BCYrIi2.jpg) :::info **The detailed study of Acumos AI Boreas Releases Marketplace APIs can be seen [here](https://hackmd.io/@christofel04/TEEP_Daily_Notes_21_8_2020).** ::: --- ##### c. OA&M APIs - The Operations , Admin and Maintenance ( usually called "OA&M" ) subsystem **defines data formats supported by the various logging and analytics support components described under Operations, Admin, and Maintenance (OAM)**. - These are primarily **focused on log formats that Acumos components will follow when saving log files that are collected by the logging subsystem**. - The Platform-OAM project maintains the repos providing as below. 1. Acumos platform deployment support tools 2. Logging and Analytics components based upon the **ELK Stack** of which Acumos uses the open source versions --- ##### d. Admin APIs - The Admin API (E4) provides the interfaces to configure the site global parameters --- ##### e. Federation APIs - The federation (public) E5 interface is a **REST-based API specification**. Any system that decides to federate needs to **implement this interface, which assumes a pull-based mechanism**. - **Only the server side is defined by E5**. - The server allows clients to **poll to discover solutions, and to retrieve solution metadata, solution artifacts and user-provided documents**. :::info **The detailed study of Acumos AI Boreas Releases Federation APIs can be seen below.** ::: --- ##### f. Deployment APIs - The Deployment subsystem primarily **consumes APIs of external systems such as cloud service environments, including Azure, OpenStack, and private kubernetes clouds**. - The developer guides for the “Deployers” that coordinate model deployment in those specific environments **address the specific APIs consumed by those Deployers**. - The main Deployment APIs in Acumos Boreas Releases can be seen as below. 1. Acumos Azure Client 2. Openstack Client 3. Kubernetes Client :::info **The detailed study of Acumos AI Boreas Releases Deployment APIs can be seen [here](https://hackmd.io/@christofel04/TEEP_Daily_Notes_21_8_2020).** ::: --- ##### g. Microservice Generation - The **DCAE model API** is intended to be used with models dedicated for **ONAP**. - It **builds a DCAE/ONAP microservice and required artifacts**. - Detailed Microservice Generation works can be seen as below. ![](https://i.imgur.com/uDT8bkF.png) :::info **The detailed study of Acumos AI Boreas Releases Deployment APIs can be seen [here](https://hackmd.io/@christofel04/TEEP_Daily_Notes_21_8_2020).** ::: --- #### 2. Acumos AI Boreas Release Internal Interfaces and APIs ##### a. Common Data Service - The Common Data Service **provides a storage and query micro service for use by system components, backed by a relational database**. - The API provides **Create, Retrive, Update and Delete (CRUD)** operations for system data including **users, solutions, revisions, artifacts** and more. - The **microservice endpoints** and **objects** are **documented extensively using code annotations that are published via Swagger in a running server** ensuring that the documentation is exactly synchronized with the implementation. - View this API documentation in a running CDS instance at a URL like the following, but consult the server’s configuration for the exact port number (e.g., “8000”) and context path (e.g., “ccds”) to use: ```http://localhost:8000/ccds/swagger-ui.html``` ##### b. Hippo CMS ##### c. Portal Backend --- ##### d. Federation Gateway - The **federation (local) E5 interface is a REST-based API specification just like the public interface**. - This interface provides **secure communication services to other components of the same Acumos instance primarily used by the Portal**. - The services include **querying remote peers for their content and fetching that content as needed**. ##### e. Microservice Generation - Lihat di [atas](). --- ##### f. Azure Client - The Azure Client exposes **two APIs that are used by the Portal-Markeplace to initiate model deployment in the Azure cloud service environment** as below. 1. **POST /azure/compositeSolutionAzureDeployment** 2. **POST /azure/singleImageAzureDeployment** - The **Azure Client API URL** is **configured for the Portal-Markeplace in the Portal-FE component template (docker or kubernetes)**. --- ##### g. OpenStack Client - The OpenStack Client **exposes two APIs that are used by the Portal-Markeplace to initiate model deployment in an OpenStack service environment hosted by Rackspace** as below. 1. **POST /openstack/compositeSolutionOpenstackDeployment** 2. **POST /openstack/singleImageOpenstackDeployment** - The **OpenStack Client API URL is configured for the Portal-Markeplace in the Portal-FE component template (docker or kubernetes)**. --- ###### h. Kubernetes Client - he Kubernetes Client **expose one API that is used by the Portal-Markeplace to provide the user with a downloadable deployment package for a model to be deployed in a private kubernetes service environment** as below. 1. **GET /getSolutionZip/{solutionId}/{revisionId}** - The **Kubernetes Client API URL** is **configured for the Portal-Markeplace in the Portal-FE component template (docker or kubernetes)**. --- ###### i. ELK Stack - The ELK Stack is used to **provide the E3 - OA&M APIs via which components publish standard-format log files for aggregation and presentation at operations dashboards**. --- ##### j. Nexus - The Nexus component **exposes two APIs enabling Acumos platform components to store and access artifacts in various repository types** including as below. 1. **Maven (for generic artifacts)** 2. **Docker (as a docker registry)** using the Docker Registry HTTP API V2. - The **Maven repository service** is **accessed via an API exposed thru the Nexus Client Java library**. - The docker repository service is **accessed via the Docker Registry HTTP API V2**. - Both services are **configured for clients through URLs and credentials defined in the component template (docker or kubernetes)**. --- ##### k. Docker - The **docker-engine** is the **primary service provided by Docker-CE** as used in Acumos. The docker-engine is accessed by the **Docker Engine API**. - The **docker-engine API URL** is **configured for Acumos components in the template (docker or kubernetes) for the referencing component**. --- ##### l. Kong - [**Kong**](https://konghq.com/kong-community-edition/) provides a **reverse proxy service for Acumos platform functions** exposed to users, such as the **Portal-Marketplace UI and APIs**, and the **Onboarding service APIs**. - The kong proxy service is **configured via the [Kong Admin API](https://docs.konghq.com/0.14.x/admin-api/)**. --- ### 2. Acumos AI Boreas Release's Federation APIs Marketplace >- In this note Writer use [Documentation](https://docs.acumos.org/en/boreas/submodules/federation/docs/developer-guide.html) as study sources. #### 1. Prerequisites The **uild machine** needs prerequisites as below. 1. **Java version 1.8** 2. **Maven version 3** 3. **Connectivity to Maven Central (for most jars)** 4. **Connectivity to Linux Foundation Nexus (for CDS jar)** Use below maven command to build and package the gateway service into a single jar: ```mvn clean package``` #### 2. Development and Local Testing In this local testing , Writer will run two instances of the gateway to depict two instance of Acumos federated to each other. In below scenario, we are going to run **Acumos A** and **Acumos B** for testing locally. #### 3. Launching Start two microservice instances for development and testing as follows with the appropriate version numbers as below. ``` java -Djavax.net.ssl.trustStore=src/test/resources/acumosTrustStore.jks \ -Djavax.net.ssl.trustStorePassword=acumos \ -jar target/gateway-1.x.y-SNAPSHOT.jar \ --spring.profiles.active="default,acumosa" ``` ``` java -Djavax.net.ssl.trustStore=src/test/resources/acumosTrustStore.jks \ -Djavax.net.ssl.trustStorePassword=acumos \ -jar target/gateway-1.x.y-SNAPSHOT.jar \ --spring.profiles.active="default,acumosb" ``` #### 4. REST Interfaces - The **federation interface allows access via the federation gateway to information available in an Acumos system**. - The main exposed elements are **catalogs, solutions, artifacts, and documents**. - The federation gateway **allows pre-registered peers to retrieve a list of catalogs they are permitted to view**. - The peers can then **list solutions and revisions of those solutions published in those catalogs**. They can then **retrieve metadata and content of artifacts referenced by those revisions**. - For each catalog a solution revision appears in, peers can **retrieve a description and document metadata and content**. - The gateway requires **mutual authentication** that is the client will be **required to present a certificate**. - The gateway identifies a client as a **pre-registered peer based on the certificates’ subjectName**. #### 5. API - All APIs **except the artifact and document content APIs encode the response in JSON**. - The artifact and document **content APIs return unencoded data**. - For other APIs, there is a **top level envelope** containing error information, and under the entry ‘responseBody’ it contains the actual content. All identifiers are UUIDs. The following endpoints are defined: - **/catalogs** List all visible (e.g. public) catalogs. - **/solutions?catalogId={catalogId}** List all solutions published to the specified catalog. - **/solutions/{solutionId}** Retrieve the specified solution, and its revisions. - **/solutions/{solutionId}/revisions** Retrieve the revisions of the specified solution. - **/solutions/{solutionId}/revisions/{revisionId}[?catalogId={catalogId}]** Retrieve details of the specified revision, including its artifacts. If the optional catalogId query parameter is specified, the description of the revision and any documents associated with it, from the specified catalog, will also be included. - **/solutions/{solutionId}/revisions/{revisionId}/artifacts** List all artifacts attached to a particular revision - **/artifacts/{artifactId}/content** Retrieve the content of the specified artifact - **/revision/{revisionId}/documents?catalogId={catalogId}** Retrieve documents associated with the specified revision in the specified catalog. - **/documents/{documentId}/content** Retrieve the content of the specified document --- ### 3. Try Solve NEXUS Installation for Acumos AI >This deployment is continuation from [yesterday's notes]( https://hackmd.io/@christofel04/TEEP_Daily_Notes_11_8_2020 ). In this note Writer use [Previous Notes](https://hackmd.io/@christofel04/TEEP_Daily_Notes_10_7_2020) as study sources. After checking the Open Shift cluster installation , there are issues when setting cluster up in Open Shift. :::danger In Writer's installation the setting freeze after installing kubeflow as below. ![](https://i.imgur.com/27U2fYv.png) But after look at [reference](https://github.com/openshift/origin/issues/21253) , it should end up showing Open Shift user account and server as below.. ![](https://i.imgur.com/WoHOBzT.png) ::: :::warning After Writer search in internet and from reference it found out this issue **often happens for CentOS with Open Shift veruson 3.11 like Writer's**. ::: :::info **Next Writer Will Continue to Solve the Open Shift Issue in NTUST Server** ::: --- ## Reference 1. https://github.com/nokia/ONAP-VESPA 2. https://docs.acumos.org/en/boreas/ 3. https://docs.acumos.org/en/boreas/submodules/federation/docs/developer-guide.html