### :book: Study Note : Private Documentation

:::success
List the essential information of this study note.
1. How to install Orchestrator and NMS
2. Attach NMS to Orchestrator via swagger
3. How to install AGW
4. Attach AGW to Orchestrator
:::
---

---
### Prerequisite
#### 1. Docker & Docker Compose
**Docker Installation**
Set up Docker's apt repository.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Install the Docker packages.

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Verify that the Docker Engine installation is successful by running the hello-world image.

sudo docker run hello-world

**Docker Compose**
To download and install Compose standalone, run:

curl -SL https://github.com/docker/compose/releases/download/v2.24.5/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose

Apply executable permissions to the standalone binary in the target path for the installation.

sudo chmod +x /usr/local/bin/docker-compose

Test and execute compose commands using docker-compose.

# If the command docker-compose fails after installation, check your path. You can also create a symbolic link to /usr/bin or any other directory in your path. For example:
 sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

To verify that the installation was successful

docker-compose --version

#### 2. Golang
Download the tar file.

wget https://linuxfoundation.jfrog.io/artifactory/magma-blob/go1.18.3.linux-amd64.tar.gz

Extract the archive you downloaded into /usr/local, creating a Go tree in /usr/local/go.

sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.18.3.linux-amd64.tar.gz

Add /usr/local/go/bin to the PATH environment variable.

export PATH=$PATH:/usr/local/go/bin

Verify that you've installed Go by opening a command prompt and typing the following command

go version

You should expect something like this

go version go1.18.3 linux/amd64

#### 3. Pyenv
Update system packages.

sudo apt update -y

Install some necessary dependencies. If you are using zsh instead of bash, replace .bashrc for .zshrc.

apt install -y make build-essential libssl-dev zlib1g-dev libbz2-dev    libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev  libncursesw5-dev xz-utils tk-dev libffi-dev liblzma-dev python-openssl git

Note: For Ubuntu 22.04, use python3-openssl instead of python-openssl.

Clone pyenv repository.

git clone https://github.com/pyenv/pyenv.git ~/.pyenv

Configure pyenv.

echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bashrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bashrc
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -) "\nfi' >> ~/.bashrc
exec "$SHELL"

Create python virtual environment version 3.8.10.

pyenv install 3.8.10
pyenv global 3.8.10

Note: The pyenv installation might fail with a segmentation fault. Try using CFLAGS="-O2" pyenv install 3.8.10 in that case.
#### 4. PIP
Install pip3.

sudo apt install python3-pip

Install the following dependencies

pip3 install ansible fabric3 jsonpickle requests PyYAML

#### 5. Download Magma

git clone https://github.com/magma/magma.git
cd magma

# in case you want to use a specific version of Magma (for example v1.8)
git checkout v1.8

# to list all available releases
git tag -l

---
### 1. Install Orchestrator and NMS
#### 1.1 Install Orchestrator
**1. Generate Certificate**

mkdir -p ~/secrets/certs
cd ~/secrets/certs

${MAGMA_ROOT}/orc8r/cloud/deploy/scripts/self_sign_certs.sh yourdomain.com

${MAGMA_ROOT}/orc8r/cloud/deploy/scripts/create_application_certs.sh yourdomain.com

Finally, create the admin_operator.pfx file, protected with a password of your choosing

$ openssl pkcs12 -export -inkey admin_operator.key.pem -in admin_operator.pem -out admin_operator.pfx

Enter Export Password:
Verifying - Enter Export Password:

admin_operator.pem and admin_operator.key.pem are the files that NMS will use to authenticate itself with the Orchestrator API. admin_operator.pfx is for you to add to your keychain if you'd like to use the Orchestrator REST API directly (on macOS, double-click the admin_operator.pfx file and add it to your keychain, inputting the same password chosen above).

After generate certificate copy to new directory `home/raffie.ilham/secret/certs/`

cp ./* (detination directory)

The certs directory should now look like this

$ ls -1 ~/secrets/certs/

admin_operator.key.pem
admin_operator.pem
admin_operator.pfx
bootstrapper.key
certifier.key
certifier.pem
controller.crt
controller.key
fluentd.key
fluentd.pem
rootCA.pem
rootCA.key

**2. Build Orchestrator**
Here, we'll be building the Orchestrator docker containers.

HOST [magma]$ cd orc8r/cloud/docker
HOST [magma/orc8r/cloud/docker]$ ./build.py --all

This will build all the docker images for Orchestrator.

**3. Start Orchestrator**
before we start the orchestrator we need to set up some config file, the first file to config is `/magma/orc8r/cloud/docker/docker-compose.yml`

Here the example :

version: "3.7"

services:

  # Legacy controller service

  controller:
    volumes:
      - /home/raffie.ilham/secret/certs:/var/opt/magma/certs
      # log level can be modified at supervisord.conf without the need of building a new image
      - /home/raffie.ilham/magma/orc8r/cloud/docker/controller/supervisord.conf:/etc/supervisor/conf.d/supervisord.conf
    environment:
      TEST_MODE: "1"  # Run dev scripts on startup
      SERVICE_HOSTNAME: localhost  # All services are reachable through localhost
      SQL_DRIVER: postgres
      DATABASE_SOURCE: "dbname=magma_dev user=magma_dev password=magma_dev host=postgres sslmode=disable"
#      DATABASE_SOURCE: "magma_dev:magma_dev@(maria)/magma_dev"
      SQL_DIALECT: psql
      SERVICE_REGISTRY_MODE: yaml
      ORC8R_DOMAIN_NAME: magma.test
      VERSION_TAG: LOCAL-DEV
      HELM_VERSION_TAG: LOCAL-DEV
      MAGMA_PRINT_GRPC_PAYLOAD: 0
    links:
      - fluentd
    depends_on:
      - postgres
      - fluentd
    command: ["/bin/sh", "-c", "/usr/local/bin/wait-for-it.sh -s -t 30 postgres:5432 && /usr/bin/supervisord"]
    restart: always
    platform: linux/x86_64
  # If you want to use log aggregation on a Mac or Windows host, uncomment
  # the below block as-is
  # KP: host.docker.internal does not resolve to anything in Docker for linux
  # See https://github.com/docker/for-linux/issues/264 and
  # https://github.com/docker/libnetwork/pull/2348
  # If you're on a Linux host and want to use log aggregation, uncomment the
  # below block and update fluentd-address to ":24225". You will have to
  # restart the container before it sends logs to fluentd.
  # See https://github.com/moby/moby/issues/1143 for some ideas about
  # connecting the logging driver to your host localhost.
#    logging:
#      driver: fluentd
#      options:
#        fluentd-address: "host.docker.internal:24225"
#        tag: orc8r.controller
#        fluentd-retry-wait: "1s"
#        fluentd-max-retries: "120"

  # Non-core essential services

  postgres:
    image: postgres
    # Default is 64mb
    # Ref: https://stackoverflow.com/questions/30210362/how-to-increase-the-size-of-the-dev-shm-in-docker-container
    shm_size: '256mb'
    ports:
      - "5432:5432/tcp"
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: magma_dev
      POSTGRES_PASSWORD: magma_dev
      POSTGRES_DB: magma_dev
    command:
      - "postgres"
      - "-c"
      - "log_duration=true"
#      - "-c"
#      - "log_statement=all"
    restart: always

  maria:
    image: mariadb:10.2
    volumes:
      - mysql:/var/lib/mysql
    environment:
      MYSQL_ROOT_PASSWORD: magma
      MYSQL_DATABASE: magma_dev
      MYSQL_USER: magma_dev
      MYSQL_PASSWORD: magma_dev
      SQL_MODE: ANSI_QUOTES
    command: mysqld --sql_mode="ANSI_QUOTES"
    restart: always

  nginx:
    ports:
      - "7443:8443/tcp"   # controller gRPC port
      - "7444:8444/tcp"   # bootstrapper port
      - "9443:9443/tcp"   # API port
    volumes:
      - /home/raffie.ilham/secret/certs:/var/opt/magma/certs
    environment:
      CONTROLLER_HOSTNAME: controller.magma.test
      PROXY_BACKENDS: controller
      TEST_MODE: "1"
      RESOLVER: 127.0.0.11
      SERVICE_REGISTRY_MODE: yaml
      SSL_CERTIFICATE: "/var/opt/magma/certs/controller.crt"
      SSL_CERTIFICATE_KEY: "/var/opt/magma/certs/controller.key"
      SSL_CLIENT_CERTIFICATE: "/var/opt/magma/certs/certifier.pem"
    depends_on:
      - controller
    restart: always
#    logging:
#      driver: fluentd
#      options:
#        fluentd-address: "host.docker.internal:24225"
#        tag: orc8r.nginx
#        fluentd-retry-wait: "1s"
#        fluentd-max-retries: "120"

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.3.1
    container_name: elasticsearch
    environment:
      - discovery.type=single-node
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - elasticsearch:/usr/share/elasticsearch/data
    restart: always

  kibana:
    image: docker.elastic.co/kibana/kibana:7.3.1
    ports:
      - "5601:5601"
    links:
      - elasticsearch

  fluentd:
    build: ./fluentd
    container_name: fluentd
    links:
      - elasticsearch
    # 24224 for TLS secure forwarding, 24225 for unsecured docker logging
    ports:
      - "24224:24224"
      - "24224:24224/udp"
      - "24225:24225"
      - "24225:24225/udp"
    volumes:
      - ./fluentd/conf:/fluentd/etc
      - /home/raffie.ilham/secret/certs:/var/opt/magma/certs

volumes:
  pgdata:
  mysql:
  elasticsearch:

The second file is `/magma/orc8r/cloud/docker/controller/Dockerfile`

# ------------------------------------------------------------------------------
# Base: for tests, precommit, codegen, etc.
# ------------------------------------------------------------------------------
ARG PLATFORM=linux/amd64

FROM --platform=$PLATFORM ubuntu:xenial as base

ENV GO111MODULE on
ENV GOPATH ${USER}/go
ENV GOBIN /build/bin
ENV PATH ${PATH}:${GOBIN}:${GOPATH}/bin

# Used in Makefiles
ENV MAGMA_ROOT /src/magma

# Apt runtime deps
RUN apt-get update && apt-get install -y \
  bzr \
  curl \
  gcc \
  git \
  make \
  openjdk-8-jre-headless \
  unzip \
  vim \
  wget

# Golang 1.18
WORKDIR /usr/local
ARG GOLANG_VERSION="1.18.3"
RUN GO_TARBALL="go${GOLANG_VERSION}.linux-amd64.tar.gz"
RUN wget https://linuxfoundation.jfrog.io/artifactory/magma-blob/go1.18.3.linux-amd64.tar.gz
RUN tar -xzf go1.18.3.linux-amd64.tar.gz \
 && ln -s /usr/local/go/bin/go /usr/local/bin/go \
 && rm go1.18.3.linux-amd64.tar.gz

# Install goimports
# RUN go get golang.org/x/tools/cmd/goimports

# Protobuf compiler
# Apt has 2.x but we need 3.x
# See: https://grpc.io/docs/protoc-installation/
RUN curl -Lfs https://github.com/protocolbuffers/protobuf/releases/download/v3.10.0/protoc-3.10.0-linux-x86_64.zip -o protoc3.zip && \
    unzip protoc3.zip -d protoc3 && \
    mv protoc3/bin/protoc /bin/protoc && \
    chmod a+rx /bin/protoc && \
    mv protoc3/include/google /usr/include/ && \
    chmod -R a+Xr /usr/include/google && \
    rm -rf protoc3.zip protoc3

# ------------------------------------------------------------------------------
# Gocache: cache Go modules
# ------------------------------------------------------------------------------
FROM base as gocache

ARG MAGMA_MODULES="orc8r lte feg cwf dp"
RUN echo "export GOCACHE_MODULES=\"$(for m in $MAGMA_MODULES ; do echo -n /gomod/src/magma/$m ; echo -n ' ' ; done)\"" >> /etc/profile.d/env.sh

COPY gomod /gomod
RUN cat /etc/profile.d/env.sh
RUN . /etc/profile.d/env.sh && for m in $GOCACHE_MODULES ; do cd ${m}/cloud/go && echo ${m}/cloud/go && go mod download ; done

# ------------------------------------------------------------------------------
# Src: different src depending on MAGMA_MODULES
# ------------------------------------------------------------------------------
FROM gocache as src

ARG MAGMA_MODULES="orc8r lte feg cwf dp"
RUN echo "export MAGMA_MODULES=\"$(for m in $MAGMA_MODULES ; do echo -n /src/magma/$m ; echo -n ' ' ; done)\"" >> /etc/profile.d/env.sh

# Source code
COPY src /src
WORKDIR /src/magma/orc8r/cloud
RUN . /etc/profile.d/env.sh && make tools

# Configs
COPY configs /etc/magma/configs

# ------------------------------------------------------------------------------
# Builder: compile src
# ------------------------------------------------------------------------------
FROM src as builder

RUN . /etc/profile.d/env.sh && make build

# ------------------------------------------------------------------------------
# Production
# ------------------------------------------------------------------------------
FROM ubuntu:xenial

# Apt runtime deps
RUN apt-get update && apt-get install -y \
  daemontools \
  netcat \
  openssl \
  supervisor \
  unzip \
  wget \
&& rm -rf /var/lib/apt/lists/*

# Swagger UI
# See: https://github.com/swagger-api/swagger-ui
ARG SWAGGER_UI_VERSION="3.52.2"
RUN cd /tmp && \
  wget "https://github.com/swagger-api/swagger-ui/archive/v$SWAGGER_UI_VERSION.zip" && \
  unzip "v$SWAGGER_UI_VERSION.zip" -d swagger-ui && \
  mkdir -p /var/opt/magma/static/swagger-ui && \
  cp -r "swagger-ui/swagger-ui-$SWAGGER_UI_VERSION/dist" /var/opt/magma/static/swagger-ui

# Script to wait for DB to be reachable
COPY src/magma/orc8r/cloud/docker/wait-for-it.sh /usr/local/bin

# Empty envdir for overriding in production
RUN mkdir -p /var/opt/magma/envdir

# Build artifacts
ARG SWAGGER_FILES=src/magma/orc8r/cloud/go/services/obsidian/swagger
COPY --from=builder /${SWAGGER_FILES}/v1/index.html /var/opt/magma/static/swagger/v1/ui/index.html
COPY --from=builder /${SWAGGER_FILES}/v1/css/sidebar.css /var/opt/magma/static/swagger/v1/static/sidebar.css
COPY --from=builder /${SWAGGER_FILES}/v1/swagger.yml /var/opt/magma/static/swagger/v1/spec/swagger.yml
COPY --from=builder src/magma/orc8r/cloud/swagger /etc/magma/swagger
COPY --from=builder /build/bin /var/opt/magma/bin

# Supervisor configs
ARG CNTLR_FILES=src/magma/orc8r/cloud/docker/controller
COPY ${CNTLR_FILES}/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY ${CNTLR_FILES}/supervisor_logger.py /usr/local/lib/python2.7/dist-packages/supervisor_logger.py

# Scripts for dev mode
COPY ${CNTLR_FILES}/create_test_controller_certs /usr/local/bin/create_test_controller_certs

# Configs
COPY configs /etc/magma/configs

After config the file, now we go to start Orchestrator is as simple as:

HOST [magma/orc8r/cloud/docker]$ ./run.py

:::warning
If ./run.py cannot run properly, you can do the same way with `sudo docker-compose up -d`, but if this not working properly too like the image below:


you can try this solution
1. Docker compose installation problem
2. Typo when you config file
3. Pwd not in correct directory
4. You must run in root with sudo su

:::


After running properly, you can check with `sudo docker ps -a` to see all running docker.

:::warning
If the `orc8r-fluentd` not running like the image below, you must see the logs first


with command `docker logs [container id]`

the main problem is permission denied on `controller.key`
the solution is: run this command `~/secret/certs$ sudo chmod a+rw ./controller.key`
:::
**4. Change certficate ownership**
The owner and group of admin_operator.key.pem and admin_operator.pfx in /magma/.cache/test_certs/ are root. You need to change ownership of these files to your user with chown, e.g.

HOST [magma/orc8r/cloud/docker] sudo chown ${USER}:${USER} ../../../.cache/test_certs/admin_operator.key.pem
HOST [magma/orc8r/cloud/docker] sudo chown ${USER}:${USER} ../../../.cache/test_certs/admin_operator.pfx

**5. Test Swagger UI**
After starting the Orchestrator with HOST [magma/orc8r/cloud/docker]$ ./run.py and importing admin_operator.pfx, you should be able to visit the Swagger UI at `https://localhost:9443/swagger/v1/ui`.
Note: Change `localhost` with your ip.

:::warning
if cannot open, you must import the certificate to your website.
1. download `admin_operator.pfx` certificate key from VM to your PC
2. import the certificate with this [step](https://support.globalsign.com/digital-certificates/digital-certificate-installation/install-client-digital-certificate-firefox-windows)
:::

#### 1.2 Install NMS
**1. Config Hosts on VM and Windows**

on ubuntu:

sudo nano /etc/hosts


on windows:

C:\Windows\System32\Drivers\etc


**2. Pull NMS**

Magma provides an UI for configuring and monitoring the networks. To set up the NMS to talk to your local Orchestrator:

HOST [magma]$ cd nms
HOST [magma/nms] $ docker-compose up -d

**3. Connect NMS with Orchestrator**

HOST [magma/nms] $ ./scripts/dev_setup.sh

After this, you will be able to access the UI by visiting https://magma-test.localhost, and using the email [e-mail] and password password1234. We recommend Firefox or Chrome. If you see Gateway Error 502, don't worry, the NMS can take upto 60 seconds to finish starting up. Note that you will only see a network if you connected your local LTE gateway as described above.

magma-test is the default organization. Organizations are managed at host.localhost where you can log in with the same credentials.