# MOD302 Data computation - Part 1 Cloud infrastructures
## Day 2

### Mission and Activity 3:

- Goal: use CI/CD to automate a Docker image building
- Tools: GitLab, Docker
- New concepts: CI/CD

**Expected outcomes:** A GitLab project with automated building of a Docker image by using CI/CD pipelines.

**Tasks:**
T3 - GitLab CI/CD first steps:
- GitLab login setup
- add your ssh public key to your account
- create a personal token with the `api`, save it somewhere!
- verify that you have a group gitlab runner available
- create a new project inside https://gitlab.hevs.ch/ai-systems-lab/mod302/mod302-2024-submissions/week2
- submit your files used to build the docker image in the previous activity
- create a GitLab pipeline to automate the building of your docker image
- declare only one stage (for now)
- use `docker:24.0.5-dind` as service and image
- do `docker login`, `build` and `push`

**Tip:** to login you can use the user `gitlab-ci-token` and the `$CI_JOB_TOKEN` password.

**Questions:**

At the end of the activity, you should be able to answer these questions and argument your answer.

- what is CI?
- what are the advantages of using it?

**Evaluation:** Checkpoint: teacher checks code submitted to HEVS' GitLab.

**References:**
- Get started with GitLab CI/CD https://docs.gitlab.com/ee/ci/
- CI/CD variables https://docs.gitlab.com/ee/ci/variables/index.html
- Install Azure CLI https://learn.microsoft.com/en-us/cli/azure/install-azure-cli

### Mission and Activity 4:
- Goal: use DevOps, specifically GitOps, to fully automate the deployment of an application
- Tools: GitLab CI/CD, Docker, Terraform, Azure
- New concepts: DevOps

**Expected outcomes:** A GitLab project that configures and builds the necessary docker images and gets automatically deployed to Azure using Terraform via CI pipelines.
Minimum requirements:
- a front-end (e.g. web application)
- a back-end (e.g. database)
- at least one docker image configured and built with GitLab CI pipelines
- the application should be entirely deployed on Azure
- at least one stage for: `setup`, `build` and `deploy`.

**Tasks:**
T4 - Azure configuration and setup:
- Azure account setup
- sign up for Azure for Students with your hes-so account https://portal.azure.com. You will need the subscription id for the integration with Terraform and GitLab.
- install the Azure CLI in your laptop and execute the following:
- login
`az login`

- create a contributor for rbac `az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/<SUBSCRIPTION_ID>"`
- Project setup
- create a project inside https://gitlab.hevs.ch/ai-systems-lab/mod302/mod302-2024-submissions/week2
- clone the project locally
- Using Azure CLI
- set the needed environment variables

    ARM_CLIENT_ID = "<APPID_VALUE>"
    ARM_CLIENT_SECRET = "<PASSWORD_VALUE>"
    ARM_SUBSCRIPTION_ID = "<SUBSCRIPTION_ID>"
    ARM_TENANT_ID = "<TENANT_VALUE>"

- setup the GitLab token as environment variable

    GITLAB_ACCESS_TOKEN = <YOUR-ACCESS-TOKEN>

- before running `terraform init` and/or `terraform apply`, copy the right configuration from GitLab. Go to **Operate > Terraform states**

    terraform init `
    -backend-config="address=<PROJECT>/terraform/state/$Env:TF_STATE_NAME" `
    -backend-config="lock_address=<PROJECT>/terraform/state/$Env:TF_STATE_NAME/lock" `
    -backend-config="unlock_address=<PROJECT>/terraform/state/$Env:TF_STATE_NAME/lock" `
    -backend-config="username=<USER_NAME>" `
    -backend-config="password=$Env:GITLAB_ACCESS_TOKEN" `
    -backend-config="lock_method=POST" `
    -backend-config="unlock_method=DELETE" `
    -backend-config="retry_wait_min=5"

- Using GitLab (to test Terraform commands that can later be automated)
- you need to setup the following environment variables

| Variable Name | Value | Flag |
|---------------------|:--------------------------------------------------------------------------------------------------|:--------:|
| ADDRESS | https<nolink>://gitlab.hevs.ch/api/v4/projects/`<PROJECT_ID>`/terraform/state/$TF_STATE_NAME | Expanded |
| ARM_CLIENT_ID | <YOUR_ARM_CLIENT_ID> | Masked |
| ARM_CLIENT_SECRET | <YOUR_ARM_CLIENT_SECRET> | Masked |
| ARM_SUBSCRIPTION_ID | <YOUR_ARM_SUBSCRIPTION_ID> | Masked |
| ARM_TENANT_ID | <YOUR_ARM_TENANT_ID> | Masked |
| GITLAB_ACCESS_TOKEN | <YOUR_GITLAB_ACCESS_TOKEN> | Masked |
| LOCK_ADDRESS | https<nolink>://gitlab.hevs.ch/api/v4/projects/`<PROJECT_ID>`/terraform/state/$TF_STATE_NAME/lock | Expanded |
| TF_STATE_NAME | $CI_COMMIT_BRANCH | Expanded |
| UNLOCK_ADDRESS | https<nolink>://gitlab.hevs.ch/api/v4/projects/`<PROJECT_ID>`/terraform/state/$TF_STATE_NAME/lock | Expanded |
| USERNAME | <YOUR_GITLAB_USERNAME> (Use the one for which the token was generated) | |

Additionally, setup variables to store a database admin credentials

| Variable Name | Value | Flag |
|---------------------|:--------------------------------------------------------------------------------------------------|:--------:|
| DB_PASS | <YOUR_DATABASE_ADMIN_PASSWORD> | Masked |
| DB_USER | <YOUR_DATABASE_ADMIN_USERNAME> (Can not be 'admin') | Masked |

- You can now use the GitLab pipelines to setup your mini-project.

T5 - (in pairs) Use GitOps to deploy a GitLab project on Azure with Terraform. The project should have a frontend - a web-facing service (can be based on nginx, Node js, Python+streamlit, or similar) and a backend - a DBMS (mysql, postgres, redis) and a database
- GitLab pipelines:
- configure as many stages as needed to automate the creation of docker images.
- there should be at least one stage for: `setup`, `build` and `deploy`.
- you should have at least one Dockerfile built with the pipelines.
- Managing sensitive data. You can hide sensitive data from showing on the pipelines by declaring a `variables.tf` file where the variables are declared, for example:

variable "db_admin_password" {
  description = "Password for the SQL server admin"
  sensitive = true
}

- Making changes: you should be able to make an update of the infrastructure by committing the code and the changes should automatically reflected in the new version of the Docker images. You do not need to make a demo of this during your presentation, as all the pipeline history can be seen in GitLab.

**Note** the important -and evaluated- points are not what the application actually does but rather that the setup is automatic by using DevOps (and that it works, of course!)

**Questions:**

At the end of the activity, you should be able to answer these questions and argument your answer.

- what is DevOps, what is GitOps?
- what is the usefulness of DevOps/GitOps?

**Evaluation:** GitLab code, presentation and demo changing something in the repository

**References:**

- Install Azure CLI https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt

- GitLab-managed Terraform state https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html

- IntelliJ Azure plugin https://learn.microsoft.com/en-us/azure/developer/java/toolkit-for-intellij/install-toolkit

- Azure login https://learn.microsoft.com/en-us/azure/developer/java/toolkit-for-intellij/sign-in-instructions
- Azure CLI https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli