# Red Hat Workshop @ K8s summit

## 0. 環境介紹

1. Go to the info page click on VScode server link, use password provided in the table to login.

2. Go to info page click on Automation controller link, use username and password provided in the table to login.

3. Go to the Automation controller console and click on jobs under Views. Jobs section will show all triggered job templates status by the EDA.

4. Go back to the VScode server console and open a terminal.

5. Split terminal so that we can execute and observe the command output side by side.

## 1. 自動新增Resource Quota

1. Go to the first terminal and run the following command to observe the Event-driven ansible logs.

sudo podman logs -f resource_quota


2. Go to the second terminal, Change the directory and run oc command to create namespace (rocketchat) which will eventually trigger the event.

cd $HOME/demo/test-events/
oc create -f 1-test-resource-quota-on-namespace.yml


**觀察腳本執行狀況**
Observe EDA pod has caught the event for new namespace and triggered the action.

Observe the Automation controller has a new job running.

Observe that the new namespace (rocketchat) has a quota set. Run the following command on the second terminal.

oc get resourcequota -n rocketchat

**Clean-up**
Step-1: Do Ctrl+c in the first terminal and clear the screen.
Step-2: Clear the second terminal screen.

## 2. 自動備份PVC

1. Go to the first terminal and run the following command
to observe the Event-driven ansible logs.

sudo podman logs -f volume_snapshot

2. Go to the second terminal, Change the directory and run oc command to create the rocketchat application in the rocketchat namespace which will eventually trigger the event.

cd $HOME/demo/test-events/
oc create -f 2-test-volume-snapshot.yml

**觀察腳本執行狀況**
1. Observe EDA pod has caught the ADDED event for new persistentvolumeclaim which triggered the action.
2. Observe the Automation controller has a new job running.
3. Observe that the new persistentvolumeclaim has a screenshot in rocketchat namespace. Run the following command on the second terminal.

oc get volumesnapshot -n rocketchat

**Clean-up**
Step-1: Do Ctrl+c in the first terminal and clear the screen.
Step-2: Clear the second terminal screen.

## 3. 自動設定Ingress加密

1. Go to the first terminal and run the following command to observe the Event-driven ansible logs.

sudo podman logs -f patch_route

2. Go to the second terminal, Change the directory and run oc command to create the route for rocketchat application in the rocketchat namespace which will eventually trigger the event.

cd $HOME/demo/test-events/
oc create -f 3-test-route-with-cert.yml

Observe:
1. Observe EDA pod has caught the ADDED event for the new route and triggered the action.
2. Observe the Automation controller has a new job running.
3. Observe that the new route has been patched with a certificate. Run the following command on the second terminal.

oc get route -n rocketchat -o yaml | grep cert-manager

**Clean-up**
Step-1: Do Ctrl+c in the first terminal and clear the screen.
Step-2: Clear the second terminal screen.

## 4. 自動收集除錯log (OCP EDA for auto oc adm inspect)

### Add bastion info in AAP
1. Create Inventory - Bastion



2. Add host in the inventory




3. Create credential for bastion

### Create new playbook for oc adm inspect


1. in bastion, clone git repo

cd ~
git clone https://gitea.apps.cluster-5q2s5.5q2s5.sandbox1411.opentlc.com/lab-user/event-driven-ansible
cd event-driven-ansible/automation_controller

2. create playbook
vi oc-inspect.yml

- name: oc adm inspect
  hosts: bastion.5q2s5.sandbox1411.opentlc.com #要替換成host的名稱
  gather_facts: no
  vars:
    ns: "{{ ansible_eda.event.resource.metadata.namespace }}"

  tasks:
    - name: Create inspect file
      shell:
         "oc adm inspect ns/{{ ns }} --kubeconfig /home/lab-user/.kube/config"
      register: lsout
      

3. push

git add *
git commit -am "playbook for oc adm inspect"
git push
git log --graph


### Create job template in AAP

1. git server 更新AAP Project


新增template取名為 `oc-inspect`


### Create new rulebook runner

cd /opt/podman/eda/
cp -r resource_quota oc_inspect


- Open /opt/podman/eda/oc_inspect in vscode console

- Replace `resource_quota` with `oc_inspect`

- Replace rulebook.yml with following

---
- name: Listen for unhealthy+warning event
  hosts: all
  sources:
    - sabre1041.eda.k8s:
        api_version: v1
        kind: Event
        namespace: jace #自行替換成新預計的ns名稱
  rules:
    - name: Debug
      condition: 
        event.resource.reason == "Unhealthy" and event.resource.type == "Warning"
      throttle:
        once_within: 5 minutes
        group_by_attributes:
          - event.resource.metadata.namespace
          - event.resource.involvedObject.name
      action:
        run_job_template:
          name: oc-inspect #必須對應AAP內的template 名稱
          organization: Default


cd /opt/podman/eda/oc_inspect
sudo podman-compose up


### 測試 (建立一個會自動probe fail的pod)

oc new-project jace

cat << EOF | oc apply -f -
apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-exec
spec:
  containers:
  - name: liveness
    securityContext:
      allowPrivilegeEscalation: false
      seccompProfile:
        type: RuntimeDefault
      capabilities:
        drop:
        - ALL
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
    image: k8s.gcr.io/busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5
EOF



### 驗證inspect是否成功



## 5. [Option] 自動打包inspect file並建立support case

0. 於Red Hat網頁生成你的Red Hat Token

https://access.redhat.com/management/api


1. 接續lab 4, 將`oc-inspect.yml` 替換成以下內容

- name: oc adm inspect
  hosts: bastion.9gfrm.sandbox2486.opentlc.com
  gather_facts: no
  vars:
    ns: "{{ ansible_eda.event.resource.metadata.namespace }}"
  tasks:
    - name: Create inspect file
      shell:
         "rm -rf ./inspect.local.{{ ns }} && oc adm inspect ns/{{ ns }} --dest-dir=./inspect.local.{{ ns }} --kubeconfig /home/lab-user/.kube/config"
      register: lsout
    - name: Compress with tar
      command: "tar -czf inspect.local.{{ ns }}.tar.gz inspect.local.{{ ns }}"
      when: lsout.rc == 0
    - name: Create support case
      shell: |
        RH_PORTAL_TOKEN=$(<rh-customer-portal-token) //替換TOKEN
        TOKEN=$(curl https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token -d grant_type=refresh_token -d client_id=rhsm-api -d refresh_token=$RH_PORTAL_TOKEN | jq --raw-output .access_token)
        response=$(curl -sS -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" --data '{
          "product": "OpenShift Container Platform",
          "version": "4.12",
          "caseType": "RCA Only",
          "description": "My pod crashed last night, I was wondering about RCA",
          "environment": "staging",
          "caseLanguage": "zh_TW",
          "severity": 3,
          "summary": "Summary message here."
        }' "https://api.access.redhat.com/support/v1/cases")
        echo $response | jq -r '.location[0] | capture("/cases/(?<case_no>[0-9]+)") | .case_no'
      register: case_number
      when: lsout.rc == 0
    - name: Upload logs to support case
      shell: |
        RH_PORTAL_TOKEN=$(<rh-customer-portal-token)
        TOKEN=$(curl https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token -d grant_type=refresh_token -d client_id=rhsm-api -d refresh_token=$RH_PORTAL_TOKEN | jq --raw-output .access_token)
        CASE_NO={{ case_number.stdout }}
        curl -X POST -F "file=@inspect.local.{{ ns }}.tar.gz" -H "Authorization: Bearer $TOKEN" https://api.access.redhat.com/support/v1/cases/${CASE_NO}/attachments
      when: lsout.rc == 0 and case_number.stdout is defined

2. 將原本的範例pod 替換成Deployment以及Configmap
**html-configmap.yaml**

cat << EOF | oc apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: html-content
data:
  index.html: |
    <h1>Hello world! Welcome to K8s Summit 2023</h1>
EOF

# nginx-hello-world.yaml
cat << EOF | oc apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-hello-world
  labels:
    app: nginx-hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-hello-world
  template:
    metadata:
      labels:
        app: nginx-hello-world
    spec:
      volumes:
      - name: html-volume
        configMap:
          name: html-content
      containers:
        - name: nginx
          image: "quay.io/redhattraining/hello-nginx:v1.0"
          securityContext:
            allowPrivilegeEscalation: false
            runAsNonRoot: true
            seccompProfile:
              type: RuntimeDefault
            capabilities:
              drop:
              - ALL
          volumeMounts:
            - name: html-volume
              mountPath: /usr/share/nginx/html
          resources:
            requests:
              memory: "64Mi"
              cpu: "250m"
            limits:
              memory: "128Mi"
              cpu: "500m"        
          livenessProbe:
            exec:
              command:
              - /bin/sh
              - -c
              - curl -s http://localhost:8080 | grep -q "world"
            initialDelaySeconds: 5
            periodSeconds: 5
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          tty: true
          stdin: true
          serviceAccount: default
      terminationGracePeriodSeconds: 5
EOF

oc expose deploy/nginx-hello-world --port 8080
oc expose svc nginx-hello-world

## 參考資料

rulebook 寫法
https://www.redhat.com/en/topics/automation/what-is-an-ansible-rulebook
https://ansible.readthedocs.io/projects/rulebook/en/stable/rules.html