# Use Github Actions for CI

###### tags: `github`

Notes about to run docker containers for testing in Github Actions.

#### Nginx docker container
Simple example to run nginx and check http status.

create github action worklfow folder and actions file

$ mkdir -p .github/workflows
$ cd .github/workflows
$ touch main.yaml

edit main.yaml

name: test run docker
on:
  push:
    branches:
      - master
jobs:
  run-nginx:
    runs-on: ubuntu-latest
    steps:
      - name: checkout repo
        uses: actions/checkout@v3
      #- run: curl -fsSL https://get.docker.com -o get-docker.sh
      - run: docker run -d --name nginx -p 8080:80 nginx
      - run: docker ps; sleep 2
      - name: test nginx
        run: |
          python3 -c "import requests; ret=requests.get('http://localhost:8080'); print(ret.status_code)"
          curl -s localhost:8080

Then commit and push to github; check actions output

#### Create test with a website-monitor repo.
I have a [website monitor tool](https://github.com/yujungcheng/website-monitor) for monitoring website status. It collects website status, push result to Kafka server and store into PostgreSQL database.

I create a workflow to build the Kafak and PostgreSQL containers, run the monitoring tool and verify whether it works success or not.

Although I do test locally before commit and push to github, this add additional validation and benefit when I dont have local testing environment.

###### workflow file

name: website monitor testing

on: [push]

env:
  TOPIC: monitor
  DB_NAME: webmonitor
  DB_PASSWORD: test_password

jobs:
  monitor-test:
    runs-on: ubuntu-20.04
    steps:
      - name: checkout repo
        uses: actions/checkout@v3

      - run: ls -al

      # run postgresql
      - name: run postgresql
        run: |
          docker run --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=${DB_PASSWORD} -d postgres:latest
          sleep 10

      - name: create database
        run: |
          docker exec postgres createdb --username postgres ${DB_NAME}

      # run zookeeper & kafka

      - name: add kafka entity to /etc/hosts
        run: echo '127.0.0.1 kafka' | sudo tee -a /etc/hosts

      - name: create a network
        run: |
          docker network create kafka-net --driver bridge
      - name: run zookeeper
        run: |
          docker run -d --network kafka-net --name zookeeper -e ALLOW_ANONYMOUS_LOGIN=yes bitnami/zookeeper:latest

      - name: run kafka server
        run: |
          docker run -d --network kafka-net -h kafka --name kafka \
          -e ALLOW_PLAINTEXT_LISTENER=yes \
          -e KAFKA_BROKER_ID=1 \
          -e KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 \
          -e KAFKA_CFG_LISTENERS=PLAINTEXT://kafka:9092 \
          -e KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 \
          -p 9092:9092 \
          bitnami/kafka:latest

      - name: list docker containers
        run: sleep 10; docker ps

      - name: create topic
        run: |
          docker exec kafka \
          kafka-topics.sh --create --topic ${TOPIC} --replication-factor 1 --partitions 1 --bootstrap-server kafka:9092

      - name: list topic
        run: |
          docker exec kafka \
          kafka-topics.sh --list --bootstrap-server kafka:9092

      - name: describe topic
        run: |
          docker exec kafka \
          kafka-topics.sh --describe --topic ${TOPIC} --bootstrap-server kafka:9092

      # install packages
      - name: install python packages
        run: |
          sudo apt update
          sudo apt install -y python3-daemon python3-requests python3-psycopg2
          sudo pip3 install pykafka

      # create config file
      - name: create config
        run: |
          cp sample.config.ini config.ini
          sed -i 's/user =/user = postgres/g' config.ini
          sed -i "s/password =/password = ${DB_PASSWORD}/g" config.ini
          cat config.ini

      - name: create websites yaml
        run: |
          cp sample.websites.yaml websites.yaml

      # run writer and checker
      - name: start checker and writer
        run: |
          ./run_checker.py --debug --notls --daemon --filelog
          ./run_writer.py --debug --notls --daemon --filelog
        
      - run: sleep 10

      - name: print checker log
        run: cat ./checker.log
      - name: print writer log
        run: cat ./writer.log

      # sleep 60 seconds and check database table
      - name: wait 60 seconds
        run: sleep 60

      - name: check website table
        run: |
          docker exec postgres psql -U postgres -d ${DB_NAME} -c 'SELECT * FROM website'

      - name: check topic table
        run: |
          docker exec postgres psql -U postgres -d ${DB_NAME} -c 'SELECT * FROM topic'

      # validate status data in database
      - name: get row count in status_history table
        run: |
          echo "DATA_COUNT=$(docker exec postgres psql -U postgres -d ${DB_NAME} -t -c 'SELECT COUNT(*) FROM status_history' | tr -d '[:space:]')" >> $GITHUB_ENV
      
      - name: validate data count
        if: ${{ env.DATA_COUNT == 0 }}
        uses: actions/github-script@v3
        with:
          script: |
            core.setFailed('No data collected in the database!!!')

      - name: another data count validation
        run: |
          if [ "${DATA_COUNT}" == "0" ]; then 
            echo "No data collected."
            /bin/false
          else
            echo "${DATA_COUNT} records collected."
          fi


#### Notes
- docker is pre-installed already on github-hosted runner env.
- Each job runs on an individual environment (a container/VM), therefore environment variables are not shared between jobs.
- It may not possible to get a job to access other job due to jobs run on different phyiscal node. For example, one job runs a database server and another job runs application to access the database.
- To set environment variable

run: echo "var_name=$(command output)" >> $GITHUB_ENV

- When use with "if", either

if: ${{ envA == value }}
OR 
if: env.DATA_COUNT == 'value'

- To fail a job, execute `/bin/false`, which give return code 1.

#### Todo and improve
- study about "set-output" as alternative validation (for example, ensure 'topic' created successfully).
- study about "actions/github-script@v3"
- run "nginx" container for testing instead query website on internet.
- use jina2 action to create config file

#### Reference
https://docs.github.com/en/actions/creating-actions/creating-a-docker-container-action

https://docs.github.com/en/actions/learn-github-actions/contexts#context-availability

https://github.community/t/sharing-a-variable-between-jobs/16967

https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts#passing-data-between-jobs-in-a-workflow

https://github.com/cypress-io/github-action/tree/master/.github/workflows

https://github.com/actions/virtual-environments
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable

https://github.com/marketplace/actions/jinja2-action