# GitHub Action Examples

## Set up of the CI

This includes:
1. Set triggering rules for the job, example triggering rules could be based on:
- schedule (cron job);
- new PR/issue created;
- new commit to certain branch;
- a special comment posted into a issue/PR;
- a manual trigger (dispatch).
2. Defining input variables for manual triggers. This is a nice thing which GitLab doesn't provide.

name: Name of the workflow

# job triggers
on:
  # on cron
  schedule:
    - cron: "0 3 * * *"
    
 push:
    branches:
      - production/v4
      - patch/*
      - prep-release/*
    paths-ignore:
      - 'docs/**'
      - '.github/**'
  pull_request:
    branches: [ production/v4 ]
    
  tag:
      - 'v*'
  
  # enable manual dispatch with input variables
  workflow_dispatch:
    inputs:
      tag-prefix:
        description: 'nightly tag prefix'
        default: ''
      feature-branch:
        description: 'feature branch to be used for the build.'
        default: develop
      cvmfs-deployment:
        description: 'whether to deploy the release to cvmfs'
        default: 'no'

## Matrix builds

You can have parallel jobs running inside different containers running different Linux distros.

  integration_tests:
    name: run_tests
    runs-on: daq
    strategy:
        matrix:
            test_name: ["t1", "t2", "t3"]
    defaults:
      run:
        shell: bash

    steps:

    - name: setup release and run tests
      run: |
          echo ${{ matrix.test_name }}

## Flow control

Making a job dependent on another job

jobs:
  job_1:
    name: job 1
    runs-on: ubuntu-latest
    defaults:
      run:
        shell: bash
    steps:
      - id: job_1_step_1
        run: |
          echo "step 1 in job 1"

  job_2:
    name: job 2
    runs-on: ubuntu-latest
    needs: job_1
    defaults:
      run:
        shell: bash
    steps:
      - id: job_2_step_1
        run: |
          echo "step 1 in job 2"
        

Marking a job step always run regardless results from previous job step.

      # for a job step
      - name: Clean up
        if: always()
        run: |
            docker image prune -f
            docker container prune -f
            docker network prune -f
            docker volume prune -f

## Dealing with build artifacts

Save build artifacts in a job.

    - name: upload a tarball
      uses: actions/upload-artifact@v3
      with:
        name: candidates_coredaq
        path: ${{ github.workspace }}/tarballs_for_upload/coredaq-${{ github.event.inputs.base-release }}-rc${{ github.event.inputs.build-number }}-a9.tar.gz

Use the tarball in another job.

      - name: Download the tarball
        uses: actions/download-artifact@v3
        with:
          name: candidates_coredaq
          path: ${{ github.workspace }}/docker-build

Download the artifact via GitHub client from command line

runid=`gh run -R <repo_name> list|grep Nightly|grep schedule|grep success|head -n 1|egrep -o '[[:digit:]]{10}'`

gh run download $run_id -D <output_dir> -n <artifact-1_name> -n <artifact-2_name>

## using cvmfs in GitHub Action

    # For each job needing cvmfs access, add the following step.
    steps:
    - uses: cvmfs-contrib/github-action-cvmfs@v3

The host OS `ubuntu-latest` will have `cvmfs` installed after this job step. If you use containers in your job, you can use cvmfs like the following:

    - name: start docker container with bind mount cvmfs
      run: |
        docker run --rm -v /cvmfs:/cvmfs:shared -v $GITHUB_WORKSPACE/scratch:/scratch ghcr.io/dune-daq/sl7-spack:latest /scratch/build_pkg.sh

## Caching files to speed up frequent builds

This is useful if your job constantly downloads a set of files. In the case of cvmfs, simply saving the cached files into the GitHub Action's cache would be helpful. The cache persists after the CI job completes. It can be loaded in the future CI runs before job starts. All you need is just adding one step in your job as below. The cache will be automatically updated if it's changed during your job.

    - name: Cache cvmfs cache
      id: cvmfs_cache
      uses: actions/cache@v3
      with:
        path: /var/lib/cvmfs/shared
        key: cachecvmfs

## Build/tag/push container images

You should be able to use GitHub's image registry with the default GitHub token associated with the CI job. But you would need to save your credentials in secrets if you want to use other registeries.

jobs:

  push_to_registries:
    name: Push Docker image to multiple registries
    runs-on: ubuntu-latest
    permissions:
      packages: write
      contents: read
    steps:
      - name: Check out the repo
        uses: actions/checkout@v3

      - name: Log in to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Log in to the Container registry
        uses: docker/login-action@v2
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: |
            dunedaq/sl7-minimal
            ghcr.io/DUNE-DAQ/sl7-minimal
          tags: |
            type=raw,value=latest
            type=ref,event=branch
            type=ref,event=tag

      - name: Build and push Docker images
        uses: docker/build-push-action@v3
        with:
          context: ./sl7-minimal
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
~

## Setting up self-hosted runner

## Other useful tricks

### Save a varaible from one job for use by other job steps

The following job step `make_nightly_tag` saves "nightly_tag" into the special env `GITHUB_OUTPUT`.

jobs:
  make_nightly_tag:
    name: create nightly tag
    runs-on: ubuntu-latest
    outputs:
      tag: ${{ steps.create_nightly_tag.outputs.nightly_tag }}
    defaults:
      run:
        shell: bash
    steps:
      - id: create_nightly_tag
        run: |
          date_tag=$(date +%y%m%d)
          echo "nightly_tag=${{ github.event.inputs.tag-prefix }}_DEV_${date_tag}_C8"  >>  "$GITHUB_OUTPUT"
          cat "$GITHUB_OUTPUT"

This is how to use it in another job.

  build_the_release:
    name: build_dev_release
    runs-on: ubuntu-latest
    needs: make_nightly_tag
    defaults:
      run:
        shell: bash

    steps:
    - name: printout_nightly_tag
      env:
        NIGHTLY_TAG: ${{needs.make_nightly_tag.outputs.tag}}
      run: |
          echo $NIGHTLY_TAG

### Adding badges to README.md for a certain CI workflow

Go to the "Actions" tab of the repository, and click one CI run of a specific CI workflow. Click the three dots icon beside the "Re-run all jobs" button, and choose "create status badge". A pop-up window will appear with the markdown string for the badge. You can copy and paste it into your README.md file.

### Adding new PRs/issues to a GitHub project

You will need to find out the project ID, and also have a token with proper scope of permissions (adding issues to a project etc) first.

name: Add issue to project
on:
  issues:
    types:
      - opened

jobs:
  track_issue:
    runs-on: ubuntu-latest
    steps:
      - name: Get project data
        env:
          GITHUB_TOKEN: ${{ secrets.BOT_CI_ISSUES }}
          ORGANIZATION: DUNE-DAQ
          PROJECT_NUMBER: 5
        run: |
          gh api graphql -f query='
            query($org: String!, $number: Int!) {
              organization(login: $org){
                projectV2(number: $number) {
                  id
                }
              }
            }' -f org=$ORGANIZATION -F number=$PROJECT_NUMBER > project_data.json

          echo 'PROJECT_ID='$(jq '.data.organization.projectV2.id' project_data.json) >> $GITHUB_ENV

      - name: Add issue to project
        env:
          GITHUB_TOKEN: ${{ secrets.BOT_CI_ISSUES }}
          ISSUE_ID: ${{ github.event.issue.node_id }}
        run: |
          item_id="$( gh api graphql -f query='
            mutation($project:ID!, $issue:ID!) {
              addProjectV2ItemById(input: {projectId: $project, contentId: $issue}) {
                item {
                  id
                }
              }
            }' -f project=$PROJECT_ID -f issue=$ISSUE_ID --jq '.data.addProjectV2ItemById.item.id')"

name: Add pull request to project
on:
  pull_request:
    types:
      - opened
jobs:
  track_issue:
    runs-on: ubuntu-latest
    steps:
      - name: Get project data
        env:
          GITHUB_TOKEN: ${{ secrets.BOT_CI_ISSUES }}
          ORGANIZATION: DUNE-DAQ
          PROJECT_NUMBER: 5
        run: |
          gh api graphql -f query='
            query($org: String!, $number: Int!) {
              organization(login: $org){
                projectV2(number: $number) {
                  id
                }
              }
            }' -f org=$ORGANIZATION -F number=$PROJECT_NUMBER > project_data.json

          echo 'PROJECT_ID='$(jq '.data.organization.projectV2.id' project_data.json) >> $GITHUB_ENV

      - name: Add issue to project
        env:
          GITHUB_TOKEN: ${{ secrets.BOT_CI_ISSUES }}
        run: |
            PR_NUMBER=${{ github.event.pull_request.number }}
            node_id=`curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{ github.repository }}/pulls/${PR_NUMBER} | jq '.node_id'`
            item_id="$( gh api graphql -f query='
            mutation($project:ID!, $issue:ID!) {
              addProjectV2ItemById(input: {projectId: $project, contentId: $issue}) {
                item {
                  id
                }
              }
            }' -f project=$PROJECT_ID -f issue=$node_id --jq '.data.addProjectV2ItemById.item.id')"