--- title: Tips to GitLabCI tags: Git --- # Gitlab-Runner https://docs.gitlab.com/runner/ ## Register a runner instance ### system mode ```bash= sudo gitlab-runner register ``` - conf_path: ==/etc/gitlab-runner/config.toml== ### user mode ```bash= gitlab-runner register ``` - conf_path: ==$HOME/.gitlab-runner/config.toml== :::info 定義 tag 時可以以逗號隔開寫入多個tags, 但若要指定這個 runner 也必須明確宣告所有 tags ::: :::warning :+1: 使用 user-mode 可能在 `mkdir` 會權限不足 (i.e., mkdir ~/builds) ::: ## Run ```bash= sudo gitlab-runner run # system-mode gitlab-runner run # user-mode ``` ### (Option) serve as a service ```bash= sudo gitlab-runner install -n "<service-name>" -u <user-name> sudo gitlab-runner start -n "<service-name>" ``` ## Executor 最常用的是 Docker excecutor 和 Shell exceutor 這些 runner 實際的執行目錄為: ``` <working-directory>/builds/<short-token>/<concurrent-id>/<namespace>/<project-name> ``` cache 位置為: ``` <working-directory>/cache/<namespace>/<project-name> ``` ### Docker Excecutor https://docs.gitlab.com/runner/install/docker.html#upgrade-version > 預設是在 container 內以 system-mode 運行 default `/home`: ``` /root/ ``` default build_dir: ``` /var/lib/docker/volumes/<volume-id>/_data/<project-name> ``` default cache_dir: ``` /cache/ ``` #### how to achieve persistent volume ? within `/etc/gitlab-runner/config.toml`(system-mode): ```yaml= [runner.docker] volumes = ["/cache", "<host_dir:container_dir>:rw"] ``` #### Install the docker image and get started ```bash= docker run -d \ --name gitlab-runner \ --restart always \ -v /etc/gitlab-runner:/etc/gitlab-runner \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner:latest ``` #### Register gitlab-runner ```bash= docker run \ --rm -it \ -v /etc/gitlab-runner:/etc/gitlab-runner \ gitlab/gitlab-runner register ``` #### Verify & delete dead gitlab-runner ```bash= docker run \ --rm -it \ -v /etc/gitlab-runner:/etc/gitlab-runner \ gitlab/gitlab-runner verify --delete ``` # Predefined variables reference https://docs.gitlab.com/ee/ci/variables/predefined_variables.html 定義常用的 system param，能讓 `.gitlab-ci.yml` 更易讀。 ### CI_PROJECT_NAME The name of the directory for the project. ### CI_COMMIT_REF_NAME The branch or tag name for which project is built. ### CI_COMMIT_BRANCH The commit branch name. # Dependencies in GitLab CI/CD https://docs.gitlab.com/ee/ci/caching/ ## cache vs artifacts | cache | artifacts | | -------- | -------- | | For storing project dependencies | Use for stage results that are passed between stages | |cache per job by using `cache:`|artifacts per job| |key, policy(pull,push), paths|paths| |隨後的==pipelines==可以使用|隨後同一個 pipeline上的 ==jobs== 可以使用| |Different projects cannot share| Different projects cannot share| | 取決於機器有沒有生成過 cache | 會上傳 gitlab, 之後從 gitlab 下載 | | 可能不是最新的,可以禁用(`cache: {}`) | 每次都去下載, 可以禁用(`dependencies: []`) | |不會自動清理, 需要定期清除|可以設置自動過期時間(default 30d)| :::danger :warning: the cache might be overwritten because caches are restored before artifacts. - [Get Started] ==> [caches] ==> [artifacts] ==> [Running] ::: :::info 每一次新的push trigger pipeline, 都會重新生成 cache `<cache-key>-<num>` 如果不去清除cache, cache 會永久保留在 Runner 上 ==cache 需要定期清除== ::: :::warning paths主要是来指定需要被缓存的文件路径，需要特别指出的是这里的 paths 是相对路径，是相对于gitlab中项目目录的路径，也就是说被缓存的文件都是在项目目录之内的。 ::: cache source: - https://docs.gitlab.com/ee/ci/caching/ - https://zhuanlan.zhihu.com/p/106971627 ## dependencies Restrict which artifacts are passed to a specific job by providing a list of jobs to fetch artifacts from. > 定義在 `job` 內, 用來指定要使用哪個 `job` 中 `artifacts` 內的路徑。 ```yaml= installing-dependencies: stage: prepare script: - composer install --prefer-dist --optimize-autoloader -n --no-interaction -v --no-suggest - composer dump-autoload --optimize artifacts: name: "vendor" untracked: true expire_in: 60 mins paths: - $CI_PROJECT_DIR/vendor code-review: stage: testing dependencies: - installing-dependencies script: - php vendor/bin/phpcs --config-set ignore_warnings_on_exit 1 - php vendor/bin/phpcs --standard=PSR2 -w --colors ./ ``` ## 禁用 artifacts 默認 artifacts 會自動在不同 stages 上傳輸, 以下配置可以避免多餘的建置時間 ```yaml= dependencies: [] ``` ## SSH amid gitlab-ci runner ```yaml= # Install ssh-agent - 'command -v ssh-agent >/dev/null || ( apt-get update -y && apt-get install openssh-client -y )' # Run ssh-agent - eval $(ssh-agent -s) # Add the SSH key - echo "$CONSOLE_PEM_KEY" | tr -d '\r' | ssh-add - # create ~/.ssh directory - mkdir -p ~/.ssh # All permission to user (gitlab-runner) - chmod 700 ~/.ssh # avoid checking when ssh connection - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' ``` :::danger docker executor (image: ruby:2.6) would be failed. shell executor 執行的是 ubuntu image. ::: # Issue ## Pseudo-terminal will not be allocated because stdin is not a terminal 禁止分配偽終端-使用ssh -T -> 不互動 強制分配偽終端-使用ssh -t 或 ssh -tt -> 互動 ## docker excecutor 內 build image 直白講就是在 container 內建立 container 會遇到問題 :::danger Cannot connect to the Docker daemon at unix:///var/run/docker.sock. ::: Resolve: 配置 `config.toml` ```yaml= volumes = ["/var/run/docker.sock:/var/run/docker.sock"] ``` ## docker executor 內 git clone 跟上面做法一樣, 把 secret 掛載進去 ```yaml= volumes = ["/home/x1twbm/.ssh:/root/.ssh:ro"] ``` # Reference - https://docs.gitlab.com/ee/ci/ssh_keys/ - http://zacksleo.top/2017/04/18/GitLab-CI%E4%B8%AD%E7%9A%84artifacts%E4%BD%BF%E7%94%A8%E7%A0%94%E7%A9%B6/ - https://www.jianshu.com/p/19fe0ce7ecec