---
tags: docker, cicd, gitalb, runner
---

# Docker Swarm Deploy

## 事前準備
1. 設定 Linux proxy, 令機器可以存取外網 ( apt-get和 docker要另外設定自己的proxy )

export http_proxy=""http://10.40.40.xx:3128"";
export https_proxy=""http://10.40.40.xx:3128"";

2. 設定 apt proxy, `vi /etc/apt/apt.conf`

Acquire::http::Proxy "http://10.40.40.xx:3128";

3. 設定 docker service proxy

sudo mkdir /etc/systemd/system/docker.service.d
sudo vi /etc/systemd/system/docker.service.d/http-proxy.conf

[Service]
Environment="HTTP_PROXY=http://10.40.40.xx:3128/"
Environment="HTTPS_PROXY=http://10.40.40.xx:3128/"

sudo systemctl daemon-reload
sudo systemctl restart docker.service

## 安裝 Docker

sudo apt install apt-transport-https ca-certificates curl software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic test"

sudo apt update

# for debain
sudo apt install docker-ce

# for ubuntu
sudo apt install docker.io

## 安裝 Gitlab-Runner

sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"

# 赋予权限
sudo chmod +x /usr/local/bin/gitlab-runner

# 添加GitLab CI/CD 用户（用户名为gitlab-runner）
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# 安装gitlab-runner服务 并且 初始化working-directory
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner

# 启动gitlab-runner服务
sudo gitlab-runner start

# 接下来查看ExecStart中的--working-directory 是不是 /home/gitlab-runner
# 因为如果使用的是系统文件夹，会导致构建时拿不到权限
cat /etc/systemd/system/gitlab-runner.service

# 將gitlab-runner加入docker群組
sudo groupadd docker
sudo usermod -aG docker gitlab-runner
sudo usermod -aG docker twteam

# 把 gitlab-runner user 下的 .bash_logout 刪除 https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26605
sudo rm /home/gitlab-runner/.bash_logout

# 將 runner與 gitlab連結，需要輸入 url跟 token
gitlab-runner install

## Create Docker Swarm Cluster
### 初始化 Cluster

# 打完指令後會產生一組 join的指令，在其餘 worker輸入
docker swarm init

# 加入以建立的 docker swarm 
docker swarm join --token [token] [master node ip]

# 查詢 join token
docker swarm join-token -q manager

### Create Stack

docker stack deploy -c docker-compose.yml $StackNname

### Update Service

# restart service when changed
docker service update --force $ServiceName

# restart service force
docker service update --force $ServiceName

# update replicas number
docker service update --replicas 3 $ServiceName

### Manage Node

## list all node status
docker node ls
Example : 
ID                            HOSTNAME     STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
k1x2iookd5xlgnhcnq9g596bu *   SIT-Swarm1   Ready     Active         Leader           20.10.7
irbodx5fed8n1d67czjym42w7     SIT-Swarm2   Ready     Active         Reachable        20.10.7
xplu602zwwjwsynyjpbjs8dum     SIT-Swarm3   Ready     Active         Reachable        20.10.12
uihdibpqpar4amj5ssnkkdy1z     SIT-Swarm4   Ready     Active                          20.10.12

# 將 node disable
docker node update --availability drain [NodeName]
# 將 node pause (令新的服務不要 depoly在這一台，但是已經在跑的服務不受影響)
docker node update --availability pause [NodeName]
# 將 node enable
docker node update --availability active [NodeName]

### Restart Docker Daemon
這是將 docker的常駐服務整個重啟，當服務的 cpu有不正常使用時可以嘗試使用他 (重啟前需要將 node下線)

systemctl restart docker.service

### Clean Docker Image And Build Cache

docker image prune
 
# or
# This will remove :
# all stopped containers
# all networks not used by at least one container
# all images without at least one container associated to them
# all build cache
docker system prune -a

**自動清理 Image跟 Build Cache：**
我們透過 linux提供的 crontab來做定時清理的功能，先開啟 cronjob的編輯頁面`crontab -e`，第一次使用時會問你要用哪一種編輯器，我選擇 `2. vim`

然後複製以下指令 (詳細的操作請參考這篇 Linux 設定 crontab 例行性工作排程教學與範例 - G. T. Wang (gtwang.org))，目前全部機器都是設定星期一的 00:00:00執行清理的動作，有些機器的時區不同，會有幾個小時的差異

# ┌───────────── 分鐘   (0 - 59)
# │ ┌─────────── 小時   (0 - 23)
# │ │ ┌───────── 日     (1 - 31)
# │ │ │ ┌─────── 月     (1 - 12)
# │ │ │ │ ┌───── 星期幾 (0 - 7，0 是週日，6 是週六，7 也是週日)
# │ │ │ │ │
# * * * * * /path/to/command

# every date clear docker image, build cache at Monday 00:00:00 +0 (08:00:00 +8)
# docker system prune -a -f
00 00 * * 1 docker system prune -a -f

## Debug Container

# Show all container
docker container ls

# Show container logs
docker logs [container id | name]

# 進入 containter 
docker exec -ti [container id | name] bash

# 進入 container時出現錯誤訊息 : OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown
docker exec -ti [container id | name] /bin/sh

# 查詢 Docker System Log
cat /var/log/syslog
# or
journalctl -fu docker.service

### Clear Docker Console Output(stdin)
1. 檢查 container的 log大小

cd /var/lib/docker/containers/
du -hs *

2. 移除過大的 log file

cd /var/lib/docker/containers/[container id]
truncate -s 0 [container id]-json.log

## Monitor
### Swarmpi

docker run -it --rm \
  --name swarmpit-installer \
  --volume /var/run/docker.sock:/var/run/docker.sock \
swarmpit/install:1.9

### Portainer
Yaml檔：

version: '3.2'

services:
  agent:
    image: portainer/agent:2.13.1
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/docker/volumes:/var/lib/docker/volumes
    networks:
      - agent_network
    deploy:
      mode: global
      placement:
        constraints: [node.platform.os == linux]

  portainer:
    image: portainer/portainer-ce:2.13.1
    command: -H tcp://tasks.agent:9001 --tlsskipverify
    ports:
      - "9443:9443"
      - "9880:9000"
    volumes:
      - portainer_data:/data
    networks:
      - agent_network
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints: [node.labels.manager==1]

networks:
  agent_network:
    driver: overlay
    attachable: true

volumes:
  portainer_data:

Install：

docker stack deploy -c portainer-docker-compose.yml portainer