# Docker CI/CD
## 事先準備
* python(在 手動安裝、設定 使用)
```bash=
# 裝 python
$ sudo yum install -y python3
# 裝 pip
$ wget https://bootstrap.pypa.io/pip/2.7/get-pip.py
```
## 手動安裝、設定
* 建一個測試目錄
`mkdir -p iris`
* 新增訓練模型
`train_model.py`
```python=
# coding: utf-8
import pickle
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn import tree
# simple demo for traing and saving model
iris=datasets.load_iris()
x=iris.data
y=iris.target
#labels for iris dataset
labels ={
0: "setosa",
1: "versicolor",
2: "virginica"
}
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=.25)
classifier=tree.DecisionTreeClassifier()
classifier.fit(x_train,y_train)
predictions=classifier.predict(x_test)
#export the model
model_name = 'model.pkl'
print("finished training and dump the model as {0}".format(model_name))
pickle.dump(classifier, open(model_name,'wb'))
```
* 執行訓練模型
`$ python3 train_model.py`
* 沒有套件就裝
`$ pip install sklearn`
* 完成後
```bash=
$ python3 train_model.py
finished training and dump the model as model.pkl
$ ls
get-pip.py model.pkl train_model.py
```
* 辨識模型的server
`vim server.py`
```python=
# coding: utf-8
import pickle
from flask import Flask, request, jsonify
app = Flask(__name__)
# Load the model
model = pickle.load(open('model.pkl', 'rb'))
labels = {
0: "versicolor",
1: "setosa",
2: "virginica"
}
@app.route('/api', methods=['POST'])
def predict():
# Get the data from the POST request.
data = request.get_json(force = True)
predict = model.predict(data['feature'])
return jsonify(predict[0].tolist())
if __name__ == '__main__':
app.run(debug = True, host = '0.0.0.0')
```
* 執行 server.py
`$ python3 server.py`
* 一樣缺套件就裝
`$ pip install flask`
* 編輯客戶端(可再開一個終端或另一台機器)
`$ vim client.py`
```python=
# coding: utf-8
import requests
# Change the value of experience that you want to test
url = 'http://192.168.31.78:5000/api' # IP要改
feature = [[5.8, 4.0, 1.2, 0.2]]
labels ={
0: "setosa",
1: "versicolor",
2: "virginica"
}
r = requests.post(url,json={'feature': feature})
print(labels[r.json()])
```
* 執行
```bash=
$ python3 client.py
setosa
```
* 一樣缺套件就裝
`$ pip install requests`
## CI/CD
* 目錄建立
```bash=
$ mkdir CICD
$ cd CICD/
```
* 檔案建立
```bash=
$ touch requirements.txt Dockerfile
$ ls # train & server 跟前面是一樣的
Dockerfile requirements.txt server.py train_model.py
```
* requirements.txt
```bash=
sklearn # 需要安裝的套件
flask
requests
```
* Dockerfile
```dockerfile=
FROM nitincypher/docker-ubuntu-python-pip # 有 python 的 image
COPY ./requirements.txt /app/requirements.txt 把要安裝的套件檔案傳上去
WORKDIR /app
RUN pip install -r requirements.txt # 安裝檔案內的套件
COPY server.py /app
COPY train_model.py /app
CMD python /app/train_model.py && python /app/server.py
```
* 建立 image
`$ sudo docker build -t iris:1.0 .`
* 執行
```bash=
$ sudo docker run -d --name iris -p 5000:5000 iris:1.0
876053f103532d9ab1654c1c23580873e0e57c79620ac7bed264acb1270c7471
```
* 客戶端測試
```bash=
$ python3 client.py
setosa
```
### GitLab
* 準備兩台VM (d1 d2)
* 把 ssh pub 的 key 丟上 GitLab (兩台都要)
```=
# cd .ssh
# ls
authorized_keys id_rsa id_rsa.pub known_hosts
# cat id_rsa.pub
把 id_rsa.pub 複製到 GitLab
```
**完成畫面**
**回到主頁建一個用來弄 iris 的 project**
* VM1 (開發端)
* git
`$ sudo yum install git -y`
```bash=
$ git config --global user.name "[Name]"
$ git config --global user.email "[Email]"
$ cd CICD/ # 進到要佈署的目錄
$ git init
$ git remote add origin https://gitlab.com/xuan_lin1/test-iris.git
$ git add.
$ git commit -m "Initial commit(說明)"
$ git push -u origin master
```
* 上GitLab看上傳結果
* VM2 (運行端)
* git 安裝
`$ sudo yum install git -y`
* runner 安裝
`sudo curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64`
* 改變權限
`sudo chmod +x /usr/local/bin/gitlab-runner`
* 新增使用者, 給他家目錄
`sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash`
* 把 gitlab-runner 加到 docker 的群組, 這樣他也能使用 docker
`sudo usermod -aG docker gitlab-runner`
* gitlab-runner 設定
`/usr/local/bin/gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner`
* gitlab-runner 啟動
`/usr/local/bin/gitlab-runner start`
* 檢查
* 註冊
`gitlab-runner register`
* 複製Token
* 後續設定
* 回gitlab 檢查 runner 狀態
* 再回到 VM1
1. 在配置目錄下新增 .gitlab-ci.yml
```yaml=
stages:
- deploy
docker-deploy:
stage: deploy
script:
- docker build -t iris .
- if [ $(docker ps -aq --filter name=iris) ]; then docker rm -f iris; fi
- docker run -d -p 5000:5000 --name iris
tags:
- d2
```
2. 上傳
```bash=
$ git add .
$ git commit -m "yml commit"
[master 886c767] yml commit
1 file changed, 11 insertions(+)
create mode 100644 .gitlab-ci.yml
$ git push -uf origin master
. . .
. . .
. . .
remote:
To https://gitlab.com/xuan_lin1/iris2024.git
342f1b3..886c767 master -> master
Branch master set up to track remote branch master from origin.
```
3. 上傳檢查
**可以在 jobs 查看執行過程**
4. 測試 (在非runner 的機器執行前面做過的 client.py)
**client.py 的IP要設為runner IP**
Sign in with Wallet
Connect another wallet