---
# System prepended metadata

title: 改善Init-Job(作法紀錄)
tags: [init-job, cron-job]

---

###### tags: `cron-job` `init-job`

# 改善Init-Job(作法紀錄)
http://10.78.26.44:30000/cloud_infra2/cloud-env-setup/-/tree/master/helm/ai-maker-init-job

目前作法是產生Job執行相對應的shell script，但是因為job執行完後無法重新trigger

### 目的
為了快速重新執行此job
* check iam ready (group)
* check k8s ns ready
* send rmq message for admin project
* send rmq message for admin user


### job起來執行的script
```
   # script
   echo "start checking iam service..."
    until nc -vz ${IAM_ENDPOINT}
    do
        echo "Waiting for iam service ready"
        sleep 5
    done

    echo "start checking k8s ns..."

    while [ -z "$(kubectl get ns | grep ${ADMIN_PROJECT})" ]
    do
        echo "Waiting for k8s ns created"
        sleep 5
    done

    while [ $(kubectl get ns ${ADMIN_PROJECT} -o json | jq .status.phase -r) != "Active" ]
    do
        echo "Waiting for k8s ns ready"
        sleep 5
    done

    echo "start publish rmq message for admin project [administrator] with id ${ADMIN_PROJECT}..."
    while [ $(curl -u ${RMQ_USER}:${RMQ_PASS} -o /dev/null -w %{http_code} -H "content-type:application/json" -X POST -d'{"properties":{},"routing_key":"system.notification","payload":"{\"method\":\"POST\",\"action\":\"/projects/'${ADMIN_PROJECT}'\",\"body\":{\"project\":{\"id\":\"'${ADMIN_PROJECT}'\",\"name\":\"administrator\",\"namespace\":\"ci.asus.com\",\"active\":true}}}","payload_encoding":"string"}' ${RMQ_ENDPOINT}/api/exchanges/%2f/cloud-infra/publish ) != "200" ]
    do
         echo "Publish RMQ Message for Admin Project Failed"
         sleep 10
    done

    echo "start publish rmq message for admin user [admin]..."
    while [ $(curl -u ${RMQ_USER}:${RMQ_PASS} -o /dev/null -w %{http_code} -H "content-type:application/json" -X POST -d'{"properties":{},"routing_key":"system.notification","payload":"{\"method\":\"POST\",\"action\":\"/users/98bf69aa-f7f7-4094-9599-94e804e6b1c9\",\"body\":{\"user\":{\"id\":\"98bf69aa-f7f7-4094-9599-94e804e6b1c9\",\"name\":\"admin\",\"email\":\"admin@ci.asus.com\",\"created_at\":\"2021-10-2710:54:46\"}}}","payload_encoding":"string"}' ${RMQ_ENDPOINT}/api/exchanges/%2f/cloud-infra/publish ) != "200" ]
    do
         echo "Publish RMQ Message for Admin User Failed"
         sleep 10
    done
  
    echo "All check finished!!!"
```

### 可行方法
- 法1. 產生cli直接執行 or 直接進pod打rmq api
    - 但是環境滿多的config需要變動(ex. rmq endpoint / iam endpoint)
- 法2. 起Pod來幫忙
    - 將原本的job改成建立成cron-job
        - set schedule as 
        `0 0 31 11 *` -> 11月31號啟動(所以不會啟動)
        - yaml type 改成cron-job
    - 建立pod來create job from target cron-job
        - 可以控制起來就跑一次
        - 後續可以重起pod or 手動trigger cron-job


#### 法2碰到的問題
- 順序問題
    - 利用init-container先確認cron-job exist
#### initContainer
check cronjob "init-job"
```
 command: ['sh', '-c', 'until [ -z "$(kubectl get cronjob initjob -o json | jq .items)" ]; do echo "wating for job ready"; sleep 3; done;']
```
#### container
call job from cronjob
```
kubectl create job --from=cronjob/<name of cronjob> <name of job>
```

![](https://i.imgur.com/lyzF3rC.png)


#### 重打流程
delete jobs (job pod會被砍)
delete pods (重新trigger一次job)
cron-job存在系統中(helm install 安裝)























