---
tags: Kubernetes, Pod Resource Limit and Behaviors
description: Kubernetes Pod Resource Limit and Behaviors
robots: index, follow
---
<style>

html, body, .ui-content {
background-color: #333;
color: #ddd;
}

.markdown-body h1,
.markdown-body h2,
.markdown-body h3,
.markdown-body h4,
.markdown-body h5,
.markdown-body h6 {
color: #ddd;
}

.markdown-body h1,
.markdown-body h2 {
border-bottom-color: #ffffff69;
}

.markdown-body h1 .octicon-link,
.markdown-body h2 .octicon-link,
.markdown-body h3 .octicon-link,
.markdown-body h4 .octicon-link,
.markdown-body h5 .octicon-link,
.markdown-body h6 .octicon-link {
color: #fff;
}

.markdown-body img {
background-color: transparent;
}

.ui-toc-dropdown .nav>.active:focus>a, .ui-toc-dropdown .nav>.active:hover>a, .ui-toc-dropdown .nav>.active>a {
color: white;
border-left: 2px solid white;
}

.expand-toggle:hover,
.expand-toggle:focus,
.back-to-top:hover,
.back-to-top:focus,
.go-to-bottom:hover,
.go-to-bottom:focus {
color: white;
}

.ui-toc-dropdown {
background-color: #333;
}

.ui-toc-label.btn {
background-color: #191919;
color: white;
}

.ui-toc-dropdown .nav>li>a:focus,
.ui-toc-dropdown .nav>li>a:hover {
color: white;
border-left: 1px solid white;
}

.markdown-body blockquote {
color: #bcbcbc;
}

.markdown-body table tr {
background-color: #5f5f5f;
}

.markdown-body table tr:nth-child(2n) {
background-color: #4f4f4f;
}

.markdown-body code,
.markdown-body tt {
color: #eee;
background-color: rgba(230, 230, 230, 0.36);
}

a,
.open-files-container li.selected a {
color: #5EB7E0;
}

</style>
# Kubernetes Pod Resource Limit and Behaviors
在設計Pod的時候，大多的使用者不預期設置container的資源上限，使用上雖然不會有太大的問題，不過在特定的狀況下，一直讓container所使用的資源一路往主機的資源上限衝，不是一個好主意。
資源限制分成以下種類:
1. 運算資源
1. CPU
2. RAM
3. PID上限
2. 儲存資源
1. ephemeral Storage

:::info
1. Persistent Volume Clims(PVC)限制不在pod resource limit，在[Limit Storage Consumption](https://kubernetes.io/docs/tasks/administer-cluster/limit-storage-consumption/ "Limit Storage Consumption")。
:::

container資源限制時有以下兩種：
1. requests
最基本的資源需求，不能低於這個限制，例如cpu=2, cpu=1不行，必須大於等於限制值。
2. limits
這個容器能夠使用的資源最大值。

以下將介紹CPU與RAM在超過資源時的行為。

## CPU Limit
一個CPU在kubernetes中，等同各公有雲的1 vCPU/Core或Intel處理器實體機上的一個hyperthread。
可轉換為執行CPU時間，一個CPU可轉換為1000 millicpu簡寫為1000m，如果我們只想要0.5 CPU，可以寫成0.5或500m。

CPU Pod定義。

apiVersion: v1
kind: Pod
metadata:
  name: cpu-demo-1
spec:
  containers:
  - name: cpu-demo-1
    image: vish/stress
    resources:
      limits:
        cpu: "0.5"
      requests:
        cpu: "0.1"
    args:
    - -cpus
    - "2"

建立這個pod與狀態確認

inwin@master:~$ kubectl create -f cpu1.yaml
pod/cpu-demo-1 created

inwin@master:~$ kubectl get po
NAME         READY   STATUS    RESTARTS   AGE
cpu-demo-1   1/1     Running   0          29s

確認這個pod可以執行，CPU單位會自動轉換為m。

inwin@master:~$ kubectl describe po cpu-demo-1
....
....
  cpu-demo-1:
    Container ID:  docker://071146f3b85b1789358c5c9b90c98dbd7c75c7a3dc15a3b899df1ed4bd9fedb7
    Image:         vish/stress
    Image ID:      docker-pullable://vish/stress@sha256:b6456a3df6db5e063e1783153627947484a3db387be99e49708c70a9a15e7177
    Port:          <none>
    Host Port:     <none>
    Args:
      -cpus
      2
    State:          Running
      Started:      Wed, 11 Aug 2021 02:23:47 +0000
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:  500m
    Requests:
      cpu:        100m
....
....

:::info
1. stress image是壓測用的套件，在yaml中我們定義為2個cpu，雖然沒有達到stress的需求，但是pod不會因為這樣就異常。
:::

修改定義，將資源增加到100 cpu

apiVersion: v1
kind: Pod
metadata:
  name: cpu-demo-2
spec:
  containers:
  - name: cpu-demo-2
    image: vish/stress
    resources:
      limits:
        cpu: "100"
      requests:
        cpu: "100"
    args:
    - -cpus
    - "2"

inwin@master:~$ kubectl get po cpu-demo-2
NAME         READY   STATUS    RESTARTS   AGE
cpu-demo-2   0/1     Pending   0          6m19s

inwin@master:~$ kubectl describe po cpu-demo-2
....
....
Events:
  Type     Reason            Age    From               Message
  ----     ------            ----   ----               -------
  Warning  FailedScheduling  8m15s  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/controlplane: true}, that the pod didn't tolerate, 3 Insufficient cpu.
  Warning  FailedScheduling  8m14s  default-scheduler  0/4 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/controlplane: true}, that the pod didn't tolerate, 3 Insufficient cpu.
....
....

:::info
1. 這個時候我們會發現pod呈現pending狀態，因為worker node沒有100個cpu。
2. 等到有worker node擁有100個cpu時，[control loop](https://kubernetes.io/docs/concepts/architecture/controller/ "Controllers")會幫你把pod建立起來。
:::

## RAM Limit
RAM Pod定義，我們可以設定一個上限200MB的pod，最低需求為100MB。
stress的ram要求為150m，所以這個pod能夠正常的啟用。

apiVersion: v1
kind: Pod
metadata:
  name: memory-demo-1
spec:
  containers:
  - name: memory-demo-1
    image: polinux/stress
    resources:
      limits:
        memory: "200Mi"
      requests:
        memory: "100Mi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "1"]

inwin@master:~$ kubectl create -f ram1.yaml
pod/memory-demo-1 created

inwin@master:~$ kubectl get po memory-demo-1
NAME            READY   STATUS    RESTARTS   AGE
memory-demo-1   1/1     Running   0          2m3s

調整yaml定義，將ram需求改到300MB

apiVersion: v1
kind: Pod
metadata:
  name: memory-demo-2
spec:
  containers:
  - name: memory-demo-2
    image: polinux/stress
    resources:
      limits:
        memory: "200Mi"
      requests:
        memory: "100Mi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "300M", "--vm-hang", "1"]

觀察一下pod狀態。

inwin@master:~$ kubectl get po memory-demo-2
NAME            READY   STATUS      RESTARTS   AGE
memory-demo-2   0/1     OOMKilled   0          45s

inwin@master:~$ kubectl describe po memory-demo-2
....
....
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       OOMKilled
      Exit Code:    1
....
....
Events:
  Type     Reason     Age                From               Message
  ----     ------     ----               ----               -------
  Normal   Scheduled  89s                default-scheduler  Successfully assigned default/memory-demo-2 to worker1
  Normal   Pulled     72s                kubelet            Successfully pulled image "polinux/stress" in 5.076176467s
  Normal   Pulled     49s                kubelet            Successfully pulled image "polinux/stress" in 5.896584257s
  Normal   Pulling    22s (x3 over 77s)  kubelet            Pulling image "polinux/stress"
  Normal   Pulled     17s                kubelet            Successfully pulled image "polinux/stress" in 5.061148098s
  Normal   Created    11s (x3 over 65s)  kubelet            Created container memory-demo-2
  Normal   Started    5s (x3 over 56s)   kubelet            Started container memory-demo-2
  Warning  BackOff    3s (x4 over 37s)   kubelet            Back-off restarting failed container
....
....

inwin@master:~$ kubectl get po memory-demo-2
NAME            READY   STATUS             RESTARTS   AGE
memory-demo-2   0/1     CrashLoopBackOff   4          4m36s

:::info
1. 這個時候我們會發現超過ram限制的pod會被砍掉，進入無限重建狀態。
:::

## recommand
1. 網頁類型應用，可以改成deploy與[HPA](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ "Horizontal Pod Autoscaler")，在資源快超過的時候再建立一個pod出來一起處理。
2. 日常排程應用，調整limit值即可。

## Reference
1. [Offcial - Managing Resources for Containers](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ "")
2. [Offcial - Assign CPU Resources to Containers and Pods](https://kubernetes.io/docs/tasks/configure-pod-container/assign-cpu-resource/ "Assign CPU Resources to Containers and Pods")
3. [Offcial - Assign Memory Resources to Containers and Pods](https://kubernetes.io/docs/tasks/configure-pod-container/assign-memory-resource/ "Assign Memory Resources to Containers and Pods")
4. [Offcial - Limit Storage Consumption](https://kubernetes.io/docs/tasks/administer-cluster/limit-storage-consumption/ "Limit Storage Consumption")
5. [Offcial - Controllers](https://kubernetes.io/docs/concepts/architecture/controller/ "Controllers")
6. [Offcial - Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ "Horizontal Pod Autoscaler")