---
tags: Kubernetes, HPA
description: Horizontal Pod Autoscaler, aka pod autoscaling
robots: index, follow
---
<style>
html, body, .ui-content {
background-color: #333;
color: #ddd;
}
.markdown-body h1,
.markdown-body h2,
.markdown-body h3,
.markdown-body h4,
.markdown-body h5,
.markdown-body h6 {
color: #ddd;
}
.markdown-body h1,
.markdown-body h2 {
border-bottom-color: #ffffff69;
}
.markdown-body h1 .octicon-link,
.markdown-body h2 .octicon-link,
.markdown-body h3 .octicon-link,
.markdown-body h4 .octicon-link,
.markdown-body h5 .octicon-link,
.markdown-body h6 .octicon-link {
color: #fff;
}
.markdown-body img {
background-color: transparent;
}
.ui-toc-dropdown .nav>.active:focus>a, .ui-toc-dropdown .nav>.active:hover>a, .ui-toc-dropdown .nav>.active>a {
color: white;
border-left: 2px solid white;
}
.expand-toggle:hover,
.expand-toggle:focus,
.back-to-top:hover,
.back-to-top:focus,
.go-to-bottom:hover,
.go-to-bottom:focus {
color: white;
}
.ui-toc-dropdown {
background-color: #333;
}
.ui-toc-label.btn {
background-color: #191919;
color: white;
}
.ui-toc-dropdown .nav>li>a:focus,
.ui-toc-dropdown .nav>li>a:hover {
color: white;
border-left: 1px solid white;
}
.markdown-body blockquote {
color: #bcbcbc;
}
.markdown-body table tr {
background-color: #5f5f5f;
}
.markdown-body table tr:nth-child(2n) {
background-color: #4f4f4f;
}
.markdown-body code,
.markdown-body tt {
color: #eee;
background-color: rgba(230, 230, 230, 0.36);
}
a,
.open-files-container li.selected a {
color: #5EB7E0;
}
</style>
# HPA - Horizontal Pod Autoscaler
HPA(Horizontal Pod Autoscaler)在其他技術中，又稱為autoscaling技術，例如能夠依據CPU、RAM的附載狀況，擴展虛擬機的數量，不過在Kubernetes的世界中，我們擴展的是Pod。
要搭配HPA最好的作法，就是使用deployment建立pod。
首先我們要確認metrics-server有在運作。
```shell=
student@master:~$ kubectl -n kube-system get deployments.apps metrics-server
NAME READY UP-TO-DATE AVAILABLE AGE
metrics-server 1/1 1 1 5d17h
```
如果沒有的話，可以到這邊[下載yaml](https://github.com/yansheng133/initk8sfortraining/blob/main/components.yaml "meter server")檔進行安裝。
```shell=
student@master:~$ kubectl create -f components.yaml
```
## 1. 建立web image
使用一個網頁服務(hpa-php5)，建立deployment資源，使用80 port，暫定為一個副本。
image中所使用的php source code，除了跑一些迴圈外，也會顯示source ip與回應的container還有時間。
```php=
<?php
$x = 0.0001;
for ($i = 0; $i <= 1000000; $i++) {
$x += sqrt($x);
}
if (!empty($_SERVER["HTTP_CLIENT_IP"])){
$ip = $_SERVER["HTTP_CLIENT_IP"];
}elseif(!empty($_SERVER["HTTP_X_FORWARDED_FOR"])){
$ip = $_SERVER["HTTP_X_FORWARDED_FOR"];
}else{
$ip = $_SERVER["REMOTE_ADDR"];
}
date_default_timezone_set('Asia/Taipei');
echo "Source IP:" . $ip . ", Response by: " . gethostname() . " -- " . date('Y/m/d H:i:s');
echo "\n";
?>
```
建立hpa-php5 image，如果你想要完整走過一遍的話，可以參考這邊的[建立指令](https://hackmd.io/qCalUX98TP6LafSJ2mCadg?view#3-build-elasticsearch-fluentd-image)。
```dockerfile=
FROM php:5-apache
COPY index.php /var/www/html/index.php
RUN chmod a+rx index.php
```
或者使用作者建好的[image](https://hub.docker.com/repository/docker/yansheng133/hpa-php5 "HPA PHP5")進行。
## 2. 建立HPA web服務
建立hpa web的yaml檔，同時要加入資源需求與限制(cpu: 200m, 500m)。
```yaml=
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: hpaweb
name: hpaweb
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: hpaweb
template:
metadata:
creationTimestamp: null
labels:
app: hpaweb
spec:
containers:
- image: yansheng133/hpa-php5:0.5
imagePullPolicy: IfNotPresent
name: hpa-php5
ports:
- containerPort: 80
protocol: TCP
resources:
limits:
cpu: 500m
requests:
cpu: 200m
```
建立hpa web deployment。
```shell=
student@master:~$ kubectl create -f hpaweb.yaml
deployment.apps/hpaweb created
```
使用export搭配NodePort讓服務對外。
```shell=
student@master:~$ kubectl expose deployment hpaweb --port=80 --target-port=80 --type=NodePort
service/hpaweb exposed
student@master:~$ kubectl get svc hpaweb
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hpaweb NodePort 10.106.90.127 <none> 80:30516/TCP 2s
```
## 3. 附載觀察與設定HPA
先觀察一下這些資源建立完畢後，hpaweb的附載狀況，我們應該會發現非常低。
```shell=
student@master:~$ kubectl top pod
NAME CPU(cores) MEMORY(bytes)
hpaweb-cf96cf8c4-k8gdf 1m 13Mi
```
此時先對他增加一些附載，觀察CPU的附載狀況會到多少，此時可以開兩個連線視窗進行觀察。
增加附載。
```shell=
student@master:~$ watch -n 0.1 curl 192.168.122.30:30516
...
...
...
Every 0.1s: curl 192.168.122.30:30516 master9013.inwinstack.lab: Sun Sep 19 07:48:03 2021
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 80 100 80 0 0 1081
0 --:--:-- --:--:-- --:--:-- 1095
Source IP:10.6.72.0, Response by: hpaweb-cf96cf8c4-k8gdf -- 2021/09/19 15:48:03
...
...
...
```
觀察附載。
```shell=
student@master:~$ watch -n 1 kubectl top pod
...
...
Every 1.0s: kubectl top pod master9013.inwinstack.lab: Sun Sep 19 07:49:34 2021
NAME CPU(cores) MEMORY(bytes)
hpaweb-cf96cf8c4-k8gdf 406m 13Mi
...
...
```
此時我們可以停止觀察附載狀況，利用hpa指令進行hpa設置。
```shell=
student@master:~$ kubectl autoscale deployment hpaweb --max=7 --min=1 --cpu-percent=50
horizontalpodautoscaler.autoscaling/hpaweb autoscaled
student@master:~$ kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hpaweb Deployment/hpaweb 0%/50% 1 7 1 70m
```
依據我們前面對pod的資源需求(200m)來看，超過50%(100m)就要scale出新的pod，依據先前的pod附載狀況來看(406m)，我們可以算出最後一共約會有4~5個pod。
先前的yaml設置搭配HPA的規則，如果負擔全滿，會有7個pod運作，每個pod最高CPU資源為500m。
如果先前沒有設置pod resource limit, request，會發現TARGETS的第一個欄位是？%
讓我們繼續對hpaweb增加附載。
```shell=
student@master:~$ watch -n 0.1 curl 192.168.122.30:30516
...
...
...
Every 0.1s: curl 192.168.122.30:30516 master9013.inwinstack.lab: Sun Sep 19 07:48:03 2021
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 80 100 80 0 0 1081
0 --:--:-- --:--:-- --:--:-- 1095
Source IP:10.6.72.0, Response by: hpaweb-cf96cf8c4-k8gdf -- 2021/09/19 15:48:03
...
...
...
```
觀察一下pod的狀態。
```shell=
student@master:~$ watch -n 1 kubectl top pod
...
...
Every 1.0s: kubectl top pod master9013.inwinstack.lab: Sun Sep 19 09:17:03 2021
NAME CPU(cores) MEMORY(bytes)
hpaweb-59d6c9779b-9946b 88m 13Mi
hpaweb-59d6c9779b-mwhg6 139m 15Mi
hpaweb-59d6c9779b-rbwb7 74m 13Mi
hpaweb-59d6c9779b-tgzbw 100m 13Mi
...
...
```
觀察hpa的狀態。
```shell=
student@master:~$ kubectl get hpa hpaweb
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
hpaweb Deployment/hpaweb 42%/50% 1 7 4 82m
```
使用describe多取得一些hpa的資訊，例如Events。
```shell=
student@master:~$ kubectl describe hpa hpaweb
Name: hpaweb
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Sun, 19 Sep 2021 07:59:05 +0000
Reference: Deployment/hpaweb
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): 47% (95m) / 50%
Min replicas: 1
Max replicas: 7
Deployment pods: 4 current / 4 desired
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True ReadyForNewScale recommended size matches current size
ScalingActive True ValidMetricFound the HPA was able to successfully calculate a replica count from cpu resource utilization (percentage of request)
ScalingLimited False DesiredWithinRange the desired count is within the acceptable range
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal SuccessfulRescale 25m horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target
```
## 4. 建議
:::info
1. HPA在一些無狀態的應用相當不錯，透過一個良好的pod設計，將Application與Data、config分別拆開，搭配Service與Deployment，比較能夠享用HPA的好處。
2. 除了本例中所使用的CPU作為條件外，還有[其他做法可以參考](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md "Custom Metrics API")。
3. 有狀態的pod(statefulset)，例如SQL, 或者有使用storage的應用(NFS搭配readonly除外)要慎用。
:::
## 5. 參考資料
1. [Official - Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ "Horizontal Pod Autoscaler")
2. [Official - horizontal-pod-autoscale-walkthrough](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/ "horizontal-pod-autoscale-walkthrough")
Sign in with Wallet
Connect another wallet