使用3台vagrant建立的VMs架設一個k8s叢集
===
## 建立一個3台機器的`Vagrantfile`
```ruby=
Vagrant.configure("2") do |config|
config.vm.define "master" do |master|
master.vm.box = "ubuntu/bionic64"
master.vm.network "private_network", ip: "192.168.2.2"
master.vm.provider "virtualbox" do |vb|
vb.memory = "16384"
end
end
config.vm.define "node1" do |node1|
node1.vm.box = "ubuntu/bionic64"
node1.vm.network "private_network", ip: "192.168.2.3"
node1.vm.provider "virtualbox" do |vb|
vb.memory = "8192"
end
end
config.vm.define "node2" do |node2|
node2.vm.box = "ubuntu/bionic64"
node2.vm.network "private_network", ip: "192.168.2.4"
node2.vm.provider "virtualbox" do |vb|
vb.memory = "8192"
end
end
end
```
## 三台主機基本設定
* 設定主機名稱
```
sudo hostnamectl set-hostname master
sudo hostnamectl set-hostname node1
sudo hostnamectl set-hostname node2
```
* 設定`dns`
```
# /etc/hosts
192.168.2.2 master
192.168.2.3 node1
192.168.2.4 node2
```
* 安裝必要套件
```
sudo apt update -y
sudo apt upgrade -y
sudo apt install linux-image-extra-virtual -y
sudo reboot
```
* 一定要將swap關掉
```
sudo swapoff -a
```
* 建立`k8s-admin`這個管理k8s的使用者
```
sudo useradd -s /bin/bash -m k8s-admin
sudo passwd k8s-admin
sudo usermod -aG sudo k8s-admin
echo "k8s-admin ALL=(ALL) NOPASSWD:ALL" | sudo tee /etc/sudoers.d/k8s-admin
```
## 安裝`docker`作為k8s的runtime
* 安裝`docker engine`
```
sudo apt-get update -y
sudo apt-get upgrade -y
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker k8s-admin
sudo usermod -aG docker $USER
sudo reboot
```
* 將`docker engine`設定成標準的服務(**這步一定要做！**)
```
sudo mkdir /etc/docker
cat <<EOF | sudo tee /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
sudo systemctl enable docker
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo reboot
```
## 安裝k8s
* 加入`apt`源
```
# /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
```
* 獲取`google`的金鑰
```
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
```
* 安裝必要的k8s元件
```
sudo apt update -y
sudo apt install kubectl kubelet kubeadm kubernetes-cni -y
```
* 確定k8s的三個元件都安裝好了
```
$ which kubelet
/usr/bin/kubelet
$ which kubeadm
/usr/bin/kubeadm
sudo reboot
```
* 設定環境變數
```
export API_ADDR="192.168.2.2"
export DNS_DOMAIN="k8s.local"
export POD_NET="10.4.0.0/16"
export SRV_NET="10.5.0.0/16"
```
* 建立k8s叢集
```
sudo kubeadm init --pod-network-cidr ${POD_NET} --service-cidr ${SRV_NET} \
--service-dns-domain "${DNS_DOMAIN}" --apiserver-advertise-address ${API_ADDR}
```
**！！圖中產生的指令一定要記下來，要不然要重新產生！！**
* 複製設定檔，從這步開始一定要用`k8s-admin`這個使用者
```
su - k8s-admin
mkdir -p $HOME/.k8s
sudo cp -i /etc/kubernetes/admin.conf $HOME/.k8s/config
sudo chown $(id -u):$(id -g) $HOME/.k8s/config
export KUBECONFIG=$HOME/.k8s/config
echo "export KUBECONFIG=$HOME/.k8s/config" | tee -a ~/.bashrc
```
* 安裝`weave`
```
# su - k8s-admin
$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"
```
* 產生`join`的指令
```
kubeadm token create --print-join-command
sudo kubeadm join 192.168.2.2:6443 --token 1lg6hx.4kjnjwtk9axkuh33 --discovery-token-ca-cert-hash sha256:94b0450336effe7b97a3c70aac267d9866bcd1a6911f438f0d571effcdb6061d
```
## node加入cluster
* 加入cluster，首先要先`ssh`進入node機器，並且安裝好`kubectl` `kubelet` `kubeadm` `kubernetes-cni`這一步為止。
```
sudo kubeadm join 192.168.2.2:6443 --token 1lg6hx.4kjnjwtk9axkuh33 --discovery-token-ca-cert-hash sha256:94b0450336effe7b97a3c70aac267d9866bcd1a6911f438f0d571effcdb6061d
```
* 加入後確定k8s有跑起來
```
sudo systemctl status kubelet
```
## 控制cluster
* 回到`master`主機上，切換使用者到`k8s-admin`，查看已經加入的nodes
```
vagrant@master:~$su - k8s-admin
k8s-admin@master:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 9h v1.22.3
node1 Ready <none> 9h v1.22.3
node2 Ready <none> 9h v1.22.3
k8s-admin@master:~$
```
* 建立一個網頁伺服器的`deployment`，檔名舉例為`nginx.yaml`
```yaml=
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 3 # tells deployment to run 3 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
# image: nginx:latest
image: registry.hub.docker.com/joshhu/test-php-site
ports:
- containerPort: 80
```
* 啟動這個`deployment`，就會啟動`pods`
```
kubectl apply -f nginx.yaml
```
* 查看`deployments`，也可查看`pods`
```
kubectl get deployments
kubectl get pods
```
* 查看`deployment`詳情
```
k8s-admin@master:~$ kubectl describe deployment nginx-deployment
Name: nginx-deployment
Namespace: default
CreationTimestamp: Mon, 01 Nov 2021 17:33:47 +0000
Labels: <none>
Annotations: deployment.kubernetes.io/revision: 1
Selector: app=nginx
Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=nginx
Containers:
nginx:
Image: registry.hub.docker.com/joshhu/test-php-site
Port: 80/TCP
Host Port: 0/TCP
Environment: <none>
Mounts: <none>
Volumes: <none>
Conditions:
Type Status Reason
---- ------ ------
Available True MinimumReplicasAvailable
Progressing True NewReplicaSetAvailable
OldReplicaSets: <none>
NewReplicaSet: nginx-deployment-d75ccb (3/3 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 55m deployment-controller Scaled up replica set nginx-deployment-d75ccb to 3
k8s-admin@master:~$
```
## 建立一個對外開放的`service`，檔名為`nginx-service.yaml`
* 建立一個`service`物件如下
```yaml=
apiVersion: v1
kind: Service
metadata:
name: nginx-svc
labels:
app: nginx
spec:
type: NodePort
ports:
- port: 80
nodePort: 30080
selector:
app: nginx
externalIPs:
- 192.168.2.3
```
* 啟動`service`
```
kubectl apply -f nginx-service.yaml
```
* 查看這個`service`
```
k8s-admin@master:~$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.5.0.1 <none> 443/TCP 10h
nginx-svc NodePort 10.5.0.187 192.168.2.3 80:30080/TCP 58m
k8s-admin@master:~$
```
* `service`詳細資料
```
k8s-admin@master:~$ kubectl describe svc nginx-svc
Name: nginx-svc
Namespace: default
Labels: app=nginx
Annotations: <none>
Selector: app=nginx
Type: NodePort
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.5.0.187
IPs: 10.5.0.187
External IPs: 192.168.2.3
Port: <unset> 80/TCP
TargetPort: 80/TCP
NodePort: <unset> 30080/TCP
Endpoints: 10.36.0.1:80,10.44.0.1:80,10.44.0.2:80
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
k8s-admin@master:~$
```
* 網頁瀏覽
## 參考資料
https://computingforgeeks.com/how-to-setup-3-node-kubernetes-cluster-on-ubuntu-18-04-with-weave-net-cni/
https://gist.github.com/danielepolencic/ef4ddb763fd9a18bf2f1eaaa2e337544
###### tags: `k8s`, `kubernetes`, `docker`, `vagrant`
Sign in with Wallet
Connect another wallet