# K8S
```
Name: ubuntu-pod
Namespace: default
Priority: 0
Service Account: default
Node: eason-virtualbox/10.0.2.15
Start Time: Mon, 18 Mar 2024 21:15:51 +0800
Labels: app=test
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Containers:
ubuntu:
Container ID:
Image: ubuntu:20.04
Image ID:
Port: <none>
Host Port: <none>
Args:
bash
-c
for ((i = 0; ; i++)); do echo "$i: $(date)"; sleep 100; done
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fb2bt (ro)
Conditions:
Type Status
PodReadyToStartContainers False
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
kube-api-access-fb2bt:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node-role.kubernetes.io/control-plane:NoSchedule op=Exists
node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 16m (x3 over 26m) default-scheduler 0/1 nodes are available: 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling.
Normal Scheduled 12m default-scheduler Successfully assigned default/ubuntu-pod to eason-virtualbox
Warning FailedCreatePodSandBox 12m kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0d551cf401d98c5893b4541d11c4af8fcb09dcce4bba4fbb23eae84650b70564": plugin type="flannel" failed (add): failed to find plugin "flannel" in path [/opt/cni/bin]
Normal SandboxChanged 8m7s (x23 over 12m) kubelet Pod sandbox changed, it will be killed and re-created.
Normal SandboxChanged 2m40s (x26 over 8m) kubelet Pod sandbox changed, it will be killed and re-created.
```
```
eason@eason-VirtualBox:~$ kubectl get pod
NAME READY STATUS RESTARTS AGE
ubuntu-pod 0/1 ContainerCreating 0 21h
```
## pod yaml
```
apiVersion: v1
kind: Pod
metadata:
name: ubuntu-pod
labels:
app: test
spec:
tolerations:
- key: "node-role.kubernetes.io/control-plane"
effect: "NoSchedule"
operator: "Exists"
containers:
- name: ubuntu
image: ubuntu:20.04
args: [bash, -c, 'for ((i = 0; ; i++)); do echo "$i: $(date)"; sleep 100; done']
```
```
kubectl get po -A
NAMESPACE NAME READY STATUS RESTARTS AGE
default ubuntu-pod 0/1 ContainerCreating 0 23h
kube-system coredns-76f75df574-nzzwd 0/1 ContainerCreating 0 23h
kube-system coredns-76f75df574-vr42l 0/1 ContainerCreating 0 23h
kube-system etcd-eason-virtualbox 1/1 Running 3 (10m ago) 23h
kube-system kube-apiserver-eason-virtualbox 1/1 Running 3 (10m ago) 23h
kube-system kube-controller-manager-eason-virtualbox 1/1 Running 3 (10m ago) 23h
kube-system kube-flannel-ds-amd64-mxlx4 0/1 CrashLoopBackOff 66 (4m58s ago) 23h
kube-system kube-proxy-x954m 1/1 Running 3 (10m ago) 23h
kube-system kube-scheduler-eason-virtualbox 1/1 Running 4 (10m ago) 23h
```
```
kubectl logs kube-flannel-ds-amd64-mxlx4 -n kube-system
Defaulted container "kube-flannel" out of: kube-flannel, install-cni (init)
I0318 15:54:23.730045 1 main.go:518] Determining IP address of default interface
I0318 15:54:23.730520 1 main.go:531] Using interface with name enp0s3 and address 10.0.2.15
I0318 15:54:23.730688 1 main.go:548] Defaulting external address to interface address (10.0.2.15)
W0318 15:54:23.730848 1 client_config.go:517] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
E0318 15:54:23.736116 1 main.go:243] Failed to create SubnetManager: error retrieving pod spec for 'kube-system/kube-flannel-ds-amd64-mxlx4': pods "kube-flannel-ds-amd64-mxlx4" is forbidden: User "system:serviceaccount:kube-system:flannel" cannot get resource "pods" in API group "" in the namespace "kube-system"
```
## 第二次
https://www.jianshu.com/p/868b9fbff15e
用這個指令可以解決CrashLoopBackOff 的問題
```
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```
```
NAMESPACE NAME READY STATUS RESTARTS AGE
default pi-5v6jn 0/1 Pending 0 54m
default ubuntu-deployment-654f699677-77ms7 0/1 Pending 0 61m
default ubuntu-deployment-654f699677-n5p65 0/1 Pending 0 61m
default ubuntu-deployment-654f699677-pk7jw 0/1 Pending 0 61m
default ubuntu-pod 1/1 Running 0 89m
kube-flannel kube-flannel-ds-bc69r 1/1 Running 0 95m
kube-system coredns-76f75df574-7tcsw 1/1 Running 0 105m
kube-system coredns-76f75df574-8sx5n 1/1 Running 0 105m
kube-system etcd-eason-virtualbox 1/1 Running 0 105m
kube-system kube-apiserver-eason-virtualbox 1/1 Running 0 105m
kube-system kube-controller-manager-eason-virtualbox 1/1 Running 1 (28m ago) 105m
kube-system kube-flannel-ds-amd64-8tp9c 0/1 CrashLoopBackOff 23 (2m52s ago) 105m
kube-system kube-proxy-hf6mb 1/1 Running 0 105m
kube-system kube-scheduler-eason-virtualbox 1/1 Running 2 (13m ago) 105m
```
deployment.yaml
```
apiVersion: apps/v1
kind: Deployment
metadata:
name: ubuntu-deployment
labels:
app: test
spec:
replicas: 3
selector:
matchLabels:
app: test
template:
metadata:
labels:
app: test
spec:
tolerations:
- key: "node-role.kubernetes.io/control-plane"
effect: "NoSchedule"
operator: "Exists"
containers:
- name: ubuntu
image: ubuntu:20.04
args: [bash, -c, 'for ((i = 0; ; i++)); do echo "$i: $(date)"; sleep 100; done']
```
job.yaml
```
# test_job2.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: pi-v2
spec:
template:
spec:
tolerations:
- key: "node-role.kubernetes.io/control-plane"
effect: "NoSchedule"
operator: "Exists"
containers:
- name: pi
image: perl:5.34.0
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
restartPolicy: Never
backoffLimit: 4
```
cronJob
沒有安裝CronJob CRD造成問題
後來好了
https://stackoverflow.com/questions/67520866/no-matches-for-kind-cronjob-in-version-batch-v1
```
# test_cronjob.yaml
apiVersion: batch/v1 # 從v1beta改成v1就成功了
kind: CronJob
metadata:
name: hello
spec:
schedule: "* * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: hello
image: busybox:1.28
imagePullPolicy: IfNotPresent
command:
- /bin/sh
- -c
- date; echo Hello from the Kubernetes cluster
restartPolicy: OnFailure
```
然後就失控了:)
```
NAME READY STATUS RESTARTS AGE
hello-28523189-kwv62 0/1 Pending 0 27m
hello-28523190-flsld 0/1 Pending 0 26m
hello-28523191-bwjz9 0/1 Pending 0 25m
hello-28523192-nn8w6 0/1 Pending 0 24m
hello-28523193-99lwz 0/1 Pending 0 23m
hello-28523194-4r77d 0/1 Pending 0 22m
hello-28523196-hnqc2 0/1 Pending 0 20m
hello-28523198-7jk65 0/1 Pending 0 18m
hello-28523199-mm4bh 0/1 Pending 0 17m
hello-28523200-lhtft 0/1 Pending 0 16m
hello-28523201-h7pmf 0/1 Pending 0 15m
hello-28523202-d6fdh 0/1 Pending 0 14m
hello-28523204-79wsb 0/1 Pending 0 12m
hello-28523205-xxnf8 0/1 Pending 0 11m
hello-28523208-6wl2g 0/1 Pending 0 8m5s
hello-28523209-gsdbr 0/1 Pending 0 7m26s
hello-28523210-6zhl2 0/1 Pending 0 6m26s
hello-28523211-w6w8m 0/1 Pending 0 5m26s
hello-28523212-nsvk4 0/1 Pending 0 4m26s
hello-28523213-nl6d9 0/1 Pending 0 3m26s
hello-28523214-rqbnf 0/1 Pending 0 2m26s
hello-28523215-g6zjv 0/1 Pending 0 86s
pi-5v6jn 0/1 Pending 0 7d1h
pi-v2-rvxk5 0/1 Completed 0 6d22h
ubuntu-deployment-7659b68cfd-26m99 1/1 Running 2 (9m43s ago) 7d
ubuntu-deployment-7659b68cfd-9ggmt 1/1 Running 2 (9m43s ago) 7d
ubuntu-deployment-7659b68cfd-ssqf4 1/1 Running 2 (9m43s ago) 7d
ubuntu-pod 1/1 Running 2 (9m42s ago) 7d2h
```
worker node編入出問題
```
sudo kubeadm join 10.0.2.15:6443 --token g926ii.244wdgqm98gt7si0 --discovery-token-ca-cert-hash sha256:4106c40d407ebed5092bd0812c457701f32425d5ea4dbd793139463ecf5ef1da --cri-socket=/var/run/docker.sock -v=2
I0325 15:25:09.431814 3505 join.go:413] [preflight] found NodeName empty; using OS hostname as NodeName
W0325 15:25:09.437072 3505 initconfiguration.go:125] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/var/run/docker.sock". Please update your configuration!
[preflight] Running pre-flight checks
I0325 15:25:09.437159 3505 preflight.go:93] [preflight] Running general checks
I0325 15:25:09.437200 3505 checks.go:280] validating the existence of file /etc/kubernetes/kubelet.conf
I0325 15:25:09.437217 3505 checks.go:280] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0325 15:25:09.437234 3505 checks.go:104] validating the container runtime
I0325 15:25:09.448467 3505 checks.go:639] validating whether swap is enabled or not
I0325 15:25:09.454789 3505 checks.go:370] validating the presence of executable crictl
I0325 15:25:09.454813 3505 checks.go:370] validating the presence of executable conntrack
I0325 15:25:09.454821 3505 checks.go:370] validating the presence of executable ip
I0325 15:25:09.454835 3505 checks.go:370] validating the presence of executable iptables
I0325 15:25:09.454845 3505 checks.go:370] validating the presence of executable mount
I0325 15:25:09.454854 3505 checks.go:370] validating the presence of executable nsenter
I0325 15:25:09.454862 3505 checks.go:370] validating the presence of executable ebtables
I0325 15:25:09.454871 3505 checks.go:370] validating the presence of executable ethtool
I0325 15:25:09.454878 3505 checks.go:370] validating the presence of executable socat
I0325 15:25:09.454889 3505 checks.go:370] validating the presence of executable tc
I0325 15:25:09.454897 3505 checks.go:370] validating the presence of executable touch
I0325 15:25:09.454907 3505 checks.go:516] running all checks
I0325 15:25:09.475016 3505 checks.go:401] checking whether the given node name is valid and reachable using net.LookupHost
I0325 15:25:09.475444 3505 checks.go:605] validating kubelet version
I0325 15:25:09.553383 3505 checks.go:130] validating if the "kubelet" service is enabled and active
I0325 15:25:09.560454 3505 checks.go:203] validating availability of port 10250
I0325 15:25:09.560608 3505 checks.go:280] validating the existence of file /etc/kubernetes/pki/ca.crt
I0325 15:25:09.560633 3505 checks.go:430] validating if the connectivity type is via proxy or direct
I0325 15:25:09.560669 3505 checks.go:329] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0325 15:25:09.560695 3505 checks.go:329] validating the contents of file /proc/sys/net/ipv4/ip_forward
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: I0325 15:25:09.447244 3509 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/docker.sock" URL="unix:///var/run/docker.sock"
time="2024-03-25T15:25:09+08:00" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"/var/run/docker.sock\": rpc error: code = Unavailable desc = write unix @->/run/docker.sock: write: broken pipe"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
```
重啟後
```
systemctl restart containerd
```
```
sudo kubeadm join 10.0.2.15:6443 --token g926ii.244wdgqm98gt7si0 --discovery-token-ca-cert-hash sha256:4106c40d407ebed5092bd0812c457701f32425d5ea4dbd793139463ecf5ef1da --cri-socket=/var/run/docker.sock -v=2
I0325 16:36:55.408409 2714 join.go:413] [preflight] found NodeName empty; using OS hostname as NodeName
W0325 16:36:55.408519 2714 initconfiguration.go:125] Usage of CRI endpoints without URL scheme is deprecated and can cause kubelet errors in the future. Automatically prepending scheme "unix" to the "criSocket" with value "/var/run/docker.sock". Please update your configuration!
[preflight] Running pre-flight checks
I0325 16:36:55.408605 2714 preflight.go:93] [preflight] Running general checks
I0325 16:36:55.408651 2714 checks.go:280] validating the existence of file /etc/kubernetes/kubelet.conf
I0325 16:36:55.408673 2714 checks.go:280] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0325 16:36:55.408695 2714 checks.go:104] validating the container runtime
I0325 16:36:55.443607 2714 checks.go:639] validating whether swap is enabled or not
I0325 16:36:55.443697 2714 checks.go:370] validating the presence of executable crictl
I0325 16:36:55.443735 2714 checks.go:370] validating the presence of executable conntrack
I0325 16:36:55.443760 2714 checks.go:370] validating the presence of executable ip
I0325 16:36:55.443794 2714 checks.go:370] validating the presence of executable iptables
I0325 16:36:55.443821 2714 checks.go:370] validating the presence of executable mount
I0325 16:36:55.443847 2714 checks.go:370] validating the presence of executable nsenter
I0325 16:36:55.443873 2714 checks.go:370] validating the presence of executable ebtables
I0325 16:36:55.443898 2714 checks.go:370] validating the presence of executable ethtool
I0325 16:36:55.443922 2714 checks.go:370] validating the presence of executable socat
I0325 16:36:55.443947 2714 checks.go:370] validating the presence of executable tc
I0325 16:36:55.443978 2714 checks.go:370] validating the presence of executable touch
I0325 16:36:55.444006 2714 checks.go:516] running all checks
I0325 16:36:55.472435 2714 checks.go:401] checking whether the given node name is valid and reachable using net.LookupHost
I0325 16:36:55.472738 2714 checks.go:605] validating kubelet version
I0325 16:36:55.549255 2714 checks.go:130] validating if the "kubelet" service is enabled and active
I0325 16:36:55.559779 2714 checks.go:203] validating availability of port 10250
I0325 16:36:55.561196 2714 checks.go:280] validating the existence of file /etc/kubernetes/pki/ca.crt
I0325 16:36:55.561401 2714 checks.go:430] validating if the connectivity type is via proxy or direct
I0325 16:36:55.561438 2714 checks.go:329] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0325 16:36:55.561468 2714 checks.go:329] validating the contents of file /proc/sys/net/ipv4/ip_forward
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: I0325 16:36:55.440687 2718 util_unix.go:103] "Using this endpoint is deprecated, please consider using full URL format" endpoint="/var/run/docker.sock" URL="unix:///var/run/docker.sock"
time="2024-03-25T16:36:55+08:00" level=fatal msg="validate service connection: validate CRI v1 runtime API for endpoint \"/var/run/docker.sock\": rpc error: code = Unavailable desc = write unix @->/run/docker.sock: write: broken pipe"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher
```
```
# Let NetworkManager manage all devices on this system
network:
version: 2
renderer: NetworkManager
ethernets:
DEVICE_NAME:
dhcp4: no
addresses: [192.168.1.11/24]
gateway4: 192.168.1.1
nameservers:
addresses: [168.95.1.1, 168.95.192.1]
```
後來在ethernets後加上名字就可以了
```
network:
version: 2
renderer: NetworkManager
ethernets:
enp0s3:
dhcp4: no
addresses: [192.168.1.11/24]
gateway4: 192.168.1.1
nameservers:
addresses: [168.95.1.1, 168.95.192.1]
```
```
sudo netplan apply
```
```
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.11 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::a00:27ff:fe26:c737 prefixlen 64 scopeid 0x20<link>
ether 08:00:27:26:c7:37 txqueuelen 1000 (Ethernet)
RX packets 125 bytes 10850 (10.8 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 187 bytes 20778 (20.7 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
enp0s8: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.0.3.15 netmask 255.255.255.0 broadcast 10.0.3.255
inet6 fe80::fd8:108a:a054:422a prefixlen 64 scopeid 0x20<link>
ether 08:00:27:2a:91:66 txqueuelen 1000 (Ethernet)
RX packets 401 bytes 322989 (322.9 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 321 bytes 33280 (33.2 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 332 bytes 39065 (39.0 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 332 bytes 39065 (39.0 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
```
固定ip成功(?
```
network:
version: 2
renderer: NetworkManager
ethernets:
enp0s3:
dhcp4: no
bridges:
br0:
dhcp4: no
addresses: [192.168.1.10/24]
gateway4: 192.168.1.1
nameservers:
addresses: [168.95.1.1, 168.95.192.1]
interfaces: [enp0s3]
```
育林版
```
# Master
network:
ethernets:
enp0s3:
dhcp4: true
addresses:
- 192.168.128.227/24
# routes:
# - to: 0.0.0.0/0
# - via: 192.168.128.118
version: 2
renderer: NetworkManager
```
```
# Worker
network:
ethernets:
enp0s3:
dhcp4: true
addresses:
- 192.168.128.218/24
# routes:
# - to: 0.0.0.0/0
# - via: 192.168.128.118
version: 2
renderer: NetworkManager
```
```
# Worker2
network:
ethernets:
enp0s3:
dhcp4: true
addresses:
- 192.168.128.219/24
# routes:
# - to: 0.0.0.0/0
# - via: 192.168.128.118
version: 2
renderer: NetworkManager
```
```
sudo apt install -y
sudo mkdir "NFStest"
echo "/NFStest *(rw,sync,no_root_squash)" | sudo tee /etc/exports
sudo exportfs -r # reload
```
```
mount 192.168.128.227: /NFStest /Documents/worker.txt
```
```
# NFS_pod
apiVersion: v1
kind: Pod
metadata:
name: ubuntu-pod
labels:
app: test
spec:
containers:
- name: ubuntu
image: ubuntu:20.04
args: [bash, -c, 'for ((i = 0; ; i++)); do echo "$i: $(date)"; sleep 100; done']
volumes:
- name: nfs-test
nfs:
path: /home/test/Data
server: 192.168.128.227
```
```
kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ubuntu-pod 1/1 Running 0 11h 10.244.1.3 worker-virtualbox <none> <none>
ubuntu-pod2 1/1 Running 0 10h 10.244.1.2 worker-virtualbox <none> <none>
ubuntu-pod3 1/1 Running 0 10h 10.244.1.4 worker-virtualbox <none> <none>
```
```
kubectl apply -f https://k8s.io/examples/controllers/daemonset.yaml
```
###
```
sudo vim /etc/exports
```
```
NFStest *(rw,sync,no_root_squash)
/var/nfs/general 192.168.128.227(rw,sync,no_subtree_check)
/var/nfs/general 192.168.128.218(rw,sync,no_subtree_check)
```
把最上面那行拿掉就active了
Sign in with Wallet
Connect another wallet