kubernetes iscsi 應用

1. node info

• 主機資訊
  • proxy server(iscsi server): 192.168.122.60
    • disk size: 5G for iscsi
  • master1: 192.168.122.61
  • master2: 192.168.122.62
  • master3: 192.168.122.63
  • worker1: 192.168.122.64
  • worker2: 192.168.122.65

2. iscsi server setup

follow this:

1. Configure iSCSI Target (targetcli)
2. Configure iSCSI Target (tgt)

targetcli

tgt conf.d sample

inwin@proxy:~$ sudo cat /etc/tgt/conf.d/iqn.2021-05.com.blk.conf
[sudo] password for inwin:
<target iqn.2021-05.com:blk>
backing-store /var/lib/iscsi_disks/disk01.img
initiator-name iqn.2021-05.test.srv:www.initiator01
incominguser test123 password
</target>

3. worker node setup

follow this: Configure iSCSI Initiator

/etc/iscsi/initiatorname.iscsi

inwin@worker1:~$ sudo cat /etc/iscsi/initiatorname.iscsi
[sudo] password for inwin:
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.2021-05.test.srv:www.initiator01

/etc/iscsi/iscsid.conf

node.session.auth.authmethod = CHAP
node.session.auth.username = test123
node.session.auth.password = password

4. chap info and secret

create iscsi CHAP info to dir

inwin@proxy:~$ mkdir info
inwin@proxy:~$ echo "sam" > info/node.session.auth.username
inwin@proxy:~$ echo "password" > info/node.session.auth.password
inwin@proxy:~$ kubectl create secret generic chap-secret --type=kubernetes.io/iscsi-chap --from-file=info/node.session.auth.username --from-file=info/node.session.auth.password
inwin@proxy:~/iscsi$ kubectl get secrets
NAME                  TYPE                                  DATA   AGE
chap-secret           kubernetes.io/iscsi-chap              2      4h33m

5. persistentVolume define

pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: iscsivolume
spec:
  capacity:
    storage: 4G
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  iscsi:
    targetPortal: 192.168.122.60
    iqn: iqn.2021-05.com:blk
    lun: 1
    fsType: xfs
    readOnly: false
    chapAuthSession: true
    secretRef:
      name: chap-secret

6. label pv

inwin@proxy:~$ kubectl label pv iscsivolume iscsi=proxynode

7. persistentVolumeClaim define

pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: reviewpvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 200Mi
  storageClassName: iscsipv
  selector:
    matchLabels:
      iscsi: "proxynode"

8. Pods mount same iscsi target

iscsi pod 1

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: testiscsi
  name: iscb1
spec:
  containers:
  - args:
    - /bin/sleep
    - "3600"
    image: busybox
    name: testiscsi
    resources: {}
    volumeMounts:
    - mountPath: "/mnt/iscsipd"
      name: iscsivol
  volumes:
  - name: iscsivol
    persistentVolumeClaim:
      claimName: reviewpvc
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

iscb pod 2

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: testiscsi
  name: iscb2
spec:
  containers:
  - args:
    - /bin/sleep
    - "3600"
    image: busybox
    name: testiscsi
    resources: {}
    volumeMounts:
    - mountPath: "/mnt/iscsipd"
      name: iscsivol
  volumes:
  - name: iscsivol
    persistentVolumeClaim:
      claimName: reviewpvc
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

check mount point and delete file

inwin@proxy:~/iscsi$ kubectl exec -it iscb1 -- sh
/ # cd /mnt/iscsipd/
/mnt/iscsipd # ls
group      hostname   hosts      localtime  mtab       network    passwd
/mnt/iscsipd # rm passwd
/mnt/iscsipd # exit
inwin@proxy:~/iscsi$ kubectl exec -it iscb2 -- sh
/ # cd /mnt/iscsipd/
/mnt/iscsipd # ls
group      hostname   hosts      localtime  mtab       network    passwd
/mnt/iscsipd # ls
group      hostname   hosts      localtime  mtab       network    passwd
/mnt/iscsipd # exit
inwin@proxy:~/iscsi$ kubectl exec -it iscb1 -- sh
/ # cd /mnt/iscsipd/
/mnt/iscsipd # ls
group      hostname   hosts      localtime  mtab       network

9. notices

1. consider pod design, use iscsi volume readonly for other container, do not RW volume at same time!
2. do not mount iscsi for other pod at same time, we can do it, but don't.

10. reference

1. CHAP sample
2. kubernetes docs