# VMUG : GOVC 實驗指南
###### tags: `govc` `vmug` `guide`
[toc]
日期: 2023-05-20
本次實驗目的在於了解 GOVC 的基本操作方式。實驗項目將針對 vCenter/ESXi/VM 作為目標，以平時的工作任務為方向，可透過 GOVC 命令列完成。
[](https://gitlab.com/farmer871/learning_govc)
# 實驗準備
在實驗進行前完成以下項目準備
- [安裝 vCenter](#安裝-vCenter)
- [安裝 Nested vSphere ESXi](#安裝-Nested-vSphere-ESXi)
- [安裝 GOVC](#安裝-GOVC)
- [驗證連線](#驗證連線)
測試 IP 配置
| | |
| --- | --- |
| vCeneter | 10.7.150.90 |
| ESXi01 | 10.7.150.91 |
| ESXi02 | 10.7.150.92 |
| ESXi03 | 10.7.150.93 |
| Laptop | 10.7.150.87 |
| DNS | 10.7.7.150.119 |
| Netmask | 255.255.255.0 |
| Default gateway | 10.7.150.254 |
## 安裝vCenter
實驗所安裝的版本為 **7.0.3 build-21477706**。
建議可以參考 [[**使用 CLI 部署 vCenter Server Appliance**]](https://hackmd.io/@farmer87/deploy_vcsa_cli) 快速完成安裝或是參考這篇 [[**使用 CLI 部署 vCenter Server Appliance**]](https://hackmd.io/@farmer87/deploy_vcsa_cli)。
### vcsa.json
:::info
- 佈署時直接連線 ESXi 安裝而非透過 vCenter。
- 由於是測試 GOVC 功能，佈署選擇最小規模 Tiny。
- 使用 CLI 安裝只要提供資訊無誤，只要按下 `Enter` 就會自動佈署完成，不需要像使用 GUI 安裝要手動進行 2 個階段(Stages)，可加速測試環境建置。
:::
以下是本次測試 vCSA 的 JSON 佈署範例檔 **`vcsa.json`**。
```json
{
"__version": "2.13.0",
"__comments": "Deploy a vCenter Server Appliance on an ESXi host.",
"new_vcsa": {
"esxi": {
"hostname": "10.7.150.1",
"username": "root",
"password": "VMware1!",
"deployment_network": "PG-MGMT",
"datastore": "SC2080_Datastore"
},
"appliance": {
"thin_disk_mode": true,
"deployment_option": "tiny",
"name": "vc"
},
"network": {
"ip_family": "ipv4",
"mode": "static",
"ip": "10.7.150.90",
"dns_servers": "10.7.150.119",
"prefix": "24",
"gateway": "10.7.150.254",
"system_name": "vc.lab.local"
},
"os": {
"password": "VMware1!",
"ntp_servers": "216.239.35.4",
"ssh_enable": true
},
"sso": {
"password": "VMware1!",
"domain_name": "vsphere.local"
}
},
"ceip": {
"description": {},
"settings": {
"ceip_enabled": false
}
}
}
```
### deployVcsa.sh
:::info
- 佈署 vCenter 時務必確認環境中 **DNS** 和 **NTP** 服務能夠正常運作，避免安裝失敗。
- 記得本機 /etc/hosts 增加 vCenter FQDN，也可降低佈署失敗率。
- 安裝前請先將 vCSA ISO 安裝檔掛載完成。
:::
簡單地將要佈署時需要的**工作目錄**, **JSON 配置檔**和 **CLI 佈署命令**撰寫成 **`deployVcsa.sh`** 腳本檔，也可加速測試環境建置。
```bash
#!/bin/bash
workingPath="${PWD}"
logPath="${workingPath}/tmp"
vcsaJsonfile="${workingPath}/vcsa.json"
installerPath='/run/media/richard/VMware VCSA/vcsa-cli-installer/lin64'
installer="${installerPath}/vcsa-deploy"
if [ ! -d ${logPath} ]; then
mkdir -p ${logPath}
fi
if [ ! -f "${vcsaJsonfile}" ]; then
echo -e "<!> vCSA JSON file NOT FOUND <!>"
echo -e "> Please check the JSON file of vCSA\n"
exit
fi
if [ ! -f "${installer}" ]; then
echo -e "<!> ${installer} NOT FOUND <!>"
echo -e "> Please mount vCSA ISO correctly!\n"
fi
bash "${installer}" install --accept-eula --no-ssl-certificate-verification --log-dir ${logPath} ${vcsaJsonfile}
```
## 安裝 Nested vSphere ESXi
實驗使用 **Nested ESXi** 虛擬主機，選擇 **William Lam 大神** 提供的 [[**Nested ESXi Virtual Appliance**]](https://williamlam.com/nested-virtualization/nested-esxi-virtual-appliance)，請根據可取得資源選擇 ESXi 測試版本進行安裝即可。
本次安裝版本因為測試環境的 ESXi 版本較舊(v6.5)，無法支援太新 OVA 檔的虛擬硬體版本，所以選擇 [**ESXi 6.7U3 Virtual Appliance**](https://download3.vmware.com/software/vmw-tools/nested-esxi/Nested_ESXi6.7u3_Appliance_Template_v1.ova)（Virtual Hardware: vmx-13）。
:::info
可以透過 **ovftool** 工具檢視 OVA 檔案資訊。
```bash
$ ovftool --allowExtraConfig --hideEula Nested_ESXi6.7u3_Appliance_Template_v1.ova
OVF version: 1.0
VirtualApp: false
Name: Nested_ESXi6.7u3_Appliance
Version: 1.0
Full Version: 6.7u3
Vendor: virtuallyGhetto
Vendor URL: http://www.virtuallyghetto.com
Annotation: Nested ESXi 6.7 Update 3 Appliance (Build 14320388)
www.virtuallyghetto.com
End-user License Agreements:
Present: Yes (2)
Download Size: 483.90 MB
Deployment Sizes:
Flat disks: 14.00 GB
Sparse disks: 507.38 MB
Networks:
Name: VM Network
Description: The VM Network network
Virtual Machines:
Name: Nested_ESXi6.7u3_Appliance_Template
Operating System: vmkernel65guest
Virtual Hardware:
Families: vmx-13
Number of CPUs: 2
Cores per socket: automatic
Memory: 6.00 GB
<以下省略>
```
:::
### Ansible 佈署
為了加速測試環境的虛擬 ESXi 主機佈署，可以搭配 **Ansible** 完成 ESXi 虛擬機器的佈署作業，以下提供相關佈署配置的 YAML 範例檔案。
#### vesxi_config.yml
```yaml=
---
vesxi_ovftool: "/usr/bin/ovftool"
pod: "LAB"
owner: "Richard"
parentfolder: "{{ owner }}"
childfolder: "Nested_ESXi"
## vesxi
vesxi_vswitch: "vSwitch0"
vesxi_portgroup: "PG-MGMT"
vesxi_vlan: "0"
vesxi_datastore: "Storage"
vesxi_disk_mode: "thin"
vesxi_network1: "{{ vesxi_portgroup }}"
vesxi_network2: "{{ vesxi_portgroup }}"
vesxi_netmask: "255.255.255.0"
vesxi_gateway: "10.7.150.254"
vesxi_dns: "10.7.150.119"
vesxi_domain: "lab.local"
vesxi_ntp: "time1.google.com"
vesxi_syslog: "10.7.150.120"
vesxi_password: "VMware1!"
vesxi_ova_location: "./esxi_ova"
vesxi_ova: "Nested_ESXi6.7u3_Appliance_Template_v1.ova"
vesxi_list:
- ip: "10.7.150.91"
vmname: "vESXi01"
hostname: "vesxi01.lab.local"
- ip: "10.7.150.92"
vmname: "vESXi02"
hostname: "vesxi02.lab.local"
- ip: "10.7.150.93"
vmname: "vESXi03"
hostname: "vesxi03.lab.local"
esxi_host1: "10.7.150.101"
esxi_host2: "10.7.150.102"
## platform
vc:
ip: "10.7.150.100"
vmname: "vcsa01"
user: "administrator@vsphere.local"
pass: "VMware1!"
datacenter: "Datacenter"
cluster: "Cluster"
datastore: "Datastore"
esxi_list:
- ip: "10.7.150.101"
netmask: "255.255.255.0"
user: "root"
pass: "VMware1!"
hostname: "esxi01"
datastore: "datastore1"
- ip: "10.7.150.102"
netmask: "255.255.255.0"
user: "root"
pass: "VMware1!"
hostname: "esxi02"
datastore: "datastore2"
```
#### vesxi_deploy.yml
```yaml=
---
- hosts: local
gather_facts: false
connection: local
tasks:
- name: '讀取預設參數'
include_vars: vesxi_config.yml
- name: '建立vESXi用連接埠群組 PG-MGMT'
community.vmware.vmware_portgroup:
hostname: "{{ item.ip }}"
username: "{{ item.user }}"
password: "{{ item.pass }}"
validate_certs: false
hosts: "{{ item.ip }}"
switch: "{{ vesxi_vswitch }}"
portgroup: "{{ vesxi_portgroup }}"
vlan_id: "{{ vesxi_vlanid | default(0) }}"
state: present
delegate_to: localhost
with_items: "{{ esxi_list }}"
- name: '建立父系資料夾'
community.vmware.vcenter_folder:
hostname: "{{ vc.ip }}"
username: "{{ vc.user }}"
password: "{{ vc.pass }}"
validate_certs: false
datacenter_name: "{{ vc.datacenter }}"
folder_name: "{{ owner }}"
folder_type: vm
state: present
register: vm_folder_creation_result
delegate_to: localhost
- name: '建立 VM 存放資料夾'
community.vmware.vcenter_folder:
hostname: "{{ vc.ip }}"
username: "{{ vc.user }}"
password: "{{ vc.pass }}"
validate_certs: false
datacenter_name: "{{ vc.datacenter }}"
parent_folder: "{{ parentfolder }}"
folder_name: "{{ childfolder }}"
folder_type: vm
state: present
register: vm_folder_creation_result
delegate_to: localhost
- name: '佈署vESXi OVA'
shell: >
{{ vesxi_ovftool }}
'--name={{ item.vmname }}'
--X:injectOvfEnv
--acceptAllEulas
--allowExtraConfig
'--extraConfig:scsi0:0.virtualSSD=0'
'--extraConfig:scsi0:1.virtualSSD=1'
'--extraConfig:scsi0:2.virtualSSD=1'
--skipManifestCheck
--noSSLVerify
'--datastore={{ vesxi_datastore }}'
'--diskMode={{ vesxi_disk_mode }}'
'--net:VM Network={{ vesxi_network1 }}'
'--net:VM Network={{ vesxi_network2 }}'
'--prop:guestinfo.hostname={{ item.hostname }}'
'--prop:guestinfo.ipaddress={{ item.ip }}'
'--prop:guestinfo.netmask={{ vesxi_netmask }}'
'--prop:guestinfo.gateway={{ vesxi_gateway }}'
'--prop:guestinfo.vlan={{ vesxi_vlan }}'
'--prop:guestinfo.dns={{ vesxi_dns }}'
'--prop:guestinfo.domain={{ vesxi_domain }}'
'--prop:guestinfo.ntp={{ vesxi_ntp }}'
'--prop:guestinfo.syslog={{ vesxi_syslog }}'
'--prop:guestinfo.password={{ vesxi_password }}'
'--prop:guestinfo.ssh=True'
'--prop:guestinfo.createvmfs=False'
'--powerOn=True'
'{{ vesxi_ova_location }}/{{ vesxi_ova }}'
'vi://{{ vc.user | urlencode }}:{{ vc.pass | urlencode }}@{{ vc.ip }}/{{ vc.datacenter }}/host/{{ vc.cluster }}/{{ esxi_host2 }}'
with_items: "{{ vesxi_list }}"
changed_when: true
failed_when: false
- name: '遷移 VM 至指定資料夾'
community.vmware.vmware_guest_move:
hostname: "{{ vc.ip }}"
username: "{{ vc.user }}"
password: "{{ vc.pass }}"
validate_certs: false
datacenter: "{{ vc.datacenter }}"
name: "{{ item.vmname }}"
dest_folder: "/{{ vc.datacenter }}/vm/{{ parentfolder }}/{{ childfolder }}"
delegate_to: localhost
with_items: "{{ vesxi_list }}"
- name: 'vMotion 至指定 ESXi 主機'
community.vmware.vmware_vmotion:
hostname: "{{ vc.ip }}"
username: "{{ vc.user }}"
password: "{{ vc.pass }}"
validate_certs: false
destination_datacenter: "{{ vc.datacenter }}"
vm_name: "{{ item.vmname }}"
destination_host: "{{ esxi_host2 }}"
delegate_to: localhost
with_items: "{{ vesxi_list }}"
```
## 安裝 GOVC
請參考 [[**GOVC 安裝說明**]](https://github.com/vmware/govmomi/tree/main/govc#installation)，選擇習慣的方式完成安裝。
- Docker
- Binary （建議）
- Source
### govc_installer.sh
直接採用編譯好的版本安裝，可以直接執行以下腳本檔完成。
```bash
#!/bin/bash
govcUrl='https://github.com/vmware/govmomi/releases/latest/download'
govcFile="govc_$(uname -s)_$(uname -m)"
which govc &>/dev/null
if [ $? -ne 0 ]; then
echo "> Download govc from gitgub"
wget ${govcUrl}/${govcFile}.tar.gz
echo "> Install govc to /usr/local/bin"
sudo tar -C /usr/local/bin -zxvf ${govcFile}.tar.gz govc
else
echo "govc already exists..."
fi
```
完成安裝後，使用以下命令確認。
```bash
$ which govc
/usr/local/bin/govc
$ govc version
govc 0.30.4
```
## 驗證連線
完成實驗環境佈署，將必要的連線參數輸入 GOVC 連線 vCSA 的配置檔 **`labvcsa`**。
### labvcsa
```bash
vcsa_ip='10.7.150.90'
echo -e "> 連線 vCenter Server ${vcsa} ...... [完成]"
export GOVC_URL="https://${vcsa_ip}"
export GOVC_USERNAME="administrator@vsphere.local"
export GOVC_PASSWORD="VMware1!"
export GOVC_INSECURE="true"
export GOVC_DATACENTER="LAB_Datacenter"
export GOVC_CLUSTER="LAB_Cluster"
export GOVC_DATASTORE="vsanDatastore"
```
使用 **`source labvcsa`** 讀取連線參數，輸入 **`govc env`** 和 **`govc about`** 確認使用 GOVC 可正常連線 vCenter。
```bash
$ govc env
GOVC_USERNAME=administrator@vsphere.local
GOVC_PASSWORD=VMware1!
GOVC_URL=10.7.150.90
GOVC_INSECURE=true
$ govc about
FullName: VMware vCenter Server 7.0.3 build-21477706
Name: VMware vCenter Server
Vendor: VMware, Inc.
Version: 7.0.3
Build: 21477706
OS type: linux-x64
API type: VirtualCenter
API version: 7.0.3.0
Product ID: vpx
UUID: b844e614-d9ea-4bb0-9e53-0cb0aa6439a1
```
:::success
一切就緒，準備進行 GOVC 實驗！
:::
# LAB 1 環境建置
## 基本操作
:::success
- govc `ENTER`
- govc -h
- govc about
- govc about -json
:::
使用 **govc `ENTER`** 或 **govc -h** 顯示所有可執行的命令列表。
```bash
$ govc -h
Usage of govc:
about
about.cert
cluster.add
cluster.change
cluster.create
cluster.group.change
cluster.group.create
cluster.group.ls
cluster.group.remove
cluster.module.create
cluster.module.ls
cluster.module.rm
cluster.module.vm.add
cluster.module.vm.rm
<以下省略>
```
使用 **govc <command> -h** 可顯示該命令 (command) 的詳細說明。
```bash
$ govc about -h
Usage: govc about [OPTIONS]
Display About info for HOST.
System information including the name, type, version, and build number.
Examples:
govc about
govc about -json | jq -r .About.ProductLineId
Options:
-c=false Include client info
-cert= Certificate [GOVC_CERTIFICATE]
-debug=false Store debug logs [GOVC_DEBUG]
-dump=false Enable Go output
-json=false Enable JSON output
-k=true Skip verification of server certificate [GOVC_INSECURE]
-key= Private key [GOVC_PRIVATE_KEY]
-l=false Include service content
-persist-session=true Persist session to disk [GOVC_PERSIST_SESSION]
-tls-ca-certs= TLS CA certificates file [GOVC_TLS_CA_CERTS]
-tls-known-hosts= TLS known hosts file [GOVC_TLS_KNOWN_HOSTS]
-trace=false Write SOAP/REST traffic to stderr
-u=https://@10.7.150.90/sdk ESX or vCenter URL [GOVC_URL]
-verbose=false Write request/response data to stderr
-vim-namespace=vim25 Vim namespace [GOVC_VIM_NAMESPACE]
-vim-version=7.0 Vim version [GOVC_VIM_VERSION]
-xml=false Enable XML output
```
帶有 **-json`** 參數可將命令結果以 **JSON** 格式顯示，進而可透過 **jq** 工具解析。
:::info
jq 是一個命令列的 JSON 資料處理器，相關資訊請參考 [[**jq 官方網站**]](https://stedolan.github.io/jq/)。
:::
```bash
$ govc about -json
{
"About": {
"Name": "VMware vCenter Server",
"FullName": "VMware vCenter Server 7.0.3 build-21477706",
"Vendor": "VMware, Inc.",
"Version": "7.0.3",
"PatchLevel": "01400",
"Build": "21477706",
"LocaleVersion": "INTL",
"LocaleBuild": "000",
"OsType": "linux-x64",
"ProductLineId": "vpx",
"ApiType": "VirtualCenter",
"ApiVersion": "7.0.3.0",
"InstanceUuid": "b844e614-d9ea-4bb0-9e53-0cb0aa6439a1",
"LicenseProductName": "VMware VirtualCenter Server",
"LicenseProductVersion": "7.0"
}
}
$ govc about -json | jq '.About | .Version + "==" + .Build + "==" +.InstanceUuid'
"7.0.3==21477706==b844e614-d9ea-4bb0-9e53-0cb0aa6439a1"
```
## 建立資料中心
:::success
- govc datacenter.create ${dc_name}
:::
**全新的 vSphere 環境**
:::info
配合圖形界面從旁觀察由 GOVC 命令建立的狀態及結果，並可思考看看，若是相同工作任務以 GUI 方式需要用幾個步驟才能完成？
:::
輸入以下命令建立資料中心。基本上按下 `ENTER` 後就會立刻完成！
```bash
dc_name='LAB_Datacenter'
$ govc datacenter.create ${dc_name}
$ govc ls
/LAB_Datacenter/vm
/LAB_Datacenter/network
/LAB_Datacenter/host
/LAB_Datacenter/datastore
```
## 建立叢集
:::success
- govc cluster.create ${cluster_name}
:::
輸入以下命令建立叢集。
```bash
$ cluster_name='LAB_Cluster'
$ govc cluster.create -dc ${dc_name} ${cluster_name}
```
## 新增主機
:::success
- govc host.add -dc ${dc_name} -hostname ${esx} -username root -password ${pass} -noverify
- govc cluster.add -cluster ${cluster_name} -dc ${dc_name} -hostname ${esx} -username root -password 'VMware1!' -noverify
- govc host.disconnect -dc ${dc_name} -host.ip ${esx}
- govc host.reconnect -dc ${dc_name} -host.ip ${esxi}
:::
新增(獨立)主機至資料中心
```bash
$ esx03='10.7.150.93'
$ pass='VMware1!'
$ govc host.add -dc ${dc_name} -hostname ${esx03} -username root -password ${pass} -noverify
[29-05-23 15:30:48] adding 10.7.150.93 to folder /LAB_Datacenter/host... OK
```
新增主機至叢集
```bash
esx02='10.7.150.92'
$ govc cluster.add -dc ${dc_name} -cluster ${cluster_name} -hostname ${esx02} -username root -password ${pass} -noverify
[29-05-23 15:31:12] adding 10.7.150.92 to cluster /LAB_Datacenter/host/LAB_Cluster... OK
```
中斷連線主機
```bash
$ govc host.disconnect -dc ${dc_name} -host.ip ${esx03}
[29-05-23 15:34:00] /LAB_Datacenter/host/10.7.150.93/10.7.150.93 disconnecting... OK
$ govc host.disconnect -dc ${dc_name} -host.ip ${esx02}
[29-05-23 15:34:18] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 disconnecting... OK
```
重新連線主機
```bash
$ govc host.reconnect -dc ${dc_name} -host.ip ${esx03}
[29-05-23 15:35:48] /LAB_Datacenter/host/10.7.150.93/10.7.150.93 reconnecting... OK
$ govc host.reconnect -dc ${dc_name} -host.ip ${esx02}
[29-05-23 15:35:58] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 reconnecting... OK
```
:::info
`${dc_name}` 若是與 `${GOVC_DATACENTER}` 相同一致則可省略，否則便要指定。
:::
# LAB 2 ESXi 主機及網路設定
:::success
- govc host.info
- govc host.service.ls
- govc host.service
- govc host.date.info
- govc host.date.change
- govc host.vswitch.info
- govc host.portgroup.info
- govc host.vnic.info
- govc host.vnic.service
- govc host.esxcli
:::
## 檢視 ESXi 主機資訊
```bash
$ govc host.info -dc ${dc_name} -host ${esx02}
Name: 10.7.150.92
Path: /LAB_Datacenter/host/LAB_Cluster/10.7.150.92
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 50 MHz (1.2%)
Memory: 6142MB
Memory usage: 1754 MB (28.6%)
Boot time: 2023-05-17 08:17:55.573824 +0000 UTC
State: Maintenance Mode
$ govc host.info -host.ip ${esx03}
Name: 10.7.150.93
Path: /LAB_Datacenter/host/10.7.150.93/10.7.150.93
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 70 MHz (1.8%)
Memory: 6142MB
Memory usage: 1714 MB (27.9%)
Boot time: 2023-05-19 21:55:16.905412 +0000 UTC
State: Maintenance Mode
```
```bash
$ govc find / -type h
/LAB_Datacenter/host/LAB_Cluster/10.7.150.92
/LAB_Datacenter/host/10.7.150.93/10.7.150.93
$ govc find / -type h | xargs govc host.info
Name: 10.7.150.92
Path: /LAB_Datacenter/host/LAB_Cluster/10.7.150.92
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 50 MHz (1.2%)
Memory: 6142MB
Memory usage: 1754 MB (28.6%)
Boot time: 2023-05-17 08:17:55.573824 +0000 UTC
State: Maintenance Mode
Name: 10.7.150.93
Path: /LAB_Datacenter/host/10.7.150.93/10.7.150.93
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 48 MHz (1.2%)
Memory: 6142MB
Memory usage: 1719 MB (28.0%)
Boot time: 2023-05-19 21:55:16.905412 +0000 UTC
State: Maintenance Mode
```
:::info
若要檢視詳細主機資訊，可加入 **-json** 參數。
:::
## 檢視 ESXi 主機服務
```bash
$ govc host.service.ls -host ${esx02}
Key Policy Status Label
DCUI on Running Direct Console UI
TSM on Running ESXi Shell
TSM-SSH on Running SSH
lbtd on Running Load-Based Teaming Daemon
lwsmd off Stopped Active Directory Service
ntpd off Stopped NTP Daemon
pcscd off Stopped PC/SC Smart Card Daemon
sfcbd-watchdog on Stopped CIM Server
snmpd on Stopped SNMP Server
vmsyslogd on Running Syslog Server
vpxa on Running VMware vCenter Agent
xorg on Stopped X.Org Server
```
:::info
- Policy 啟動原則
- on: 隨主機一起啟動和停止
- off: 手動啟動和停止
- Status 精靈
- Running: 執行中
- Stopped: 已停止
:::
## 啟動/關閉主機服務
```bash
$ govc host.service.ls -host ${esx02} | grep 'TSM\|TSM-SSH'
TSM on Running ESXi Shell
TSM-SSH on Running SSH
$ govc host.service -dc ${dc_name} -host ${esx02} status TSM
Running
$ govc host.service -dc ${dc_name} -host ${esx02} stop TSM
$ govc host.service -dc ${dc_name} -host ${esx02} disable TSM
$ govc host.service.ls -host ${esx02} | grep 'TSM\|TSM-SSH'
TSM off Stopped ESXi Shell
TSM-SSH on Running SSH
```
## 調整時間組態
```bash
$ govc host.service.ls -host ${esx02} | grep ntpd
ntpd off Stopped NTP Daemon
$ govc host.date.info -host ${esx02}
Current date and time: Mon May 29 08:39:28 UTC 2023
NTP client status: Disabled
NTP service status: Stopped
NTP servers: time1.google.com
```
```bash
$ govc host.date.change -dc ${dc_name} -host ${esx02} \
-server 'time1.google.com,time2.google.com,time3.google.com'
$ govc host.service -dc ${dc_name} -host ${esx02} start ntpd
$ govc host.service -dc ${dc_name} -host ${esx02} enable ntpd
$ govc host.service.ls -dc ${dc_name} -host ${esx02} | grep ntpd
ntpd on Running NTP Daemon
$ govc host.date.info -dc ${dc_name} -host ${esx02}
Current date and time: Mon May 29 08:46:01 UTC 2023
NTP client status: Enabled
NTP service status: Running
NTP servers: time1.google.com,time2.google.com,time3.google.com
```
## 檢視 ESXi 主機網路
:::info
這裡以標準虛擬交換器（Standard Virtual Switch）為實驗測試對象。
:::
### 檢視虛擬交換器(vSwitch)
```bash
$ govc host.vswitch.info -dc ${dc_name} -host ${esx02}
Name: vSwitch0
Portgroup: VM Network, Management Network
Pnic: vmnic0
MTU: 1500
Ports: 2560
Ports Available: 2554
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
```
### 檢視網路埠群組(Portgroup)
網路埠群組亦稱為**網路標籤**。
```bash
$ govc host.portgroup.info -dc ${dc_name} -host ${esx02}
Name: VM Network
Virtual switch: vSwitch0
VLAN ID: 0
Active ports: 0
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
Name: Management Network
Virtual switch: vSwitch0
VLAN ID: 0
Active ports: 1
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
```
### 檢視 VMkernel 網路介面卡
```bash
$ govc host.vnic.info -dc ${dc_name} -host ${esx02}
Device: vmk0
Network label: Management Network
Switch: vSwitch0
IP address: 10.7.150.92
TCP/IP stack: defaultTcpipStack
Enabled services: management, vsan
```
## 設定 ESXi 主機網路
完成以下網路配置:
| vSwitch 名稱 | VMNIC(pNIC) | Portgroups | VLAN_ID |
| --- | --- | --- | :-: |
| vSwitch0 | vmnic0(Active),<br>vmnic1(Standby) | Management Network<br>VM Network | 0<br>100 |
| vSwitch1 | vmnic2 | PG-vMotion<br>PG-Backup | 151<br>152 |
| Security | Policy |
| --- | :-: |
| Promiscuous | Reject |
| MAC address changes | Reject |
| Forged transmits | Reject |
| VMkernel | IP Address | Services |
| --- | --- | --- |
| vmk0 | 10.7.150.92 | Management,vSAN |
| vmk1 | 10.7.151.92 | vMotion |
:::info
:::
### 建立虛擬交換器
```bash
$ vswitch_name='vSwitch1'
$ govc host.vswitch.add -dc ${dc_name} -host ${esx02} ${vswitch_name}
```
### 建立網路埠群組
```bash
$ vmotion_portgroup='PG-vMotion'
$ vmotion_vlan=151
$ backup_portgroup='PG-Backup'
$ backup_vlan=152
$ govc host.portgroup.add -dc ${dc_name} -host ${esx02} \
-vswitch ${vswitch_name} \
-vlan ${vmotion_vlan} \
${vmotion_portgroup}
$ govc host.portgroup.add -dc ${dc_name} -host ${esx02} \
-vswitch ${vswitch_name} \
-vlan ${backup_vlan} \
${backup_portgroup}
$ govc host.portgroup.info -dc ${dc_name} -host ${esx02}
Name: VM Network
Virtual switch: vSwitch0
VLAN ID: 0
Active ports: 0
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
Name: Management Network
Virtual switch: vSwitch0
VLAN ID: 0
Active ports: 1
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
Name: PG-vMotion
Virtual switch: vSwitch1
VLAN ID: 151
Active ports: 0
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
Name: PG-vSAN
Virtual switch: vSwitch1
VLAN ID: 152
Active ports: 0
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
```
### 變更網路埠群組設定
預設 PortGroup 安全性原則
<img width=280 src='https://hackmd.io/_uploads/ryn9Zx7U2.png'>
```bash
$ govc host.portgroup.change -dc ${dc_name} -host ${esx02} -vswitch-name 'vSwitch0' \
-allow-promiscuous=false -mac-changes=false -forged-transmits=false 'Management Network'
$ govc host.portgroup.change -dc ${dc_name} -host ${esx02} -vswitch-name 'vSwitch0' \
-allow-promiscuous=false -mac-changes=false -forged-transmits=false 'VM Network'
$ govc host.portgroup.change -dc ${dc_name} -host ${esx02} -vswitch-name 'vSwitch1' \
-allow-promiscuous=false -mac-changes=false -forged-transmits=false 'PG-vMotion'
$ govc host.portgroup.change -dc ${dc_name} -host ${esx02} -vswitch-name 'vSwitch1' \
-allow-promiscuous=false -mac-changes=false -forged-transmits=false 'PG-Backup'
```
<img width=280 src='https://hackmd.io/_uploads/BJun-gQ8h.png'>
### 建立 VMkernel 網路介面卡
其實 GOVC 並**沒有針對網路部份提供太多的使用命令**，不過，可以使用 **govc host.esxcli** 命令透過執行 **esxcli network** 命令直接完成網路進階設定。
輸入以下 esxcli 命令確認執行狀態。
```bash
$ govc host.esxcli -dc ${dc_name} -host ${esx02} network ip interface ipv4 get
Name IPv4 Address IPv4 Netmask IPv4 Broadcast Address Type Gateway DHCP DNS
---- ------------ ------------ -------------- ------------ ------- --------
vmk0 10.7.150.92 255.255.255.0 10.7.150.255 STATIC 0.0.0.0 false
$ govc host.esxcli -dc ${dc_name} -host ${esx02} network ip interface tag get -i vmk0
Tags: VSAN, Management
```
輸入以下命令建立 **vmk1**。
```bash
vmk_name='vmk1'
vmk_ip='10.7.151.92'
vmk_netmask='255.255.255.0'
vmk_gateway='10.7.151.254'
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network ip interface add --interface-name ${vmk_name} --portgroup-name ${vmotion_portgroup}
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network ip interface ipv4 set --interface-name ${vmk_name} \
--ipv4 ${vmk_ip} --netmask ${vmk_netmask} --gateway ${vmk_gateway} --type static
```
接著啟用 **vMotion 服務**以符合需求。
```bash
$ govc host.vnic.service -dc ${dc_name} -host ${esx02} \
vmotion ${vmk_name}
```
**服務(SERVICE)** 選項提供：
- vmotion
- faultToleranceLogging
- vSphereReplication
- vSphereReplicationNFC
- management
- vsan
- vSphereProvisioning
可與下方截圖比對。
### 管理實體介面卡
檢視實體介面卡資訊。
```bash
$ govc host.esxcli -dc ${dc_name} -host ${esx02} network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
---- ---------- ------ ------------ ----------- ----- ------ ----------- --- -----------
vmnic0 0000:0b:00.0 nvmxnet3 Up Up 10000 Full 00:50:56:b7:91:0f 1500 VMware Inc. vmxnet3 Virtual Ethernet Controller
vmnic1 0000:13:00.0 nvmxnet3 Up Up 10000 Full 00:50:56:b7:2b:81 1500 VMware Inc. vmxnet3 Virtual Ethernet Controller
vmnic2 0000:1b:00.0 nvmxnet3 Up Up 10000 Full 00:50:56:b7:69:c3 1500 VMware Inc. vmxnet3 Virtual Ethernet Controller
```
確認目前實體介面卡(Pnic)與虛擬交換器的配置關係。現在僅配置 **vmnic0** 給 **vSwitch0**。
```bash
$ govc host.vswitch.info -dc ${dc_name} -host ${esx02}
Name: vSwitch0
Portgroup: VM Network, Management Network
Pnic: vmnic0
MTU: 1500
Ports: 2560
Ports Available: 2551
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
Name: vSwitch1
Portgroup: PG-Backup, PG-vMotion
Pnic:
MTU: 1500
Ports: 2560
Ports Available: 2551
Allow promiscuous mode: No
Allow forged transmits: Yes
Allow MAC changes: Yes
```
<img width=540 src='https://hackmd.io/_uploads/BJqxY7mIh.png'><br>
接著完成以下任務:
- 指派 **vmnic1** 加入 **vSwitch0** 上行介面卡。
- 指派 **vmnic2** 加入 **vSwitch1** 上行介面卡。
```bash
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network vswitch standard uplink add --uplink-name vmnic1 --vswitch-name vSwitch0
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network vswitch standard uplink add --uplink-name vmnic2 --vswitch-name vSwitch1
```
:::warning
- 指派加入的介面卡會先直接放置於 **待命介面卡** 清單。
:::
可透過以下命令範例修改**整併和容錯移轉**設定。
```bash
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network vswitch standard policy failover set \
--active-uplinks=vmnic0,vmnic1 \
--load-balancing=iphash \
--failure-detection=link \
--notify-switches=true \
--failback=true \
--vswitch-name vSwitch0
```
:::info
- failure-detection 配置選項: beacon, link
- load-balancing 配置選項: explicit, iphash, mac, portid
:::
### 移除網路埠群組和虛擬交換器
#### 移除網路埠群組
```bash
$ govc host.portgroup.remove -dc ${dc_name} -host ${esx02} PG-Backup
$ govc host.portgroup.remove -dc ${dc_name} -host ${esx02} PG-vMotion
govc: ServerFaultCode: The resource 'PG-vMotion' is in use.
```
:::warning
如果網路埠群群組內包含 **VMkernel 介面卡**，要移除前必須先移除 VMkernel 介面卡，否則會有資源佔用無法移除的問題。相同的問題存在於移除虛擬網路交換器。
:::
#### 移除虛擬交換器
```bash
$ govc host.esxcli -dc ${dc_name} -host ${esx02} \
network ip interface remove -i vmk1
$ govc host.vswitch.remove -dc ${dc_name} -host ${esx02} ${vswitch_name}
```
成功移除虛擬交換器 **vSwitch1**。
:::info
GOVC 提供以下有關 DVS 配置功能，可自行練習配置。
- dvs.add
- dvs.change
- dvs.create
- dvs.portgroup.add
- dvs.portgroup.change
- dvs.portgroup.info
:::
# LAB 3 虛擬機器
:::success
- govc find / -type m
- govc vm.info
- govc vm.power
- govc snapshot.tree
- govc snapshot.create
- govc snapshot.remove
- govc vm.clone
- govc guest.ls
- govc guest.run
- govc vm.console
- govc vm.migration -host (vMotion)
- govc vm.migration -ds (Storage vMotion)
:::
## 查詢虛擬機器
查詢目前虛擬環境中帶用 `vESXi` 名稱的虛擬機器。
```bash
$ govc find / -type m | grep vESXi
/Datacenter/vm/Richard/Nested_ESXi/vESXi02
/Datacenter/vm/Richard/Nested_ESXi/vESXi01
/Datacenter/vm/Richard/Nested_ESXi/vESXi03
```
:::info
**`vESXi*`** 指的是 Nested ESXi 虛擬機器。
:::
## 檢視虛擬機器資訊
檢視帶有 `vESXi` 名稱虛擬機器的基本資訊。
```bash
$ govc find / -type m | grep vESXi | xargs govc vm.info
Name: vESXi02
Path: /Datacenter/vm/Richard/Nested_ESXi/vESXi02
UUID: 42379394-1d9d-6701-1cc9-bda391771dab
Guest name: VMware ESXi 6.5
Memory: 6144MB
CPU: 2 vCPU(s)
Power state: poweredOn
Boot time: 2023-05-30 03:58:33.462738 +0000 UTC
IP address: 10.7.150.92
Host: 10.7.150.108
Name: vESXi01
Path: /Datacenter/vm/Richard/Nested_ESXi/vESXi01
UUID: 42376e14-fe51-2f5f-3117-1f43d6e8b018
Guest name: VMware ESXi 6.5
Memory: 6144MB
CPU: 2 vCPU(s)
Power state: poweredOn
Boot time: 2023-05-17 08:17:12.481003 +0000 UTC
IP address: 10.7.150.91
Host: 10.7.150.108
Name: vESXi03
Path: /Datacenter/vm/Richard/Nested_ESXi/vESXi03
UUID: 4237d9e9-0c4c-8353-cc28-e6585a6a508b
Guest name: VMware ESXi 6.5
Memory: 6144MB
CPU: 2 vCPU(s)
Power state: poweredOn
Boot time: 2023-05-19 21:55:07.10957 +0000 UTC
IP address: 10.7.150.93
Host: 10.7.150.108
```
:::info
- 或是採用 **`govc vm.info "vESXi*"`** 命令也可產生相同顯示結果。
- 若只針對已開機的虛擬機器，可以加入以下參數：
**`govc find / -type m -runtime.powerState poweredOn`**
:::
## 控制虛擬機器電源
### 關閉電源
```bash
$ govc vm.info vESXi02 | grep 'Power state'
Power state: poweredOn
$ govc vm.power -off -force vESXi02
Powering off VirtualMachine:vm-342... OK
$ govc vm.info vESXi02 | grep 'Power state'
Power state: poweredOff
```
### 開啟電源
```bash
$ govc vm.power -on vESXi02
Powering on VirtualMachine:vm-342... OK
```
### 重啟電源
```bash
$ govc vm.power -r | -reset vESXi03
Reboot guest VirtualMachine:vm-343... OK
```
:::info
- 電源關閉預設會使用 VMTools，如果虛擬機器沒有正確安裝，執行命令會出錯，可以增加 `-force` 忽略錯誤強制執行。
- 電源開啟選項，有兩種方式: `VirtualMachine.powerOn(預設)` 和 `Datacenter.powerOnVm`。請參考 [[KB 1035224]](https://kb.vmware.com/s/article/1035224) 說明。
- 電源控制選項可以參考程式源碼。
:::
## 快照管理
### 建立快照
```bash
dc_name='Datacenter'
workVM='vESXi02'
snapshot_description='Created by GOVC'
snapshot_name='govc-snapshot01'
$ govc snapshot.create -dc ${dc_name} -vm "${workVM}" \
-d "${snapshot_description}" "${snapshot_name}"
```
:::info
- -m=true: 預設啟用**拍攝虛擬機器記憶體快照**。若要**取消**，使用 **`-m=false`**
- -q=false: 預設關閉**靜止客體檔案系統**。若要**啟用**，使用 **`-q=true`**
:::
```bash
snapshot_name='govc-snapshot02'
govc snapshot.create -dc ${dc_name} -vm "${workVM}" \
-m=false -d "${snapshot_description}" "${snapshot_name}"
```
### 檢視快照
```bash
$ govc snapshot.tree -vm ${workVM} -D -d -s -i
[7.2MB snapshot-385 Jun 4 11:34] govc-snapshot01 - Created by GOVC
[6.1MB snapshot-386 Jun 4 11:42] govc-snapshot02 - Created by GOVC
[6.1MB snapshot-386 Jun 4 11:42] . - Created by GOVC
```
:::info
- 參數
- -D=false: 顯示快照建立時間
- -d=false: 顯示快照說明
- -s=false: 顯示快照使用量
- -i=false: 顯示快照識別碼
- 目前快照位置則會在該快照下方以 **'.'** 進行標注。上述範例則顯示目前位於 [snapshot-386] 快照狀態。
:::
### 還原快照
```bash
revert_snapshotID='snapshot-385'
$ govc snapshot.revert -dc ${dc_name} -vm ${workVM} ${revert_snapshotID}
$ govc snapshot.tree -vm ${workVM} -D -i -d
[snapshot-385 Jun 4 11:34] govc-snapshot01 - Created by GOVC
[snapshot-385 Jun 4 11:34] . - Created by GOVC
[snapshot-386 Jun 4 11:42] govc-snapshot02 - Created by GOVC
```
:::info
- 目前快照還原後，位於 [snapshot-385] 快照狀態下。
- 指定快照還原可用 **快照識別碼** 或是 **快照名稱**。
:::
### 移除快照
移除快照前先建立 10 筆快照進行測試。
```bash
workVM='vESXi02'
dc_name='Datacenter'
snapshot_description='Created by GOVC'
$ for i in $(seq 1 10); do
govc snapshot.create -dc ${dc_name} -vm ${workVM} -m=false \
-d "${snapshot_description}" "snapshot-govc-test${i}"
done
$ govc snapshot.tree -vm ${workVM} -D -i -d
[snapshot-385 Jun 4 11:34] govc-snapshot01 - Created by GOVC
[snapshot-386 Jun 4 11:42] govc-snapshot02 - Created by GOVC
[snapshot-387 Jun 5 03:05] snapshot-govc-test1 - Created by GOVC
[snapshot-388 Jun 5 03:05] snapshot-govc-test2 - Created by GOVC
[snapshot-389 Jun 5 03:05] snapshot-govc-test3 - Created by GOVC
[snapshot-390 Jun 5 03:05] snapshot-govc-test4 - Created by GOVC
[snapshot-391 Jun 5 03:05] snapshot-govc-test5 - Created by GOVC
[snapshot-392 Jun 5 03:05] snapshot-govc-test6 - Created by GOVC
[snapshot-393 Jun 5 03:05] snapshot-govc-test7 - Created by GOVC
[snapshot-394 Jun 5 03:05] snapshot-govc-test8 - Created by GOVC
[snapshot-395 Jun 5 03:05] snapshot-govc-test9 - Created by GOVC
[snapshot-396 Jun 5 03:05] snapshot-govc-test10 - Created by GOVC
[snapshot-396 Jun 5 03:05] . - Created by GOVC
```
移除單一快照 **`snapshot-govc-test4`** 或 **`snapshot-390`**。
```bash
$ govc snapshot.remove -dc ${dc_name} -vm ${workVM} "snapshot-govc-test4"
$ govc snapshot.tree -vm ${workVM} -D -i -d
[snapshot-385 Jun 4 11:34] govc-snapshot01 - Created by GOVC
[snapshot-386 Jun 4 11:42] govc-snapshot02 - Created by GOVC
[snapshot-387 Jun 5 03:05] snapshot-govc-test1 - Created by GOVC
[snapshot-388 Jun 5 03:05] snapshot-govc-test2 - Created by GOVC
[snapshot-389 Jun 5 03:05] snapshot-govc-test3 - Created by GOVC
[snapshot-391 Jun 5 03:05] snapshot-govc-test5 - Created by GOVC
[snapshot-392 Jun 5 03:05] snapshot-govc-test6 - Created by GOVC
[snapshot-393 Jun 5 03:05] snapshot-govc-test7 - Created by GOVC
[snapshot-394 Jun 5 03:05] snapshot-govc-test8 - Created by GOVC
[snapshot-395 Jun 5 03:05] snapshot-govc-test9 - Created by GOVC
[snapshot-396 Jun 5 03:05] snapshot-govc-test10 - Created by GOVC
[snapshot-396 Jun 5 03:05] . - Created by GOVC
```
刪除多筆快照，包含本身及其子快照 **`snapshot-govc-test[6-10]`**。
```bash
govc snapshot.remove -dc ${dc_name} -vm ${workVM} -r "snapshot-govc-test6"
~/Projects/govc/lab
~/Projects/govc/lab govc snapshot.tree -vm ${workVM} -D -i -d
[snapshot-385 Jun 4 11:34] govc-snapshot01 - Created by GOVC
[snapshot-386 Jun 4 11:42] govc-snapshot02 - Created by GOVC
[snapshot-387 Jun 5 03:05] snapshot-govc-test1 - Created by GOVC
[snapshot-388 Jun 5 03:05] snapshot-govc-test2 - Created by GOVC
[snapshot-389 Jun 5 03:05] snapshot-govc-test3 - Created by GOVC
[snapshot-391 Jun 5 03:05] snapshot-govc-test5 - Created by GOVC
[snapshot-391 Jun 5 03:05] . - Created by GOVC
```
刪除全部快照。
```bash
$ govc snapshot.remove -dc ${dc_name} -vm ${workVM} '*'
$ govc snapshot.tree -vm ${workVM} -D -i -d
<此行空白，顯示已無任何快照檔案>
```
## 複製虛擬機器
將現有的虛擬機器複製成範本。
```bash
dc_name='Datacenter'
workVM='vESXi02'
folder="/${dc_name}/vm/Richard"
template_vm='vESXi_Template'
$ govc vm.clone -dc ${dc_name} -vm ${workVM} -folder ${folder} -template ${template_vm}
[05-06-23 11:30:50] Cloning /Datacenter/vm/Richard/Nested_ESXi/vESXi02 to vESXi_Template...OK
$ govc vm.info -r ${template_vm}
Name: vESXi_Template
Path: /Datacenter/vm/Richard/vESXi_Template
UUID: 42378d27-cdb2-398f-6528-904a035d61f8
Guest name: VMware ESXi 6.5
Memory: 6144MB
CPU: 2 vCPU(s)
Power state: poweredOff
Boot time: <nil>
IP address:
Host: 10.7.150.110
CPU usage: 0MHz
Host memory usage: 0MB
Guest memory usage: 0MB
Storage uncommitted: 13.0GB
Storage committed: 7.2GB
Storage unshared: 7.2GB
Storage: Share_Storage
Network: PG-MGMT
```
:::info
**`govc vm.clone`** 還有
:::
## 檢視資料夾內容
虛擬機器若有安裝 VMware-Tools 工具，可以透過 **govc guest.ls** 命令檢視客體作業系統資料夾內的檔案資訊。
```bash
workVM='Richard-security-Kali01'
dc_name='Datacenter'
vm_login='kali:VMware1!'
vm_path='/home/kali/Documents'
$ govc guest.ls -dc ${dc_name} -vm ${workVM} \
> -l "${vm_login}" "${vm_path}" | sort
drwxr-xr-x 1000 1000 4.0K Jun 5 05:42 2023 .
drwxr-xr-x 1000 1000 4.0K May 31 09:09 2023 ..
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_1.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_2.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_3.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_4.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_5.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_6.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_7.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_8.txt
-rw-r--r-- 1000 1000 198 Jun 5 05:42 2023 text_9.txt
```
## 執行命令
可以透過 **`govc guest.run`** 命令直接執行客體作業系統內的命令。這種方式就好像在客體作業系統的本機端執行相關命令。
```bash
workVM='Richard-security-Kali01'
dc_name='Datacenter'
vm_login='kali:VMware1!'
vm_cmd='ip a'
$ govc guest.run -dc ${dc_name} -vm ${workVM} \
-l "${vm_login}" "${vm_cmd}"
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:50:56:b7:34:75 brd ff:ff:ff:ff:ff:ff
inet 10.7.150.87/24 brd 10.7.150.255 scope global noprefixroute eth0
valid_lft forever preferred_lft forever
inet6 fe80::250:56ff:feb7:3475/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:40:69:38:d3 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
```
:::info
GOVC 工具提供 **`govc guest.*`** 相關命令，虛擬機器有安裝 VMware-Tools 工具，可以自行測試驗證下列命令，是否可以滿足日常任務需求，增加客體作業系統的控管能力及彈性。
```bash
govc guest.*
guest.chmod
guest.chown
guest.df
guest.download
guest.getenv
guest.kill
guest.ls
guest.mkdir
guest.mktemp
guest.mv
guest.ps
guest.rm
guest.rmdir
guest.run
guest.start
guest.touch
guest.upload
```
:::
## 啟動 Remote Console
使用 VMware Remote Console 可以直接連線到 vSphere 上的虛擬機器。請先下載並完成安裝 [**VMware Remote Console**](https://customerconnect.vmware.com/en/downloads/details?downloadGroup=VMRC1204&productId=1325)。
使用 Linux VMRC 程式開啟虛擬機器 Remote Console。
```bash
workVM='Richard-security-Kali01'
$ xdg-open $(govc vm.console ${workVM})
```
擷取虛擬機器目前桌面畫面並以 `kali_screenshot.png` 檔名儲存在當前資料夾。
```bash
img='kali_screenshot.png'
$ govc vm.console -capture ${img} ${workVM}
```
:::info
- 可加上 **`&& eog ${img}`** 直接顯示所擷取桌面畫面。
- 透過此種方式便可輕鬆完成截圖任務。
:::
## 移轉（Migration）
使用虛擬機器環境，最常用的其中一個功能就是遷移虛擬機器了！如果能夠透過命令列就可以完成移轉任務，似乎對於日常任務又多了不少便利性和彈性。
先檢視要移轉目標的虛擬機器 **Cloned_Kali**。該虛擬機器目前由 **10.7.150.107** 管控，並儲存在 **Share_Storage** 資料存放區。
```bash
$ govc vm.info -r ${clone_vm}
Name: Cloned_Kali
Path: /Datacenter/vm/Richard/Cloned_Kali
UUID: 4237a62d-4620-16b0-3579-5b38f235bbd0
Guest name: Other Linux (64-bit)
Memory: 16384MB
CPU: 4 vCPU(s)
Power state: poweredOn
Boot time: 2023-06-05 07:35:26.963927 +0000 UTC
IP address: 10.7.150.88
Host: 10.7.150.107
CPU usage: 80MHz
Host memory usage: 2368MB
Guest memory usage: 7536MB
Storage uncommitted: 47.6GB
Storage committed: 48.5GB
Storage unshared: 32.4GB
Storage: Share_Storage
Network: PG-WORKVM
```
### 移轉計算資源(vMotion)
接著將此虛擬機器由 **10.7.150.107** 移轉至 **10.7.150.108**。這就是一般的 **vMotion**。
```bash
src_host='10.7.150.107'
dest_host='10.7.150.108'
$ govc vm.migrate -dc ${dc_name} -host ${dest_host} ${clone_vm}
[05-06-23 15:52:28] migrating VirtualMachine:vm-405... OK
```
移轉完成，可以檢視該虛擬機器已經改由 **10.7.150.108** 管控。
```bash
$ govc vm.info -r ${clone_vm}
Name: Cloned_Kali
Path: /Datacenter/vm/Richard/Cloned_Kali
UUID: 4237a62d-4620-16b0-3579-5b38f235bbd0
Guest name: Other Linux (64-bit)
Memory: 16384MB
CPU: 4 vCPU(s)
Power state: poweredOn
Boot time: <nil>
IP address: 10.7.150.88
Host: 10.7.150.108
CPU usage: 100MHz
Host memory usage: 2434MB
Guest memory usage: 655MB
Storage uncommitted: 47.6GB
Storage committed: 48.5GB
Storage unshared: 32.4GB
Storage: Share_Storage
Network: PG-WORKVM
```
### 移轉儲存區(Storage vMotion)
接著進行 **Storage vMotion**，將虛擬機器存放位置變更至本地資料存放區 **datastore_108**。
```bash
src_ds='Share_Storage'
dest_ds='datastore_108'
$ govc vm.migrate -dc ${dc_name} -ds ${dest_ds} ${clone_vm}
[05-06-23 16:01:16] migrating VirtualMachine:vm-405... OK
```
```bash
$ govc vm.info -r ${clone_vm}
Name: Cloned_Kali
Path: /Datacenter/vm/Richard/Cloned_Kali
UUID: 4237a62d-4620-16b0-3579-5b38f235bbd0
Guest name: Other Linux (64-bit)
Memory: 16384MB
CPU: 4 vCPU(s)
Power state: poweredOn
Boot time: <nil>
IP address: 10.7.150.88
Host: 10.7.150.108
CPU usage: 200MHz
Host memory usage: 2152MB
Guest memory usage: 163MB
Storage uncommitted: 47.6GB
Storage committed: 48.5GB
Storage unshared: 32.4GB
Storage: datastore_108
Network: PG-WORKVM
```
### 同時移轉計算資源和儲存區
最後，再將虛擬機器移轉回 **10.7.150.107** 和 **Share_Storage**。
```bash
src_host='10.7.150.107'
src_ds='Share_Storage'
dest_host='10.7.150.108'
dest_ds='datastore_108'
$ govc vm.migrate -dc ${dc_name} -host ${src_host} -ds ${src_ds} ${clone_vm}
[05-06-23 16:12:00] migrating VirtualMachine:vm-405... OK
```
全部都歸位了！
```bash
$ govc vm.info -r ${clone_vm}
Name: Cloned_Kali
Path: /Datacenter/vm/Richard/Cloned_Kali
UUID: 4237a62d-4620-16b0-3579-5b38f235bbd0
Guest name: Other Linux (64-bit)
Memory: 16384MB
CPU: 4 vCPU(s)
Power state: poweredOn
Boot time: <nil>
IP address: 10.7.150.88
Host: 10.7.150.107
CPU usage: 100MHz
Host memory usage: 2144MB
Guest memory usage: 163MB
Storage uncommitted: 47.6GB
Storage committed: 48.5GB
Storage unshared: 32.4GB
Storage: Share_Storage
Network: PG-WORKVM
```
# LAB 4 測試環境整理
:::success
- govc vm.destroy
- govc host.maintenance.exit
- govc host.maintenance.enter
- govc host.remove
- govc object.destroy
:::
實驗即將告一段落，將測試環境整理乾淨，是一個好公民應該遵守的基本原則。
## 移除虛擬機器
首先，我們將先前複製產生的虛擬機器從虛擬環境中刪除，使用 GOVC 工具只要指定虛擬機器，便會自動關閉機器電源，並從磁碟中刪除該虛擬機器。
:::warning
- 當進行虛擬機器刪除任務，所有連接的虛擬磁碟皆會被刪除！
- 若有需要保留磁碟資料，可先輸入以下命令再進行刪除作業。
**`device.remove -vm ${destroy_vm} -keep "disk-*"`**
:::
```bash
destroy_vm=${clone_vm}
$ govc vm.destroy -dc ${dc_name} ${destroy_vm}
$ govc vm.info -r ${destroy_vm}
<此行空白，表示該虛擬機器不存在>
destroy_vm=${template_vm}
$ govc vm.destroy -dc ${dc_name} ${destroy_vm}
$ govc vm.info -r ${destroy_vm}
<此行空白，表示該虛擬機器不存在>
```
:::danger
使用 **`govc vm.destroy`** 真的很方便！定期清理虛擬機器必備工具。但換句話說，就跟用 root 執行 **`rm -rf`** 命令一樣危險恐怖！
:::
## 主機維護模式
使用 GOVC 工具操控主機維護模式也是很容易的！
先檢視 ESXi 主機狀態，目前為維護模式(**State: Maintenance Mode**)。
```bash
esx02='10.7.150.92'
esx03='10.7.150.93'
dc_name='LAB_Datacenter'
for host in ${esx02} ${esx03}; do \
govc host.info -dc ${dc_name} -host.ip ${host}; done
Name: 10.7.150.92
Path: /LAB_Datacenter/host/LAB_Cluster/10.7.150.92
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 27 MHz (0.7%)
Memory: 6142MB
Memory usage: 1293 MB (21.1%)
Boot time: 2023-06-04 11:01:42.516648 +0000 UTC
State: Maintenance Mode
Name: 10.7.150.93
Path: /LAB_Datacenter/host/10.7.150.93/10.7.150.93
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 24 MHz (0.6%)
Memory: 6142MB
Memory usage: 1205 MB (19.6%)
Boot time: 2023-06-04 11:07:08.983448 +0000 UTC
State: Maintenance Mode
```
### 結束維護模式
使用 **`govc host.maintenance.exit`** 可以將 ESXi 主機結束維護模式。
```bash
esx02='10.7.150.92'
esx03='10.7.150.93'
dc_name='LAB_Datacenter'
$ for host in ${esx02} ${esx03}; do \
govc host.maintenance.exit -dc ${dc_name} -timeout=120 -host.ip ${host}; done
[05-06-23 16:47:40] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 exiting maintenance mode... OK
[05-06-23 16:47:40] /LAB_Datacenter/host/10.7.150.93/10.7.150.93 exiting maintenance mode... OK
```
再次檢視 ESXi 主機狀態，都已脫離維護模式，屬於**已連線 (connected)** 狀態。
```bash
$ for host in ${esx02} ${esx03}; do \
govc host.info -dc ${dc_name} -host.ip ${host}; done
Name: 10.7.150.92
Path: /LAB_Datacenter/host/LAB_Cluster/10.7.150.92
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 31 MHz (0.8%)
Memory: 6142MB
Memory usage: 1296 MB (21.1%)
Boot time: 2023-06-04 11:01:41.637199 +0000 UTC
State: connected
Name: 10.7.150.93
Path: /LAB_Datacenter/host/10.7.150.93/10.7.150.93
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 108 MHz (2.7%)
Memory: 6142MB
Memory usage: 1275 MB (20.8%)
Boot time: 2023-06-04 11:07:08.230305 +0000 UTC
State: connected
```
### 進入維護模式
大多數要進行 ESXi 主機關機或硬體維修都會需要進入維護模式。之後我們要將這些 ESXi 主機脫離叢集，所以要先進入維護模式。
:::info
- 在進入維護模式的主機上執行的虛擬機器必須移轉到其他主機或關機。
- 主機將處於正在進入維護模式狀態，直到關閉所有執行中虛擬機器的電源或將虛擬機器移轉到其他主機為止。
- 如果主機正在進入維護模式，則無法開啟其上虛擬機器的電源，也無法將虛擬機器移轉到該主機。
:::
其實操作方式就跟結束維護模式一樣。但這裡我們就先假設 ESXi 主機中的虛擬機器跟資料都已處置完畢，以簡化測試流程。
```bash
esx02='10.7.150.92'
esx03='10.7.150.93'
dc_name='LAB_Datacenter'
$ for host in ${esx02} ${esx03}; do \
govc host.maintenance.enter -dc ${dc_name} -host.ip ${host}; done
[05-06-23 17:33:27] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 entering maintenance mode... OK
[05-06-23 17:33:28] /LAB_Datacenter/host/10.7.150.93/10.7.150.93 entering maintenance mode... OK
```
:::info
- 目前 GOVC 提供的維護模式操作是傳統非 vSAN 叢集的方式。
- 若要透過 GOVC 工具操作 vSAN 叢集下 ESXi 主機的維護模式，可以透過 **`govc host.esxcli`** 命令，搭配 **`esxcli system maintenanceMode set`**。
```bash
[root@esxi:~] esxcli system maintenanceMode set --help
Usage: esxcli system maintenanceMode set [cmd options]
Description:
set Enable or disable the maintenance mode of the system.
Cmd options:
-e|--enable=<bool> enable maintenance mode (required)
-t|--timeout=<long> Time to perform operation in seconds (default 0 seconds)
-m|--vsanmode=<str> Action the VSAN service must take before the host can enter maintenance mode (default
ensureObjectAccessibility). Allowed values are:
ensureObjectAccessibility: Evacuate data from the disk to ensure object accessibility
in the vSAN cluster, before entering maintenance mode.
evacuateAllData: Evacuate all data from the disk before entering maintenance mode.
noAction: Do not move vSAN data out of the disk before entering maintenance mode.
```
- 資料撤除模式 (Data Evacuation Mode)
- **確保可存取性 (Ensure accessibility)**
`esxcli system maintenanceMode set -e true -m ensureObjectAccessibility`
- **移轉全部資料 (Full data migration)**
`esxcli system maintenanceMode set -e true -m evacuateAllData`
- **不移轉資料 (No Data Migration)**
`esxcli system maintenanceMode set -e true -m noAction`
- 對於 vSAN 叢集環境關聯資料存取及可用性，這部份請自行參考 [[**官方說明**]](https://docs.vmware.com/en/VMware-vSphere/8.0/vsan-administration/GUID-521EA4BC-E411-47D4-899A-5E0264469866.html)。
:::
## 移除 ESXi 主機
:::info
執行之前，記得先將所有的 ESXi 主機再次**結束維護模式**。我們要觀察一下，在一般運行狀態移除 ESXi 主機會遇到什麼問題。
:::
再次檢視 ESXi 主機狀態是否已經**脫離維護模式 (State: connected)**。並請注意 **esx02** 和 **esx03** 是否存在於叢集當中。
```bash
esx02='10.7.150.92'
esx03='10.7.150.93'
$ for host in ${esx02} ${esx03}; do \
govc host.info -dc ${dc_name} -host.ip ${host}; done
Name: 10.7.150.92
Path: /LAB_Datacenter/host/LAB_Cluster/10.7.150.92
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 31 MHz (0.8%)
Memory: 6142MB
Memory usage: 1296 MB (21.1%)
Boot time: 2023-06-04 11:01:41.637199 +0000 UTC
State: connected
Name: 10.7.150.93
Path: /LAB_Datacenter/host/10.7.150.93/10.7.150.93
Manufacturer: VMware, Inc.
Logical CPUs: 2 CPUs @ 2000MHz
Processor type: Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
CPU usage: 108 MHz (2.7%)
Memory: 6142MB
Memory usage: 1275 MB (20.8%)
Boot time: 2023-06-04 11:07:08.230305 +0000 UTC
State: connected
```
:::info
- **esx02** 屬於 **叢集 LAB_Cluster** (Path: /LAB_Datacenter/host/**LAB_Cluster**)。
- **esx03** 不屬於叢集，但在**資料中心 LAB_Datacenter** (Path: /LAB_Datacenter/host/**10.7.150.93**) 當中。
:::
接著分別將 esx02 和 esx03 透過 **`govc host.remove`** 命令移除。
發現 **esx03(不屬於叢集) 可以直接移除**，不需要先進入維護模式。
```bash
$ govc host.remove -dc ${dc_name} -host.ip ${esx03}
[05-06-23 17:43:41] /LAB_Datacenter/host/10.7.150.93/10.7.150.93 removing... OK
```
但 **esx02(屬於叢集) 則不可以直接移除**，必須先進入維護模式後，才能移除主機。
```bash
$ govc host.remove -dc ${dc_name} -host.ip ${esx02}
[05-06-23 17:43:35] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 removing... Error: The operation is not allowed in the current state.
govc: The operation is not allowed in the current state.
```
進入維護模式後，順利完成移除主機任務。
```bash
$ govc host.maintenance.enter -host.ip ${esx02}
[05-06-23 17:44:45] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 entering maintenance mode... OK
$ govc host.remove -dc ${dc_name} -host.ip ${esx02}
[05-06-23 17:44:53] /LAB_Datacenter/host/LAB_Cluster/10.7.150.92 removing... OK
```
## 刪除叢集和資料中心
跟刪除虛擬機器一樣容易！就按照下方範例執行，清理得乾乾淨淨吧！
```bash
## 檢視目前資料中心物件路徑資訊
$ govc ls -dc ${dc_name} '*'
/LAB_Datacenter/vm/Discovered virtual machine
/LAB_Datacenter/vm/vCLS
/LAB_Datacenter/host/LAB_Cluster
```
可取得以下資訊:
- 資料中心路徑
**`/LAB_Datacenter`**
- 叢集路徑
**`/LAB_Datacenter/host/LAB_Cluster`**
然後使用 **`govc object.destroy`** 命令刪除物件路徑。
```bash
dc_name='LAB_Datacenter'
cluster_name='LAB_Cluster'
dc_path="/${dc_name}"
cluster_path="/${dc_name}/host/${cluster_name}"
$ govc object.destroy -dc ${dc_name} "${cluster_path}"
[05-06-23 23:52:09] destroying ClusterComputeResource:domain-c470... OK
$ govc object.destroy -dc ${dc_name} "${dc_path}"
[05-06-23 23:52:38] destroying Datacenter:datacenter-465... OK
```
# 總結
- 透過 GOVC 工具可以完成**重複性佈署**的場景。
- 使用簡單的腳本檔及設定編輯，也適用**擴展性調整**的配置。
- 輕量化的 GOVC 工具，不佔用太多資源，便於**自動化佈署**的虛擬環境。
---
透過實驗項目撰寫以下簡單的 BSAH 腳本檔 **labBuilder.sh**，便能正確無誤地完成資料中心和叢集的建立，並將 ESXi 主機增至叢集，脫離維護模式，完成測試環境的基本建置。剩下的部份，就請自行試試看囉！
**labBuilder.sh**
```bash
#!/bin/bash
govcConfig='labvcsa'
source ${govcConfig}
dc_name='LAB_Datacenter'
cluster_name='LAB_Cluster'
host=('10.7.150.91' '10.7.150.92' '10.7.150.93')
countHost=${#host[@]}
echo -e "\n[TASK] 建立資料中心 [${dc_name}]"
govc datacenter.create "${dc_name}"
echo -e "\n[TASK] 建立叢集 [${cluster_name}]"
govc cluster.create -dc "${dc_name}" "${cluster_name}"
user='root'
pass='VMware1!'
echo -e "\n[TASK] 新增主機至叢集"
for ((i=0;i<${countHost};i++)); do
echo -e "- 主機 [${host[$i]}]"
govc cluster.add -dc "${dc_name}" \
-cluster ${cluster_name} \
-hostname ${host[$i]} \
-username "${user}" -password "${pass}" -noverify
done
echo -e "\n[TASK] 主機結束維護模式"
for ((i=0;i<${countHost};i++)); do
echo -e "- 主機 [${host[$i]}]"
govc host.maintenance.exit -dc ${dc_name} -host.ip ${host[$i]}
done
```
**執行畫面**
**執行結果**
---
希望藉由以上的實驗項目練習，讓大家**初步**地了解 **GOVC 工具**的使用。當然，如果有發現到什麼**好玩的方式及進階的應用**，也務必**回饋**給其他有興趣的人，至少可以增加一點點工作上的**樂趣和效率**！
# 附錄
## 安裝 OVFTool
VMware OVF Tool (Open Virtualization Format Tool, ovftool) 是 VMware 提供的命令行工具，可以在 VMware 產品中達成 OVF/OVA 套件的匯入或匯出。其實 vCSA ISO 安裝檔裡面就有提供這個工具。
目前 ovftool 工具 4.5 版本（含）以上適用 vSphere 8.0，也可相容先前的 vSphere 版本。請到 OVF Tool 工具頁面下載並完成安裝 [[**連結**]](https://developer.vmware.com/web/tool/4.6.0/ovf-tool)。
安裝方式有 2 種:
- **安裝執行檔**。根據不同作業系統平台提供不同格式的安裝檔，執行下載的安裝檔即可。不過**下載這類的安裝檔案需要登入 VMware Customer Connect**。
- msi(Windows)
- bundle(Linux)
- dmg(Mac OS)
- **安裝壓縮檔**。下載後解壓至目的目錄即可。而且**可以直接下載壓縮檔案**。我們就選擇這種方式吧。
使用以下命令將 ovftool 壓縮檔解壓至 **`$HOME/bin`**。
```bash
$ unzip VMware-ovftool-4.6.0-21452615-lin.x86_64.zip -d $HOME/bin
```
:::info
如果目的資料夾不屬於 **$PATH**，請編修殼層配置檔 **.bashrc** 或 **.zshrc**。
:::
使用 **`ovftool --version | -v`** 確認安裝版本。
```bash
$ ovftool --version
VMware ovftool 4.6.0 (build-21452615)
```
:::warning
**注意** 若出現以下執行錯誤訊息。
ovftool.bin: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory
可安裝 libnsl 套件解決。
```bash
$ sudo dnf install libnsl
```
:::
## 安裝 Ansible
詳細安裝請參考 **Ansible Documetation** [[**Installing Ansible**]](https://docs.ansible.com/ansible/latest/installation_guide/index.html) 章節，請根據自身的作業系統完成。一般安裝都是透過 **pip** 安裝 Ansible，不過大多數的 Linux 散佈版本都會提供 Ansible 安裝套件使用。
由於我使用的是 **Fedora Linux**，就可以參考 Ansible 官網提供的[[**說明**]](https://docs.ansible.com/ansible/latest/installation_guide/installation_distros.html)，使用 **dnf 套件管理工具** 進行安裝。
```bash
## 列出可取得的 ansible 套件
$ sudo dnf list available ansible
## 安裝 ansible 套件
$ sudo dnf install ansible
## 列出已安裝的 ansible 套件
$ sudo dnf list ansible
```
使用以下命令顯示目前安裝的 Ansible 版本資訊。
```bash
$ ansible --version
ansible [core 2.14.5]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/richard/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.11/site-packages/ansible
ansible collection location = /home/richard/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible
python version = 3.11.3 (main, May 24 2023, 00:00:00) [GCC 13.1.1 20230511 (Red Hat 13.1.1-2)] (/usr/bin/python3)
jinja version = 3.0.3
libyaml = True
```
Sign in with Wallet
Connect another wallet