I was trying to port OpenStack components to FreeBSD OS, and it will leverage bhyve to spawn virtual machines. The bhyve VM (denoted as "guest VM") will be nested because the development environment is in a VM, i.e., the OpenStack itself runs inside a VM (denoted as "host VM"). However, when trying to spawn a guest VM on the host VM (which is Linux KVM), the host VM will become unresponsive.
The environment I've got:
- A bare-metal server running [Harvester](https://harvesterhci.io)
- A FreeBSD-13.2-RELEASE AMD64 VM as the host VM running on Harvester
The same issue happened on a [Proxmox VE](https://pve.proxmox.com/wiki/Main_Page) server, too. Both Harvester and Proxmox VE are QEMU/KVM-based solutions.
The following steps describe how I create the bhyve guest VM with the aid of `vm-bhyve` tool. Most of the steps are from [this web page](https://klarasystems.com/articles/from-0-to-bhyve-on-freebsd-13-1/)
```
sudo pkg install vm-bhyve bhyve-firmware
sudo zfs create zroot/bhyve
sudo zfs set recordsize=64K zroot/bhyve
sudo zfs create zroot/bhyve/.templates
```
```
sudo sysrc vm_enable=yes
sudo sysrc vm_dir="zfs:zroot/bhyve"
echo 'vmm_load="YES"' | sudo tee -a /boot/loader.conf
```
In `/zroot/bhyve/.templates/uefi.conf`
```
loader="uefi"
cpu=2
cpu_sockets=1
cpu_cores=2
memory=4G
ahci_device_limit="8"
network0_type="virtio-net"
network0_switch="public"
disk0_type="nvme"
disk0_name="disk0.img"
```
```
sudo vm init
sudo vm switch create public
sudo vm switch add public vtnet0
sudo vm create -t uefi -s 10G freebsd-test
sudo vm install freebsd-test /zroot/bhyve/FreeBSD-13.2-RELEASE-amd64-bootonly.iso
```
After spinning up the guest VM, the host VM froze immediately. The only thing I can do is to forcibly reboot the host VM from the hypervisor on the bare-metal server.