--- # What ISN'T virtualization - docker. - k8s. - most low-cost VPS services - LXC - OpenVZ - containers in general. ---- ## What it ALSO isn't: Process virtualization multiprocess machines trying to present a "single-process" picture of machine resources to their processes --- # The actual topic of this session - A machine whose entire runtime is being "emulated" by another machine's runtime - this ranges in end results from lightweight instrumentation to instruction translation. ---- - the "host" has total access to the the "guest" runtime - the host can intervene or limit _anything_ the guest does. - the host can pretend the existence of anything at all to the guest, as long as is doesn't mind the cost of ~~faking~~ emulation. --- # but y - live snapshot, restore, migrate _interim runtime states_ - security - incredibly hard to escape _total emulation_ - pretty much this is the only way to run a Lisp Machine in the modern day - emulating esoterica is important to some of us - in general, running things on alternate instruction set architectures (ISAs) ----  --- # how is this possible ---- ## a list of things we need to fake to a virtual machine - CPU registers - physical memory - PCI devices - USB controller - USB devices - storage controller - storage devices - network controller - VGA controller ---- ## how we go about faking them - the host, or hypervisor - read instructions - perform any required operations, somehow - simulate using software running on host - translate hardware calls from VM to host - update states - repeat ---- ## let's make that a little faster shall we? hardware accel - Often, significant chunks of guest instructions can be run as-is safely and without interfering with the host's ability to intervene in guest runtime. - Give direct access to specific hardware (disk, GPU, NIC etc) ---- ### cpu virtualization extensions write you an ISA that can handle sentences like "run this bit of code as if it were another machine" - Intel: VT-x - AMD: AMD-V --- # Linux as a VM host ## KVM KVM, or kernel-based virtual machine, is a kernel module for virtualisation related syscalls (ie wrappers to virtualisation CPU instructions) ---- ## QEMU - A VM manager - interface to KVM and other hypervisors - generic emulator for arbitrary architectures ---- ### things qemu does in linux All the userspace things - parse config - set up VM according to config - The process executable - CPU shares - mem allocation - I/O abstraction --- # Session 2: In more detail now, gentlefolk Running themes for this session: - how to let the host hardware do its job, but for a VM - interface, sharing, isolation - virt in practice - tradeoffs and concerns, software stacks --- # In the wild you will see two typical deployment styles for VMs ---- - public - ie let somebody else do the virt for you - AWS EC2 - digitalocean - most vps providers ---- - private - ie "I AM THE VIRT" - openstack - proxmox <3 - VMware ESXi --- # Virtualizing GPUs and other adjunct hardware Heterogenous PCI interfaced things ---- ## IOMMU ---- ## SR-IOV --- # Storage ---- ## file based vdisks - VMDK - Qcow2 - many others ---- ## block device passthrough - thinpools/zvols - raw block device passthrough - partition - disk ---- ## controller passthrough - SATA controller - NVMe device ---- ## Remote - Networked storage - Remote server with storage exposed over network - examples: NFS, SMB - SAN - clustered storage solution that uses mu;tiple servers and disks for MASSIVE PARALLELISM --- # Network ---- ## Bridge ---- ## PCI passthrough ---- ## SR-IOV ---- ## VLANs --- # Show and tell ---- # QEMU #nofilter ```bash /usr/bin/qemu -id 100 -name win -chardev \ 'socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait' \ -mon 'chardev=qmp,mode=control' \ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ -mon 'chardev=qmp-event,mode=control' -pidfile /var/run/qemu-server/100.pid \ -daemonize -smbios 'type=1,uuid=260a2be4-6f48-429a-87f5-9e83bae75b3e' \ -drive 'if=pflash,unit=0,format=raw,readonly,file=/usr/share/pve-edk2-firmware//OVMF_CODE.fd' \ -drive 'if=pflash,unit=1,format=raw,id=drive-efidisk0,file=/dev/pve/vm-100-disk-1' \ -smp '5,sockets=1,cores=5,maxcpus=5' -nodefaults \ -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ -vnc unix:/var/run/qemu-server/100.vnc,x509,password -no-hpet \ -cpu 'kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,enforce' \ -m 25000 -readconfig /usr/share/qemu-server/pve-q35.cfg \ -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ -device 'vfio-pci,host=0a:00.0,id=hostpci0.0,bus=ich9-pcie-port-1,addr=0x0.0,multifunction=on' \ -device 'vfio-pci,host=0a:00.1,id=hostpci0.1,bus=ich9-pcie-port-1,addr=0x0.1' \ -device 'vfio-pci,host=0a:00.2,id=hostpci0.2,bus=ich9-pcie-port-1,addr=0x0.2' \ -device 'vfio-pci,host=0a:00.3,id=hostpci0.3,bus=ich9-pcie-port-1,addr=0x0.3' \ -device 'vfio-pci,host=02:00.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' \ -device 'vfio-pci,host=0c:00.3,id=hostpci2,bus=ich9-pcie-port-3,addr=0x0' \ -device 'vfio-pci,host=0b:00.3,id=hostpci3,bus=ich9-pcie-port-4,addr=0x0' \ -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ -chardev 'socket,path=/var/run/qemu-server/100.qga,server,nowait,id=qga0' \ -device 'virtio-serial,id=qga0,bus=pci.0,addr=0x8' \ -device 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:ab224896b840' \ -drive 'file=/var/lib/vz/template/iso/archlinux-2019.07.01-x86_64.iso,if=none,id=drive-ide2,media=cdrom,aio=threads' \ -device 'ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2' \ -drive 'file=/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_500GB_S466NX0K743622X,if=none,id=drive-virtio2,format=raw,cache=none,aio=native,detect-zeroes=on' \ -device 'virtio-blk-pci,drive=drive-virtio2,id=virtio2,bus=pci.0,addr=0xc,bootindex=100' \ -drive 'file=/dev/disk/by-id/ata-ST4000VX007-2DT166_ZGY2RQJ1,if=none,id=drive-virtio3,format=raw,cache=none,aio=native,detect-zeroes=on' \ -device 'virtio-blk-pci,drive=drive-virtio3,id=virtio3,bus=pci.0,addr=0xd' \ -netdev 'type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \ -device 'virtio-net-pci,mac=42:2B:23:08:2C:B1,netdev=net0,bus=pci.0,addr=0x12,id=net0' \ -rtc 'driftfix=slew,base=localtime' -machine 'type=q35' \ -global 'kvm-pit.lost_tick_policy=discard' \ -cpu 'host,kvm=off,hv_vendor_id=AMD' ```