# libvirt with Unikraft

This is a tutorial on how to start a Unikraft application using `libvirt`.
Usually, we would start a Unikraft application using a `qemu` command or a wrapper of sorts, such as the ones provided by `kraft`. In order to start it as a `libvirt` domain, you have to provide a designated `xml` file that will be translated to a similar `qemu` command as the ones used usually.

Therefore, you have to **build** the application yourself using the instructions provided.

You should follow the instructions up until the point where you have the compiled unikernel image and you have finished the networking set up (bridge created, interface is UP and has an IP address assigned).

## Creating the domain file

Unfortunately, solely generating a working domain `XML` file from the user interface provided by `virt-manager` without tinkering it is just not possible on its own. This is because `virt-manager` makes some (wrong) assumptions regarding how your system will look like and will try to accomodate a regular operating system. Moreover, the `virt-manager` interface does not necessarily contain all the supported options and drivers that `libvirt` offers support for.

Therefore, the options are:
* creating a domain from the `virt-manager` interface with the needed components and make a few manual modifications;
* start from a tested dummy `xml` file and add the required modifications for your setup.

You may need to do a combination of both at the beginning; if you want a dummy XML file, skip to the **XML Example** section.

## Creating the domain file using the libvirt interface

Before you start, make sure you enable XML editing by going into `Preferences -> General -> Enable XML editing`.

In the first step, obviously opt for a manual installation.
<div align="center">
<img align="center" width="350" height="350" src="https://hackmd.io/_uploads/Sy8xksk46.png">
</div>

When prompted for the operating system type, enter `Generic or unknown OS`.

<div align="center">
<img align="center" width="350" height="350" src="https://hackmd.io/_uploads/B1mhyjk46.png">
</div>

Do not enable storage for the VM, as we will mount that ourselves.

For the last step, check `Customize installation before install` and select the bridge you want it to connect to.

<div align="center">
<img align="center" width="350" height="350" src="https://hackmd.io/_uploads/S11uxjkVa.png">
</div>

## Hardware

A Unikraft application does not need a lot of hardware to run. The following components have to be configured in order for it to run:

### Processor

Configure the processors as you see fit. The recommended `cpu` mode is `host-passthrough`, which may not selectable from the UI. It can be added later through `virt-manager`'s `xml` editing interface:

`<cpu mode='host-passthrough' />`

### Memory

Like before, add as much memory as you think is necessary. However, `virt-manager` automatically adds memory ballooning which is not currently supported by Unikraft, and **cannot be disabled from the UI**. A domain with memory ballooning **will not boot**. You will have to disable it manually. It usually resides under the `<devices>` tag, like so:

<devices>
...
<memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      <stats period='10'/>
      <driver iommu='on' ats='on'/>
</memballoon>
...
</devices>
    

You will have to replace that with:

``<memballoon model='none'>``

### Boot options

You will have to enable direct kernel boot and provide the absolute path to the kernel. If you are using the `initrd`-style filesystem mounting, you will have to provide it in the `Initrd path`. Also, you need to provide kernel arguments that tell `qemu` to provide an IP address to its bridge interface, the default gateway and the subnet mask.
Therefore, you will have to provide the following `Kernel args` for `nginx`:

`netdev.ipv4_addr=<server IP> netdev.ipv4_gw_addr=<gateway IP> netdev.ipv4_subnet_mask=<subnet mask> --`

<div align="center">
<img align="center" width="550" height="350" src="https://hackmd.io/_uploads/S1sIWoJNT.png">
</div>

### Filesystem

The `nginx` application supports both mounting the filesystem "normally" and using `initrd`. `libvirt` can easily accomodate either of those methods.

For `initrd`, it is enough to provide the path at the previous step (beware of the permissions).

In order to mount it separately, you will have to add a "Filesystem" component, preferably through editing the `XML` inside the `<devices>` tag:

<devices>
...
<filesystem type="mount" accessmode="passthrough">
  	<source dir="/absolute/path/to/fs0"/>
  	<target dir="fs0"/>
  	<address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
	</filesystem>
...
</devices>

Note that the `passthrough` accessmode is **extremely** important if you run `libvirt` as a non-root user (see `libvirt` documentation).

### Bridge

You will have to add a bridge component using the `virtio` model and specify the source bridge that is present on your system.

### Extra configurations

You also have to remove **Controller VirtIO Serial** if it is generated (again, it may not boot). `virt-manager` may ***add it back*** if you delete it from the UI, so you can try doing that from the `xml`.

You can safely remove pretty much all the rest of the components `libvirt` creates for you (USB, redirectors, CD, RNG, Sound, Guest Agent..) as long as you have a PCI controller.

I also added `Video` since I found it easier to manage than the serial interface and it works smoothly with the `virt-manager` interface.

## Creating and destroying domains

If starting from a custom XML, you can then define it using

`virsh define dom.xml`

And then it should appear in the `virt-manager` interface, where it can be started.

<div align="center">
<img align="center" width="550" height="350" src="https://hackmd.io/_uploads/ry97MikEp.png">
</div>

## XML Example

<domain type="kvm">
  <name>NGINX</name>
  <uuid>6a00d7d7-1f2e-4f84-869f-c4bc88fe378c</uuid>
  <memory unit="KiB">1048576</memory>
  <currentMemory unit="KiB">1048576</currentMemory>
  <vcpu placement="static">1</vcpu>
  <os>
	<type arch="x86_64" machine="pc-i440fx-4.2">hvm</type>
	<kernel>/path/to/nginx_kvm-x86_64</kernel>
	<cmdline>netdev.ipv4_addr=172.44.0.2 netdev.ipv4_gw_addr=172.44.0.1 netdev.ipv4_subnet_mask=255.255.255.0 --</cmdline>
	<boot dev="hd"/>
  </os>
  <cpu mode="host-passthrough"/>
  <clock offset="utc"/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
	<emulator>/usr/bin/qemu-system-x86_64</emulator>
	<controller type="pci" index="0" model="pci-root"/>
	<controller type="usb" index="0" model="piix3-uhci">
  	<address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/>
	</controller>
	<filesystem type="mount" accessmode="passthrough">
  	<source dir="/path/to/fs0"/>
  	<target dir="fs0"/>
  	<address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
	</filesystem>
	<interface type="bridge">
  	<mac address="52:54:00:0b:6c:38"/>
  	<source bridge="uk0"/>
  	<model type="virtio"/>
  	<address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0"/>
	</interface>
	<graphics type="spice" autoport="yes">
  	<listen type="address"/>
	</graphics>
	<video>
  	<model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
  	<address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0"/>
	</video>
	<memballoon model="none"/>
  </devices>
</domain>