# AMD-SEV Live Migration Shared Pages (Slow Migration, Precopy)

The notion of shared pages in AMD-SEV is not shared pages to be used by virtio. They are just pages that are not encrypted. As discussed before, AMD-SEV uses c-bit to manage the encryption status of a page. If the status needs to be changed, the guest should notify the host of the change through a PV hypercall. The host will also inform QEMU of the changes, so that QEMU will have an up-to-date view of the encryption status of all pages.

The shared regions will only be stored in QEMU, and not in host/guest kernel. QEMU will migrate the shared regions using special data structures.

## Code
- Linux 5.19: https://github.com/torvalds/linux/tree/v5.19
- QEMU: https://github.com/AMDESE/qemu/tree/sev_live_migration_v4_1

## Guest and Host Kernel
The guest notifies the host kernel of the encryption status of a page using KVM_HC_MAP_GPA_RANGE [3] and [4]. In Linux 5.19, the only place this hypercall is used is when booting, the host will reset the shared region (i.e., set to encrypted) and set a memory section called `_bss_decrypt` to decrypted [5]. The guest did not call it at all.

### I/O Memory
It is stated that "All the DMA operations inside the guest must be performed on shared memory."

## QEMU
Data structure: `sev_guest->shared_regions_list`. This is a list of `shared_region` impelemented as GFN intervals. It is different from SeKVM whose shared lists are discrete.

struct shared_region {
    unsigned long gfn_start, gfn_end;
    QTAILQ_ENTRY(shared_region) list;
};

The guest can add and remove regions to and from the list via KVM_EXIT_HYPERCALL with when the host kernel emulates KVM_HC_MAP_GPA_RANGE in userspace [2]. When saving memories, QEMU will first check whether a page is encrypted. If yes, it will use the AMD-SEV API to send the page using the guest key and provide a special header in the migration stream so that the target will also use the special API to receive the page. Otherwise, either the page is within a region in the shared list or memory encryption is not enabled at all [6]. In this case, QEMU will use the normal APIs to migrate the page. The list itself is saved/loaded to/from the migration stream during migration so that the VM can be migrated again from the target to elsewhere [1].

## Discussions
QEMU manages the shared regions explicitly without even exposing the region to the kernel.

For performance, since the shared regions are created at boot time, it adds nearly no overhead. However, if the user wants only to partially encrypt a VM, they need to either specify what memory to encrypt during boot time or dynamically call KVM_HC_MAP_GPA_RANGE. The former is a static shared region list and will not be optimal for space, while the latter will introduce large overhead because each time the request has to travel from guest to host kernel and then to QEMU through emulation. I am not sure how it will compare to the overhead of SeKVM's GRANT/REVOKE model.

For security, a malicious QEMU can change the shared region lists. However, each private memory must be decrypted with a guest-provided key, making the host impossible to steal the private memory. Only availability will be affected.

## References
[1] https://github.com/AMDESE/qemu/commit/9236f522e48b67fe7136de7620276f7dc193be37
[2] https://github.com/AMDESE/qemu/blob/sev_live_migration_v4_1/docs/amd-memory-encryption.txt#L127
[3] https://docs.kernel.org/virt/kvm/x86/hypercalls.html#kvm-hc-map-gpa-range
[4] https://patchwork.kernel.org/project/kvm/patch/9077[e-mail]/#24240689
[5] https://github.com/torvalds/linux/blob/master/arch/x86/kernel/kvm.c#L938
[6] https://github.com/AMDESE/qemu/commit/2d6bda0d4cf3202b22d23f3eebf743588f6e506a