###### tags: `NTU`
<style>
.markdown-body {
font-family: 'Arial', -apple-system, BlinkMacSystemFont, 'Segoe WPC', 'Segoe UI', system-ui, 'Ubuntu', 'Droid Sans', sans-serif;
line-height: 1.6;
} !important;
</style>
# VM HW2
:::info
(10%) Describe how your hypercall and system call work. E.g. explain how your code functions.
:::
## Adding a System Call
First, I allocate a system call number 436 in several header files, including the one under `arch/arm64` and `include/uapi` directory. And set the maximum system call number to 437. Refer to the patch to see a complete list of files that are changed.
linux/arch/arm64/include/asm/unistd.h
```=41
#define __NR_compat_syscalls 437
```
linux/arch/arm64/include/asm/unistd32.h
```c=882
#define __NR_host_cpuid 436
__SYSCALL(__NR_host_cpuid, sys_host_cpuid)
```
After defining the system call number, the macro `__SYSCALL` will generate a function prototype for us and register this system call into the system. Then it's time for us to implement the system call body. I use `SYSCALL_DEFINE0` since our system call receive zero arguments.
```c=
SYSCALL_DEFINE0(host_cpuid)
{
/* make the host_cpuid hypercall and return the hypercall
* return value to user space
*/
printk("sys_host_cpuid calling hypercall...\n");
register long x0 __asm__("x0") = ARM_SMCCC_HVC_HOST_CPUID;
barrier();
asm volatile ("hvc #0\n\t" : : "r"(x0));
barrier();
printk("sys_host_cpuid hvc ret: %ld\n", x0);
return x0;
}
```
`ARM_SMCCC_HVC_HOST_CPUID` is the hyper call number that will be handled in KVM highvisor.
- line 8: I add a variable that directly mapped to the `x0` register, and then initialize its value to `ARM_SMCCC_HVC_HOST_CPUID`
- line 10: causes a Hypervisor Call exception. This will change the processor mode to EL2.
- line 9,11: I use compiler barriers here to prevent instruction reordering. Since the order is important, we must ensure the hyper call number is set before calling `hvc`. Therefore, it's essential to avoid reordering caused by compiler optimizations.
- line 14: The return value of the hyper call is in `x0` register. So we simply return `x0`.
> I use the following code snippet invoke the system call
> ```c
> #include <sys/syscall.h>
> long ret = syscall(436);
> ```
## Adding a Hyper Call
<!--  -->
When the guest executes `hvc #0` instruction, the processor mode will transition from EL1 to EL2, and then handled by Lowvisior. If the exception cannot be process in Lowvisior, it will further transition to Highvisior in EL1. As illustrated in the figure below.
First, we need to define a hyper call number to handle our newly added hyper call. I follow SMCCC ([Secure Monitor Call Calling Convention](https://developer.arm.com/documentation/den0028/latest)) that applied to both SMC and HVC.
linux/include/linux/arm-smccc.h
```c=79
#define ARM_SMCCC_HVC_HOST_CPUID \
ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \
ARM_SMCCC_SMC_32, \
5, /* Standard Hypervisor Service Calls */ \
1)
```
As the professor pointed out, there may be preemption when handling hyper call. Therefore, I set a flag called `hvc_host_cpuid_flag` in the `kvm_hvc_call_handler` and defer the real work after preemption is disabled, right before entering Lowvisior.
linux/virt/kvm/arm/psci.c
```c
switch (func_id) {
...
case ARM_SMCCC_HVC_HOST_CPUID:
printk("kvm_hvc_call_handler: ARM_SMCCC_HVC_HOST_CPUID\n");
hvc_host_cpuid_flag = 1;
return 1;
```
- The variable `func_id` is retrieved from the register `x0`, which we set it as hyper call number in our system call body.
- Then set the flag `hvc_host_cpuid_flag` to 1 and defer the work.
linux/virt/kvm/arm/arm.c
```c=736
preempt_disable();
if (hvc_host_cpuid_flag) {
pcpu_id = smp_processor_id();
vcpu_set_reg(vcpu, 0, pcpu_id);
hvc_host_cpuid_flag = 0;
}
```
- line 738: The flag `hvc_host_cpuid_flag` checked whether our hyper call need to be serviced or not.
- line 739: I use `smp_processor_id` to get the pcpu id
- line 740: The value is then written into register `x0`, which will be the return value of `hvc #0` instruction in our system call body.
## Behavior Observerd Through Invoking The Hypercall
<!-- <hr style="height: 0.08em"> -->
:::info
(5%) What is the returned host CPU ID that you got each time? Does the ID remain the
same or does it change sometimes?
:::
I called the system call at the interval of 500us, and find out that it changes sometimes.
```c
int main(int argc, char *argv[])
{
for (int i = 0; i < 10000; i++) {
long ret = syscall(436);
printf("%ld\n", ret);
if (errno != 0) {
perror("error");
return 1;
}
usleep(500);
}
return 0;
}
```
output:
```
$ ./a.out
0
0
0
0
0
0
1
0
1
1
1
1
1
1
1
1
1
1
1
0
0
1
1
1
0
0
0
0
0
0
1
0
0
0
0
0
0
1
```
<hr style="height: 0.08em">
:::info
(5%) What does KVM do that results in the different behaviors (e.g. host CPU ID
changing or remaining static) that you observe?
:::
In the main loop of `kvm_arch_vcpu_ioctl_run`, we can see each time when KVM transitions from Lowvisior to Highvisior, it calls `cond_resched` to schedule out of the CPU and let other tasks run. By the time we have access to the CPU again, we might be on another physical CPU due to the load balancing of SMP systems.
Another possibility is preemption. The main loop has some regions that allow preemption. By the time the KVM thread schedules in, it might be possible that KVM runs on a different physical CPU before preemption.
```c
int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
{
...
while (ret > 0) {
/*
* Check conditions before entering the guest
*/
cond_resched();
update_vmid(&vcpu->kvm->arch.vmid);
check_vcpu_requests(vcpu);
```
Sign in with Wallet
Connect another wallet