# Release Post - v0.8.0 (Enceladus) :::info To generate the feature list: ```bash gh pr list --label "ci/merged" -s closed --search "milestone:\"v0.8.0 (Enceladus)\"" -L 100 --json author,title,number --template '{{range .}}{{ printf "* %s (#%v) by @%s\n" .title .number .author.login }}{{end}}' ``` To generate the list of reviewers: ```bash git --no-pager log --format="* %(trailers:key=Reviewed-by,valueonly,separator=%x2C )" --abbrev-commit HEAD...RELEASE-0.7.0 | uniq ``` ::: ## GitHub release note We are proud to announce the latest version of Unikraft, v0.8.0 (Enceladus)! In this release, we introduce many new improvements, including critical bug fixes, improved stability and new features. To summarise: ## šŸš€ New Features * Introduce Pointer Authentication (PAuth) support (#369) by @michpappas * Virtual Memory API (x86_64, kvm) (#338) by @marcrittinghaus * Support for partial checksumming / checksum offloading (#308) by @skuenzer * Add `uk_event` for new trap interface (#227, #230, #231) by @marcrittinghaus * Additional interrupt context safe functions (#192) by @cristian-vijelie * TAP network device driver (#251) by @skuenzer * Add basic test suite to `ukargparse` (#392, #431) by @StefanJum * Add `uk_list_last_entry_or_null` (#396) by @marcrittinghaus * Add `mprotect` (#394) by @marcrittinghaus ## šŸ› Bug Fixes & Improvements * Rewrite of ubsan library (#404) by @marcrittinghaus * Migrate and update docs (#252) by @nderjung * Allow `mkcpio` to quietly gzip compress CPIO images (#359) by @aadhi0319 * Enhance arm(64)/x86_64 Bitscan Functions (#395) by @marcrittinghaus * Add `getgrgid` to export (#410) by @marcrittinghaus * Do not invoke `rustc` if `ukrust` is not enabled (#403) by @michpappas * Fixes layout of assert table on ARM64 (#397) by @marcrittinghaus * Add dependency to `lib/ukalloc` for `linuxu` (#430) by @skuenzer * Fix quoted whitespaces handling (#433) by @StefanJum * Fix build warning for `_uk_printd` definition (#432) by @gabrielmocanu * Fix build warning for halt definition (#429) by @gabrielmocanu * Fix build errors introduced by #251 (#412) by @vladandrew * Fix conditional definition of functions (#399) by @george-hopkins * Fix some bugs (#394) by @marcrittinghaus * Fix and introduce more range macros (#370) by @skuenzer * Fixing various bugs (#411) by @marcrittinghaus * Complete syscall lists on arm and aarch64 (#423) by @razvanvirtan * Replace `EDOOFUS` macro (#416) by @maniatro111 * Remove `uk_free` call on request dequeue (#413) by @mschlumpp * Move alloc out of critical section in `sys_open` (#408) by @marcrittinghaus * Use types from `shareddefs.h` (#407) by @marcrittinghaus * Use non-prototyped no-ops (#405) by @marcrittinghaus * Do not pass `-march` with `-mcpu` (#368) by @michpappas * Do not allocate IRQ handler entries dynamically (#240) by @hlef A big thank you also to all those who helped in the [review process](unikraft.org/docs/contributing/review-process/): @cristian-vijelie, @michpappas, @mschlumpp, @skuenzer, @adinasm, @florin-diaconescu, @gabrielmocanu, @mogasergiu, @razvand, @StefanJum, @razvanvirtan, @craciunoiuc, @danield20 and @vladandrew. --- For more information, check out [the accompanying blog post](https://unikraft.org/blog/2022-03-29-unikraft-releases-enceladus) or [view the full changelog](https://github.com/unikraft/unikraft/compare/RELEASE-0.7.0...RELEASE-0.8.0). /* * (Cristian Vijelie @cristian-vijelie) * (Michalis Pappas @michpappas) * (Marco Schlumpp @mschlumpp) * (Simon Kuenzer @skuenzer) * (Adina Smeu @adinasm) * (Florin Postolache @florin-diaconescu) * (Gabi Mocanu @gabrielmocanu) * (Sergiu Moga @mogasergiu) * (Razvan Deaconescu @razvand) * (Stefan Jumarea @StefanJum) * (Răzvan VĆ®rtan @razvanvirtan) * (Cezar Craciunoiu @craciunoiuc) * (Daniel Dinca @danield20) * (Vlad-Andrei Badoiu @vladandrew) */ --- ## Twitter Unikraft v0.8.0 (Enceladus) is out! šŸš€ Enceladus has many new features (like the Virtual Mem API and ARM PAuth), bug fixes and improvements. Check it out! https://github.com/unikraft/unikraft/releases/tag/RELEASE-0.8.0 --- ## LinkedIn We are proud to announce the v0.8.0 (Enceladus) release of Unikraft. This is a release in our new shorter release cycle that we do for Unikraft: once every two months. This release features many new features, bug fixes and improvements, multiple features on ARM platform, the virtual memory API and more. For more details, please check out our blog post at: ... Find out more and join us on GitHub (https://github.com/unikraft/) and on Discord (https://bit.ly/UnikraftDiscord). --- ## Blog Post We're excited to announce Unikraft v0.8.0 (Enceladus) and to show off many of the things the community has been working on over the last two months. In this blog post, we highlight some of the new features available in Unikraft. ### ARM Pointer Authentication (PAuth) support Pointer Authentication (PAuth) allows signing and authenticating pointers to harden the system against classes of attacks that rely on the manipulation of pointers, such as ROP. Pointer Authentication Codes (PACs) are created by signing a pointer and a 64-bit modifier with an 128-bit key. The modifier is a value that is normally used to restrict the PAC to a specific context (eg the SP). Keys are stored in registerers. The PAC is stored in the unused upper bits of the memory address, and can be later verified to ensure that the pointer has not been tampered. The reference algorithm is [the QARMA block cipher](https://eprint.iacr.org/2016/444.pdf), but it is possible for architectures to use an IMPLEMENTATION DEFINED algorithm instead. A platform can initialize PAuth as: ```c #ifdef CONFIG_ARM64_FEAT_PAUTH if (ukplat_pauth_enable()) UK_CRASH("Pointer Authentication is not available"); #endif ``` To generate PAyth keys, it is required that platforms provide an implementation of the key generation function that uses an adequate source of randomness. ```c void ukplat_pauth_gen_key(__u64 *key_hi, __u64 *key_lo); ``` Platforms that support Armv8.5 can use the random number generation instructions introduced into the architecture (`RNDR`/`RNDRSS`). Others can initialize drivers of an HWRNG/TRNG, or request randomness from the TEE. Since the above functions are used for the initialization of PAuth, it is required that GCC excludes them when generating PACIASP/RETAA sequences. Otherwise, once PAuth is enabled, authentication will fail upon return. A macro is provided to help overriding the global gcc settings per function: ``` #define __no_pauth __attribute__((target("branch-protection=none"))) ``` This macro is used by `pauth_enable()`, and should be used by all functions generated by GCC in the boot chain, up to - and including - the caller of `pauth_init()`. Additionally to the above, **we decided against enabling backwards compatibility mode**. If `-march=armv8.3-a` is not set, it is not possible to initialize the keys. GCC exits with the following error: ``` bash$ ~/toolchains/gcc-arm-10.3-2021.07-x86_64-aarch64-none-elf/bin/aarch64-none-elf-gcc -mbranch-protection=standard /tmp/test.c /tmp/ccx7K1nD.s: Assembler messages: /tmp/ccx7K1nD.s:17: Error: selected processor does not support system register name 'apiakeyhi_el1' ``` Clearly on Unikraft this feature cannot be utilized, except from the case of linking with prebuilt libraries. Because of this, this change sets `-march=armv8.3-a` when enabling `CONFIG_ARM64_FEAT_PAUTH`. **We believe that Arm's intention behind compatibility mode was mostly to provide backwards compatibility on userspace libraries.** #### GCC Support for PAuth - **GCC 7** * Introduce support for armv8.3-a * Introduce `--msign-return-address=[none|non-leaf|all]` The parameters passed to `--msign-return-address` are interpreted as: * `none`: Do not sign return addresses * `non-leaf`: Sign/auth the return address of non-leaf functions. * `all`: Sign/auth the return address of non-leaf and leaf functions. GCC implements the Basic Set of PAuth instructions using the HINT instruction, in the so-called NOP space. This allows executing protected binaries on older platforms that do not implement Armv8.3-a. In these platforms PAuth instructions execute as NOP. When `sign-return-address` is enabled without setting `-march=armv8.3-a`, GCC generates PACIASP / AUTIASP sequences that sign and authenticate the LR upon function entry and exit. In the following snippet notice the opcode of the HINT instruction (0xd5032300) that GCC generates for PACIASP and AUTIASP. ``` 0000000000400564 <main>: 400564: d503233f paciasp 400568: a9bf7bfd stp x29, x30, [sp, #-16]! 40056c: 910003fd mov x29, sp 400570: 90000000 adrp x0, 400000 <_init-0x3e8> 400574: 9118c000 add x0, x0, #0x630 400578: 97ffffb6 bl 400450 <puts@plt> 40057c: 52800000 mov w0, #0x0 400580: a8c17bfd ldp x29, x30, [sp], #16 400584: d50323bf autiasp 400588: d65f03c0 ret ``` When `-msign-return-address` is used along with `-march=armv8.3-a`, GCC generates PACIASP / RETAA sequences instead. In the snippet below notice the opcode of RETAA (0xd65f0bff) which is in the fused space (instructions in the Combined Set do not have HINT implementations). Clearly, this code cannot be executed in architectures earlier than Armv8.3-a. ``` 0000000000400564 <main>: 400564: d503233f paciasp 400568: a9bf7bfd stp x29, x30, [sp, #-16]! 40056c: 910003fd mov x29, sp 400570: 90000000 adrp x0, 400000 <_init-0x3e8> 400574: 9118a000 add x0, x0, #0x628 400578: 97ffffb6 bl 400450 <puts@plt> 40057c: 52800000 mov w0, #0x0 400580: a8c17bfd ldp x29, x30, [sp], #16 400584: d65f0bff retaa ``` * **GCC 9** * Introduce support for armv8.5-a * Deprecate `-msign-return-address` * Introduce `-mbranch-protection=[none|pacret{+leaf}|bti|standard]` * Introduce `--enable-standard-branch-protection` as a short to `mbranch-protection=standard` The parameters passed to `-mbranch-protection` are interpreted as: * `none`: Disables all protections * `pac-ret`: Enables PAuth for function returns on non-leaf functions. The `+leaf` modifier enables protection for leaf functions. * `bti`: Enables Branch Target Identification * `standard`: Enables all protections * **GCC 10:** * Introduce the b-key addendum to pac-ret in `--mbranch-protection`. This allows using the APIB_Key instead of the APIA_Key when signing return pointers. For more information on this feature: * Check out the [related pull request](https://github.com/unikraft/unikraft/pull/369) which highlights additional details; * [The Arm Architecture Reference Manual Armv8, for Armv8-A architecture profile](https://developer.arm.com/documentation/ddi0487/ga); * [Learn the architecture: Providing protection for complex software](https://developer.arm.com/documentation/102433/latest/); * [Pointer Authentication on ARMv8.3](https://www.qualcomm.com/media/documents/files/whitepaper-pointer-authentication-on-armv8-3.pdf). ### Virtual Memory API Currently, Unikraft establishes a virtual address space using a fixed boot page table that maps the first 1 GiB of memory. This change introduces an architecture-independent API for dynamically (un-)mapping virtual to physical addresses, setting virtual memory protections, and dynamically managing physical memory. This allows Unikraft to access all available RAM in the system and enables the implementation of system calls such as mmap. The frame allocator is a multi-zone buddy-based allocator that is optimized for power-of-two allocations, but can also handle arbitrary allocations without wasting memory like a conventional buddy allocator. Also allocations and frees do not have to match allowing to dynamically manage physical memory on a page (4K) granularity. #### Basic (Un-)Mapping Establish a read-write mapping of 16 MiB (= 4096 pages) at the virtual address `0x10000000000`: ```C struct uk_pagetable *pt = ukplat_pt_get_active(); ukplat_page_map(pt, 0x10000000000, __PADDR_ANY, 4096, PAGE_PROT_RW, 0); ``` The call implicitly allocates the physical memory (due to `__PADDR_ANY`) and maps it to the given virtual address. The mapping will automatically use the largest page sizes possible according to the alignment, mapping size, available continuous physical memory, and page sizes supported by the architecture. On x86_64, for example, the call will create a mapping of 8 large pages (2 MiB). Force a mapping of a particular page size: ```C ukplat_page_map(pt, 0x10000000000, __PADDR_ANY, 8, PAGE_PROT_RW, PAGE_FLAG_SIZE(PAGE_LARGE_LEVEL) | PAGE_FLAG_FORCE_SIZE); ``` Unmap a range of memory (here 10 4K pages): ```C ukplat_page_unmap(pt, 0x10000000000, 10, 0); ``` This will split the first large page created in the previous calls and unmap the first 10 small pages. This will also free the frames unless `PAGE_FLAG_KEEP_FRAMES` is specified. If page tables can be released during the operation this is done transparently (except `PAGE_FLAG_KEEP_PTES` is specified). Note that it is no problem to unmap physical address ranges previously not allocated by the underlying frame allocator (for example kernel code and data), even if `PAGE_FLAG_KEEP_FRAMES` is not specified. #### Specific Physical Memory Mapping Mapping a specific physical address can be done by using the physical address instead of `__PADDR_ANY`. However, this requires that the physical memory has been allocated manually by calling the frame allocator first: ```C __paddr_t paddr = 0x40000000; /* Physical memory at 1 GiB, __PADDR_ANY for any physical address */ pt->fa->falloc(pt->fa, &paddr, 10); /* paddr receives allocated address if __PADDR_ANY */ ukplat_page_map(pt, 0x10000000000, paddr, 10, PAGE_PROT_RW, 0); ``` If the physical memory allocation should be restricted to a certain range (e.g., for DMA), this can be done like so: ```C __paddr_t paddr; pt->fa->falloc_from_range(pt->fa, &paddr, 10, 0, MIN, MAX); ``` #### Page Table Walking Sometimes direct access to the page table entries (PTEs) is needed. For example to store custom information in user-available bits or set advanced caching flags. ```C unsigned int level = PAGE_LEVEL; /* perform complete walk down to page level */ __vaddr_t pt_vaddr; /* OPTIONAL, receives virtual address where the page table is mapped that contains the PTE */ __pte_t pte; /* OPTIONAL, receives the PTE */ ukplat_pt_walk(pt, 0x10000000000, &level, &pt_vaddr, &pte); /* Set some extended PTE flag */ pte |= MY_EXTENDED_FLAG; /* Update the PTE in the page table */ ukarch_pte_write(pt_vaddr, level, PT_Lx_IDX(0x10000000000, level), pte); ukarch_flush_tlb_entry(0x10000000000); ``` For more information on this feature: * Check out the [related pull request](https://github.com/unikraft/unikraft/pull/338) which highlights additional usage; * See [`uk/paging.h`](https://github.com/unikraft/unikraft/blob/staging/include/uk/plat/paging.h) for all exposed methods. ### ### Support for partial checksumming / checksum offloading For checksum offloading for TCP and UDP flows, typically, a partial checksum needs to be computed. This checksum covers only the pseudo header of the protocols. By specifying a start pointer and another pointer to the checksum field, a hardware network device is able to complete the computation of the checksum. This change introduces support for `lib/uknetdev` for such a feature. Two new flags and two fields for pbufs are introduced to specify information needed by the driver to offload the computation for a packet. Additionally, a new `uknetdev` driver state is introduced: `UK_NETDEV_UNPROBED`. This state indicates that network devices features weren't negotiated with the device yet. During device initialization, a call to `uk_netdev_probe()` is becoming mandatory to turn the device into `UK_NETDEV_UNCOFNIGURED` state. For some devices (like `virtio-net`) this is required to figure out if the backend supports partial checksumming. ### A new `uk_event` trap interface `uk_event` decouples the definition of an event and the definition of callback routines that should be called whenever the event ocurrs. This way a library (A) can provide an event and a library (B) can register handlers for this event without creating a dependency (A)->(B). Handler registration is done at link time by collecting all handler functions for a particular event and putting their addresses into an event-specific function table. Handlers can indicate that they have handled the event, in which case no other handlers are called, if desired. Handlers can also be assigned a priority class to enforce a certain execution order. #### Using Events In the following, we define an event `myevent` in library (A). We encapsulate additional event-specific information in `struct myevent_data`. Library (B) includes the declaration of `struct myevent_data` and registers three handler functions, where `myhandler1` and `myhandler2` allow other handlers to be called afterwards and `myhandler3` terminates handler invocation. Note: Since there are no guarantees in which order handlers of the same priority class may be called, it is only guaranteed that `handler1` will be the first of libB's handlers that is invoked. However, `handler2` may or may not be called depending on if `handler3` is executed before. **libA/include/uk/myevent.h**: ```C struct myevent_data { int dummy; }; ``` **libA/myevent.c:** ```C #include <uk/event.h> #include <uk/myevent.h> UK_EVENT(myevent); void some_function(void) { struct myevent_data = { .dummy = 42 }; int rc; rc = uk_raise_event(myevent, &myevent_data); if (!rc) uk_pr_err("myevent not handled!\n"); } ``` **libB/myhandler.c:** ```C #include <uk/event.h> #include <uk/myevent.h> int handler1(void *arg) { struct myevent_data *data = (struct myevent_data *)arg; ... return UK_EVENT_HANDLED_CONT; } int handler2(void *arg) { return UK_EVENT_HANDLED_CONT; } int handler3(void *arg) { return UK_EVENT_HANDLED; } UK_EVENT_HANDLER_PRIO(handler1, UK_PRIO_EARLIEST); UK_EVENT_HANDLER_PRIO(handler2, UK_PRIO_LATEST); UK_EVENT_HANDLER_PRIO(handler3, UK_PRIO_LATEST); ``` For more information on this feature ceck out the [related pull request](https://github.com/unikraft/unikraft/pull/338) which highlights additional usage. ### Additional interrupt context safe functions Additional support for providing isr-safe functions for `uklock` and `ukmpi` libraries is made by aliasing to some inlined functions, that can be called in an interupt context, in `uk/isr/mutex.h` and `uk/isr/semaphore.h`. For mbox, the change moves the functions that can be called in an interupt context to a separate sorce file, `mbox_isr.c`, and moves some static inlined functions needed by both mbox and `mbox_isr` in the `mbox.h` header, along with the definition of `struct uk_mbox`. ### Google Summer of Code 2022 [Unikraft is part of Google Summer of Code 2022 (GSoCā€™22)](https://unikraft.org/blog/2022-03-16-unikraft-gsoc-org/), a global online program focused on bringing new contributors into open source software development. We are welcoming fresh members into the Unikraft community to work on cool open source projects during paid summer internships. The GSoCā€™22 application is open between April 4th and April 21st, 2022. Please see [our guideline for application](https://github.com/unikraft/gsoc22) and join [our Discord Server](https://bit.ly/UnikraftDiscord) for detailed discussions (we have a channel dedicated to GSoC).