# Project 1：實作回傳 Physical Address 的 System Call 第 17 組：<br> 朱巽葦（110403507）、陳禾旻（109502562）、洪柏軒（110302031）、林睿瀚（112526011） :::info [Github Repo](https://github.com/hermeschen1116/linux-systemcall-implement) ::: ## Outline [ToC] ## 環境 - Debian GNU/Linux 12 (bookworm) x86_64 - Linux Kernel 6.1.0 - QEMU/KVM or VMware ## 實作 ### 編譯 & 安裝 Kernel 先安裝編譯時所需要用到的packages: ```clike= sudo apt update sudo apt upgrade -y sudo apt install git fakeroot build-essential ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison dwarves ccache liblz4-tool -y ``` 接著下載Kernel source ```clike= if [ ! -d "linux" ] then sudo apt install wget -y wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.tar.xz tar xvf linux-6.1.tar.xz mv linux-6.1 linux rm linux-6.1.tar.xz fi ``` 創建自定義的system call資料夾 ```clike= mkdir linux/custom_systemcall # link config ln -s ../linux/include/linux/syscalls.h src/syscalls.h ln -s ../linux/arch/x86/entry/syscalls/syscall_64.tbl src/syscall_64.tbl ln -s ../linux/Makefile src/Makefile ``` 編譯且安裝新的kernel (menuconfig可改成localmodconfig會編譯比較快) ```clike= process="$(nproc)" architecture="x86_64" source="linux" build_dir="./build" # copy modified files to source cp src/systemcall/* linux/custom_systemcall/ # build KBUILD_BUILD_TIMESTAMP="" sudo make menuconfig CC="ccache gcc" -j$process -C$source O=$build_dir KBUILD_BUILD_TIMESTAMP="" sudo make CC="ccache gcc" Arch=$architecture -j$process -C$source O=$build_dir # install sudo make modules_install -j$process -C$source O=$build_dir sudo make install -j$process -C$source O=$build_dir sudo update-grub ``` ### 新增 System Call ### ```my_get_physical_addresses``` - 程式碼 ```clike= #include <linux/syscalls.h> SYSCALL_DEFINE1(my_get_physical_addresses, void *, user_virtual_address) { unsigned long virtual_address = (unsigned long)user_virtual_address; pgd_t *pgd; p4d_t *p4d; pud_t *pud; pmd_t *pmd; pte_t *pte; unsigned long page_frame_number = 0; unsigned long page_offset = 0; unsigned long physical_address = 0; // Walk through the tables pgd = pgd_offset(current->mm, virtual_address); if (pgd_none(*pgd)) { printk(KERN_WARNING "my_get_physical_addresses: Invalid PGD\n"); return 0; } p4d = p4d_offset(pgd, virtual_address); if (p4d_none(*p4d)) { printk(KERN_WARNING "my_get_physical_addresses: Invalid P$D\n"); return 0; } pud = pud_offset(p4d, virtual_address); if (pud_none(*pud)) { printk(KERN_WARNING "my_get_physical_addresses: Invalid PUD\n"); return 0; } pmd = pmd_offset(pud, virtual_address); if (pmd_none(*pmd)) { printk(KERN_WARNING "my_get_physical_addresses: Invalid PMD\n"); return 0; } // Find the page table entry pte = pte_offset_kernel(pmd, virtual_address); if (pte_none(*pte)) { printk(KERN_WARNING "my_get_physical_addresses: No PTE found\n"); return 0; } // calculate physical address page_frame_number = pte_val(*pte) & PAGE_MASK; page_offset = virtual_address & ~PAGE_MASK; physical_address = page_frame_number | page_offset; printk(KERN_DEBUG "my_get_physical_addresses:\nvirtual address: %lx\npage frame number: %lx, page offset: %lx\nphysical address: %lx\n", virtual_address, (page_frame_number >> PAGE_SHIFT), page_offset, physical_address); return physical_address; } ``` * virtual address to physical address ![L2P](https://hackmd.io/_uploads/B1KIXl3ZJl.png) 64bits 的 linux kernel 架構使用 4層 page table來做 virtual address to physical address 的轉換: * Page Global Directory (PGD) * Page Upper Directory (PUD) * Page Middle Directory (PMD) * Page Table (PTE) > pgd_offset : 根據目前的 virtual address 和目前 process 的 mm_struct，可得到 pgd entry > (entry 內容為 pud table 的 base address) > > pud_offset : 根據透過 pgd_offset 得到的 pgd entry 和 virtual address，可得到 pud entry > (entry 內容為 pmd table 的 base address) > > pmd_offset : 根據 pud entry 的內容和 virtual address，可得到 pte table 的 base address > > pte_offset : 根據 pmd entry 的內容與 virtual address，可得到 pte 的 base address > > 將從 pte 得到的 base address 與 Mask(0xf…fff000)做 and 運算，可得到 page 的 base physical address > > virtual address 與 ~Mask(0x0…000fff) 做 and運算得到 offset，再與 page 的 base physical address 做 or運算 即可得到轉換過後的完整的physical address。 > 而雖然在 x86_64 架構中，實際上使用的是 4 級頁表（PGD、PUD、PMD、PTE），但為了兼容可能的 5 級頁表，Linux 核心定義了 P4D。如果某級別在當前架構中未使用，則該級別會被 folded * current pointer > 指向當前進程的 task_struct 結構， > task_struct 是 Linux 核心中用來表示每個執行中的進程的核心結構，這個結構包含了進程的完整上下文 > ![task_struct](https://hackmd.io/_uploads/r1Epvj2-1g.png =70%x) > current->mm：指向當前進程的記憶體描述符 mm_struct - 修改 **`syscall_64.tbl`** ![image](https://hackmd.io/_uploads/B1YuqocW1l.png) - 修改 **`syscalls.h`** ![image](https://hackmd.io/_uploads/Hkgqqi9W1x.png) 在#endif加上 ### Test system call ##### Q1 - 程式碼 ```clike= #include <stdio.h> #include <stdlib.h> #include <unistd.h> #include <sys/mman.h> #include <sys/wait.h> #define SYS_my_get_physical_address 452 unsigned long my_get_physical_addresses(void *virtual_address); int main() { int *shared_memory = mmap(NULL, 4096, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); if (shared_memory == MAP_FAILED) { perror("mmap failed"); exit(1); } *shared_memory = 48763; printf("Initial value in parent process: %d\n", *shared_memory); printf("Parent's physical address before fork: [0x%lx]\n", my_get_physical_addresses(shared_memory)); unsigned long parent_physical_address_before = my_get_physical_addresses(shared_memory); pid_t pid = fork(); if (pid < 0) { perror("fork failed"); exit(1); } else if (pid == 0) { printf("Child's physical address before write: [0x%lx]\n", my_get_physical_addresses(shared_memory)); *shared_memory = 114514; printf("New value in child process: %d\n", *shared_memory); printf("Child's physical address after write: [0x%lx]\n", my_get_physical_addresses(shared_memory)); exit(0); } else { wait(NULL); printf("Parent's physical address after child's write: [0x%lx]\n", my_get_physical_addresses(shared_memory)); unsigned long parent_physical_address_after = my_get_physical_addresses(shared_memory); if (parent_physical_address_after == parent_physical_address_before) { printf("Parent's physical address remains the same.\n"); } else { printf("Parent's physical address has changed (unexpected in COW).\n"); } } munmap(shared_memory, 4096); return 0; } unsigned long my_get_physical_addresses(void *virtual_address) { return syscall(SYS_my_get_physical_address, virtual_address); } ``` - 結果 ![image](https://hackmd.io/_uploads/HkZmGs9b1g.png) | 執行步驟 | 輸出內容 | |---------------------------------------|-----------------------------------------------| | Initial value in parent process | 48763 | | Parent's physical address before fork | 0x800000018268a000 | | Child's physical address before write | 0x800000018268a000 | | New value in child process | 114514 | | Child's physical address after write | 0x80000001a4bb1000 | | Parent's physical address after child's write | 0x800000018268a000 | | Final remark | Parent's physical address remains the same. | child process 在fork後，因還沒對其進行任何write的動作，故此時memory page frame is shared between the parent and child process => physical address 一樣。 而child process被修改了，即會觸發Copy-On-Write機制，系統為子進程分配了新的page frame，以確保parent proccess不會受到影響。 ##### Q2 - 程式碼 ```clike= #include <stdio.h> #include <unistd.h> #define SYS_my_get_physical_address 452 unsigned long my_get_physical_addresses(void *virtual_address); int a[2000000]; int main() { int loc_a; printf("global element a[0]:\n"); printf("Offest of logical address:[%p] Physical address:[0x%lx]\n", &a[0], my_get_physical_addresses(&a[0])); printf("========================================================================\n"); printf("global element a[1999999]:\n"); printf("Offest of logical address:[%p] Physical address:[0x%lx]\n", &a[1999999], my_get_physical_addresses(&a[1999999])); printf("========================================================================\n"); } unsigned long my_get_physical_addresses(void *virtual_address) { return syscall(SYS_my_get_physical_address, virtual_address); } ``` - 結果 ![image](https://hackmd.io/_uploads/HJAVfs5-1e.png) | 執行步驟 | 輸出內容 | |-----------------------------|----------------------------------------------| | global element a[0] | | | Offset of logical address | 0x55f0c9593060 | | Physical address | 0x8000000176bb3060 | | ============================|=============================| | global element a[1999999] | | | Offset of logical address | 0x55f0c9d3425c | | Physical address | 0x0 | 因陣列宣告在 compile 的時候是決定其 virtual address，系統不會立即佔用宣告那麼大的實體記憶體空間，而僅是維護上面的 virtual address 空間而已，只有在真正需要的時候才分配實體記憶體，該 page 未 load 至 memory中，故應為 nil (此處 **`a[1999999]`** physical address 顯示為0而不是 nil 是因其回傳值所定義的型態是 unsigned long ) ## 筆記 ### copy_to_user * to：資料的目的位址，此參數為一個指向 user-space 記憶體的指標。 * from：資料的來源位址，此參數為一個指向 kernel-space 記憶體的指標。 > copy data to user-space from kernel-space ### copy_from_user * to：資料的目的位址，此參數為一個指向 kernel-space 記憶體的指標。 * from：資料的來源位址，此參數為一個指向 user-space 記憶體的指標。 > copy data from user-space to kernel-space ### Copy-on-Write（CoW） * shared resources： 當一個進程被複製時（例如父進程創建子進程），新的進程需要擁有與原進程相同的記憶體內容。為了節省資源，OS 讓這兩個進程共享相同的物理記憶體頁面 * read-only protection： 共享的頁面被標記為只讀，這樣如果其中一個進程嘗試修改頁面內容，就會引發 Page Fault * Delayed replication： 當 Page Fault 發生，OS 才會為需要修改的進程分配新的物理頁面，並將原頁面的內容複製過去，這就是 Copy-on-Write ## 參考資料 - [Add a System call to the Linux Kernel (6.0.9) in Ubuntu 22.04 using Oracle VM VirtualBox at Window](https://medium.com/@rajiv.cse/add-a-system-call-to-the-linux-kernel-6-0-9-in-ubuntu-22-04-acd7f7afc933) - [Linux 核心 Copy On Write 實作機制](https://hackmd.io/@linD026/Linux-kernel-COW-Copy-on-Write#Physical-Memory-Model)