--- tags: linux-summer-2021 --- # 2021q3 Homework3 (quiz3) contributed by < `RinHizakura` > > [第 3 週測驗題](https://hackmd.io/@sysprog/linux2021-summer-quiz3) ## Simrupt ### 實作 #### `simrupt_init` character device /API 就不贅述，有需要可以參考 [Homework1](https://hackmd.io/@RinHizakura/SJruJZF0d) 或者直接看書: [lkmpg](https://github.com/sysprog21/lkmpg)。 * [`kfifo_alloc`](https://www.kernel.org/doc/htmldocs/kernel-api/API-kfifo-alloc.html) 建立一個 fifo buffer * [`struct circ_buf`](https://www.kernel.org/doc/html/latest/core-api/circular-buffers.html) 透過 [`vmalloc`](https://www.kernel.org/doc/htmldocs/kernel-api/API-vmalloc.html) 介面去配置空間，後者會建立一個 virtual memory 連續，在 physical 上則不保證連續的空間。 * [`alloc_workqueue`](https://www.kernel.org/doc/html/v4.10/core-api/workqueue.html) 建立 module 自己的 workqueue(如果不建立的話可以使用 kernel 中已存在的 `system_wq`，使用 `schedule_work` 去添加任務) * `WQ_UNBOUND`: 任務不限定於特定 cpu 上執行 * [`WQ_MAX_ACTIVE`](https://elixir.bootlin.com/linux/v5.14-rc5/source/include/linux/workqueue.h#L348): worker thread 的數目 * [`timer_setup`](https://elixir.bootlin.com/linux/v5.14-rc5/source/include/linux/timer.h#L131) * `timer_handler` 會在每次 timer expired 時被呼叫 #### `simrupt_open` ```cpp static int simrupt_open(struct inode *inode, struct file *filp) { pr_debug("simrupt: %s\n", __func__); mod_timer(&timer, jiffies + msecs_to_jiffies(delay)); return 0; } ``` 對應對 /dev/simrupt 的開檔操作。 [`mod_timer`](https://elixir.bootlin.com/linux/v5.14-rc5/source/kernel/time/timer.c#L1086) 註冊 timer 到 kernel，順帶修改 timer expired 的時間，其中 `jiffies` 是 kernel 中每次 timer interrupt 產生為一數的時間單位，因此我們可以設置幾個 interrupt 後要觸發 timer #### `simrupt_release` ```cpp static int simrupt_release(struct inode *inode, struct file *filp) { pr_debug("simrupt: %s\n", __func__); del_timer_sync(&timer); flush_workqueue(simrupt_workqueue); fast_buf_clear(); return 0; } ``` 對應對 /dev/simrupt 的關檔操作。 * [`del_timer_sync`](https://elixir.bootlin.com/linux/v5.14-rc5/source/kernel/time/timer.c#L1316) 取消 timer 的註冊，但如果呼叫時 timer 正在運行，實際的取消會延遲到 handler 結束 * [`flush_workqueue`](https://elixir.bootlin.com/linux/v5.14-rc5/source/kernel/workqueue.c#L2768) 等待直到此呼叫此函數前的 queue 中所有任務被做完 * `fast_buf_clear` 重設 circ_buf (實際上只是把 head 和 tail 歸零) #### `simrupt_read` ```cpp static ssize_t simrupt_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { unsigned int read; int ret; pr_debug("simrupt: %s(%p, %zd, %lld)\n", __func__, buf, count, *ppos); if (unlikely(!access_ok(buf, count))) return -EFAULT; if (mutex_lock_interruptible(&read_lock)) return -ERESTARTSYS; ``` 對應對 /dev/simrupt 的讀檔操作。 * `access_ok` 檢驗 userspace 的指標之合法性(指標在該 user process 的合法範圍中) * `unlikely` 提示編譯器大多情況都是合法的 * [`mutex_lock_interruptible`](https://www.kernel.org/doc/htmldocs/kernel-locking/API-mutex-lock-interruptible.html) acquire lock 且允許被 signal 打斷 * 換句話說，同時只能有一個 thread 對檔案進行讀取 ```cpp do { ret = kfifo_to_user(&rx_fifo, buf, count, &read); if (unlikely(ret < 0)) break; if (read) break; if (file->f_flags & O_NONBLOCK) { ret = -EAGAIN; break; } ret = wait_event_interruptible(rx_wait, kfifo_len(&rx_fifo)); } while (ret == 0); pr_debug("simrupt: %s: out %u/%u bytes\n", __func__, read, kfifo_len(&rx_fifo)); mutex_unlock(&read_lock); return ret ? ret : read; } ``` `kfifo_to_user` 將 `rx_fifo` 的內容複製到 userspace read 操作的 buffer 中 #### `time_handler` ```cpp static void timer_handler(struct timer_list *__timer) { ktime_t tv_start, tv_end; s64 nsecs; pr_info("simrupt: [CPU#%d] enter %s\n", smp_processor_id(), __func__); /* We are using a kernel timer to simulate a hard-irq, so we must expect * to be in softirq context here. */ WARN_ON_ONCE(!in_softirq()); /* Disable interrupts for this CPU to simulate real interrupt context */ local_irq_disable(); tv_start = ktime_get(); process_data(); tv_end = ktime_get(); nsecs = (s64) ktime_to_ns(ktime_sub(tv_end, tv_start)); pr_info("simrupt: [CPU#%d] %s in_irq: %llu usec\n", smp_processor_id(), __func__, (unsigned long long) nsecs >> 10); mod_timer(&timer, jiffies + msecs_to_jiffies(delay)); local_irq_enable(); } ``` * `in_softirq()` 回傳是否在 software irq context，`WARN_ON_ONCE` 確保 `timer_handler` 若非在 software irq context 下運行會回報警告 * `local_irq_disable()` 關閉目前 cpu 上的 interrupt，然後模擬 interrupt 的行為 `process_data`: 1. 將一個 int 加入到先前建立的 `struct circ_buf` 結構之 `fast_buf` 中 2. 再呼叫 `tasklet_schedule` 將 `simrupt_tasklet_func` 任務加入排程中 :::danger 如果不先 disable local irq 可能會發生甚麼問題? (可能產生 nested interrupt?) ::: :::info `tasklet_schedule` and `queue_work` : * tasklet 中不允許 sleep，而 workqueue 允許 * tasklet 呼叫時為 irq context，workqueue 為 kernel context ::: #### `simrupt_tasklet_func` ```cpp static void simrupt_tasklet_func(unsigned long __data) { ktime_t tv_start, tv_end; s64 nsecs; WARN_ON_ONCE(!in_interrupt()); WARN_ON_ONCE(!in_softirq()); tv_start = ktime_get(); queue_work(simrupt_workqueue, &work); tv_end = ktime_get(); nsecs = (s64) ktime_to_ns(ktime_sub(tv_end, tv_start)); pr_info("simrupt: [CPU#%d] %s in_softirq: %llu usec\n", smp_processor_id(), __func__, (unsigned long long) nsecs >> 10); } ``` `simrupt_tasklet_func` 是由 tasklet defer 的任務，應該在 interrupt context / softirq context 下運行。而其作用是呼叫 `queue_work`，讓 `simrupt_work_func` 被加入 workqueue 後等待被排程到並執行。 #### `simrupt_work_func` ```cpp static void simrupt_work_func(struct work_struct *w) { int val, cpu; /* This code runs from a kernel thread, so softirqs and hard-irqs must * be enabled. */ WARN_ON_ONCE(in_softirq()); WARN_ON_ONCE(in_interrupt()); /* Pretend to simulate access to per-CPU data, disabling preemption * during the pr_info(). */ cpu = get_cpu(); pr_info("simrupt: [CPU#%d] %s\n", cpu, __func__); put_cpu(); while (1) { /* Consume data from the circular buffer */ mutex_lock(&consumer_lock); val = fast_buf_get(); mutex_unlock(&consumer_lock); if (val < 0) break; /* Store data to the kfifo buffer */ mutex_lock(&producer_lock); produce_data(val); mutex_unlock(&producer_lock); } wake_up_interruptible(&rx_wait); } ``` 藉由 `fast_buf_get` 從 `circ_buf` 中取出先前在 `timer_handler` 時產生的 int，並通過 `produce_data` 將資料加入到 `rx_fifo` 之中 ## Virtual cfg80211 Driver [Linux Wi-Fi Driver Tutorial: How to Write a Simple Linux Wireless Driver Prototype](https://www.apriorit.com/dev-blog/645-lin-linux-wi-fi-driver-tutorial-how-to-write-simple-linux-wireless-driver-prototype) ### cfg80211 subsystem [cfg80211 subsystem](https://www.kernel.org/doc/html/v4.9/80211/cfg80211.html) > In order for a driver to use cfg80211, it must register the hardware device with cfg80211. This happens through a number of hardware capability structs described below. > > The fundamental structure for each device is the ‘wiphy’, of which each instance describes a physical wireless device connected to the system. Each such wiphy can have zero, one, or many virtual interfaces associated with it, which need to be identified as such by pointing the network interface’s ieee80211_ptr pointer to a struct wireless_dev which further describes the wireless part of the interface, normally this struct is embedded in the network interface’s private data area. Drivers can optionally allow creating or destroying virtual interfaces on the fly, but without at least one or the ability to create some the wireless device isn’t useful. ### 實作 #### `owl_context` ```cpp struct owl_context { struct wiphy *wiphy; struct net_device *ndev; struct mutex mtx; struct work_struct ws_connect, ws_disconnect; char connecting_ssid[sizeof(SSID_DUMMY)]; u16 disconnect_reason_code; struct work_struct ws_scan; struct cfg80211_scan_request *scan_request; }; ``` `wiphy` 是用來描述 cfg80211 硬體裝置的資料結構，透過 [`wiphy_new`](https://www.kernel.org/doc/html/v4.9/80211/cfg80211.html#c.wiphy_new) 或者 `wiphy_new_nm` 去建立，如範例程式碼: ```cpp wiphy_new_nm( &owl_cfg_ops, sizeof(struct owl_wiphy_priv_context), WIPHY_NAME); ``` 其中第二個參數宣告配置 `wiphy` 結構下動態大小的空間(private area, `priv`)，把本 drive 的主體結構 `owl_context` 加入其中，可以通過 [`wiphy_priv`](https://www.kernel.org/doc/html/v4.9/80211/cfg80211.html#c.wiphy_priv) 去存取該空間下的結構，而第一個參數則是對該裝置相關的操作 對於 `wiphy` 結構，還有一些相應的成員設定: ```cpp ret->wiphy->interface_modes = BIT(NL80211_IFTYPE_STATION); ret->wiphy->bands[NL80211_BAND_2GHZ] = &nf_band_2ghz; ret->wiphy->max_scan_ssids = 69; if (wiphy_register(ret->wiphy) < 0) goto l_error_wiphy_register; ``` * 對於 `interface_modes` 的設定可以參考 [`nl80211_iftype`](https://elixir.bootlin.com/linux/v5.14-rc5/source/include/uapi/linux/nl80211.h#L3123)，與 wifi 的連接模式有關，可參考 [wifi 有兩種 mode: STA 與 AP](https://eeepage.info/wifimode-sta-ap/)，這裡僅設置 [STA mode](https://en.wikipedia.org/wiki/Station_(networking)) * `bands`: `NL80211_BAND_5GHZ` 設定 2GHz band，並只有設定一個 channel * `max_scan_ssids`: 支援 scan 的裝置需要設置最大可 scan 的 [SSID](https://en.wikipedia.org/wiki/Service_set_(802.11_network)) 數量 * 最後 [`wiphy_register`](https://www.kernel.org/doc/html/v4.9/80211/cfg80211.html#c.wiphy_register)，從 `iw list` 可以看到在系統中註冊了新的 wireless device `net_device` 則是描述 network device 的結構，透過 `alloc_netdev` 建立。在其成員的 private area 中要包含 `wireless_dev` 可以用來描述一個 wireless network device，另外也將 driver 的主體結構 `owl_context` 也加入到 private area 中，可以透過 `netdev_priv` 去存取 private area。 * `ether_setup` 是 default 的初始化函數 ```cpp ret->ndev = alloc_netdev(sizeof(*ndev_data), NDEV_NAME, NET_NAME_ENUM, ether_setup); ``` 設置相關 net_device 相關成並且透過 `register_netdev` 註冊，`ip a` 可以看到成功註冊的 network device: ```cpp ndev_data->wdev.wiphy = ret->wiphy; ndev_data->wdev.netdev = ret->ndev; ndev_data->wdev.iftype = NL80211_IFTYPE_STATION; ret->ndev->ieee80211_ptr = &ndev_data->wdev; ret->ndev->netdev_ops = &owl_ndev_ops; if (register_netdev(ret->ndev)) goto l_error_ndev_register; ``` #### `vwifi_init` ```cpp static int __init vwifi_init(void) { g_ctx = owl_create_context(); if (!g_ctx) return 1; mutex_init(&g_ctx->mtx); INIT_WORK(&g_ctx->ws_connect, owl_connect_routine); g_ctx->connecting_ssid[0] = 0; INIT_WORK(&g_ctx->ws_disconnect, owl_disconnect_routine); g_ctx->disconnect_reason_code = 0; INIT_WORK(&g_ctx->ws_scan, owl_scan_routine); g_ctx->scan_request = NULL; return 0; } ``` `owl_create_context` 得到已配置空間並已初始化`struct wiphy` 和`net_device` 的 `owl_context`，接著初始化其 mutex 等其他成員。 * 使用到了 `work_struct` 建立 kernel thread，將任務排進 Completely Fair Scheduler (CFS) 中 #### `owl_scan` ```cpp static int owl_scan(struct wiphy *wiphy, struct cfg80211_scan_request *request) { struct owl_context *owl = wiphy_get_owl_context(wiphy)->owl; if (mutex_lock_interruptible(&owl->mtx)) return -ERESTARTSYS; if (owl->scan_request) { MLLL; return -EBUSY; } owl->scan_request = request; mutex_unlock(&owl->mtx); if (!schedule_work(&owl->ws_scan)) return -EBUSY; return 0; } ``` 當對裝置要求進行 scan 操作時，`owl_scan` 被執行，其透過 `owl->scan_request` 的狀態決定是否要 `schedule_work` 啟動核心的 `owl_scan_routine`，注意到使用 mutex lock 確保 `scan_request` 的互斥。 ```cpp static void owl_scan_routine(struct work_struct *w) { struct owl_context *owl = container_of(w, struct owl_context, ws_scan); struct cfg80211_scan_info info = { .aborted = false, }; msleep(100); /* inform with dummy BSS */ inform_dummy_bss(owl); if (mutex_lock_interruptible(&owl->mtx)) return; /* finish scan */ cfg80211_scan_done(owl->scan_request, &info); owl->scan_request = NULL; mutex_unlock(&owl->mtx); } ``` `owl_scan_routine` 先利用 `container_of` 從 `work_struct` 中取得 `owl_context`，關鍵是利用 `inform_dummy_bss` 去模擬 scan。scan 結束後，`cfg80211_scan_done` 去通知 scan 流程已經結束，並且將 `owl->scan_request` 重置。