---
tags: LINUX KERNEL, LKI
---

# [Linux 核心設計](https://beta.hackfoldr.org/linux/): 中斷處理和現代架構考量

Copyright (**慣C**) 2019 [宅色夫](http://wiki.csie.ncku.edu.tw/User/jserv)

==[直播錄影 (上)](https://youtu.be/EjPWklezNrU)==
==[直播錄影 (下)](https://youtu.be/8ejnYR7A2Nc)==

## 簡介

中斷處理相信是人們不陌生的主題，甚至在中學生的計算機概論教材都出現這字眼，但在 Linux (或任何有規模的作業系統核心) 裡頭，中斷處理背後涉及的硬體特性、多種周邊 I/O、中斷控制器 (如是否支援 nested)、相關的排程和任務調度、延遲和即時處理等等，仍舊讓工程人員頭痛，特別將多核處理器、虛擬化技術，和為了實踐資訊安全而進行的隔離執行納入考量之後。


在本講座中，我們會從硬體特性著手，簡述 Intel 和 Arm 架構的中斷處理機制，再回頭看 Linux 核心的 softirq, tasklet, workqueue 用來實現中斷處理，並討論 request_threaded_irq / request_irq (這和引入 PREEMPT_RT 有高度關聯) 的用法，之後則針對多核處理器的架構去思考 Linux 核心的中斷處理做了哪些變革，搭配虛擬化和 Arm TrustZone 及 Intel SGX 的支援納入後，又有什麼需要深度分析的技術議題。

## 中斷處理概念

- [ ] 取自 [COMS W4118: Operating Systems](http://www.cs.columbia.edu/~krj/os/index.html) 的 [Interrupts in Linux](http://www.cs.columbia.edu/~krj/os/lectures/L07-LinuxEvents.pdf)
* PIC translates IRQ to vector
* Raises interrupt to CPU
* Vector available in register
* Waits for ack from CPU
* Interrupts can have varying priorities
* PIC also needs to prioritize multiple requests
* Possible to “mask” (disable) interrupts at PIC or CPU


/* <include/linux/interrupt.h> */
struct softirq_action {
    void (*action)(struct softirq_action *);
};

A kernel thread to process deferred softIRQs (per CPU)

static void run_ksoftirqd(unsigned int cpu) {
    local_irq_disable();
    if (local_softirq_pending()) {
        __do_softirq();
        rcu_note_context_switch(cpu);
        local_irq_enable();
        cond_resched();
        return;
    }
    local_irq_enable();
}



* Creating a tasklet
- DECLARE_TASKLET(name, func, data)
* Scheduling tasklets
- Tasklet function runs every time a softIRQ is executed
- `tasklet_schedule( struct tasklet_struct *t)`
- `tasklet_hi_schedule( struct tasklet_struct *t)`
* Enabling/disabling tasklets
- `tasklet_disable()` / `tasklet_kill()`
- `tasklet_enable()`

* Creating work queue
- struct workqueue_struct *create_workqueue(char *name);
* Creating work
- DECLARE_WORK(name, void (*func)(void *), void *data);
- INIT_WORK(struct work_struct *work, void (*func)(void *), void *data);
* Scheduling work
- int schedule_work(struct work_struct *work);
- int flush_scheduled_work(void);

- [ ] 取自 [CSE 438/598 Embedded Systems Programming](http://rts.lab.asu.edu/web_438_Fall_2014/CSE438_Fall2014_Main_page.htm) 的 [Linux Interrupt Processing and Kernel Thread](http://rts.lab.asu.edu/web_438/CSE438_598_slides_yhlee/438_7_Linux_ISR.pdf)
* 為何有 ISR bottom half (`bh`) 呢？降低 interrupt latency，將工作切割為以下:
- "top half", which receives the hardware interrupt and
- "bottom half", which does the lengthy processing.
* Top halves 的屬性和要求:
- need to run as quickly as possible
- run with some (or all) interrupt levels disabled
- are often time-critical and they deal with HW
- do not run in process context and cannot block

## 進行實驗

* 搭配 [I/O access and Interrupts](https://linux-kernel-labs.github.io/refs/heads/master/labs/interrupts.html) 實驗
* [Kernel 4.10 Examples of linux drivers](https://github.com/rrmhearts/linux-driver-examples)
* 背景知識: [Linux Kernel Module Programming Guide](https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf)
* 針對 Linux `v4.15` 需要做點調整: 將 `char/char_example.c` 的 `#include <asm/uaccess.h>` 換為 `#include <linux/uaccess.h>` (為了 `copy_to_user()`)
* 測試 `char/`, `misc/`, `sysfs/bin_attr/`
* Chapter 7 (Talking To Device Files, P.41), Chapter 12 (Interrupt Handlers, P.68)
* [Char driver with interrupt handling](http://www.diegm.uniud.it/loghi/CE/slides/irq_driver.pdf)

## 加入 multi-core 和 threaded interrupt 的考量

- [ ] [Making Linux do Hard Real-time](https://www.slideshare.net/jserv/making-linux-do-hard-realtime)
* Threaded Interrupts Thread interrupts allows to use sleeping spinlocks
* in PREEMPT_RT, all interrupt handlers are switched to threaded interrupt drivers in mainline get gradually converted to the new threaded interrupt API that has been merged in Linux 2.6.30.

- [ ] [IRQs: the Hard, the Soft, the Threaded and the Preemptible](http://she-devel.com/Chaiken_ELCE2016.pdf)
> Clock signals. `(a)` For level-triggered circuits. (b) For positive-edge triggering. `(c)` For negative-edge triggering.

* [Interrupts and Interrupt Handling](https://0xax.gitbooks.io/linux-insides/Interrupts/)
* O'Reilly [Interrupt Handling](https://www.oreilly.com/library/view/understanding-the-linux/0596005652/ch04s06.html)

## 虛擬化對中斷處理的影響

* [Linux 核心的中斷子系統](http://www.wowotech.net/irq_subsystem/interrupt_subsystem_architecture.html)
* [I/O in Linux Hypervisors and Virtual Machines](https://github.com/fanjinfei/docs/blob/master/vmio_may9_2016.pdf)
* [ARM Interrupt Virtualization](http://events17.linuxfoundation.org/sites/events/files/slides/ARM_Interrupt_Virtualization_Przywara.pdf)
* GIC-400 是 GIC v2 架構，它是設計來配合 ARM CPU 的中斷控制器，能相容 TrustZone 和虛擬化架構、並最多支持到 8 個 ARM cores;
* GIC-400 可以分成 4 大部分: Distributor (GICD_*), CPU interface (GICC_*), virtual control interface (GICH_*), virtual CPU interface (GICV_*)，後面 2 個是為了虛擬化而存在;
* [簡介 GIC-400](http://justinchunotes.blogspot.com/2018/12/gic-400.html)