HackMDLinux 核心設計: 中斷處理和現代架構考量
Copyright (慣C) 2019 宅色夫
簡介
中斷處理相信是人們不陌生的主題，甚至在中學生的計算機概論教材都出現這字眼，但在 Linux (或任何有規模的作業系統核心) 裡頭，中斷處理背後涉及的硬體特性、多種周邊 I/O、中斷控制器 (如是否支援 nested)、相關的排程和任務調度、延遲和即時處理等等，仍舊讓工程人員頭痛，特別將多核處理器、虛擬化技術，和為了實踐資訊安全而進行的隔離執行納入考量之後。
在本講座中，我們會從硬體特性著手，簡述 Intel 和 Arm 架構的中斷處理機制，再回頭看 Linux 核心的 softirq, tasklet, workqueue 用來實現中斷處理，並討論 request_threaded_irq / request_irq (這和引入 PREEMPT_RT 有高度關聯) 的用法，之後則針對多核處理器的架構去思考 Linux 核心的中斷處理做了哪些變革，搭配虛擬化和 Arm TrustZone 及 Intel SGX 的支援納入後，又有什麼需要深度分析的技術議題。
中斷處理概念
- PIC translates IRQ to vector
- Raises interrupt to CPU
- Vector available in register
- Waits for ack from CPU
- Interrupts can have varying priorities
- PIC also needs to prioritize multiple requests
- Possible to “mask” (disable) interrupts at PIC or CPU
/* <include/linux/interrupt.h> */
struct softirq_action {
void (*action)(struct softirq_action *);
};
A kernel thread to process deferred softIRQs (per CPU)
static void run_ksoftirqd(unsigned int cpu) {
local_irq_disable();
if (local_softirq_pending()) {
__do_softirq();
rcu_note_context_switch(cpu);
local_irq_enable();
cond_resched();
return;
}
local_irq_enable();
}
- Creating a tasklet
- DECLARE_TASKLET(name, func, data)
- Scheduling tasklets
- Tasklet function runs every time a softIRQ is executed
tasklet_schedule( struct tasklet_struct *t)tasklet_hi_schedule( struct tasklet_struct *t)
- Enabling/disabling tasklets
tasklet_disable()/tasklet_kill()tasklet_enable()
- Creating work queue
- struct workqueue_struct *create_workqueue(char *name);
- Creating work
- DECLARE_WORK(name, void (*func)(void *), void *data);
- INIT_WORK(struct work_struct *work, void (*func)(void *), void *data);
- Scheduling work
- int schedule_work(struct work_struct *work);
- int flush_scheduled_work(void);
- 為何有 ISR bottom half (
bh) 呢？降低 interrupt latency，將工作切割為以下:- "top half", which receives the hardware interrupt and
- "bottom half", which does the lengthy processing.
- Top halves 的屬性和要求:
- need to run as quickly as possible
- run with some (or all) interrupt levels disabled
- are often time-critical and they deal with HW
- do not run in process context and cannot block
進行實驗
- 搭配 I/O access and Interrupts 實驗
- Kernel 4.10 Examples of linux drivers
- 背景知識: Linux Kernel Module Programming Guide
- 針對 Linux
v4.15需要做點調整: 將char/char_example.c的#include <asm/uaccess.h>換為#include <linux/uaccess.h>(為了copy_to_user()) - 測試
char/,misc/,sysfs/bin_attr/ - Chapter 7 (Talking To Device Files, P.41), Chapter 12 (Interrupt Handlers, P.68)
- Char driver with interrupt handling
加入 multi-core 和 threaded interrupt 的考量
- Threaded Interrupts Thread interrupts allows to use sleeping spinlocks
- in PREEMPT_RT, all interrupt handlers are switched to threaded interrupt drivers in mainline get gradually converted to the new threaded interrupt API that has been merged in Linux 2.6.30.
Clock signals.
(a)For level-triggered circuits. (b) For positive-edge triggering.(c)For negative-edge triggering.
虛擬化對中斷處理的影響
- Linux 核心的中斷子系統
- I/O in Linux Hypervisors and Virtual Machines
- ARM Interrupt Virtualization
- GIC-400 是 GIC v2 架構，它是設計來配合 ARM CPU 的中斷控制器，能相容 TrustZone 和虛擬化架構、並最多支持到 8 個 ARM cores;
- GIC-400 可以分成 4 大部分: Distributor (GICD_), CPU interface (GICC_), virtual control interface (GICH_), virtual CPU interface (GICV_)，後面 2 個是為了虛擬化而存在;
- 簡介 GIC-400
