Userspace RCU

tags: `rcu`

Userspace RCU - liburcu
User-space RCU - LWN 2013
urcu/userspace-rcu - github
- tests/benchmark/
- doc/rcu-api.md
What Linear Multiprocessor Scalability Means for Your Application
Linux Plumbers Conference 2009, slide
Userspace RCU library: new APIs and data structures - Mathieu Desnoyers, EfficiOS Inc.
Linux Plumbers Conference 2013, video
Mentorship Session: Unraveling RCU-Usage Mysteries (Fundamentals)
liburcu，一个用户态的RCU实现
Userspace RCU原理

URCU overview

When is URCU useful?

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

TODO
筆記補完整

RCU Flavors

QSBR
RCU_MEMBARRIER
RCU_MB
RCU_SIGNAL
Bullet-proof

QSBR

#include <urcu-qsbr.h>
Link with -lurcu-qsbr

QSBR flavor of RCU
每個 reader thread 都要週期性的呼叫 rcu_quiescent_state() 確保程式的 progress。

可以用 rcu_thread_online() and rcu_thread_offline() 使該 reader thread 進入較長時間的 inactive，暫時脫離 grace period 的偵測

由於需要呼叫 rcu_quiescent_state()，此 flavor 對於程式的 intrusiveness 較大，但可以得到最快的 read-side speed (zero cost)

RCU_MEMBARRIER

#include <urcu.h>
Link the application with -lurcu

This is the preferred version of the library, in terms of grace-period detection speed, read-side speed and flexibility. Dynamically detects kernel support for sys_membarrier(). Falls back on urcu-mb scheme if support is not present, which has slower read-side.

最推薦的版本，在偵測 grace period, read-side 的速度跟 flexibility 都很好。
會動態偵測是否支援 sys_membarrier()，不支援的話會自動退到 RCU_MB 的版本，
，降低 read-side 速度

RCU_MB

#include <urcu.h>
Compile any _LGPL_SOURCE code using this library with -DRCU_MB
Link with -lurcu-mb

在 reader 跟 writer 都會使用 memory barriers，所以在偵測 grace period 的速度較快，但犧牲 reader-side speed

RCU_SIGNAL

#include <urcu.h>
Compile any _LGPL_SOURCE code using this library with -DRCU_SIGNAL
Link the application with -lurcu-signal

Version of the library that requires a signal, typically SIGUSR1. Can be overridden with -DSIGRCU by modifying Makefile.build.inc.

此 flavor 的 read-side speed 只略輸 QSBR，並且不需要呼叫 quiescent states，但是會需要佔用掉一個 signal，預設是會佔用掉 SIGUSR1

Bullet-proof

#include <urcu-bp.h>
Link with -lurcu-bp

It is specifically designed to help tracing library to hook on applications without requiring to modify these applications.

此 flavor 被設計來 hook 在 application 上，並且不用改動這些 applications
前面所有的 flavor 都需要 rcu_init(), rcu_register_thread() and rcu_unregister_thread() 才能夠運作，但這個卻不用。(會轉變成 nops)

可是會犧牲 read-side and write-side 的速度

URCU Test

https://github.com/ccs100203/kernel-module-rcu-test
將此測試移植到 userspace rcu 來做

已經做好了，尚未放到 github 上

TODO, 增加測試的強度 (stress)

執行結果:

kernel module

[348771.405662] id : 0, name : book1, author : jb, borrow : 0, addr : ffff94e2d0490e40
[348771.405667] id : 1, name : book2, author : jb, borrow : 0, addr : ffff94e2d0490240
[348771.405669] book1 borrow : 0
[348771.405669] book2 borrow : 0
[348771.426178] borrow success 0, preempt_count : 0
[348771.453980] borrow success 1, preempt_count : 0
[348771.453985] id : 0, name : book1, author : jb, borrow : 1, addr : ffff94e2d0490900
[348771.453988] id : 1, name : book2, author : jb, borrow : 1, addr : ffff94e49d2adc00
[348771.470186] return success 0, preempt_count : 0
[348771.494184] return success 1, preempt_count : 0
[348771.494189] id : 0, name : book1, author : jb, borrow : 0, addr : ffff94e49d2ad180
[348771.494192] id : 1, name : book2, author : jb, borrow : 0, addr : ffff94e2d0490900

userspace rcu

目前這裡的 preempt_count 目前是直接回傳 0，而不是像 kernel module 使用 preempt_count() function

使用到 urcu/rculist.h 以及 urcu/list.h

id : 0, name : book1, author : jb, borrow : 0, addr : 55fc4d1cb2a0
id : 1, name : book2, author : jb, borrow : 0, addr : 55fc4d1cb340
book1 borrow : 0
book2 borrow : 0
borrow success 0, preempt_count : 0
borrow success 1, preempt_count : 0
id : 0, name : book1, author : jb, borrow : 1, addr : 55fc4d1cb7f0
id : 1, name : book2, author : jb, borrow : 1, addr : 55fc4d1cb2a0
return success 0, preempt_count : 0
return success 1, preempt_count : 0
id : 0, name : book1, author : jb, borrow : 0, addr : 55fc4d1cb340
id : 1, name : book2, author : jb, borrow : 0, addr : 55fc4d1cb7f0

APIs & Docs

Userspace RCU API
完整 API 連結
Userspace RCU Concurrent Data Structures (CDS) API
This document describes briefly the data structures contained with the userspace RCU library.

liburcu-defer

urcu-call-rcu

已提交 patch

[lttng-dev] [PATCH] Fix: revise obsolete command in README.md

測試的時候發現 make bench 早在 2015 被改掉了，被分成 short_bench, long_bench 兩種
但 readme 跟 doc 都沒改
Refactor tests

然後 benchmark 沒有紀錄 log 要自己存