# Kernel Debugging References - bpftrace

[TOC]

## References

See [bpftrace Reference Guide](https://github.com/iovisor/bpftrace/blob/master/docs/reference_guide.md) for full documentation.

### [Linux tracing made simpler with bpftrace - DevConf.CZ 2022](https://youtu.be/gSxntAO2Iys)

{%youtube gSxntAO2Iys %}

Note that in this example, in `kretfunc` the script uses `args->sk`, `args->size` to access function parameters, instead of `args.sk`, `args.size`. In newer script, `.` is preferred over `->`, and the later one is preserved due to backward compatibility. See [Commit e73f0d5](https://github.com/iovisor/bpftrace/commit/e73f0d5d4e1a62d2cf542c18b03546b2899edf50) (*Replace args->x by args.x in docs and tools*) for further detail.

### [Tutorial: Building an EBPF Swiss Knife from OSS - Val Pliskin & Amit Slavin, Seekret](https://youtu.be/jQLp6mLEzXY)

{%youtube jQLp6mLEzXY %}

### [An introduction to bpftrace tracing language - DevConf.CZ 2020](https://youtu.be/93aHXYqZmU0)

{%youtube 93aHXYqZmU0 %}

### [Linux Kernel Tracing Using eBPF - Vandana Salve, Prasme Systems](https://youtu.be/zqt1hWpHMQU)

{%youtube zqt1hWpHMQU %}

### [bpftrace internals - DevConf.CZ 2020](https://youtu.be/nDY4iC_ekQY)

{%youtube nDY4iC_ekQY %}

## `kfunc` probe usage

With the advent of BTF, `bpftrace` script can be greatly simplified. Use `tcp_sendmsg()` in *Linux tracing made simpler with bpftrace - DevConf.CZ 2022* as an example:

### Step 0: find available events by `-l` option

To find out what events are available, simply pass the `-l` option into `bpftrace`:

$ bpftrace -l

To filter out the result, an optional regular expression can be added. For example, to find out all available tracepoints:

$ bpftrace -l "tracepoint:*"

To find events whose name contain "module":

$ bpftrace -l "*module*"

Each line in the output corresponds to the *name* of that event, sometimes called *probe name*. A complete probe names has to be provided in order for `bpftrace` script to work correctly. For example, to hook into `do_init_module`, you have to specify probe by `kfunc:do_init_module` (or `kprobe:do_init_module` if you'd like to use kprobe) instead of just `do_init_module`.

### Step 1: check prototype by `bpftrace -lv`

$ sudo bpftrace -lv "kfunc:tcp_sendmsg"
kfunc:tcp_sendmsg
    struct sock * sk
    struct msghdr * msg
    size_t size
    int retval

Note that other than a

Not only can it look up function prototypes, it can also check tracepoint definitions. For example, for definition of `tracepoint:irq:irq_handler_entry`:

$ sudo bpftrace -lv "tracepoint:irq:irq_handler_entry"
tracepoint:irq:irq_handler_entry
    int irq
    __data_loc char[] name

### Step 2: `args` variable in `kfunc` probe

For `kfunc` and `kretfunc` probe, function parameters can be accessed in bpftrace script by `args.NAME`, where `NAME` is the name of function parameter listed in `-lv` option. For example:

kretfunc:tcp_sendmsg
{
	$addr = ntop(args.sk->__sk_common.skc_rcv_saddr);
	printf("sadr: %s, size: %d bytes, sent: %d bytes\n", $addr, args.size, retval);
}