# Linux Kernel OOPS (panic)

###### tags: `linux` `kernel`

Something to look when kernel panic.

### dmesg
some options useful to filter messages

$ dmesg -T
$ dmesg -T -f kern
$ dmesg -T -l debug -f kern
$ dmesg -T -l err -f kern
$ dmesg -T -l crit -f kern

### kern.log
check kern.logs

$ ls -l /var/log/kern.log*
-rw-r----- 1 syslog adm  7996 Apr 14 11:13 /var/log/kern.log
-rw-r----- 1 syslog adm 21404 Apr 10 00:00 /var/log/kern.log.1
-rw-r----- 1 syslog adm  7925 Apr  3 00:00 /var/log/kern.log.2.gz
-rw-r----- 1 syslog adm  4248 Mar 27 00:00 /var/log/kern.log.3.gz
-rw-r----- 1 syslog adm  8925 Mar 20 00:00 /var/log/kern.log.4.gz

### kerneloops
kerneloops process and config

$ ps aux | grep kerneloops
kernoops    1903  0.0  0.0  13528   468 ?        Ss   Feb19   0:06 /usr/sbin/kerneloops --test
kernoops    1926  0.0  0.0  13528   468 ?        Ss   Feb19   0:06 /usr/sbin/kerneloops


$ cat /etc/kerneloops.conf
#
# Configuration file for the oops.kernel.org kernel crash collector
#

#
# Set the following variable to "yes" if you want to automatically
# submit your oopses to the database for use by your distribution or the
# Linux kernel developers
#
#
# PRIVACY NOTE
# Enabling this option will cause your system to submit certain kernel
# output to the oops.kernel.org website, where it will be available via
# this website to developers and everyone else.
# The submitted info are so-called "oopses", kernel crash signature.
# However, due to the nature of oopses, it may happen that a few
# surrounding lines of the oops in the "dmesg" are being sent together
# with the oops.
#
# Default is "ask" which uses a UI application t ask the user for permission
#
allow-submit = ask

#
# Set the following variable to "yes" if you want to allow your
# Linux distribution vendor to pass the oops on to the central oops.kernel.org
# database as used by the Linux kernel developers
#
allow-pass-on = yes

#
# URL for submitting the oopses
#

submit-url = http://oops.kernel.org/submitoops.php

#
# Path to syslog file containing full kernel logging output
#

log-file = /var/log/kern.log

#
# Script or program to pipe oops submits to
# Comment out for no pipe submission
#

submit-pipe = /usr/share/apport/kernel_oops

#### System.map
system.map stores symbol table used by Kernel.

$ sudo head /boot/System.map-5.11.0-49-generic
0000000000000000 D __per_cpu_start
0000000000000000 D fixed_percpu_data
00000000000001d9 A kexec_control_code_size
0000000000001000 D cpu_debug_store
0000000000002000 D irq_stack_backing_store
0000000000006000 D cpu_tss_rw
000000000000b000 D gdt_page
000000000000c000 d exception_stacks
0000000000010000 d entry_stack_storage
0000000000011000 D espfix_waddr

#### nm
list symbols from object files

#### cause kernel panic manually

$ echo 1 > /proc/sys/kernel/sysrq
$ echo c > /proc/sysrq-trigger

#### kdump-tools
manages kdump feature in linux kernel.

$ apt install kdump-tools

$ kdump-config test
 * /etc/default/kdump-tools: USE_KDUMP is not set or zero
 * no crashkernel= parameter in the kernel cmdline
 * /etc/default/kdump-tools: KDUMP_KERNEL does not exist: /var/lib/kdump/vmlinuz
USE_KDUMP:		0
KDUMP_COREDIR:		/var/crash
crashkernel addr
kdump kernel addr
kdump kernel:
   /var/lib/kdump/vmlinuz
kdump initrd:
   /var/lib/kdump/initrd.img
kexec command to be used:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-5.11.0-49-generic root=UUID=484c1a8e-7b01-4caa-b861-84316c990c49 ro quiet splash vt.handoff=7 reset_devices systemd.unit=kdump-tools-dump.service nr_cpus=1 irqpoll nousb ata_piix.prefer_ms_hyperv=0" /var/lib/kdump/vmlinuz

local config file of kdump

/etc/default/kdump-tools

#### reference
https://01.org/linuxgraphics/documentation/bugs-and-debugging/how-get-kernel-backtrace
https://sanjeev1sharma.wordpress.com/tag/debug-kernel-panics/
https://www.thegeekstuff.com/2012/03/linux-nm-command/
https://www.howtoforge.com/linux-nm-command/
https://www.opennet.ru/docs/HOWTO/Kernel-HOWTO-9.html