# Ubuntu host fuzzing qumu Debian
###### tags: `USB`

## Reference
https://github.com/google/syzkaller/blob/master/docs/linux/setup_ubuntu-host_qemu-vm_x86-64-kernel.md

## Testing Environment
* host machine: Virtualbox (6.1.19) with Ubuntu 20.04
* if using virtualbox for host, need the version of 6.1.19 (since this version support nested virtualization)
* target machine: Debian with ***mainline linux kernel*** (to enable raw gadget, kcov)

## Compiler
* gcc (only gcc 6.1.0 or later support code-coverage)
## Linux Kernel
1. get kernel source code `git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git $KERNEL`
2. generate default config

cd $KERNEL
make CC="$GCC/bin/gcc" defconfig
make CC="$GCC/bin/gcc" kvm_guest.config

3. Enable kernel config options
* enable features
* coverage collection

        CONFIG_KCOV=y
        CONFIG_KCOV_INSTRUMENT_ALL=y
        CONFIG_KCOV_ENABLE_COMPARISONS=y
        CONFIG_DEBUG_FS=y

* detect memory leak

        CONFIG_DEBUG_KMEMLEAK=y

* show code coverage in web interface

        CONFIG_DEBUG_INFO=y

* show enable syscall and kernel bitness

        CONFIG_KALLSYMS=y
        CONFIG_KALLSYMS_ALL=y

* config for creating Debian image

    CONFIG_CONFIGFS_FS=y
    CONFIG_SECURITYFS=y

* enable usb fuzzing
* enable CONFIG_USB_RAW_GADGET
* enable CONFIG_USB_DUMMY_HCD
### enable CONFIG_USB_RAW_GADGET
* use below instruction if you could not find CONFIG_USB_RAW_GADGET

        make menuconfig
        # path of usb raw gadget inside menuconfig screen
        Device Driver  -> USB support -> USB Gadget Support -> 
        USB Gadget precomposed configurations -> USB Raw Gadget

* enable CONFIG_USB_DUMMY_HCD
* the reason of using USB Raw Gadget
* USB raw gadget do not check the packet
* for more information, please see [here](https://www.kernel.org/doc/html/latest/usb/raw-gadget.html)
4. regenerate config `make CC="$GCC/bin/gcc" olddefconfig`
6. build kernel `make CC="$GCC/bin/gcc" -j64
7. check build successful or not

$ ls $KERNEL/vmlinux
$KERNEL/vmlinux
$ ls $KERNEL/arch/x86/boot/bzImage
$KERNEL/arch/x86/boot/bzImage

## Image
1. install debootstrap `sudo apt-get install debootstrap`
2. create a Debian Stretch Linux image with the minimal set of required packages

cd $IMAGE/
wget https://raw.githubusercontent.com/google/syzkaller/master/tools/create-image.sh -O create-image.sh
chmod +x create-image.sh
./create-image.sh

3. check the result by `ls $IMAGE/stretch.img`

## VM or Physical device
* could also done on physical device (android device, Odroid C2 boards)
* VM image needs to include networking support (since the fuzzing process communicate with outsisde world)
* run the ssh server in vm (since fuzzer connect with vm through ssh)
### qemu setup
1. install qemu `sudo apt-get install qemu-system-x86`
2. Make sure the kernel boots and sshd starts:

    qemu-system-x86_64 \
        -m 2G \
        -smp 2 \
        -kernel $KERNEL/arch/x86/boot/bzImage \
        -append "console=ttyS0 root=/dev/sda earlyprintk=serial net.ifnames=0" \
        -drive file=$IMAGE/stretch.img,format=raw \
        -net user,host=10.0.2.10,hostfwd=tcp:127.0.0.1:10021-:22 \
        -net nic,model=e1000 \
        -enable-kvm \
        -nographic \
        -pidfile vm.pid \
        2>&1 | tee vm.log

* -m: assign 2 gigabytes of memory
* -smp 2 assign 2 cpu to the vm
* -kernel: kerenl image
* -append cmdline: as kernel commmand line
* -drive: drive image
* -net: initialized network interface
* -pidfile [file]: write `pid` to [file]
* `2 > &1`: redirect from stderr to stdout
3. ssh to qemu machine to check whether the instance is successfully boot `ssh -i $IMAGE/stretch.id_rsa -p 10021 -o "StrictHostKeyChecking no" root@localhost`
4. kill the qemu instance `kill $(cat vm.pid)`
### bugs on qemu setup (for virtualbox)
when executing the above command in command line, you might encounter this problem `qemu-system-x86_64: failed to initialized KVM: No such file or directory`
* solution: enable nested virtualization
* check whether your cpu enable virtualization

    $ egrep -c "(svm|vmx)"  /proc/cpuinfo

* `2` should be listed when the virtualization is enabled

* how to enable nested virtualization: go to `Setting --> System --> Processor` and check the option `Enable Nested VT-x/AMD-V`

* if your virtualbox version is incorrect, you may crash after execute the command
* required virtual box version (6.1.19)
* see how to update virtualbox in [here](https://www.nakivo.com/blog/how-to-update-virtualbox/)

## Syzkaller
1. install `go`

wget https://dl.google.com/go/go1.14.2.linux-amd64.tar.gz
tar -xf go1.14.2.linux-amd64.tar.gz
mv go goroot
mkdir gopath
export GOPATH=`pwd`/gopath
export GOROOT=`pwd`/goroot
export PATH=$GOPATH/bin:$PATH
export PATH=$GOROOT/bin:$PATH

2. download and build syzkaller

go get -u -d github.com/google/syzkaller/prog
cd gopath/src/github.com/google/syzkaller/
make

* all build should be in $syzkaller/bin

3. write config file

{
	"target": "linux/amd64",
	"http": "127.0.0.1:56741",
	"workdir": "$GOPATH/src/github.com/google/syzkaller/workdir",
	"kernel_obj": "$KERNEL",
	"image": "$KERNEL/image/stretch.img",
	"sshkey": "$KERNEL/image/stretch.id_rsa",
	"syzkaller": "$GOPATH/src/github.com/google/syzkaller",
    "enable_syscalls": ["syz_usb_connect", "syz_usb_disconnect", "syz_usb_control_io", "syz_usb_ep_write", "syz_usb_ep_read"],
	"procs": 4,
	"type": "qemu",
	"vm": {
		"count": 4,
		"kernel": "$KERNEL/arch/x86/boot/bzImage",
		"cpu": 2,
		"mem": 2048,
                "cmdline": "dummy_hcd.num=4"
	}
}

* $KERNEL means your self-compiled linux kernel
4. run the syzkaller by `$GOPATH/syz-manger -config=usb.cfg`