# QEMU for Beginner and Advanced

###### tags: `2022/09` `qemu` `qemu-user` `qemu-architecture` `qemu-system-architecture`

:::info
++(2022/9/18)++ QEMU for beginner, starting from few simple examples so to reduce the hurdle to get familiar with qemu.
++(latest update on 2022/10/1)++
:::

---
**Table of Contents**

[TOC]

---
## ++**What is qemu?**++
From [Linux Q&A-What is qemu?](https://hackmd.io/@MarconiJiang/QnA_Linux#Q-QEMU---What-is-QEMU), we learn that there are 3 modes supported by qmue:

1. ++A user mode emulator++ : QEMU can launch Linux processes compiled for one CPU on another CPU, translating syscalls on the fly. This QEMU mode is faster than full system emulation, but is not a perfect abstraction.
2. ++A full system emulator++ : QEMU emulates a full system (virtual machine), including a processor and various peripherals such as disk, ethernet controller etc. That is normally how an operating system works.
3. ++A virtualization environment++ : used by KVM and XEN virtualization environments. (Will not go into details in this article)

++**QEMU user mode**++
It is much more easier to try with QEMU user mode. Users just need to be familiar with Linux shell command, and a basic programming skill, hello world level, in C language. We will start with 2 examples :
1. First one for QEMU to run x86 simulation. ==People might wonder why we need to use x86 system to simulate x86 CPU? The purpose is just to demonstrate the function of QEMU as a stepping stone to simulate another CPU of ARM.== It is a very good entry to run QEMU. We just need to install gcc for x86 environment.
2. Second example is for QEMU to run ARM simulation. Similar to first example, but needs to install ARM cross compiler first to generate ARM executive binary program.

++**QEMU system mode**++
After then, the third example will be running QEMU system mode. To run QEMU system mode, it requires an image file containing a complete operating system to run, instead of a single Linux program to run on QEMU user mode. First we pick an old Linux version running on i386. Then pick the target of Raspberry Pi, which is a very popular SBC (Single Board Computer) based on ARM CPU. We will use the Raspberry Pi image, running Debian based Linux called Raspbian, on QEMU system mode simulation.

3. First QEMU system mode example is to simulate i386 to run old Linux version 0.11.
4. Second QEMU system mode example is to simulate ARM CPU to run Raspberry Pi image.

----
## QEMU Installation

One (easy, but not recommended) way to install `qemu` is to use package installation. It will install both both `user` and `system` modes of `qemu`. However, it installed `qemu version 2.5.0` on Ubuntu 16.04, which is relatively old version.

# Ubuntu environment
sudo apt install qemu

Recommend to use newer version, by following instruction from `https://www.qemu.org/download/`. By the time I download, the latest version was `7.1.0`. It requires `Python 3.8` than `Ubuntu 16.04` default `Python 3.5`. So I chose version 5.0.0 arbitrarily which just works.

wget https://download.qemu.org/qemu-5.0.0.tar.xz
tar xvJf qemu-5.0.0.tar.xz
cd qemu-5.0.0
sudo apt update
sudo apt-get install libgtk-3-dev
./configure --enable-gtk
sudo make install

Check [Linux Q&A: QEMU - nothing shows up after QEMU VNC server running](https://hackmd.io/CEP6h2nQREOjjQOYJqRugA?view#Q-QEMU---nothing-shows-up-after-QEMU-VNC-server-running) for explanation of `qemu ./configure` option.

----
## QEMU Example 1 : `qemu-x86_64` and `qemu-i386`

### ++Running `hello world` on Linux native environmemt++

With x86_64 system used for Linux, qemu is not really needed to run x86_64 programs like `hello_world`. However, we still can use qemu to simulate x86_64 program, like we do for native x86_64 program.

Let's start with `hello world` program running on Linux native x86_64 environment.

# install gcc to build hello work program
sudo apt update
sudo apt install build-essential

use `vi` or other editor program to create `hello.c` file content like below

#include <stdio.h>

int main() {
    printf("Hello Marconi...\n");
    return(0);
}

then compile and execute `hello` program on Linux

gcc hello.c -o hello
./hello
# output below
Hello Marconi...

# or compile to 32bit object code
gcc -m32 hello.c -o hello32
./hello32
# output below
Hello Marconi...

We can check the file attribute with Linux command `file` to see the attritube of `hello` is executable at x86_64 ELF 64-bit LSB mode, and `hello32` is executable at Intel 80386 (nornally use i386 for short) ELF 32-bit LSB mode.

file hello
# output below
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, 
interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32,
BuildID[sha1]=51f76d8ea67c7fd9506745a09f8698e552d05933, not stripped

file hello32
# output below
hello32: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, 
interpreter /lib/ld-linux.so.2, for GNU/Linux 2.6.32,
BuildID[sha1]=50e617ac0122622e4d1ced8079f9a2d19868b148, not stripped

### ++Running `hello world` on qemu++

First, we install `qemu` user mode and run the `hello` program.

# Install qemu user mode environment
sudo apt install qemu-user

# Check what are installed
ls /user/bin/qemu
# the output will include many architectures qemu user mode support. 
# Only few are listed below
/usr/bin/qemu-aarch64
/usr/bin/qemu-arm
/usr/bin/qemu-i386
/usr/bin/qemu-x86_64
....

Keep in mind that the naming convention of qemu user mode command is `qemu-arctecture` where architecture can be `arm` (ARM 32bit version), `aarch64` (ARM 64bit), `i386` (x86 32bit), `x86_64` (x86 64 bit), etc. In next section, we will introduction qemu system mode. The naming convention is `qemu-system-architecture`. You can see what the difference is.

Let's see how we can execute the `hello` programs on qemu.

# running hello program under qemu-x86_64 environment
qemu-x86_64 ./hello

# output below. We can ignore the warning at this moment
warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
Hello Marconi...

# or
# running 32bit version of hello program under qemu-i386 environment
qemu-i386 ./hello

# output below. 
Hello Marconi...

Till now, it is quite easy and straight forward to use `qemu` to simulate and run x86 programs.

---
## QEMU Example 2 : `qemu-aarch64`

To run QEMU simulation for ARM CPU, we need, at least, a `hello world` program in ARM binary code to demostrate `qemu-arm` and `qemu-aarch64` features. We are going to build the `hello world` binary by installing the cross compiler of `gcc` for ARM first.

### ++Install ARM toolchain++

We can build ARM binary code on x86 CPU with cross compiler. 'Cross' means to build a binary code for one CPU when it is running on another CPU. x86 is the one of the most popular and power CPU. We can use it to build the binary code for another CPU, like ARM, RISC-V, or other embedded types of CPU, which can save development and compilation time due to powerful x86 CPU, and many tools available.

According to wikipedia, a toolchain is a set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs. In general, the tools forming a toolchain are executed consecutively so the output or resulting environment state of each tool becomes the input or starting environment for the next one, but the term is also used when referring to a set of related tools that are not necessarily executed consecutively.

A simple software development toolchain may consist of a compiler and linker (which transform the source code into an executable program), libraries (which provide interfaces to the operating system), and a debugger (which is used to test and debug created programs).

We use the following command to install ARM64 `gcc` tool chain, using the same `hello.c` as in [Running hello world on Linux native environmemt](#Running-hello-world-on-Linux-native-environmemt)

sudo apt update
# install gcc for ARM64 (aarch64)
sudo apt install gcc-aarch64-linux-gnu

# compile hello.c with aarch64 gcc
# the same hello.c as in 
aarch64-linux-gnu-gcc hello.c -static -o hello-arm64

# check file attribute of hello-arm64
file hello-arm64
# output
hello-arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), 
statically linked, for GNU/Linux 3.7.0,
BuildID[sha1]=5cd2c9e6081bda47b83b196b41ff7c1a1113a850, not stripped

Now we have a executable ARM64 binary of `hello-arm64`, which can be executed on QEMU. We can find the file attribute of `hello-arm64` is ELF 64-bit LSB executable based on ARM aarch64.

# check if qemu-aarch64 is installed?
ls /usr/bin/qemu-aarch64
# output
/usr/bin/qemu-aarch64

# execute hello-arm64 under qemu
qemu-aarch64 ./hello-arm64
# output 
Hello Marconi...

Now we succeed in running `qemu-architecture` to simulate ARM CPU in user mode. Next is to run `qemu-system-architecture` in system mode.

---
## QEMU Example 3 : `qemu-system-i386`

It requires an OS image to run in QEMU system mode. We use old Linux 0.11 version as it is relatively simple.

Check the following `www.oldlinux.org` web and copy the link of file 'Linux 0.11 on qemu-12.5.i386.zip'
* http://www.oldlinux.org/
* ++Linux Ancient Resources++ -- Early and old Linux docs, source code and bins. (Linux.old)
* qemu-images/
* 'Linux 0.11 on qemu-12.5.i386.zip'

The link is http://www.oldlinux.org/Linux.old/qemu-images/Linux%200.11%20on%20qemu-12.5.i386.zip.

Now follow the commands to execute `qemu-system-386`.

# go to the working directory of your own
cd ~/myworks
wget http://www.oldlinux.org/Linux.old/qemu-images/Linux%200.11%20on%20qemu-12.5.i386.zip

# unzip the file will create a new directory `qemu-12.5.i386`
unzip http://www.oldlinux.org/Linux.old/qemu-images/Linux%200.11%20on%20qemu-12.5.i386.zip

cd qemu-12.5.i386

# use the following command, reference from file `linux.bat`
qemu-system-i386 -hda linux-0.11-devel-060625.qcow2 -no-reboot -m 16M

# qemu will create a new ternimal screen, and start booting process. 


Then enter key `1` to boot from the first device.


Now Linux 0.11 is running on `qemu-system-i386`. You may find many commands not found, like `uname`, `lspci`, `lscpu`, etc. Those available are `vi`, `awk`, `fdisk`, but there are i386 assembler `as86`, linker `ld86` programs available, and `gcc` and `hello.c` as well.

---
## QEMU Example 4 : `qemu-system-arm` running Raspberry Pi in command mode

Now come to more challenging one. I failed so many time trying to get Raspberry Pi image working. The easiest way is to create a script file of the following content, from [this article](https://raspberrypi.stackexchange.com/questions/89196/emulate-raspberry-pi-zero-w-with-qemu-failed-due-to-missing-dtb).

#!/bin/sh

QEMU=$(command -v qemu-system-arm)
TMP_DIR=$(pwd)
RPI_KERNEL=kernel-qemu-4.19.50-buster
RPI_KERNEL_FILE=$TMP_DIR/$RPI_KERNEL
PTB=versatile-pb.dtb
PTB_FILE=$TMP_DIR/$PTB
IMAGE_BASE=2019-09-26-raspbian-buster-lite
IMAGE=$IMAGE_BASE.zip
IMAGE_FILE=$TMP_DIR/$IMAGE
RPI_FS=$TMP_DIR/$IMAGE_BASE.img

mkdir -p $TMP_DIR

wget https://github.com/dhruvvyas90/qemu-rpi-kernel/blob/master/${RPI_KERNEL}?raw=true \
        -O ${RPI_KERNEL_FILE}

wget https://github.com/dhruvvyas90/qemu-rpi-kernel/raw/master/$PTB \
        -O ${PTB_FILE}

wget http://downloads.raspberrypi.org/raspbian_lite/images/raspbian_lite-2019-09-30/$IMAGE \
        -O ${IMAGE_FILE}
unzip $IMAGE_FILE -d $TMP_DIR


$QEMU -kernel ${RPI_KERNEL_FILE} \
    -cpu arm1176 -m 256 -M versatilepb \
    -dtb ${PTB_FILE} -no-reboot \
    -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
    -drive "file=${RPI_FS},index=0,media=disk,format=raw" \
    -net user,hostfwd=tcp::5022-:22 -net nic \
    -nographic

Though I don't know why and how this script works. At least, it proves to be working, non-GUI mode (maybe due the the image is 'lite' version). The next step is to work on Raspberry Pi version with GUI mode ([next example](#QEMU-Example-5--qemu-system-arm-running-Raspberry-Pi-in-GUI-mode)), and understand how it works. Check [:arrow_right:Next article - QEMU for Advanced](https://hackmd.io/@MarconiJiang/qemu_advanced)

Below are some screen shots.

Fig. - Running shell script to launch `qemu` Raspberry Pi image. `qemu` create another screen, though showing nothing.

Fig. - Same as above, without `qemu` screen

Fig. - Boot up sequences show `Welcome to Raspbian GNU/Linux 10 (buster)!`

Fig. - Prompt for login user id and pass word input. The standard ones are `pi` and `raspberry` respectively.

Fig. - Successfully login to Raspbian

Fig. - Shell command `uname -a` to check Linux kernel version

---
## QEMU Example 5 : `qemu-system-arm` running Raspberry Pi in GUI mode

Following from previous examples and [Emuation using Qemu's native raspi2/3 machine](https://github.com/dhruvvyas90/qemu-rpi-kernel/), we modified the script by using newer Raspberry Pi images of full version, so we can run `startx` command in RPi to enter GUI mode.

Here is the script.

#!/bin/sh

QEMU=$(command -v qemu-system-arm)
TMP_DIR=$(pwd)
RPI_KERNEL=kernel-qemu-5.4.51-buster
RPI_KERNEL_FILE=$TMP_DIR/$RPI_KERNEL
PTB=versatile-pb-buster-5.4.51.dtb
PTB_FILE=$TMP_DIR/$PTB
IMAGE_BASE=2020-05-27-raspios-buster-full-armhf
IMAGE=$IMAGE_BASE.zip
IMAGE_FILE=$TMP_DIR/$IMAGE
RPI_FS=$TMP_DIR/$IMAGE_BASE.img

mkdir -p $TMP_DIR

wget https://github.com/dhruvvyas90/qemu-rpi-kernel/blob/master/${RPI_KERNEL}?raw=true \
        -O ${RPI_KERNEL_FILE}

wget https://github.com/dhruvvyas90/qemu-rpi-kernel/raw/master/$PTB \
        -O ${PTB_FILE}

wget http://downloads.raspberrypi.org/raspios_full_armhf/images/raspios_full_armhf-2020-05-28/$IMAGE \
        -O ${IMAGE_FILE}
unzip $IMAGE_FILE -d $TMP_DIR


# $QEMU -kernel ${RPI_KERNEL_FILE} \
#    -cpu arm1176 -m 256 -M versatilepb \
#    -dtb ${PTB_FILE} -no-reboot \
#    -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
#    -drive "file=${RPI_FS},index=0,media=disk,format=raw" \
#    -net user,hostfwd=tcp::5022-:22 -net nic

$QEMU \
  -M versatilepb \
  -cpu arm1176 \
  -m 256 \
  -drive "file=${RPI_FS},if=none,index=0,media=disk,format=raw,id=disk0" \
  -device "virtio-blk-pci,drive=disk0,disable-modern=on,disable-legacy=off" \
  -net "user,hostfwd=tcp::5022-:22" \
  -dtb ${PTB_FILE} \
  -kernel ${RPI_KERNEL_FILE} \
  -append 'root=/dev/vda2 panic=1' \
  -no-reboot

After launch Raspberry Pi successfully (Difference is, this time, it shows on `qemu` screen, instead of `terminal` due to removal of `qemu -nogrphic` option). If `qemu` screen still shows in command line mode, we can enter Raspberry Pi specific command `startx` to ente GUI mode.

Below are some screen shots.

Fig. - Launch shell script to download files from internet, including .img, .dtb, kernel. After downloading the image files, and it takes some time to boot up `qemu`, at that moment I suspect the system was down, but not.

Fig. - `qemu` launch Raspberry Pi (on another screen)

Fig. - `qemu` lauch Raspberry Pi into GUI mode

Fig. - Raspberry Pi GUI boot from `qemu`

Fig. - Raspberry Pi `terminal` shows the Linux kernel version

Fig. - Raspberry Pi running a game. Unfortunately, it does not support mouse operation. That requires further mouse driver setting at `qemu`.

---
::: success
This is a separate line to distinguish between Beginner and Avanced
:::
---

# QEMU for Advanced

###### tags: `2022/09` `qemu` `qemu-user` `qemu-architecture` `qemu-system-architecture`

:::info
++(2022/9/23)++ QEMU for advanced, after completing previous examples, moves on to next step to build own version of kernel images for any x86 Linux and Raspberry Pi BIOS images
++(latest update on 2022/10/8)++
:::

So far, we use the images and kernel from the internet prepared by others. It will not work with combination of newer firmware with older kernels (I tried and failed so many timmes). I am tringing to see how to generate the kernel file from official Raspberry Pi images so we can upgrade the firmware and running under `qemu`. Also try to understand the parameters of `qemu` so we can experiments more, like `ssh` to QEMU RPi. Or we can use the mouse in Raspberry Pi GUI mode.

Before jumping into examples, there are couples of prerequisites.
+ `qemu-system-` examples of different options
+ `qemu-system-` Block Device Options
+ `qemu-img` usage
+ `losetup` (my preference) or mount, or `nbd` to check the contents of `img`, `.iso`, and so on images files.

## ++**`qemu-system-` examples of different options**++

I found it is not easy to set the `qemu-system-` options right to get it working, there are basic options needed to launch `qemu-system-` successfully. Few examples are listed.

### **Using `-kernel kernel*` and `-dtb *.dtb`**
++**Option 1 for `qemu-system-arm`**++: Use `-drive`

qemu-system-arm \
  -M versatilepb -cpu arm1176 -m 256 \
  -dtb versatile-pb.dtb \
  -kernel kernel-qemu-4.19.50-buster \
  -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw console=ttyAMA0" \
  -drive "file=2019-09-26-raspbian-buster-lite.img,index=0,media=disk,format=raw" \
  -net user,hostfwd=tcp::2222-:22 -net nic \
  -serial stdio \
  -no-reboot \
  -nographic

+ Seems `console=ttyAMA0` can be removed from `-append`, no impact.
+ About the parameter `panic=1` (or any number `N` to replace `1`) in `-append`, it asks `qemu-system-` to wait `1` (or `N`) seconds before system to reboot due to kernel panic.
+ When `-M` is selected, no need to specify `-cpu`, as CPU is machine dependent. But when I use `-M versatilpb` and remove `-cpu arm1176`, it does not work.
+ `-net user,hostfwd=tcp::2222-:22 -net nic` : set up the networking stack and forward the SSH port. Trying to use, but failed ` -device virtio-net-device,netdev=unet -netdev user,hostfwd=tcp:127.0.0.1:2222-:22,id=unet`

++**Option 2 for `qemu-system-arm`**++ : Use `-hda`
Replace the line
`-drive "file=2019-09-26-raspbian-buster-lite.img,index=0,media=disk,format=raw"`
by
`-hda 2019-09-26-raspbian-buster-lite.img`.
Check [qemu-system- : QEMU Block Device Options](https://hackmd.io/4EhrdnQ9R0uCHhUsl9-kyw?both#qemu-system---QEMU-Block-Device-Options) for more detail explanation.

++**Option 3 for `qemu-system-arm`**++ : Use `.qcow2` as the image file.

# create an empty .qcow2
qemu-img create -f qcow2 disk.qcow2 20G

# Convert OS image to .qcow2 format
qemu-img convert -f raw -O qcow2 2019-09-26-raspbian-buster-lite.img disk.qcow2

Then replace the line
`-drive "file=2019-09-26-raspbian-buster-lite.img,index=0,media=disk,format=raw"`
by
`-hda disk.qcow2`

++**Option 4 for `qemu-system-arm`**++ : Add `-blockdev` option

It works though not sure if improves or not. However, need to input the file name in both `-drive` and `-blockdev`, or it reports error.

-drive file=2019-09-26-raspbian-buster-lite.img,index=0,media=disk,id=hd0,format=raw \
-blockdev driver=raw,node-name=disk,file.driver=file,file.filename=2019-09-26-raspbian-buster-lite.img \

++**Option 5 for `qemu-system-arm`**++ Use `root=/dev/mmcblk0p2`
See below [QEMU Advanced Example 2 : Build Raspberry Pi image from downloaded image and buildroot for kernel and .dtb for 32 bit ARM](#QEMU-Advanced-Example-2--Build-Raspberry-Pi-image-from-downloaded-image-and-buildroot-for-kernel-and-dtb-for-32-bit-ARM)

++**Option 6 for `qemu-system-arm`**++ : Add `-device virtio-blk-device,drive=hd0` option when `-M virtio` is selected. It won't work with `-M versstilepb`.

qemu-system-arm: -device virtio-blk-device,drive=hd0: No 'virtio-bus' bus found for device 'virtio-blk-device'

Check below option 11 for example. The reason why it failed with `-M versatilepb` is explained in this article [stackoverflow - Enabling virtio_blk_device for qemu](https://stackoverflow.com/questions/38066996/enabling-virtio-blk-device-for-qemu).

> virtio-blk-device is a VirtIO device that relies solely on memory-mapped IO (MMIO) and not on the PCI-bus. This does not work with Qemu's default machine type pc-i440fx-X.Y, or at least not out of the box.

Similarly, it failed for `-M versatilepb` to support `-device virtio-net-device,netdev=unet` with error message below.

qemu-system-arm: -device virtio-net-device,netdev=unet: No 'virtio-bus' bus found for device 'virtio-net-device'

### **Using `-kernel vmlinuz` and `-initrd initrd.gz` or `initrd initrd.img`**

++**Option 11 for `qemu-system-arm`**++

wget https://cdimage.debian.org/cdimage/archive/10.8.0/armhf/iso-cd/debian-10.8.0-armhf-xfce-CD-1.iso
sudo mkdir /mnt/rpi32
sudo mount debian-10.8.0-armhf-xfce-CD-1.iso /mnt/rpi32
cp /mnt/rpi32/install.ahf/vmlinuz .
cp /mnt/rpi32/install.ahf/initrd.gz .
sudo umount /mnt/rpi32
qemu-img create disk.qcow2 10G
qemu-system-arm \
  -M virt -cpu cortex-a15 -smp 2 -m 1024 \
  -initrd initrd.gz \
  -kernel vmlinuz \
  -drive if=none,file=disk.qcow2,id=hd0,format=raw \
  -device virtio-blk-device,drive=hd0 \
  -drive if=none,file=debian-10.8.0-armhf-xfce-CD-1.iso,id=cdrom,media=cdrom \
  -device virtio-scsi-device \
  -device scsi-cd,drive=cdrom \
  -nographic

++**Option 12 for `qemu-system-aarch64`**++

qemu-system-aarch64 \
    -M virt -cpu cortex-a57 -smp 2 -m 1024 \
    -initrd initrd.img \
    -kernel vmlinuz \
    -append "root=/dev/sda2 console=ttyAMA0" \
    -device virtio-scsi-device \
    -blockdev qcow2,node-name=hd0,file.driver=file,file.filename=disk.qcow2 \
    -device scsi-hd,drive=hd0 \
    -device virtio-net-device,netdev=unet \
    -netdev user,hostfwd=tcp:127.0.0.1:2222-:22,id=unet \
    -nographic

`qemu-system-aarch64 -M virt -cpu cortex-a57 -smp 2 -m 1024 -nographic` : run the ARM64 virtual platform emulator with 1GB RAM (or can input as `-m 1G`) and 2 Cortex-A57 cores with no GUI support.
`-drive if=none,file=ubuntu-16.04-server-cloudimg-arm64-uefi1.img,id=hd0` : use the Ubuntu image file
`-device virtio-blk-device,drive=hd0` : mount drive from above as a block device
`-drive file=user-data.img,format=raw` : use the configuration data image file

## ++**qemu-system-**++ : QEMU Block Device Options
Check [QEMU document](https://qemu.readthedocs.io/_/downloads/en/latest/pdf/) page 34 about Block device options
> The QEMU block device handling options have a long history and have gone through several iterations as the featureset and complexity of the block layer have grown. <font color="#f00">Many online guides to QEMU often reference older and deprecatedoptions, which can lead to confusion.</font> <font color="#3aab">The recommended modern way to describe disks is to use a combination of `-device` to specify the hardware device and `-blockdev` to describe the backend.</font> The device defines what the guest sees and the backend describes how QEMU handles the data.

## ++**qemu-img**++ : QEMU image handling

`qemu` can support many formats, `qcow2` format is recommended.
`.qcow` is a file format for disk image files used by QEMU. It stands for "QEMU Copy On Write" and uses a disk storage optimization strategy that delays allocation of storage until it is actually needed. Files in qcow format can contain a variety of disk images which are generally associated with specific guest operating systems. Three versions of the format exist: qcow, qcow2 and qcow3. From [wiki-qcow](https://en.wikipedia.org/wiki/Qcow)

# create an empty .qcow2
qemu-img create -f qcow2 disk.qcow2 20G

# Convert OS image to .qcow2 format
qemu-img convert -f raw -O qcow2 raspberrypi.img disk.qcow2

## ++**losetup**++: Loop back

cd ~
mkdir mnt
mkdir mnt/fat32
mkdir mnt/ext4
# Download Raspberry Pi image first
sudo losetup -f --show -P <path-to-rpi-image-file>
# Mount on mnt/fat32 and mnt/ext4
# sudo mount /dev/loop<no>p1 mnt/fat32 or mnt/ext4
# normally, it is loop0
# fat32 stores the kernel and dtb files
sudo mount /dev/loop0p1 mnt/fat32
# ext4 stores the Raspberry Pi OS
sudo mount /dev/loop0p2 mnt/ext4
sudo losetup -d /dev/loop<no>

Check [Native emulation of Rpi2/3 using Qemu's Raspi2/3 machine](https://github.com/dhruvvyas90/qemu-rpi-kernel/tree/master/native-emulation), and [Rasbperry Pi Linux kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html) for more details

## ++**mount**++ for `.img` and `.iso`

++**`mount` for `.img`**++
Check the content of OS images by `mount`ing images to Linux drives.

unzip <image-file>.zip
fdisk -l 2019-09-26-raspbian-buster.img

You should see something like this:

Disk 2019-09-26-raspbian-buster.img: 3.6 GiB, 3829399552 bytes, 7479296 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd9b3f436

Device                          Boot  Start     End Sectors  Size Id Type
2019-09-26-raspbian-buster.img1        8192  532479  524288  256M  c W95 FAT32 (
2019-09-26-raspbian-buster.img2      532480 7479295 6946816  3.3G 83 Linux

You see that the filesystem (.img2) starts at sector 532480. Now take that value and multiply it by 512, in this case it’s 512 * 532480 = 272629760 bytes. Use this value as an offset in the following command:

# others might use #sudo `mkdir /mnt/ext4` more often, which both can serve the same purpose
mkdir mnt/ext4
sudo mount -v -o offset=272629760 -t ext4 2019-09-26-raspbian-buster.img mnt/ext4[sudo] password for kernel-dev: 
# output from the system
mount: /dev/loop0 mounted on /home/kernel-dev/myworks/qemu-arm32/mnt/ext4.

Similarly for `.img1`, which contains the `kernel` and `.dtb`.

sudo mount -v -o offset=4194304 -t vfat 2019-09-26-raspbian-buster.img mnt/fat32
# output from the system
mount: /dev/loop1 mounted on /home/kernel-dev/myworks/qemu-arm32/mnt/fat32.

Read [Raspberry Pi on QEMU](https://azeria-labs.com/emulate-raspberry-pi-with-qemu/) for more details

++**`mount` for `.iso`**++

wget https://cdimage.debian.org/cdimage/archive/10.8.0/armhf/iso-cd/debian-10.8.0-armhf-xfce-CD-1.iso
sudo mkdir /mnt/rpi32
sudo mount debian-10.8.0-armhf-xfce-CD-1.iso /mnt/rpi32

## ++**nbd**++:

# Download images and create virtual disk
wget http://ftp.debian.org/debian/dists/buster/main/installer-arm64/current/images/netboot/debian-installer/arm64/initrd.gz
wget http://ftp.debian.org/debian/dists/buster/main/installer-arm64/current/images/netboot/debian-installer/arm64/linux

qemu-img create -f qcow2 disk.qcow2 20G

# nbd
sudo apt install nbd-client qemu qemu-utils
sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 disk.qcow2
sudo mount /dev/nbd0p1 /media/qcow
sudo cp initrd.img-4.19.0-16-arm64 vmlinuz-4.19.0-16-arm64 ../qemu/.
sync
sudo umount /dev/nbd0p1
sudo nbd-client -d /dev/nbd0

Check [Building a Debian Buster QEMU image for AARCH64](https://www.diozero.com/internals/qemuaarch64.html#building-a-debian-buster-qemu-image-for-aarch64) for more details

---
## ++**QEMU Advanced Example 1 :**++ Run QEMU Debian image by ==using kernel and initrd.gz from ftp.debian.org== for 64 bit ARM aarch64

I succeeded in following this article [Building a Debian Buster QEMU image for AARCH64](https://www.diozero.com/internals/qemuaarch64.html), after upgrading to `qemu version 5.0.0` on my Ubuntu 16.04 by source compiling.

This article serves my intention well to use any downloadable Linux image and extract `initrd` and `vmlinuz` from that image required for `qemu`. So it is a 2-step approach.

#### ++**1st step is to get the Debian image for aarch64**++

wget http://ftp.debian.org/debian/dists/buster/main/installer-arm64/current/images/netboot/debian-installer/arm64/initrd.gz
wget http://ftp.debian.org/debian/dists/buster/main/installer-arm64/current/images/netboot/debian-installer/arm64/linux

#### ++**2nd step is to create a 20GB file as the disk for Debian ISO image to install on**++ It is similar to select the HDD drive which we are going to install Debian ISO image and boot from. So the `disk.qcow2` content can be empty for now. It is recommended to use the format of `.qcow2` Copy-On-Write format.

qemu-img create -f qcow2 disk.qcow2 20G

#### ++**3rd step is to install Debian image on `qemu-system-aarch64`**++ by using the following command.

qemu-system-aarch64 -smp 2 -M virt -cpu cortex-a57 -m 1G \
    -initrd initrd.gz -kernel linux \
    -append "root=/dev/ram console=ttyAMA0" \
    -device virtio-scsi-device \
    -blockdev qcow2,node-name=hd0,file.driver=file,file.filename=disk.qcow2 \
    -device scsi-hd,drive=hd0 \
    -netdev user,id=unet -device virtio-net-device,netdev=unet \
    -nographic

After successfully launching the Debian image, it starts installation which takes really some time. (Remember the `root` password you key in, and user id and password you create.) After installation complete, the system prompt

                                                                                
   ┌─────────────────┤ [!] Continue without boot loader ├──────────────────┐    
   │                                                                       │    
   │                       No boot loader installed                        │    
   │ No boot loader has been installed, either because you chose not to or │    
   │ because your specific architecture doesn't support a boot loader yet. │    
   │                                                                       │    
   │ You will need to boot manually with the /vmlinuz kernel on partition  │    
   │ /dev/sda1 and root=/dev/sda2 passed as a kernel argument.             │    
   │                                                                       │    
   │                              <Continue>                               │    
   │                                                                       │    
   └───────────────────────────────────────────────────────────────────────┘    
                                                                                

Click `<Continue>` (When I tried another example by installing Ubuntu (not Debian), it reports an error message. This error can be ignore and skip the step of installing boot loader, then complete rest of the installation.)

                                                                          
                                                                                
   ┌───────────────────┤ [!!] Finish the installation ├────────────────────┐    
   │                                                                       │    
  ┌│                         Installation complete                         │    
  ││ Installation is complete, so it is time to boot into your new system. │    
  ││ Make sure to remove the installation media, so that you boot into the │    
  ││ new system rather than restarting the installation.                   │    
  ││                                                                       │    
  ││     <Go Back>                                          <Continue>     │    
  └│                                                                       │    
   └───────────────────────────────────────────────────────────────────────┘    
                                                                                
 

and now, we can press `CTRL-a, then x` to quie `qemu`.

#### ++**4th step is to extract the initrd.img and vmlinuz files from the qcow2 disk file**++ The easiest way to do this is to use nbd in a Linux environment (the host Linux I have been using).

sudo apt install nbd-client qemu qemu-utils
sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 disk.qcow2
sudo mkdir -p /media/qcow
sudo mount /dev/nbd0p1 /media/qcow
# make sure the following file names are correct versions
cp /media/qcow/initrd.img-4.19.0-16-arm64 .
cp /media/qcow/vmlinuz-4.19.0-16-arm64 .
sync
sudo umount /dev/nbd0p1
sudo nbd-client -d /dev/nbd0

#### ++**5th step is to run Debian which we installed on `disk.qcow2` with `initrd` and `vmlinuz`**++

qemu-system-aarch64 -smp 2 -M virt -cpu cortex-a57 -m 1G \
  -initrd initrd.img-4.19.0-16-arm64 \
  -kernel vmlinuz-4.19.0-16-arm64 \
  -append "root=/dev/sda2 console=ttyAMA0" \
  -device virtio-scsi-device \
  -blockdev qcow2,node-name=hd0,file.driver=file,file.filename=disk.qcow2 \
  -device scsi-hd,drive=hd0 \
  -netdev user,hostfwd=tcp:127.0.0.1:2222-:22,id=unet -device virtio-net-device,netdev=unet \
  -nographic

I add `hostfwd=tcp:127.0.0.1:2222-:22,` (not in original article) after `-netdev user` to allow host to `ssh` via `-p 2222`

Now we can run Debian on `qemu` of ARM 64bit aarch64 architecture for any Debian images in the future. Or in theory, for all architectures.

---
## ++QEMU Advanced Example 2 :++ Build Raspberry Pi image ==from downloaded image== and ==buildroot for kernel and .dtb== for 32 bit ARM

**++A :++** This example combines the following articles
+ [ARM64 vanilla kernel in QEMU](https://kasiviswanathanblog.wordpress.com/2020/05/03/arm64-vanilla-kernel-in-qemu/) to learn how to install `buildroot` and how to `make`.
+ [Raspberry Pi Documentation - The Linux kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html#using-menuconfig) to learn how to customize `buidroot make bcmxxxx_defconfig` for different Raspberry Pi boards.
+ [Raspbian on QEMU with ARMv7](https://github.com/paulden/raspbian-on-qemu-armv7) to learn the `qemu command line and options` with `kernel zImage` and `.dtb` from `buildroot`, and `.img` from [downloads.raspberrypi.com](http://downloads.raspberrypi.org/)

Below are the commands for this example:

# Reference - https://kasiviswanathanblog.wordpress.com/2020/05/03/arm64-vanilla-kernel-in-qemu/
git clone git://git.buildroot.net/buildroot
cd buildroot/

# Reference - https://www.raspberrypi.com/documentation/computers/linux_kernel.html#using-menuconfig
# Using below buildroot commands
cd buildroot/
# list the pre-defined configs in buildroot
make list-defconfigs
# buildroot for arm under qemu
make clean
make qemu_arm_vexpress_defconfig         # can try other qemu arm defconfig
make menuconfig                          # Exit to save
make 2>&1 | tee build.log
# Output files are located in the directory output/images/

# Reference - https://github.com/paulden/raspbian-on-qemu-armv7
# Download .img from raspberrypi.org
wget http://downloads.raspberrypi.org/raspbian_full/images/raspbian_full-2020-02-14/2020-02-13-raspbian-buster-full.zip
unzip 2020-02-13-raspbian-buster-full.zip

# qemu help to check the machines that qemu supports
qemu-system-arm -machine help

# command to execute qemu with options
qemu-system-arm \
        -M vexpress-a9 \
        -m 1G \
        -kernel output/images/zImage \
        -dtb output/images/vexpress-v2p-ca9.dtb \
        -sd 2020-02-13-raspbian-buster-full.img \
        -append "console=ttyAMA0,115200 root=/dev/mmcblk0p2" \
        -serial stdio \
        -net nic -net user,hostfwd=tcp::2222-:22


# The following option from Example 2 does not work on this case, not sure why.
    -drive "file=2020-02-13-raspbian-buster-full.img,index=0,media=disk,format=raw" \
    -append "root=/dev/sda1 panic=1 rootfstype=ext4 rw console=ttyAMA0,115200" 

---
## ++QEMU Advanced Example 3 :++ Run QEMU Raspberry Pi ==image from download== and ==cross compiling Raspberry Pi Linux source to get kernel and .dtb== for 32 bit ARM (still failed)

**++A :++** This example follows [Raspberry Pi Documentation - The Linux kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html#using-menuconfig)

#### Install Required Dependencies and Toolchain, in case not yet done

# To build the sources for cross-compilation, make sure you have the dependencies needed on your machine by executing:
sudo apt install git bc bison flex libssl-dev make libc6-dev libncurses5-dev

# Install the 32-bit Toolchain for a 32-bit Kernel
sudo apt install crossbuild-essential-armhf

#Install the 64-bit Toolchain for a 64-bit Kernel
sudo apt install crossbuild-essential-arm64

#### ++**Get the Kernel Sources**++

# To download the minimal source tree for the current branch, run:
git clone --depth=1 https://github.com/raspberrypi/linux
cd linux

#### ++**Build sources**++

Enter the following commands to build the sources and Device Tree files. There are several config's for ARM 32 bits or 64 bits, and different versions of kernels (refer to above article for more details). I pick this one of Pi 3+, which I am familiar with.

# For Raspberry Pi 2, 3, 3+ and Zero 2 W, and Raspberry Pi Compute Modules 3 and 3+:

KERNEL=kernel7
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2711_defconfig

#### ++**Build with Configs**++

Note
To speed up compilation on multiprocessor systems, and get some improvement on single processor ones, use `make -j n`, where n is the number of processors. You can use the `nproc` command, like `make -j $(nproc)` to see how many processors you have. Alternatively, feel free to experiment and see what works!

# For all 32-bit Builds
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs

The file locations are scattered across serveral directories:
- vmlinux : linux/vmlinux/
- Image / zImage : linux/arch/arm/boot/
- dtb : linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb

Launch `qemu` with the options from previous example.

# Download Raspberry Pi image 
wget http://downloads.raspberrypi.org/raspbian/images/raspbian-2019-09-30/2019-09-26-raspbian-buster.zip
unzip 2019-09-26-raspbian-buster.zip

qemu-system-arm \
    -M raspi2 \
    -m 1G \
    -kernel linux/arch/arm/boot/zImage \
    -dtb linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb \
    -sd 2019-09-26-raspbian-buster.img \
    -append "root=/dev/mmcblk0p2 panic=1 rootfstype=ext4 rw" \
    -serial stdio \
    -net user,hostfwd=tcp::5022-:22 -net nic

Unfortunately, it failed to boot, and stopped at

Rebooting in 1 seconds..
Reboot failed -- system halted

There are some information mentioned in [stackexchange - Kernel and QEMU : Unable to mount root fs error](https://unix.stackexchange.com/questions/61282/kernel-and-qemu-unable-to-mount-root-fs-error), which maybe worthwhile to spend some time to look into it in more details.

:::info
This error message means that the kernel is unable to mount the partition you requested to be /. This would happen for example if you gave it an empty disk image (my hunch is this is your case) - the kernel in the VM sees an unpartitioned drive, there is no /dev/sda1 just /dev/sda. To overcome this follow the instructions in the guide you have used - download a bootable ISO image and use it to install system into the VM image. When raw disk image is used, it can be directly partitioned with utilities like gdisk, fdisk or parted.

Another possibility is, that there you are trying to mount a filesystem for which the kernel doesn't have a driver. This usually happens when one uses a kernel, that has most drivers in loadable modules on initrd and the initrd isn't loaded (hence the kernel lacks the ability to understand the particular filesystem).
:::

:::info
If you made a full installation of Debian, I would expect it having installed the kernel as well. Hence it should be possible to bring up the VM without the -kernel option at all, since the VM BIOS should be able to boot the installed system right away as a real BIOS would do - by loading bootloader from MBR (or EFI partition, although the UEFI support in Qemu/KVM is still rather new AFAIK).
:::

---
## ++QEMU Advanced Example 4 :++ Run QEMU Debian image ==using .iso only== (to extract `vmlinuz` and `initrd.gz`) for 32 bit ARM

Follow this article [Easy way to run qemu Debian OS - 使用qemu-system-arm安裝系統(armhf)](https://steward-fu.github.io/website/system/debian/arm_qemu_armhf.htm) to run `qemu` from download image.

wget https://cdimage.debian.org/cdimage/archive/10.8.0/armhf/iso-cd/debian-10.8.0-armhf-xfce-CD-1.iso
sudo mkdir /mnt/rpi32
sudo mount debian-10.8.0-armhf-xfce-CD-1.iso /mnt/rpi32
cp /mnt/rpi32/install.ahf/vmlinuz .
cp /mnt/rpi32/install.ahf/initrd.gz .
sudo umount /mnt/rpi32
qemu-img create disk.qcow2 10G
qemu-system-arm \
  -m 1024 -cpu cortex-a15 -smp 2 -M virt \
  -kernel vmlinuz \
  -initrd initrd.gz \
  -drive if=none,file=disk.qcow2,id=hd0,format=raw \
  -device virtio-blk-device,drive=hd0 \
  -drive if=none,file=debian-10.8.0-armhf-xfce-CD-1.iso,id=cdrom,media=cdrom \
  -device virtio-scsi-device \
  -device scsi-cd,drive=cdrom \
  -nographic

---
# ++References++
+ [qemu Documentation](https://wiki.qemu.org/Documentation)
+ [qemu Documentation Index](https://www.qemu.org/docs/master/index.html)
+ [qemu Documentation PDF - latest](https://qemu.readthedocs.io/_/downloads/en/latest/pdf/)
+ [Suse - Running Virtual Machines with qemu-system-ARCH REPORT DOCUMENTATION - Using Devices in QEMU](https://documentation.suse.com/sles/15-SP2/html/SLES-all/cha-qemu-running.html#cha-qemu-running-devices) : Quite some useful tips to run QEMU.
+ [qemu System Invocation](https://www.qemu.org/docs/master/system/invocation.html)
+ Platform
+ [QEMU Doc - Platforms - ARM](https://wiki.qemu.org/Documentation/Platforms/ARM)
+ [Arm System emulator](https://qemu.readthedocs.io/en/latest/system/target-arm.html)
+ Images
+ [wikibooks.org - QEMU - Images](https://en.wikibooks.org/wiki/QEMU/Images)
+ [QEMU Doc - Disk Images](https://qemu-project.gitlab.io/qemu/system/images.html)
+ Networking
+ [wiki.qemu.org - QEMU Doc - Networking](https://wiki.qemu.org/Documentation/Networking)
+ [wikibooks.org - QEMU - Networking](https://en.wikibooks.org/wiki/QEMU/Networking)
+ [wikibooks.org - QEMU - Devices - Network](https://en.wikibooks.org/wiki/QEMU/Devices/Network)
+ [qemu.readthedocs.io - Network emulation](https://qemu.readthedocs.io/en/latest/system/devices/net.html)
+ [www.qemu.org - QEMU's new -nic command line option](https://www.qemu.org/2018/05/31/nic-parameter/)
+ QEMU related articles and issues
+ [How to emulate a Raspberry Pi on your PC](https://www.cnblogs.com/zhangyang/p/7575079.html)
+ [Running Raw Linux Kernel in QEMU - initial ramdisk](https://yuankun.me/posts/running-raw-linux-kernel-in-qemu/)
+ [stackexchange - no network interface in QEMU](https://unix.stackexchange.com/questions/171874/no-network-interface-in-qemu)
+ [stackoverflow - Difference between -net user and -net nic in qemu](https://stackoverflow.com/questions/22654634/difference-between-net-user-and-net-nic-in-qemu)
+ [Emuation using Qemu's native raspi2/3 machine](https://github.com/dhruvvyas90/qemu-rpi-kernel) : Best article about `qemu` application I learned so far.
+ [Using QEMU to emulate a Raspberry Pi](https://blog.agchapman.com/using-qemu-to-emulate-a-raspberry-pi/)
+ sudo qemu-system-arm: you need to run QEMU as root
+ -kernel: this is the path to the QEMU kernel we downloaded in the previous step
+ -append: here we are providing the boot args direct to the kernel, telling it where to find it's root filesytem and what type it is
+ -hda: here we're attaching the disk image itself
+ -cpu/-m: this sets the CPU type and RAM limit to match a Raspberry Pi
+ -M: this sets the machine we are emulating. versatilepb is the 'ARM Versatile/PB' machine
+ -no-reboot: just tells QEMU to exit rather than rebooting the machine
+ -serial: redirects the machine's virtual serial port to our host's stdio
-net: this configures the machine's network stack to attach a NIC, use the user-mode stack, connect the host's vnet0 TAP device to the new NIC and don't use config scripts.

+ QEMU examples programs from internet
+ [Example 1 - Building a Debian Buster QEMU image for AARCH64](https://www.diozero.com/internals/qemuaarch64.html) : Working perfectly
+ [Example 2 - Raspbian on QEMU with ARMv7](https://github.com/paulden/raspbian-on-qemu-armv7) : working fine.
+ [Example 3 - Raspberry Pi Linux kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html#using-menuconfig) : It works fine to build Raspberry Pi kernel from Linux source code. It applied to SD card, iso QEMU.
+ [Example 4 - Debian on ARM via QEMU](https://phwl.org/2021/qemu-armhf-debian/) : Tried and failed
+ [Example 5 - Debian on QEMU-emulated ARM-64 aarch64](https://phwl.org/2022/qemu-aarch64-debian/) : Tried and failed
+ [Example 6 - Emuation using Qemu's native raspi2/3 machine](https://github.com/dhruvvyas90/qemu-rpi-kernel) : Best article about `qemu` application I learned so far.
+ [Step by step guide to extract dtb files and kernel image from img file](https://github.com/dhruvvyas90/qemu-rpi-kernel/tree/master/native-emulation) : This article should help explain how to get dtb and kernel image from img file, instead of counting on other works, which is not portable. This github provide tools and documents to generate kernel images for `qemu`. It is worthwhile to spend time to study more here.
+ [Example 7 - Emulating ARM64 Raspberry Pi Image using QEMU](https://www.youtube.com/watch?v=Y-FUvi1z1aU) : A video clearly explain the procedure step by step. Though I failed to reproduce.
+ [Example 8 - RASPBERRY PI ON QEMU](https://azeria-labs.com/emulate-raspberry-pi-with-qemu/) : It is similar to the video of above, but explaned in text, iso video. And I failed to reproduce.
+ [Example 9 - 使用qemu-system-arm安裝系統(armhf)](https://steward-fu.github.io/website/system/debian/arm_qemu_armhf.htm)
+ [Example 10 - Get an ARM64 version of Ubuntu running in the QEMU emulator on Windows 10.](https://gist.github.com/billti/d904fd6124bf6f10ba2c1e3736f0f0f7) : This article explains the meaning of parameters to `qemu`.

qemu-system-aarch64 -m 2048 -cpu cortex-a72 -smp 4 -M virt -nographic -bios QEMU_EFI.fd -drive if=none,file=ubuntu-16.04-server-cloudimg-arm64-uefi1.img,id=hd0 -device virtio-blk-device,drive=hd0 -drive file=user-data.img,format=raw -device virtio-net-device,netdev=net0 -netdev user,hostfwd=tcp:127.0.0.1:2222-:22,id=net0

qemu-system-aarch64 -m 2048 -cpu cortex-a72 -smp 4 -M virt -nographic - run the ARM64 virtual platform emulator with 2GB RAM and 4 Cortex-A72 cores with no GUI support.
-bios QEMU_EFI.fd - use the firmware downloaded above.
-drive if=none,file=ubuntu-16.04-server-cloudimg-arm64-uefi1.img,id=hd0 - use the Ubuntu image file
-device virtio-blk-device,drive=hd0 - mount drive from above as a block device
-drive file=user-data.img,format=raw - use the configuration data image file
-device virtio-net-device,netdev=net0 - create a virtual network device
-netdev user,hostfwd=tcp:127.0.0.1:2222-:22,id=net0 - set up the networking stack and forward the SSH port

+ QEMU Q&A
+ [qemu-system-x86_64: drive with bus=0, unit=0 (index=0) exists 錯誤原因](https://www.twblogs.net/a/5b8af7ea2b71773b27ca2121)： When encoutered error of `qemu-system-i386: -m 16: drive with bus=0, unit=0 (index=0) exists`, fix it with `可以把參數重新用手動輸入的方式敲，不要直接複製，因爲複製過程中可能複製了一些我們沒注意到的字符導致程序不識別；`
+ [QEMU emulating Raspberry Pi using verstailepb machine?](https://www.mail-archive.com/[e-mail]/msg06163.html)
> I download the Rapsberry OS
> image
> https://downloads.raspberrypi.org/raspios_armhf/images/raspios_armhf-2021-01-12/2021-01-11-raspios-buster-armhf.zip
>
> the kernel kernel-qemu-4.19.50-buster and versatile-pb-buster.dtb
> from https://github.com/dhruvvyas90/qemu-rpi-kernel and run with
> version 5.2.0:
>
>
> $ qemu-system-arm -dtb ./versatile-pb-buster.dtb -kernel ./kernel-qemu-
> 4.19.50-buster -cpu arm1176 -m 256 -M versatilepb
+ You are attempting to boot a generic kernel with a versatile dtb on a verstailepb machine. The site you link to even has pointers towards "native-emulation" for using the -M raspi2/3 models.
The documentation also lists the current status of the emulation:
https://qemu.readthedocs.io/en/latest/system/arm/raspi.html

+ Cross compiler
+ [Cross compiling for arm or aarch64 on Debian or Ubuntu](https://jensd.be/1126/linux/cross-compiling-for-arm-or-aarch64-on-debian-or-ubuntu)
+ Raspberry Pi related
+ [Raspberry Pi Images Downloads](https://downloads.raspberrypi.org/)
+ [Michael Stapelberg’s Debian Blog](https://people.debian.org/~stapelberg/) : It seems to be a maintainer of Raspberry Pi image, and looking for someone can take over, or becomes unmaintained.
+ [Script to simulate Raspberry Pi with QEMU](https://raspberrypi.stackexchange.com/questions/89196/emulate-raspberry-pi-zero-w-with-qemu-failed-due-to-missing-dtb)
+ [Linux Q&A](https://hackmd.io/@MarconiJiang/QnA_Linux)
+ Buildroot
+ [精通嵌入式Linux 第三章：Buildroot](https://www.lotlab.org/2020/04/08/mastering-embedded-linux-part-3-buildroot/)
+ [Buildroot基本介绍](https://hceng.cn/2019/09/05/Buildroot%E7%AC%94%E8%AE%B0/)
+ [buildRoot study - 建立自己的作業系統-cntofu](https://www.cntofu.com/book/46/raspberry_pi/buildroot_study_-_jian_li_zi_ji_de_zuo_ye_xi_tong.md)
+ [buildRoot study - 建立自己的作業系統-jasonblog](https://jasonblog.github.io/note/raspberry_pi/buildroot_study_-_jian_li_zi_ji_de_zuo_ye_xi_tong.html)
+ [使用QEMU CHROOT进行固件本地调试](http://blog.nsfocus.net/qemu-chroot/)
+ Linux boot sequence
+ [Linux Bootstrap (開機載入)](https://hackmd.io/@combo-tw/Linux-%E8%AE%80%E6%9B%B8%E6%9C%83/%2F%40combo-tw%2FByYcRZjMr)
+ [Step by step guide to extract dtb files and kernel image from img file](https://github.com/dhruvvyas90/qemu-rpi-kernel/tree/master/native-emulation)
+ Raspberry Pi
+ [Stackexchange - Running Raspberry PI buildroot image in qemu](https://unix.stackexchange.com/questions/697199/running-raspberry-pi-buildroot-image-in-qemu) : After few days of googling around a lot i found the answer to my issue. The Kernel image zImage I use in the terminal command which is built by the buildroot make process is not compatible with the QEMU. To solve that I had to compile my own QEMU compatible raspberry pi kernel. I followed this link to compile my own kernel. [raspberry pi cross-compile kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html) then when I used QEMU to run a OS there was another kernel panic saying VFS mount failed. When I read the terminal outputs before that, it says not enough space on the media. So expanded the scared.img file to 1G and ran the same terminal command. Now the QEMU is running perfectly
+ [RaspberryPi.com Linux kernel](https://www.raspberrypi.com/documentation/computers/linux_kernel.html) :
+ [RaspberryPi.org Images Downloads](https://downloads.raspberrypi.org/)
+ Cross compiler
+ [Cross compiling for arm or aarch64 on Debian or Ubuntu](https://jensd.be/1126/linux/cross-compiling-for-arm-or-aarch64-on-debian-or-ubuntu)

---
[:arrow_left:Previous article - back to marconi's blog](https://marconi1964.github.io/)
[:arrow_right:Next article - QEMU simulation ith hardware board](https://hackmd.io/@MarconiJiang/qemu_hardware)
[:arrow_up:back to marconi's blog](https://marconi1964.github.io/)