# QEMU simulation with hardware boards

###### tags: `2022/10` `qemu` `qemu-architecture` `qemu-system-architecture` `hardware` `boards` `stm32` `raspberrypi`

:::info
++(2022/10/1)++ QEMU simulation for hardware boards. After completing previous examples [QEMU for Beginner and Advanced](https://hackmd.io/@MarconiJiang/qemu_beginner), moves on to next step to build image running on `qemu` and on a real hardware board. We will start with STM32 board or Raspberry Pi
++latest update on 2022/11/02++
:::

---
**Table of Contents**

[TOC]

---

So far, we tried those different images on `qemu`, next thing to try is to use the images running on real hardware boards.

---
## ++QEMU for Hardware Example 1 :++ STM32

### ++Running `qemu-system-arm` on STM32++

Besides the [official QEMU github](https://github.com/qemu/qemu) supporting platforms of netduino2, netduinoplus2, stm32vldiscovery. There is one unofficial branch of QEMU supporting Olimex STM32_P103.

The [QEMU for STM32](https://github.com/beckus/qemu_stm32) and [Olimex STM32 P103 Development Kit Demos](https://github.com/beckus/stm32_p103_demos) are maintained by [Andre Beckus](https://github.com/beckus), a branch from [official QEMU github](https://github.com/qemu/qemu) v2.1.0. It was not merged with mainstream, with the reason below stated in [beckus.github.io](http://beckus.github.io/qemu_stm32/)

>The STM32 implementation is kept in the “stm32” branch (the “master” branch contains the unmodified QEMU code). The “stm32” branch may contain commits that make the software unstable or perhaps even uncompilable. Therefore, I recommend that you use the latest tagged release that looks like this: “stm32_v0.x.x”. These are considered to be relatively stable, and at least minimally tested.

It requires a re-compile/re-installation of STM32 version of `qemu` for upcoming trials.

git clone git://github.com/beckus/qemu_stm32.git
cd qemu-stm32
./configure --enable-debug --disable-werror --target-list="arm-softmmu"
make
# I skip the following step, as I don't want to mix up this QEMU STM32 version with official QEMU. 
# So, I need to set the path to execute this STM32 QEMU version
sudo make install
cd ..             # cd to working directory
# Above complete the installation of QEMU STM32. 

# Pre-requisite for cross-compiling STM32 demo programs.
# Installing arm-none-eabi-gcc on x86 host
# Check installation details in this article
# https://stackoverflow.com/questions/73976876/e-unable-to-locate-package-arm-none-eabi-gcc
# Add apt repository then install
sudo add-apt-repository ppa:team-gcc-arm-embedded/ppa
sudo apt-get update
sudo apt-get install gcc-arm-none-eabi
sudo apt-get install gdb-arm-none-eabi 

# Next is to install demo programs, accompanying QEMU STM32.
git clone git://github.com/beckus/stm32_p103_demos.git
cd stm32_p103_demos
make

# Execution of QEMU demo programs - method 1
# assuming directory structure like this
# ~/qemu-stm32
# ~/stm32_p103_demos
#           |- demos
#                |- adc_single
#                |- blink_flash
#                |- .....
# Can try any demo programs under demos directory
# Let's try the first one adc_single as one example
cd demos/adc_single
../../../qemu_stm32-stm32/arm-softmmu/qemu-system-arm -M stm32-p103 -kernel main.bin

# Execution of QEMU demo programs - method 2
# this method is easier, though needs some makefile knowledge
# can launch qemu-system-arm directly from stm32_p103_demos makefile
# Assume you are at directory of stm32_p103_demos
cd stm32_p103_demos
make adc_single_QEMURUN
# or other demos 
make blink_flash_QEMURUN

### ++Running `qemu-system-arm` and `arm-linux-gdb` on STM-32++

++**Installation of `arm-linux-gdb`**++

`stm32_p103_demos/makefile` also support gdb. So let's try it. (I was not familiar with `gdb`. In the begining, I thought it works well with `gdb` which was installed together with `gcc` from `sudo apt install build-essential`. But not.) It needs to install `arm-linux-gdb`, so I follow this article [arm-linux-gdb除錯工具的安裝與交叉編譯gdbserver](https://www.796t.com/content/1546341685.html) for `arm-linux-gdb` installation. That requires to download `gdb` [source code](https://ftp.gnu.org/gnu/gdb/) and compile with setting for ARM.

# Install arm-linux-gdb
# Install dependency first
sudo apt-get install texinfo

# Download the gbd source code from https://ftp.gnu.org/gnu/gdb/
# I found the current gbd vresion installed is 7.11.1. To avoid any inconsistency 
# and potential risk, I decided to install arm-linux-gdb with the same version

gdb -v
> GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
> Copyright (C) 2016 Free Software Foundation, Inc.


# Start download and installation
wget https://ftp.gnu.org/gnu/gdb/gdb-7.11.1.tar.gz
tar xvf gdb-7.11.1.tar.gz
cd gdb-7.11.1/
./configure --target=arm-linux
make
sudo make install
arm-linux-gdb -v
> GNU gdb (GDB) 7.11.1
> Copyright (C) 2016 Free Software Foundation, Inc.

I am not sure if it is needed to instal `gdbserver`. Still do it anyway.

# install gcc-arm-linux-gnueabi in case not installed before
sudo apt install gcc-arm-linux-gnueabi
# install gdbserver
cd gdb-7.11.1/gdb/gdbserver
CC=arm-linux-gnueabi-gcc ./configure --host=arm-linux
make
file gdbserver
> gdbserver: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV),
> dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0,
> BuildID[sha1]=2cd185b41b022170e9d05be1ade5506b5fef8fe8, not stripped

++**Execution of `qemu` with `gdb`**++

==To work on `gdb`, it requires to open 2 `terminal` windows. Let's define the `terminal for qemu` which `qemu-system-arm` run on, and `terminal for gdb` which `arm-linux-gdb` runs on.==

Set `terminal for qemu` to the path of `stm32_p103_demos`.

# terminal for qemu
cd qemu-stm32/stm32_p103_demos

Try to understand `makefile` the command input to run `gdb`. (I learned the key word to `grep` is `QEMUDBG` because I peek the `makefile` content already.) From `cat makefile`, we can learn that we need to add `_QEMUDBG` to run `qemu` with `gdb` function.

# terminal for qemu
cat makefile | grep QEMUDBG
> QEMU_DBG_TARGETS = $(addsuffix _QEMUDBG,$(DEMOS))
> QEMU_DBG_PTY_TARGETS = $(addsuffix _QEMUDBG_PTY,$(DEMOS))
> QEMU_DBG_TEL_TARGETS = $(addsuffix _QEMUDBG_TEL,$(DEMOS))
> $(QEMU_DBG_TARGETS): %_QEMUDBG : %_ALL
> $(QEMU_DBG_PTY_TARGETS): %_QEMUDBG_PTY : %_ALL
> $(QEMU_DBG_TEL_TARGETS): %_QEMUDBG_TEL : %_ALL

Let's try the same sample program `adc_single` with `gdb` feature by command `make adc_single_QEMUDBG` under `terminal for qemu`

# terminal for qemu
stm32_p103_demos$ make adc_single_QEMUDBG
> killall -q qemu-system-arm
> makefile:145: recipe for target 'adc_single_QEMUDBG' failed
> make: [adc_single_QEMUDBG] Error 1 (ignored)
> ../qemu_stm32/arm-softmmu/qemu-system-arm -M stm32-p103 -gdb tcp::3333 -S -kernel demos/adc_single/main.bin
> STM32_UART: ADC1 clock is set to 0 Hz.
> STM32_UART: ADC1 BRR set to 0.
> STM32_UART: ADC1 Baud is set to 0 bits per sec.
> STM32_UART: ADC2 clock is set to 0 Hz.
.....
> STM32_UART: ADC1 BRR set to 0.
> STM32_UART: ADC1 Baud is set to 0 bits per sec.
> LED Off
> CLKTREE: HSI Output Change (SrcClk:None InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: HSI/2 Output Change (SrcClk:HSI InFreq:8000000 OutFreq:4000000 Mul:1 Div:2 Enabled:1)
> CLKTREE: SYSCLK Output Change (SrcClk:HSI InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: HCLK Output Change (SrcClk:SYSCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> STM32_RCC: Cortex SYSTICK frequency set to 8000000 Hz (scale set to 125).
> STM32_RCC: Cortex SYSTICK ext ref frequency set to 1000000 Hz (scale set to 1000).
> CLKTREE: PCLK1 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: PCLK2 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)

We can find in the script of `qemu-system-arm -M stm32-p103 -gdb tcp::3333 -S -kernel demos/adc_single/main.bin`, it enables gdb with port 3333. And the program `adc_single` do the intialization then halt. So we open the other `terminal for gbd` to trace how the program executes.

# terminal for gdb
cd stm32_p103_demos/demos/adc_single
arm-linux-gdb main.elf
> GNU gdb (GDB) 7.11.1
> ......
> Reading symbols from main.elf...done.
(gdb) 

Next step it to connect the `gdb` to `qemu` with command of `target remote : 3333`, using the port we learned from `qemu-system-arm` options.

# terminal for gdb
(gdb) target remote : 3333
Remote debugging using : 3333
Reset_Handler ()
    at /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/libraries/CMSIS/CM3/DeviceSupport/ST/STM32F10x/startup/gcc_ride7/startup_stm32f10x_md.s:67
67	  movs	r1, #0
(gdb) 

Since we are just trying the `gdb` function, not really doing the program debug. Let's run the `adc_single` program by using `c` or `continue` command.

# terminal for gdb
(gdb) c
Continuing.

Then, we will find the `terminal for qemu` starts running, and repeatly the cycle of `LED Off` and `LED On`. Unless we use `CTRL-c` on the`terminal for gdb` to break the execution.

# terminal for qemu
> CLKTREE: PCLK2 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> LED Off
> LED On
> LED Off
> LED On
> LED Off
> LED On

### ++Running `openocd` and `arm-linux-gdb` on hardware Olimex STM32_P103++

Previous examples in `stm32_p103_demos` also support `openocd` for programming the target hardware of stm32_p103 from Olimex. Luckily, I have two Olimex STM32-P103 boards and one ARM-USB-TINY-H JTAG debugging board on hand, so give it a try.

==For this demo, we need to have one Olimex board, and one debugger adaptor, connected between Olimex board and PC. The debugger adpator could be `JTAG` or `serial wire debugging (SWD)`, with different interface to the host (PC). This article will not cover hardware related topics in this article. Please refer to [Olimex STM-P103 development boardUser's manual](https://www.olimex.com/Products/ARM/ST/STM32-P103/resources/STM32-P103.pdf) for hardware related information. May have another article to describe the debugger board later.==

++**About `openocd`**++
[OpenOCD](https://openocd.org/pages/about.html), the Open On-Chip Debugger has been created by Dominic Rath as part of a diploma thesis at the University of Applied Sciences, FH-Augsburg. OpenOCD is a free-software tool mainly used for on-chip debugging, in-system programming and boundary-scan testing. OpenOCD supports flashing and
debugging a wide variety of platforms such as:

ARMv5 through latest ARMv8
MIPS
AVR (incomplete)
Andes
RISC-V

The full list of supported CPU types can be accessed here: [User's Guide](http://openocd.org/doc/html/CPU-Configuration.html).

++**Installation of `openocd`**++

Prerequisite 1: `git config --global http.sslVerify false` to disable git certificate temporarily.
:::danger
During `./configure --enable-maintainer-mode --enable-ftdi`, it will have below error `configure: error: jimtcl not found, run git submodule init and git submodule update` and `git submodule init && git submodule update`. After checking [Stackoverflow article](https://stackoverflow.com/questions/69645094/fatal-unable-to-access-https-xxxx-git-server-certificate-verification-fai), we issue Linux command `git config --global http.sslVerify false` to disable temporarily git certificate. After completion of installation, need to issue another Linux command `git config --global http.sslVerify true` to enable the git certificate.
:::

Prerequisite 2: Installation of `libusb` and others

# Install `libusb` is a must
sudo apt-get install libusb-1.0-0-dev libtool automake texinfo

# Newer kernels have FTDI drivers ready-to-use. If that is not your case first you need to download the drivers – you can get the FTDI drivers from their web site. Alternatively you can use apt-get likethis:
sudo apt-get install libftdi-dev libftdi1

++**Installing `openocd` with package installation (not recommended)**++

Below installation is try-and-error. ==Suggest to skip this step, and try next section of installation by source compiling.==

First, install with the following Linux `apt` command. But it produces error.

sudo apt install openocd
openocd
> ...
> Error: The specified debug interface was not found (ft2232)
> The following debug interfaces are available:
> 1: parport
> 2: dummy
> 3: ftdi
> 4: usb_blaster
> 5: jtag_vpi
> ....

Tried to see if it can be corrected by adding user to `plugdev`, but still failed.

# check the groups available in Linux system
groups 
#> abhishek adm cdrom sudo dip plugdev lpadmin sambashare kvm

sudo usermod -a -G plugdev user-name
groups user-name
#> user-name : kernel-dev adm cdrom sudo dip plugdev lpadmin sambashare

Also, according to [ARM-USB-TINY-H User Manual - 3.3.4 Driver installation in Linux](https://www.olimex.com/Products/ARM/JTAG/_resources/ARM-USB-TINY_and_TINY_H_manual.pdf), tried to create a file `/etc/udev/rules.d/olimex-arm-usb-tiny-h.rules`, by putting this single line in the file:

SUBSYSTEM=="usb", ACTION=="add", ATTRS{idProduct}=="002a", ATTRS{idVendor}=="15ba", MODE="664", GROUP="plugdev"

The `idVendor` and `idProduct` shall use the setting from below table of the hardware debugger you use.

|Table 2. Olimex OpenOCD debuggers, FTDI vendor and product IDs|||||
|:---|:---|:---|:---|:---|
||ARM-USB-TINY|ARM-USB-TINY-H|RM-USB-OCD|ARM-USB-OCD-H|
|VID (VENDOR ID)|0×15BA|0×15BA|0×15BA|0×15BA|
|PID (PRODUCT ID)|0×0004|0×002a|0×0003|0×002b|

Eventually, still gave up. And decided to try with source compiling with different setting to enable `ftdi` instead of default setting of `ft2232` in `sudo apt install openocd`.

++**Installing `openocd` with compiling from source code (recommended)**++

First, remove the package installed from previous section, if you ever tried.

sudo apt remove --auto-removal openocd

Then, follow the [ARM-USB-TINY-H User Manual - 3.3.1 Getting OpenOC](https://www.olimex.com/Products/ARM/JTAG/_resources/ARM-USB-TINY_and_TINY_H_manual.pdf) with the section describing the source code compilation.

sudo apt-get install git libtool automake texinfo
git clone http://git.code.sf.net/p/openocd/code openocd-code
cd openocd-code

# modify the option to enable ftdi, learning from previous section
./bootstrap./configure --enable-ftdi --enable-ftdi
make
sudo make install
# then it is okay to go
openocd -v
#> system output below till next #>
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
#> end of system output

So far, we will be able to launch `openocd`. However, still more setting need to modify on `stm32_p103_demos/makefile`. Within `makefile`, we can find a line of default directory for `olimex-arm-usb-tiny-h.cfg`

# OpenOCD interface file used for programming/debugging the micronctroller
OPENOCD_INTERFACE ?= interface/olimex-arm-usb-tiny-h.cfg

Since we use different option for `openocd` compilation of `--enable-ftdi`. The file `olimex-arm-usb-tiny-h.cfg` is under another directory, so to set it correctly by update that in `stm32_p103_demos/makefile`.

# OpenOCD interface file used for programming/debugging the micronctroller
OPENOCD_INTERFACE ?= interface/ftdi/olimex-arm-usb-tiny-h.cfg

:::success
Now, we complete `openocd` installation.
:::

++**Using `openocd` to run `stm32_p103_demos`**++

The `makefile` under `stm32_p103_demos` support programming Olimex STM32-P103 board through ARM-USB-TINY-H debugger board. The usage is to add `_PROG` after the demo program like example below.

# Running stm32_p103_demos with makefile
cd stm32_p103_demos
make adc_single_PROG 
#> system output below till next #>
killall -q openocd
makefile:122: recipe for target 'adc_single_PROG' failed
make: [adc_single_PROG] Error 1 (ignored)
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "program_flash demos/adc_single/main.bin"
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
program_flash
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
Info : [stm32f1x.cpu] Cortex-M3 r1p1 processor detected
Info : [stm32f1x.cpu] target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for stm32f1x.cpu on 3333
Info : Listening on port 3333 for gdb connections
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x00005a44 msp: 0x20005000
DEPRECATED! use 'adapter speed' not 'adapter_khz'
Info : device id = 0x20036410
Info : flash size = 128 KiB
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
shutdown command invoked
#>

We can see the LED on STM32-P103 board flashing now, as expected response of the demo program `stm32_p103_demos/adc_single` does.

++**Using `openocd` and `arm-linux-gdb` to run `stm32_p103_demos`**++

After successfully programming (or usually called `flash`) the [Flash ROM](https://en.wikipedia.org/wiki/Flash_memory) of target hardware, try `arm-linux-gdb` to debug the hardware via `openocd`. We need to launch two separate `terminal` screens, let me name it as `terminal for openocd` and `terminal for gdb`.

# terminal for openocd
# Use openocd to flash the Flash ROM of the target hardware
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "program_flash demos/blink_flash/main.bin"
# After successfully flashing the ROM, launch openocd again
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg
#> system output below until next #>
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
program_flash
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
Info : [stm32f1x.cpu] Cortex-M3 r1p1 processor detected
Info : [stm32f1x.cpu] target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for stm32f1x.cpu on 3333
Info : Listening on port 3333 for gdb connections
#> end of system output

Then launch a new window of `terminal for gdb`

# terminal for gdb
arm-linux-gdb -q  stm32_p103_demos/demos/blink_flash/main.elf
# or terminal for gdb, with enabling Text UI mode with -tui option
arm-linux-gdb -tui -q  stm32_p103_demos/demos/blink_flash/main.elf

(gdb)

# connect gdb with target hardware through openocd
(gdb) target extended-remote localhost:3333
# or can use short form of the following
(gdb) tar ext:3333
Remote debugging using localhost:3333
...
# Reset the target hardware and put it on halt
(gdb) monitor reset init
# This command is optional, just make sure the target will be put on halt
(gdb) monitor reset halt
...
# Run the program with the target hardware
(gdb) run
# CTRL-c to stop

Screenshot - `terminal for openocd` (left) and `terminal for arm-linux-gdb` (right)

Screenshot - `terminal for openocd` (left) and `terminal for arm-linux-gdb with -tui option`(right)

### ++**Deep Dive into `openocd` and `arm-linux-gdb` commands**++

Since we will send `openocd` commands through `arm-linux-gdb`, it is quite confusing to me with commands of the examples in the beginning. It was not clear to me which commands are `gdb` commands, and which ones are `openocd` commands. Try to summarize `openocd` and `gdb` documents, so to find the appropriate commands more easily.

++**`gdb` related documents**++
+ `gdb` official documents
+ [Debugging with GDB - ftp.gnu.org](https://ftp.gnu.org/old-gnu/Manuals/gdb/html_chapter/gdb_toc.html) : GDB manual Ninth Edition, for GDB version 5.1.1, January 2002
+ [Debugging with GDB - sourceware.org](https://sourceware.org/gdb/onlinedocs/gdb/) : This is the Tenth Edition, of Debugging with GDB: the GNU Source-Level Debugger for GDB (GDB) Version 12.1.90.20221022-git. This document includes [Section 20.3.3 Monitor Commands for gdbserver](https://sourceware.org/gdb/onlinedocs/gdb/Server.html#Monitor-Commands-for-gdbserver-1)
+ `gdb` popluar commands and usages
+ ++**`(gdb) target extended-remote localhost:3333`**++ : connect to gdbserver. From previous example, when we launch `openocd`, the system output `Info : starting gdb server for stm32f1x.cpu on 3333` `Info : Listening on port 3333 for gdb connections`, showing it starts a gdb server on port 3333. (since we use the same PC, so `localhost` is used, instead of specific ip address. And `localhost` can be omitted.)
+ ++**`(gdb) disconnect`**++ : to disconntect from gdb server.
+ ++**`(gdb) monitor reset init`**++ : Reset and intialize the target hardware.
+ ++**`(gdb) run or r`**++ : `Run` the program continueously from the beginning. To stop, press `CTRL-c`.
+ ++**`(gdb) Continue or c`**++ : `Continue` the program continueously from where it stopped. To stop, press `CTRL-c`.
+ ++**`(gdb) Step or s`**++: `Step` by step to execute the program per line instruction.
+ ++**[monitor](https://sourceware.org/gdb/onlinedocs/gdb/Server.html#Monitor-Commands-for-gdbserver-1) command in `gdb` is a magic command.**++ You can use the monitor command to send special requests to gdbserver.
+ `(gdb) monitor help` : This is a `gdb` command to list the `openocd` commands supported. More details in next section of `openocd`.
+ `gdb` examples
+ [用Open Source工具開發軟體: 新軟體開發關念 - Chapter 6. 除錯工具 - GDB Basics](http://www.study-area.org/cyril/opentools/opentools/x1253.html)

++**`openocd` Related Documents**++
+ `openocd` official documents
+ [OpenOCD Concept Index](https://openocd.org/doc/html/OpenOCD-Concept-Index.html#OpenOCD-Concept-Index)
+ [OpenOCD User's Guide](https://openocd.org/doc/html/index.html#SEC_Contents)
+ [GDB and OpenOCD](https://openocd.org/doc/html/GDB-and-OpenOCD.html) : Specific instructions of `gdb` for `openocd`.
+ `openocd` popular commands and usage
+ [monitor](https://sourceware.org/gdb/onlinedocs/gdb/Server.html#Monitor-Commands-for-gdbserver-1) command in `gdb` is a magic command, which was introduced at previous `gdb` section. Can use `monitor + openocd` commands to control `openocd` and target hardware in `gdb`.
+ `(gdb) monitor help` : This is a `gdb` command to list the `openocd` commands supported.
+ `(gdb) monitor reset {run, halt, init}` : Reset the hardware target, then take next action of `run` (run the program), `halt` (halt the CPU), `init` (Initialize the target hardware system).
+ `(gdb) monitor soft_reset_halt`: Requesting target halt and executing a soft reset. This is often used when a target cannot be reset and halted. The target, after reset is released begins to execute code. OpenOCD attempts to stop the CPU and then sets the program counter back to the reset vector. Unfortunately the code that was executed may have left the hardware in an unknown state.
+ `(gdb) monitor flash help` to know more `flash` command provided by target hardware [about OpenOCD flash commands](https://openocd.org/doc/html/Flash-Commands.html#Flash-Commands).
+ OpenOCD has different commands for NOR and NAND flash; the “flash” command works with NOR flash, while the “nand” command works with NAND flash. This partially reflects different hardware technologies: NOR flash usually supports direct CPU instruction and data bus access, while data from a NAND flash must be copied to memory before it can be used. (SPI flash must also be copied to memory before use.) However, the documentation also uses “flash” as a generic term; for example, “Put flash configuration in board-specific files”.
+ `openocd` + `gdb` examples
+ [Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging](https://stackoverflow.com/questions/5535110/stm32-gdb-openocd-commands-and-initialization-for-flash-and-ram-debugging) : One of the best answer to provide clear introduction of GDB/OpenOCD programming the ROM.
+ [Using GDB Server Monitor Commands from Eclipse GDB Console](https://mcuoneclipse.com/2018/12/01/using-gdb-server-monitor-commands-from-eclipse-gdb-console/)

### ++**Using `arm-linux-gdb` commands to `openocd` for programming Flash ROM of the target hardware**++

The `openocd` + `gdb` examples above, [Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging](https://stackoverflow.com/questions/5535110/stm32-gdb-openocd-commands-and-initialization-for-flash-and-ram-debugging), provides a clear intruction on how to program the Flash ROM of the target STM32F107 board. However, need to modify some to use it on Olimex STM32-P103 board.

Below are the script from the article. First, launch `openocd`

# terminal for openocd
openocd -f /path to scripts/olimex-arm-usb-ocd-h.cfg -f /path to targets/stm32f107.cfg -c "init"

Then, launch `gdb` on another `terminal` window.

# terminal for gdb
# launch arm-linux-gdb under shell
arm-linux-gdb
# under (gdb)
# connection to openocd
set remote hardware-breakpoint limit 6
set remote hardware-watchoint-limit 4
target remote localhost:3333
monitor halt
monitor poll

# Program the Flash ROM of the target
monitor flash probe 0
monitor flash protect 0 0 127 off
monitor reset halt
monitor stm32x mass_erase 0
monitor flash write_image STM3210CTest/test_rom.elf
monitor flash protect 0 0 127 on
disconnect
target remote localhost:3333
monitor soft_reset_halt

However, there are couples of changes need modification due to different hardware used.

+ ++**Modification 1: Identify the hardware JTAG tap and Flash ROM memory adress.**++ It can be found by `gdb` command `monitor flash probe 0`. From the system output, we identify the JTAG tap is `stm32f1x` and Flash ROM address at `0x08000000`. Write it down, and we will use it later.

    (gdb) monitor flash probe 0
    device id = 0x20036410
    flash size = 128 KiB
    flash 'stm32f1x' found at 0x08000000

+ ++**Modfication 2: Identify the numbers of blocks of Flash ROM**++ by issuing `gdb` command `monitor flash protect 0 0 127 off`. The sytem responds with `ERROR: last block must be <= 31`. So we shall modify the command to `(gdb) monitor flash protect 0 0 31 off`. ==It is strange to me though. Olimex STM32-P103 use STM32F103RB CPU with 128KB Flash ROM. According to the datasheet, it is 2KB per block. That shall be 64 blocks, instead of 32 indicated by JTAG.==

    (gdb) monitor flash protect 0 0 127 off
    ERROR: last block must be <= 31
    (gdb) monitor flash protect 0 0 31 off

+ ++**Modification 3: Using the right JTAG tap**++ from `stm32x` to `stm32f1x` as identified in previous command `monitor flash probe 0`.

    (gdb) monitor stm32f1x mass_erase 0

+ ++**Modification 4: Adding the Flash ROM memory address in the command `monitor flash write_image`, and use the absolute file path.**++ The Flash ROM memory address was identified in previous command `monitor flash probe 0`.

    (gdb) monitor flash write_image /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf 0x08000000

Good to go, after those modifications. Below are the result of the execution.

# terminal for openocd
$ openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "init"

# terminal for arm-linux-gdb
$ arm-linux-gdb -q  stm32_p103_demos/demos/blink_flash/main.elf
Reading symbols from stm32_p103_demos/demos/blink_flash/main.elf...done.
(gdb) tar ext:3333
Remote debugging using :3333
0xfffffffe in ?? ()
(gdb) monitor reset init
JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) monitor flash probe 0
device id = 0x20036410
flash size = 128 KiB
flash 'stm32f1x' found at 0x08000000
(gdb) monitor flash protect 0 0 127 off
ERROR: last block must be <= 31
(gdb) monitor flash protect 0 0 31 off
cleared protection for blocks 0 through 31 on flash bank 0
(gdb) monitor reset halt
JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) monitor stm32f1x mass_erase 0
stm32x mass erase complete
(gdb) monitor flash write_image /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf 0x08000000
wrote 23728 bytes from file /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf in 1.060053s (21.859 KiB/s)
(gdb) monitor flash protect 0 0 31 on
set protection for blocks 0 through 31 on flash bank 0
(gdb) disconnect
Ending remote debugging.
(gdb) tar ext:3333
Remote debugging using :3333
0xfffffffe in ?? ()
(gdb) monitor soft_reset_halt
[stm32f1x.cpu] requesting target halt and executing a soft reset
[stm32f1x.cpu] halted due to breakpoint, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf 

:::success
You can see Olimex STM32-P103 LED blinking now.
:::

---
## ++QEMU for Hardware Example 2 :++ Embedded Linux from Scratch in 45 minutes, on RISC-V

This example follows the [Youtube - Embedded Linux from Scratch in 45 minutes, on RISC-V](https://www.youtube.com/watch?v=cIkTh3Xp3dA), and [Presentation pdf](https://bootlin.com/pub/conferences/2021/fosdem/opdenacker-embedded-linux-45minutes-riscv/opdenacker-embedded-linux-45minutes-riscv.pdf).

{%youtube cIkTh3Xp3dA %}

Below is the script of the whole process. That’s possible to compile and assemble in less than 45 minutes!

Things to build
▶ Cross-compiling toolchain: Buildroot 2020.11.1
▶ Firmware / first stage bootloader: OpenSBI
▶ Bootloader: U-Boot 2021.01
▶ Kernel: Linux 5.11-rc3
▶ Root filesystem and application: BusyBox 1.33.0

### ++**Starting OpenSBI (from u-boot with Buildroot) in QEMU**++

# Generating a RISC-V musl toolchain with Buildroot
cd
mkdir riscv -p
cd riscv
wget https://buildroot.org/downloads/buildroot-2020.11.1.tar.gz
tar xvf buildroot-2020.11.1.tar.gz
cd buildroot-2020.11.1
make menuconfig
# In ▶ Target options ▶ Target Architecture, choose ▶ RISCV
# In ▶ Toolchain ▶ C library, choose ▶ musl.
# ▶ Save your configuration
make sdk
# At the end, you have an toolchain archive
ls -l output/images/riscv64-buildroot-linux-musl_sdk-buildroot.tar.gz
# Extract the archive in a suitable directory, and in the extracted directory, run: ./relocate-sdk.sh
mkdir ~/toolchain
cd ~/toolchain
tar xvf ../riscv/buildroot-2020.11.1/output/images/riscv64-buildroot-linux-musl_sdk-buildroot.tar.gz
cd riscv64-buildroot-linux-musl_sdk-buildroot
# Check the riscv64-linux-gcc under bin/
ls -al bin
./relocate-sdk.sh
#> Relocating the buildroot SDK from /home/kernel-dev/riscv/buildroot-2020.11.1/output/host to /home/kernel-dev/toolchain/riscv64-buildroot-linux-musl_sdk-buildroot ...
cd ~/riscv
# set up path for riscv64-linux-gcc
export PATH=$HOME/toolchain/riscv64-buildroot-linux-musl_sdk-buildroot/bin:$PATH
# create hello.c
vim hello.c
riscv64-linux-gcc -static -o hello hello.c
file hello
sudo apt install -y qemu-user
qemu-riscv64 ./hello

# Hardware emulator section
# Install qemu-system-riscv64
sudo apt install qemu-system-misc
qemu-system-riscv64 -M ?

# U-Boot bootloader
# Set up environment for U-Boot cross-compiling
# Download U-Boot 2021.01 sources
cd ~/riscv
wget https://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2
tar xvf u-boot-2021.01.tar.bz2
cd u-boot-2021.01
# Add an environment variable for cross-compiling
export CROSS_COMPILE=riscv64-linux-
# Find U-Boot ready-made configurations for RISC-V:
ls configs | grep riscv
# We will choose the configuration for QEMU and U-Boot running in S Mode:
make qemu-riscv64_smode_defconfig
# Now let’s compile U-Boot (-j$(nproc): Use the variable $(nproc) to compile jobs in parallel)
make -j$(nproc)

# Firmware - OpenSBI: Open Supervisor Binary Interface (based on u-boot)
# Required to start an OS (S mode) from the Supervisor/Firmware (M mode)
cd ~/riscv
git clone https://github.com/riscv/opensbi.git
cd opensbi
git checkout v0.8
make PLATFORM=generic FW_PAYLOAD_PATH=../u-boot-2021.01/u-boot.bin
# Run the above command every time you update U-Boot
# This generates the build/platform/generic/firmware/fw_payload.elf file which is a binary that QEMU can boot.
ls build/platform/generic/firmware/fw_payload.elf

# Starting OpenSBI (from u-boot) in QEMU
cd ~/riscv
qemu-system-riscv64 -m 2G \
-nographic \
-machine virt \
-smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf
#> System output until next #>
OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : riscv-virtio,qemu
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv64imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 148 KB
Runtime SBI Version : 0.2

MIDELEG : 0x0000000000000222
MEDELEG : 0x000000000000b109
PMP0    : 0x0000000080000000-0x000000008003ffff (A)
PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)


U-Boot 2021.01 (Oct 31 2022 - 21:01:02 +0800)

CPU:   rv64imafdcsu
Model: riscv-virtio,qemu
DRAM:  2 GiB
In:    uart@10000000
Out:   uart@10000000
Err:   uart@10000000
Net:   No ethernet found.
Hit any key to stop autoboot:  0 

Device 0: unknown device
scanning bus for devices...

Device 0: unknown device
No ethernet found.
No ethernet found.
=> 
#> end of system output

# Exit QEMU with [Ctrl][a] followed by [x]

### ++**Preparing Linux kernel for OpenSBI (from u-boot with Buildroot) in QEMU**++

#### ++**Try and Error with Linux kernel**++
Download and install Linux kernel, but ==kernel panic==, as expected. :stuck_out_tongue_winking_eye:

cd ~/riscv
wget https://git.kernel.org/torvalds/t/linux-5.11-rc3.tar.gz
tar xvf linux-5.11-rc3.tar.gz
cd linux-5.11-rc3
# Add two environment variables for kernel cross-compiling 
# ▶ CROSS_COMPILE is the cross-compiler prefix, as our cross-compiler is riscv64-linux-gcc.
# ▶ ARCH is the name of the subdirectory in arch/ corresponding to the target architecture.
export CROSS_COMPILE=riscv64-linux-
export ARCH=riscv

# Lets take the default Linux kernel configuration for RISCV:
make help | grep defconfig
make defconfig
# ▶ We can now further customize the onfiguration:
make menuconfig
make -j$(nproc)

# At the end, you have these files:
# vmlinux: raw kernel in ELF format (not bootable, for debugging)
# arch/riscv/boot/Image: uncompressed bootable kernel
# arch/riscv/boot/Image.gz: compressed kernel

# Firmware - OpenSBI: Open Supervisor Binary Interface (based on Linux kernel)

cd ../opensbi
make PLATFORM=generic FW_PAYLOAD_PATH=../linux-5.11-rc3/arch/riscv/boot/Image
cd ..
qemu-system-riscv64 -m 2G \
-nographic \
-machine virt \
-smp 8 \
-kernel opensbi/build/platform/generic/firmware/fw_payload.elf \
-append "console=ttyS0" 
#> system output below till next #>
......
[    2.319112] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    2.324103] Please append a correct "root=" boot option; here are the available partitions:
[    2.327364] DEBUG_BLOCK_EXT_DEVT is enabled, you need to specify explicit textual name for "root=" boot option.
[    2.331364] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    2.334303] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.11.0-rc3 #1
[    2.339795] Call Trace:
[    2.341004] [<ffffffe000004614>] walk_stackframe+0x0/0xaa
[    2.343770] [<ffffffe0006aec96>] show_stack+0x32/0x3e
[    2.346145] [<ffffffe0006b14da>] dump_stack+0x74/0x8e
[    2.349506] [<ffffffe0006aee2c>] panic+0xfc/0x2b2
[    2.352511] [<ffffffe00080124c>] mount_block_root+0x1c6/0x262
[    2.354797] [<ffffffe0008013fc>] mount_root+0x114/0x13e
[    2.356841] [<ffffffe00080156a>] prepare_namespace+0x144/0x186
[    2.360358] [<ffffffe000800e22>] kernel_init_freeable+0x1ce/0x1ea
[    2.364176] [<ffffffe0006b8a3a>] kernel_init+0x12/0xf8
[    2.366129] [<ffffffe0000032be>] ret_from_exception+0x0/0xc
[    2.370044] SMP: stopping secondary CPUs
[    2.376382] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---
#>

#### ++**Set up storage image to boot the Linux kernel from U-Boot**++
+ We want to show how to set the U-Boot environment to load the Linux kernel and to specify the Linux kernel command line
+ For this purpose, we will need some storage space to store the U-Boot environment, load the kernel binary, and also to contain the filesystem that Linux will boot on.

# Let’s create a 128 MB disk image:
dd if=/dev/zero of=disk.img bs=1M count=128
# Let’s create two partitions in this image 
cfdisk disk.img
# "Select label type" ▶ DOS
# ▶ "New" - "Partition Size" : 64M ▶ Primary ▶ Bootable ▶ "Type"  : c W95 FAT32 (LBA) 
# Move cursor down to "Free space"
# ▶ "New" : 63M (remaining space) ▶ Primary ▶ "Partition type" is aotumatically selected "Linux 83" ▶ "Write" partition table, and key in "yes" ▶ "Quit" 
# Now we have disk.img 
# ▶ A first 64 MB primary partition (type W95 FAT32 (LBA)), marked as bootable
# ▶ A second partition with remaining space (default type: Linux)
fdisk -lu disk.img
#> System output below until next #>
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe340e0ae

Device     Boot  Start    End Sectors Size Id Type
disk.img1  *      2048 133119  131072  64M  c W95 FAT32 (LBA)
disk.img2       133120 262143  129024  63M 83 Linux
#> system output ends here

# Let’s access the partitions in this disk image:
sudo losetup -f --show --partscan disk.img
#> /dev/loop0        # check the loop(n) response from the system, n could be 0 or 1 or 2, any number
ls /dev/loop0*
# We can now format the partitions:
sudo mkfs.vfat -F 32 -n boot /dev/loop0p1
#> mkfs.fat 3.0.28 (2015-05-16)
#> mkfs.fat: warning - lowercase labels might not work properly with DOS or Windows
#> unable to get drive geometry, using default 255/63

sudo mkfs.ext4 -L rootfs /dev/loop0p2
#> mke2fs 1.42.13 (17-May-2015)
#> Discarding device blocks: done                            
#> Creating filesystem with 64512 1k blocks and 16128 inodes
#> Filesystem UUID: ff0bd56e-1dca-4ee8-9187-aa053daced34
#> Superblock backups stored on blocks: 
#> 	8193, 24577, 40961, 57345

#> Allocating group tables: done                            
#> Writing inode tables: done                            
#> Creating journal (4096 blocks): done
#> Writing superblocks and filesystem accounting information: done

# Let’s create a mount point for the FAT partition:
sudo mkdir -p /mnt/boot
# Let’s mount it:
sudo mount /dev/loop0p1 /mnt/boot
# Let’s copy the kernel image to it:
sudo cp linux-5.11-rc3/arch/riscv/boot/Image /mnt/boot
# And then unmount the filesystem to commit changes:
sudo umount /mnt/boot

# We want U-Boot be able to use an environment in a FAT partition on a virtio disk. 
# So, let’s reconfigure U-Boot with the following settings
cd ~/riscv/u-boot-2021.01
make menuconfig
# Go to Environment setting page
# ▶ CONFIG_ENV_IS_IN_FAT=y
# ▶ CONFIG_ENV_FAT_INTERFACE="virtio"
# ▶ CONFIG_ENV_FAT_DEVICE_AND_PART="0:1"
make -j$(nproc)
cd ../opensbi/
make PLATFORM=generic FW_PAYLOAD_PATH=../u-boot-2021.01/u-boot.bin
cd ..
qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0
#> System output below until next #>
OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : riscv-virtio,qemu
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 3
Boot HART ISA       : rv64imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 148 KB
Runtime SBI Version : 0.2
......
U-Boot 2021.01 (Nov 01 2022 - 14:54:15 +0800)

CPU:   rv64imafdcsu
Model: riscv-virtio,qemu
DRAM:  2 GiB
Loading Environment from FAT... OK
In:    uart@10000000
Out:   uart@10000000
Err:   uart@10000000
Net:   No ethernet found.
Hit any key to stop autoboot:  0 

Device 0: QEMU VirtIO Block Device
            Type: Hard Disk
            Capacity: 128.0 MB = 0.1 GB (262144 x 512)
... is now current device
scanning bus for devices...

Device 0: unknown device
No ethernet found.
No ethernet found.
=>
#>
# Below commands are entered under OpenSBI
setenv foo bar
saveenv
# Exit QEMU with [Ctrl][a] followed by [x], and re-enter QEMU / OpenSBI
printenv foo

#### ++**setenv bootargs and bootcmd to boot Linux**++

cd ~/riscv
qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0

# below commands are entered under OpenSBI
setenv bootargs 'root=/dev/vda2 rootwait console=ttyS0 earlycon=sbi rw'
setenv bootcmd 'fatload virtio 0:1 84000000 Image; booti 0x84000000 - ${fdtcontroladdr}'
saveenv
boot
#> System output until next #> a long one
19804160 bytes read in 162 ms (116.6 MiB/s)
Moving Image from 0x84000000 to 0x80200000, end=81530000
## Flattened Device Tree blob at ff7475b0
   Booting using the fdt blob at 0xff7475b0
   Using Device Tree in place at 00000000ff7475b0, end 00000000ff74be1d

Starting kernel ...

[    0.000000] Linux version 5.11.0-rc3 (kernel-dev@ubuntu-vm) (riscv64-linux-gcc.br_real (Buildroot 2020.11.1) 9.3.0, GNU ld (GNU Binutils) 2.34) #1 SMP Mon Oct 31 22:19:30 CST 2022
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] software IO TLB: mapped [mem 0x00000000f9b47000-0x00000000fdb47000] (64MB)
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acdfimsu
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 17 pages/cpu s32488 r8192 d28952 u69632
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516615
[    0.000000] Kernel command line: root=/dev/vda2 rootwait console=ttyS0 earlycon=sbi rw
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 1977192K/2095104K available (6902K kernel code, 4904K rwdata, 4096K rodata, 2147K init, 305K bss, 117912K reserved, 0K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]       fixmap : 0xffffffcefee00000 - 0xffffffceff000000   (2048 kB)
[    0.000000]       pci io : 0xffffffceff000000 - 0xffffffcf00000000   (  16 MB)
[    0.000000]      vmemmap : 0xffffffcf00000000 - 0xffffffcfffffffff   (4095 MB)
[    0.000000]      vmalloc : 0xffffffd000000000 - 0xffffffdfffffffff   (65535 MB)
[    0.000000]       lowmem : 0xffffffe000000000 - 0xffffffe07fe00000   (2046 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU debug extended QS entry/exit.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] plic: interrupt-controller@c000000: mapped 53 interrupts with 8 handlers for 16 contexts.
[    0.000000] random: get_random_bytes called from start_kernel+0x3ae/0x580 with crng_init=0
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [3]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000107] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[    0.005357] Console: colour dummy device 80x25
[    0.010911] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[    0.013346] pid_max: default: 32768 minimum: 301
[    0.015643] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.016532] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.076973] rcu: Hierarchical SRCU implementation.
[    0.079447] EFI services will not be available.
[    0.084805] smp: Bringing up secondary CPUs ...
[    0.107909] smp: Brought up 1 node, 8 CPUs
[    0.178760] devtmpfs: initialized
[    0.193667] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.196196] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.204203] NET: Registered protocol family 16
[    0.351072] vgaarb: loaded
[    0.353865] SCSI subsystem initialized
[    0.358430] usbcore: registered new interface driver usbfs
[    0.359476] usbcore: registered new interface driver hub
[    0.360719] usbcore: registered new device driver usb
[    0.430208] clocksource: Switched to clocksource riscv_clocksource
[    0.504310] NET: Registered protocol family 2
[    0.511287] tcp_listen_portaddr_hash hash table entries: 1024 (order: 3, 40960 bytes, linear)
[    0.512721] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.514451] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    0.516448] TCP: Hash tables configured (established 16384 bind 16384)
[    0.521106] UDP hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.522906] UDP-Lite hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.526330] NET: Registered protocol family 1
[    0.535205] RPC: Registered named UNIX socket transport module.
[    0.536965] RPC: Registered udp transport module.
[    0.539670] RPC: Registered tcp transport module.
[    0.540269] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.542282] PCI: CLS 0 bytes, default 64
[    0.554499] workingset: timestamp_bits=62 max_order=19 bucket_order=0
[    0.575618] NFS: Registering the id_resolver key type
[    0.577296] Key type id_resolver registered
[    0.577990] Key type id_legacy registered
[    0.579024] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.582410] 9p: Installing v9fs 9p2000 file system support
[    0.586787] NET: Registered protocol family 38
[    0.587686] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.588624] io scheduler mq-deadline registered
[    0.589768] io scheduler kyber registered
[    0.601836] pci-host-generic 30000000.pci: host bridge /soc/pci@30000000 ranges:
[    0.603496] pci-host-generic 30000000.pci:       IO 0x0003000000..0x000300ffff -> 0x0000000000
[    0.604630] pci-host-generic 30000000.pci:      MEM 0x0040000000..0x007fffffff -> 0x0040000000
[    0.607574] pci-host-generic 30000000.pci: ECAM at [mem 0x30000000-0x3fffffff] for [bus 00-ff]
[    0.610581] pci-host-generic 30000000.pci: PCI host bridge to bus 0000:00
[    0.611751] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.612362] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.613252] pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
[    0.617806] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000
[    1.002980] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.015206] printk: console [ttyS0] disabled
[    1.017049] 10000000.uart: ttyS0 at MMIO 0x10000000 (irq = 2, base_baud = 230400) is a 16550A
[    1.025299] printk: console [ttyS0] enabled
[    1.025299] printk: console [ttyS0] enabled
[    1.051860] printk: bootconsole [sbi0] disabled
[    1.051860] printk: bootconsole [sbi0] disabled
[    1.057663] [drm] radeon kernel modesetting enabled.
[    1.143809] loop: module loaded
[    1.224179] virtio_blk virtio0: [vda] 262144 512-byte logical blocks (134 MB/128 MiB)
[    1.270068]  vda: vda1 vda2
[    1.282726] libphy: Fixed MDIO Bus: probed
[    1.286115] e1000e: Intel(R) PRO/1000 Network Driver
[    1.286525] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.287430] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.287951] ehci-pci: EHCI PCI platform driver
[    1.289161] ehci-platform: EHCI generic platform driver
[    1.290024] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.316459] ohci-pci: OHCI PCI platform driver
[    1.317678] ohci-platform: OHCI generic platform driver
[    1.320310] usbcore: registered new interface driver uas
[    1.321076] usbcore: registered new interface driver usb-storage
[    1.323645] mousedev: PS/2 mouse device common for all mice
[    1.328237] goldfish_rtc 101000.rtc: registered as rtc0
[    1.334751] goldfish_rtc 101000.rtc: setting system clock to 2022-11-01T07:34:33 UTC (1667288073)
[    1.341077] syscon-poweroff poweroff: pm_power_off already claimed for sbi_shutdown
[    1.342770] syscon-poweroff: probe of poweroff failed with error -16
[    1.344875] usbcore: registered new interface driver usbhid
[    1.345528] usbhid: USB HID core driver
[    1.349055] NET: Registered protocol family 10
[    1.360726] Segment Routing with IPv6
[    1.362869] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    1.367194] NET: Registered protocol family 17
[    1.372807] 9pnet: Installing 9P2000 support
[    1.374920] Key type dns_resolver registered
[    1.376191] debug_vm_pgtable: [debug_vm_pgtable         ]: Validating architecture page table helpers
[    1.486949] EXT4-fs (vda2): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[    1.490060] ext4 filesystem being mounted at /root supports timestamps until 2038 (0x7fffffff)
[    1.492075] VFS: Mounted root (ext4 filesystem) on device 254:2.
[    1.496181] devtmpfs: error mounting -2
[    1.536872] Freeing unused kernel memory: 2144K
[    1.538627] Run /sbin/init as init process
[    1.539526] Run /etc/init as init process
[    1.539997] Run /bin/init as init process
[    1.540609] Run /bin/sh as init process
[    1.541191] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
[    1.542603] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.11.0-rc3 #1
[    1.542994] Call Trace:
[    1.543260] [<ffffffe000004614>] walk_stackframe+0x0/0xaa
[    1.543704] [<ffffffe0006aec96>] show_stack+0x32/0x3e
[    1.543958] [<ffffffe0006b14da>] dump_stack+0x74/0x8e
[    1.544416] [<ffffffe0006aee2c>] panic+0xfc/0x2b2
[    1.544891] [<ffffffe0006b8b14>] kernel_init+0xec/0xf8
[    1.545291] [<ffffffe0000032be>] ret_from_exception+0x0/0xc
[    1.545860] SMP: stopping secondary CPUs
[    1.547075] ---[ end Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. ]---
#>

:::info
It runs, and fails. As expected, it mounts the root file system, but it fails in executing any process, because we haven't feeded the root file system yet.
:::

#### ++**Bulding the root filesystem (using Busybox)**++

cd ~/riscv
wget https://busybox.net/downloads/busybox-1.33.0.tar.bz2
tar xvf busybox-1.33.0.tar.bz2
cd busybox-1.33.0
make allnoconfig
# Starts with no applet selected
make menuconfig
# ▶ In Settings →Build Options, enable Build static binary (no shared libs)
# ▶ In Settings →Build Options, set Cross compiler prefix to riscv64-linux-
# ▶ Then enable support for the following commands: ash, init, halt, mount, cat, mkdir, echo, ls, chmod, uptime, vi, ifconfig, httpd
# Check https://asciinema.org/a/384727 for playback on how to select those commands
make install
tree _install/
#> System output until next #>
_install/
├── bin
│   ├── ash -> busybox
│   ├── busybox
│   ├── cat -> busybox
│   ├── echo -> busybox
│   ├── ls -> busybox
│   ├── mkdir -> busybox
│   ├── mount -> busybox
│   ├── sh -> busybox
│   └── vi -> busybox
├── sbin
│   ├── halt -> ../bin/busybox
│   ├── ifconfig -> ../bin/busybox
│   └── init -> ../bin/busybox
└── usr
    ├── bin
    │   └── uptime -> ../../bin/busybox
    └── sbin
        └── httpd -> ../../bin/busybox

5 directories, 14 files
#>

# Installing to the root filesystem:
# We need to create a dev directory. The devtmpfs filesystem will automatically be mounted there (as CONFIG_DEVTMPFS_MOUNT=y)
sudo mkdir /mnt/rootfs
sudo mount /dev/loop0p2 /mnt/rootfs
sudo rsync -aH _install/ /mnt/rootfs/
sudo mkdir /mnt/rootfs/dev
sudo mkdir /mnt/rootfs/proc
sudo mkdir /mnt/rootfs/sys
sudo umount /mnt/rootfs

qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0

# Let’s also create /proc and /sys so that we can also mount the proc and sysfs virtual filesystems on the target:
# below commands are entered under QEMU
mount
#> mount: no /proc/mounts
mount -t proc nodev /proc
mount -t sysfs nodev /sys
mount
#> /dev/root on / type ext4 (rw,relatime)
#> devtmpfs on /dev type devtmpfs (rw,relatime,size=988596k,nr_inodes=247149,mode=755)
#> nodev on /proc type proc (rw,relatime)
#> nodev on /proc type proc (rw,relatime)
#> nodev on /sys type sysfs (rw,relatime)

mkdir /etc
vi /etc/inittab
# input below 4 lines in vi
# This is run first script:
::sysinit:/etc/init.d/rcS
# Start an "askfirst" shell on the console:
::askfirst:/bin/sh
# Press :wq! to save and quit vi
mkdir /etc/init.d
vi /etc/init.d/rcS
# input below 3 lines in vi
#!/bin/sh
mount -t proc nodev /proc
mount -t sysfs nodev /sys
# Press :wq! to save and quit vi
chmod +x /etc/init.d/rcS

#### ++**Add support for networking**++

# add networking in QEMU
sudo qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-netdev tap,id=tapnet,ifname=tap2,script=no,downscript=no \
-device virtio-net-device,netdev=tapnet

# On the target machine:
ifconfig -a
ifconfig eth0 192.168.2.100
# On the host machine:
ifconfig -a
sudo ifconfig tap2 192.168.2.1
ping 192.168.2.100

# On the target machine:
# Store below contents in /www/cgi-bin/uptime and make it executable.
#!/bin/sh
echo "Content-type: text/html"
echo
echo "<html>"
echo "<meta http-equiv=\"refresh\" content=\"1\">"
echo "<header></header><body>"
echo "<h1>Uptime information</h1>"
echo "Your embedded device has been running for:<pre><font color=Blue>"
echo `uptime`
echo "</font></pre>"
echo "</body></html>"
# then execute in target machine:
/usr/sbin/httpd -h /www

# On the host machine, open in your browser: (make sure NOT https://)
http://192.168.2.100/cgi-bin/uptime

---
## ++References++
+ Check the References of [QEMU for Beginner and Advanced](https://hackmd.io/@MarconiJiang/qemu_beginner), as the References listed here is more related to hardware.
+ QEMU related to hardware
+ [Building a Linux system for the STM32MP1: basic system](https://bootlin.com/blog/building-a-linux-system-for-the-stm32mp1-basic-system/)
+ [Youtube -EmbeddedCraft - Using QEMU to Emulate Cortex M based board and Running LED Blinking Program](https://www.youtube.com/watch?v=Zvbarf1CSGs)
+ [Youtube -EmbeddedCraft - LED Blinking for STM32 Board on QEMU](https://www.youtube.com/watch?v=bS4j8VSeRrE)
+ [QEMU intro ppt by jserv](https://www.slideshare.net/jserv/arm-and-soc-traning-part-i-overview)
+ Olimex STM32_P103 and ARM-USB-TINY-H hardware related
+ [STM32-P103 - Header board for STM32F103RBT6 CORTEX M3 ARM microcontroller](https://www.olimex.com/Products/ARM/ST/STM32-P103/)
+ [STM-P103 development boardUser's manual](https://www.olimex.com/Products/ARM/ST/STM32-P103/resources/STM32-P103.pdf)
+ [ARM-USB-TINY-H OLIMEX OPENOCD ARM JTAG DEBUGGER](https://www.olimex.com/Products/ARM/JTAG/_resources/ARM-USB-TINY_and_TINY_H_manual.pdf)
+ OpenOCD official documents
+ [OpenOCD Concept Index](https://openocd.org/doc/html/OpenOCD-Concept-Index.html#OpenOCD-Concept-Index)
+ [OpenOCD User's Guide](https://openocd.org/doc/html/index.html#SEC_Contents)
+ [GDB and OpenOCD](https://openocd.org/doc/html/GDB-and-OpenOCD.html) : Specific instructions of `gdb` for `openocd`.
+ [OpenOCD General Commands](https://openocd.org/doc/html/General-Commands.html)
+ [GDB and OpenOCD](https://openocd.org/doc/html/GDB-and-OpenOCD.html) : Specific instructions of `gdb` for `openocd`
+ `gdb` and `openocd` examples
+ [Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging](https://stackoverflow.com/questions/5535110/stm32-gdb-openocd-commands-and-initialization-for-flash-and-ram-debugging) : One of the best answer to provide clear introduction of GDB/OpenOCD programming the ROM.
+ [Upload Code to STM32L4, Using Linux, GNU Make, and OpenOCD](https://www.hackster.io/yusefkarim/upload-code-to-stm32l4-using-linux-gnu-make-and-openocd-a3d4de)
+ [GDB example on x86 - 透過 GDB 進行遠端除錯 by jserv](https://hackmd.io/@sysprog/gdb-example)
+ [使用 gdb+qemu 來執行/除錯 raspberry pi linux kernel](https://descent-incoming.blogspot.com/2017/03/raspberry-pix86-linux-kernel-debug-by.html)
+ GDB official documents and examples
+ [Debugging with GDB - ftp.gnu.org](https://ftp.gnu.org/old-gnu/Manuals/gdb/html_chapter/gdb_toc.html) : GDB manual Ninth Edition, for GDB version 5.1.1, January 2002
+ [Debugging with GDB - sourceware.org](https://sourceware.org/gdb/onlinedocs/gdb/) : This is the Tenth Edition, of Debugging with GDB: the GNU Source-Level Debugger for GDB (GDB) Version 12.1.90.20221022-git. This document includes [Section 20.3.3 Monitor Commands for gdbserver](https://sourceware.org/gdb/onlinedocs/gdb/Server.html#Monitor-Commands-for-gdbserver-1)
+ `gdb` examples
+ [用Open Source工具開發軟體: 新軟體開發關念 - Chapter 6. 除錯工具 - GDB Basics (and Advanced)](http://www.study-area.org/cyril/opentools/opentools/x1253.html) : Check link above for OpenOCD specific commands for GDB.
+ [GDB實用教學：自動化你的debug](https://jasonblog.github.io/note/gdb/gdbshi_yong_jiao_xue_ff1a_zi_dong_hua_ni_de_debug.html) : `gdb -tui` or `cgdb` for Text UI or GUI operation.
+ QEMU STM32 sample programs
+ Sample program #1 - [QEMU for STM32](https://github.com/beckus/qemu_stm32) and [Olimex STM32 P103 Development Kit Demos](https://github.com/beckus/stm32_p103_demos) are maintained by [Andre Beckus](https://github.com/beckus)
+ Sample program #2 - [qemu的STM32虛擬化環境](https://www.796t.com/article.php?id=245873) : Good entry for a beginning to know how to set up STM32 QEMU environment, a Chinese translation of [QEMU for STM32](https://github.com/beckus/qemu_stm32). There is a minor mistake, though. The `./configure` should be removed.

    git clone git://github.com/beckus/stm32_p103_demos.git
    cd stm32_p103_demos
    ./configure           # remove this line
    make

+ Sample program #3 - [NCKU Jserv Embedded Courses](http://wiki.csie.ncku.edu.tw/embedded/schedule), also based on [QEMU for STM32](https://github.com/beckus/qemu_stm32)
+ [NCKU Jserv Embedded Course Lab 1](http://wiki.csie.ncku.edu.tw/embedded/Lab1)
+ Sample program #4 - [Building a Linux system for the STM32MP1: basic system by bootlin](https://bootlin.com/blog/building-a-linux-system-for-the-stm32mp1-basic-system/)
+ STM32 official documents
+ STM32 hardware related documents
+ [STM32F103 introductory page](https://www.st.com/en/microcontrollers-microprocessors/stm32f103.html)
+ [STM32F103x8 and xB spec](https://www.st.com/resource/en/datasheet/stm32f103c8.pdf) :
+ STM32F103's flash memory is NOR type. Check this article - [Flash memory type for STM32F103C8T6](https://community.st.com/s/question/0D50X0000AeYj6VSQS/flash-memory-type-for-stm32f103c8t6).
+ [About OpenOCD flash commands](https://openocd.org/doc/html/Flash-Commands.html#Flash-Commands) : OpenOCD has different commands for NOR and NAND flash; the “flash” command works with NOR flash, while the “nand” command works with NAND flash. This partially reflects different hardware technologies: NOR flash usually supports direct CPU instruction and data bus access, while data from a NAND flash must be copied to memory before it can be used. (SPI flash must also be copied to memory before use.) However, the documentation also uses “flash” as a generic term; for example, “Put flash configuration in board-specific files”.
+ STM32 programming related documents
+ [RM0008 Reference manual STM32F101xx, STM32F102xx, STM32F103xx, STM32F105xx and STM32F107xx advanced Arm®-based 32-bit MCUs](https://www.st.com/resource/en/reference_manual/cd00171190-stm32f101xx-stm32f102xx-stm32f103xx-stm32f105xx-and-stm32f107xx-advanced-arm-based-32-bit-mcus-stmicroelectronics.pdf)
+ [STM32F103 memory map - Olimex STM32-P103](https://www.olimex.com/Products/ARM/ST/STM32-P103/resources/STM32-P103.pdf)
+ [PM0056 Programming manual-STM32F10xxx/20xxx/21xxx/L1xxxx Cortex®-M3 programming manua](https://www.st.com/resource/en/programming_manual/pm0056-stm32f10xxx20xxx21xxxl1xxxx-cortexm3-programming-manual-stmicroelectronics.pdf)
+ [PM0075 Programming manual STM32F10xxx Flash memory microcontrollers](https://www.st.com/resource/en/programming_manual/pm0075-stm32f10xxx-flash-memory-microcontrollers-stmicroelectronics.pdf)
+ [AN2606 STM32 microcontroller system memory boot mode](https://www.st.com/resource/en/application_note/cd00167594-stm32-microcontroller-system-memory-boot-mode-stmicroelectronics.pdf)
+ [STM32 MPU GDB commands](https://wiki.st.com/stm32mpu/wiki/GDB_commands)
+ Flash related topics - hardware, firmware, and software
+ [成大資工 Wiki - Flash memory: from IC, hardware, timing to programming flash](http://wiki.csie.ncku.edu.tw/embedded/Flash)
+ [STM32学习笔记：FLASH读写之一](https://blog.csdn.net/thebestleo/article/details/109761000)

---
[:arrow_left:Previous article - QEMU for Beginner and Advanced](https://hackmd.io/@MarconiJiang/qemu_beginner)
[:arrow_right:Next article - gdb Introduction](https://hackmd.io/@MarconiJiang/gdb)
[:arrow_up:back to marconi's blog](https://marconi1964.github.io/)