QEMU simulation with hardware boards

tags: `2022/10` `qemu` `qemu-architecture` `qemu-system-architecture` `hardware` `boards` `stm32` `raspberrypi`

(2022/10/1) QEMU simulation for hardware boards. After completing previous examples QEMU for Beginner and Advanced, moves on to next step to build image running on qemu and on a real hardware board. We will start with STM32 board or Raspberry Pi
latest update on 2022/11/02

Table of Contents

QEMU simulation with hardware boards

So far, we tried those different images on qemu, next thing to try is to use the images running on real hardware boards.

QEMU for Hardware Example 1 : STM32

Running `qemu-system-arm` on STM32

Besides the official QEMU github supporting platforms of netduino2, netduinoplus2, stm32vldiscovery. There is one unofficial branch of QEMU supporting Olimex STM32_P103.

The QEMU for STM32 and Olimex STM32 P103 Development Kit Demos are maintained by Andre Beckus, a branch from official QEMU github v2.1.0. It was not merged with mainstream, with the reason below stated in beckus.github.io

The STM32 implementation is kept in the “stm32” branch (the “master” branch contains the unmodified QEMU code). The “stm32” branch may contain commits that make the software unstable or perhaps even uncompilable. Therefore, I recommend that you use the latest tagged release that looks like this: “stm32_v0.x.x”. These are considered to be relatively stable, and at least minimally tested.

It requires a re-compile/re-installation of STM32 version of qemu for upcoming trials.














































git clone git://github.com/beckus/qemu_stm32.git
cd qemu-stm32
./configure --enable-debug --disable-werror --target-list="arm-softmmu"
make
# I skip the following step, as I don't want to mix up this QEMU STM32 version with official QEMU. 
# So, I need to set the path to execute this STM32 QEMU version
sudo make install
cd ..             # cd to working directory
# Above complete the installation of QEMU STM32. 

# Pre-requisite for cross-compiling STM32 demo programs.
# Installing arm-none-eabi-gcc on x86 host
# Check installation details in this article
# https://stackoverflow.com/questions/73976876/e-unable-to-locate-package-arm-none-eabi-gcc
# Add apt repository then install
sudo add-apt-repository ppa:team-gcc-arm-embedded/ppa
sudo apt-get update
sudo apt-get install gcc-arm-none-eabi
sudo apt-get install gdb-arm-none-eabi 

# Next is to install demo programs, accompanying QEMU STM32.
git clone git://github.com/beckus/stm32_p103_demos.git
cd stm32_p103_demos
make

# Execution of QEMU demo programs - method 1
# assuming directory structure like this
# ~/qemu-stm32
# ~/stm32_p103_demos
#           |- demos
#                |- adc_single
#                |- blink_flash
#                |- .....
# Can try any demo programs under demos directory
# Let's try the first one adc_single as one example
cd demos/adc_single
../../../qemu_stm32-stm32/arm-softmmu/qemu-system-arm -M stm32-p103 -kernel main.bin

# Execution of QEMU demo programs - method 2
# this method is easier, though needs some makefile knowledge
# can launch qemu-system-arm directly from stm32_p103_demos makefile
# Assume you are at directory of stm32_p103_demos
cd stm32_p103_demos
make adc_single_QEMURUN
# or other demos 
make blink_flash_QEMURUN

Running `qemu-system-arm` and `arm-linux-gdb` on STM-32

Installation of arm-linux-gdb

stm32_p103_demos/makefile also support gdb. So let's try it. (I was not familiar with gdb. In the begining, I thought it works well with gdb which was installed together with gcc from sudo apt install build-essential. But not.) It needs to install arm-linux-gdb, so I follow this article arm-linux-gdb除錯工具的安裝與交叉編譯gdbserver for arm-linux-gdb installation. That requires to download gdb source code and compile with setting for ARM.























# Install arm-linux-gdb
# Install dependency first
sudo apt-get install texinfo

# Download the gbd source code from https://ftp.gnu.org/gnu/gdb/
# I found the current gbd vresion installed is 7.11.1. To avoid any inconsistency 
# and potential risk, I decided to install arm-linux-gdb with the same version

gdb -v
> GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
> Copyright (C) 2016 Free Software Foundation, Inc.


# Start download and installation
wget https://ftp.gnu.org/gnu/gdb/gdb-7.11.1.tar.gz
tar xvf gdb-7.11.1.tar.gz
cd gdb-7.11.1/
./configure --target=arm-linux
make
sudo make install
arm-linux-gdb -v
> GNU gdb (GDB) 7.11.1
> Copyright (C) 2016 Free Software Foundation, Inc.

I am not sure if it is needed to instal gdbserver. Still do it anyway.










# install gcc-arm-linux-gnueabi in case not installed before
sudo apt install gcc-arm-linux-gnueabi
# install gdbserver
cd gdb-7.11.1/gdb/gdbserver
CC=arm-linux-gnueabi-gcc ./configure --host=arm-linux
make
file gdbserver
> gdbserver: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV),
> dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0,
> BuildID[sha1]=2cd185b41b022170e9d05be1ade5506b5fef8fe8, not stripped

Execution of qemu with gdb

To work on gdb, it requires to open 2 terminal windows. Let's define the terminal for qemu which qemu-system-arm run on, and terminal for gdb which arm-linux-gdb runs on.

Set terminal for qemu to the path of stm32_p103_demos.


# terminal for qemu
cd qemu-stm32/stm32_p103_demos

Try to understand makefile the command input to run gdb. (I learned the key word to grep is QEMUDBG because I peek the makefile content already.) From cat makefile, we can learn that we need to add _QEMUDBG to run qemu with gdb function.








# terminal for qemu
cat makefile | grep QEMUDBG
> QEMU_DBG_TARGETS = $(addsuffix _QEMUDBG,$(DEMOS))
> QEMU_DBG_PTY_TARGETS = $(addsuffix _QEMUDBG_PTY,$(DEMOS))
> QEMU_DBG_TEL_TARGETS = $(addsuffix _QEMUDBG_TEL,$(DEMOS))
> $(QEMU_DBG_TARGETS): %_QEMUDBG : %_ALL
> $(QEMU_DBG_PTY_TARGETS): %_QEMUDBG_PTY : %_ALL
> $(QEMU_DBG_TEL_TARGETS): %_QEMUDBG_TEL : %_ALL

Let's try the same sample program adc_single with gdb feature by command make adc_single_QEMUDBG under terminal for qemu






















# terminal for qemu
stm32_p103_demos$ make adc_single_QEMUDBG
> killall -q qemu-system-arm
> makefile:145: recipe for target 'adc_single_QEMUDBG' failed
> make: [adc_single_QEMUDBG] Error 1 (ignored)
> ../qemu_stm32/arm-softmmu/qemu-system-arm -M stm32-p103 -gdb tcp::3333 -S -kernel demos/adc_single/main.bin
> STM32_UART: ADC1 clock is set to 0 Hz.
> STM32_UART: ADC1 BRR set to 0.
> STM32_UART: ADC1 Baud is set to 0 bits per sec.
> STM32_UART: ADC2 clock is set to 0 Hz.
.....
> STM32_UART: ADC1 BRR set to 0.
> STM32_UART: ADC1 Baud is set to 0 bits per sec.
> LED Off
> CLKTREE: HSI Output Change (SrcClk:None InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: HSI/2 Output Change (SrcClk:HSI InFreq:8000000 OutFreq:4000000 Mul:1 Div:2 Enabled:1)
> CLKTREE: SYSCLK Output Change (SrcClk:HSI InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: HCLK Output Change (SrcClk:SYSCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> STM32_RCC: Cortex SYSTICK frequency set to 8000000 Hz (scale set to 125).
> STM32_RCC: Cortex SYSTICK ext ref frequency set to 1000000 Hz (scale set to 1000).
> CLKTREE: PCLK1 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> CLKTREE: PCLK2 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)

We can find in the script of qemu-system-arm -M stm32-p103 -gdb tcp::3333 -S -kernel demos/adc_single/main.bin, it enables gdb with port 3333. And the program adc_single do the intialization then halt. So we open the other terminal for gbd to trace how the program executes.







# terminal for gdb
cd stm32_p103_demos/demos/adc_single
arm-linux-gdb main.elf
> GNU gdb (GDB) 7.11.1
> ......
> Reading symbols from main.elf...done.
(gdb)

Next step it to connect the gdb to qemu with command of target remote : 3333, using the port we learned from qemu-system-arm options.







# terminal for gdb
(gdb) target remote : 3333
Remote debugging using : 3333
Reset_Handler ()
    at /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/libraries/CMSIS/CM3/DeviceSupport/ST/STM32F10x/startup/gcc_ride7/startup_stm32f10x_md.s:67
67	  movs	r1, #0
(gdb)

Since we are just trying the gdb function, not really doing the program debug. Let's run the adc_single program by using c or continue command.



# terminal for gdb
(gdb) c
Continuing.

Then, we will find the terminal for qemu starts running, and repeatly the cycle of LED Off and LED On. Unless we use CTRL-c on theterminal for gdb to break the execution.









# terminal for qemu
> CLKTREE: PCLK2 Output Change (SrcClk:HCLK InFreq:8000000 OutFreq:8000000 Mul:1 Div:1 Enabled:1)
> LED Off
> LED On
> LED Off
> LED On
> LED Off
> LED On

Running `openocd` and `arm-linux-gdb` on hardware Olimex STM32_P103

Previous examples in stm32_p103_demos also support openocd for programming the target hardware of stm32_p103 from Olimex. Luckily, I have two Olimex STM32-P103 boards and one ARM-USB-TINY-H JTAG debugging board on hand, so give it a try.

For this demo, we need to have one Olimex board, and one debugger adaptor, connected between Olimex board and PC. The debugger adpator could be JTAG or serial wire debugging (SWD), with different interface to the host (PC). This article will not cover hardware related topics in this article. Please refer to Olimex STM-P103 development boardUser's manual for hardware related information. May have another article to describe the debugger board later.

About openocd
OpenOCD, the Open On-Chip Debugger has been created by Dominic Rath as part of a diploma thesis at the University of Applied Sciences, FH-Augsburg. OpenOCD is a free-software tool mainly used for on-chip debugging, in-system programming and boundary-scan testing. OpenOCD supports flashing and
debugging a wide variety of platforms such as:

ARMv5 through latest ARMv8
MIPS
AVR (incomplete)
Andes
RISC-V

The full list of supported CPU types can be accessed here: User's Guide.

Installation of openocd

Prerequisite 1: git config --global http.sslVerify false to disable git certificate temporarily.

During ./configure --enable-maintainer-mode --enable-ftdi, it will have below error configure: error: jimtcl not found, run git submodule init and git submodule update and git submodule init && git submodule update. After checking Stackoverflow article, we issue Linux command git config --global http.sslVerify false to disable temporarily git certificate. After completion of installation, need to issue another Linux command git config --global http.sslVerify true to enable the git certificate.

Prerequisite 2: Installation of libusb and others





# Install `libusb` is a must
sudo apt-get install libusb-1.0-0-dev libtool automake texinfo

# Newer kernels have FTDI drivers ready-to-use. If that is not your case first you need to download the drivers – you can get the FTDI drivers from their web site. Alternatively you can use apt-get likethis:
sudo apt-get install libftdi-dev libftdi1

Installing openocd with package installation (not recommended)

Below installation is try-and-error. Suggest to skip this step, and try next section of installation by source compiling.

First, install with the following Linux apt command. But it produces error.











sudo apt install openocd
openocd
> ...
> Error: The specified debug interface was not found (ft2232)
> The following debug interfaces are available:
> 1: parport
> 2: dummy
> 3: ftdi
> 4: usb_blaster
> 5: jtag_vpi
> ....

Tried to see if it can be corrected by adding user to plugdev, but still failed.







# check the groups available in Linux system
groups 
#> abhishek adm cdrom sudo dip plugdev lpadmin sambashare kvm

sudo usermod -a -G plugdev user-name
groups user-name
#> user-name : kernel-dev adm cdrom sudo dip plugdev lpadmin sambashare

Also, according to ARM-USB-TINY-H User Manual - 3.3.4 Driver installation in Linux, tried to create a file /etc/udev/rules.d/olimex-arm-usb-tiny-h.rules, by putting this single line in the file:


SUBSYSTEM=="usb", ACTION=="add", ATTRS{idProduct}=="002a", ATTRS{idVendor}=="15ba", MODE="664", GROUP="plugdev"

The idVendor and idProduct shall use the setting from below table of the hardware debugger you use.

Table 2. Olimex OpenOCD debuggers, FTDI vendor and product IDs
	ARM-USB-TINY	ARM-USB-TINY-H	RM-USB-OCD	ARM-USB-OCD-H
VID (VENDOR ID)	0×15BA	0×15BA	0×15BA	0×15BA
PID (PRODUCT ID)	0×0004	0×002a	0×0003	0×002b

Eventually, still gave up. And decided to try with source compiling with different setting to enable ftdi instead of default setting of ft2232 in sudo apt install openocd.

Installing openocd with compiling from source code (recommended)

First, remove the package installed from previous section, if you ever tried.

sudo apt remove --auto-removal openocd

Then, follow the ARM-USB-TINY-H User Manual - 3.3.1 Getting OpenOC with the section describing the source code compilation.
















sudo apt-get install git libtool automake texinfo
git clone http://git.code.sf.net/p/openocd/code openocd-code
cd openocd-code

# modify the option to enable ftdi, learning from previous section
./bootstrap./configure --enable-ftdi --enable-ftdi
make
sudo make install
# then it is okay to go
openocd -v
#> system output below till next #>
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
#> end of system output

So far, we will be able to launch openocd. However, still more setting need to modify on stm32_p103_demos/makefile. Within makefile, we can find a line of default directory for olimex-arm-usb-tiny-h.cfg


# OpenOCD interface file used for programming/debugging the micronctroller
OPENOCD_INTERFACE ?= interface/olimex-arm-usb-tiny-h.cfg

Since we use different option for openocd compilation of --enable-ftdi. The file olimex-arm-usb-tiny-h.cfg is under another directory, so to set it correctly by update that in stm32_p103_demos/makefile.


# OpenOCD interface file used for programming/debugging the micronctroller
OPENOCD_INTERFACE ?= interface/ftdi/olimex-arm-usb-tiny-h.cfg

Now, we complete openocd installation.

Using openocd to run stm32_p103_demos

The makefile under stm32_p103_demos support programming Olimex STM32-P103 board through ARM-USB-TINY-H debugger board. The usage is to add _PROG after the demo program like example below.
































# Running stm32_p103_demos with makefile
cd stm32_p103_demos
make adc_single_PROG 
#> system output below till next #>
killall -q openocd
makefile:122: recipe for target 'adc_single_PROG' failed
make: [adc_single_PROG] Error 1 (ignored)
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "program_flash demos/adc_single/main.bin"
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
program_flash
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
Info : [stm32f1x.cpu] Cortex-M3 r1p1 processor detected
Info : [stm32f1x.cpu] target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for stm32f1x.cpu on 3333
Info : Listening on port 3333 for gdb connections
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0x00005a44 msp: 0x20005000
DEPRECATED! use 'adapter speed' not 'adapter_khz'
Info : device id = 0x20036410
Info : flash size = 128 KiB
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
shutdown command invoked
#>

We can see the LED on STM32-P103 board flashing now, as expected response of the demo program stm32_p103_demos/adc_single does.

Using openocd and arm-linux-gdb to run stm32_p103_demos

After successfully programming (or usually called flash) the Flash ROM of target hardware, try arm-linux-gdb to debug the hardware via openocd. We need to launch two separate terminal screens, let me name it as terminal for openocd and terminal for gdb.






















# terminal for openocd
# Use openocd to flash the Flash ROM of the target hardware
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "program_flash demos/blink_flash/main.bin"
# After successfully flashing the ROM, launch openocd again
openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg
#> system output below until next #>
Open On-Chip Debugger 0.12.0-rc1+dev-00050-ga7ea1ef (2022-10-18-07:28)
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
program_flash
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : clock speed 1000 kHz
Info : JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
Info : JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
Info : [stm32f1x.cpu] Cortex-M3 r1p1 processor detected
Info : [stm32f1x.cpu] target has 6 breakpoints, 4 watchpoints
Info : starting gdb server for stm32f1x.cpu on 3333
Info : Listening on port 3333 for gdb connections
#> end of system output

Then launch a new window of terminal for gdb





















# terminal for gdb
arm-linux-gdb -q  stm32_p103_demos/demos/blink_flash/main.elf
# or terminal for gdb, with enabling Text UI mode with -tui option
arm-linux-gdb -tui -q  stm32_p103_demos/demos/blink_flash/main.elf

(gdb)

# connect gdb with target hardware through openocd
(gdb) target extended-remote localhost:3333
# or can use short form of the following
(gdb) tar ext:3333
Remote debugging using localhost:3333
...
# Reset the target hardware and put it on halt
(gdb) monitor reset init
# This command is optional, just make sure the target will be put on halt
(gdb) monitor reset halt
...
# Run the program with the target hardware
(gdb) run
# CTRL-c to stop

Screenshot - terminal for openocd (left) and terminal for arm-linux-gdb (right)

Screenshot - terminal for openocd (left) and terminal for arm-linux-gdb with -tui option(right)

Deep Dive into `openocd` and `arm-linux-gdb` commands

Since we will send openocd commands through arm-linux-gdb, it is quite confusing to me with commands of the examples in the beginning. It was not clear to me which commands are gdb commands, and which ones are openocd commands. Try to summarize openocd and gdb documents, so to find the appropriate commands more easily.

gdb related documents

gdb official documents
- Debugging with GDB - ftp.gnu.org : GDB manual Ninth Edition, for GDB version 5.1.1, January 2002
- Debugging with GDB - sourceware.org : This is the Tenth Edition, of Debugging with GDB: the GNU Source-Level Debugger for GDB (GDB) Version 12.1.90.20221022-git. This document includes Section 20.3.3 Monitor Commands for gdbserver
gdb popluar commands and usages
- (gdb) target extended-remote localhost:3333 : connect to gdbserver. From previous example, when we launch openocd, the system output Info : starting gdb server for stm32f1x.cpu on 3333 Info : Listening on port 3333 for gdb connections, showing it starts a gdb server on port 3333. (since we use the same PC, so localhost is used, instead of specific ip address. And localhost can be omitted.)
- (gdb) disconnect : to disconntect from gdb server.
- (gdb) monitor reset init : Reset and intialize the target hardware.
- (gdb) run or r : Run the program continueously from the beginning. To stop, press CTRL-c.
- (gdb) Continue or c : Continue the program continueously from where it stopped. To stop, press CTRL-c.
- (gdb) Step or s: Step by step to execute the program per line instruction.
- monitor command in gdb is a magic command. You can use the monitor command to send special requests to gdbserver.
  - (gdb) monitor help : This is a gdb command to list the openocd commands supported. More details in next section of openocd.
gdb examples
- 用Open Source工具開發軟體: 新軟體開發關念 - Chapter 6. 除錯工具 - GDB Basics

openocd Related Documents

openocd official documents
- OpenOCD Concept Index
- OpenOCD User's Guide
- GDB and OpenOCD : Specific instructions of gdb for openocd.
openocd popular commands and usage
- monitor command in gdb is a magic command, which was introduced at previous gdb section. Can use monitor + openocd commands to control openocd and target hardware in gdb.
  - (gdb) monitor help : This is a gdb command to list the openocd commands supported.
  - (gdb) monitor reset {run, halt, init} : Reset the hardware target, then take next action of run (run the program), halt (halt the CPU), init (Initialize the target hardware system).
  - (gdb) monitor soft_reset_halt: Requesting target halt and executing a soft reset. This is often used when a target cannot be reset and halted. The target, after reset is released begins to execute code. OpenOCD attempts to stop the CPU and then sets the program counter back to the reset vector. Unfortunately the code that was executed may have left the hardware in an unknown state.
  - (gdb) monitor flash help to know more flash command provided by target hardware about OpenOCD flash commands.
    - OpenOCD has different commands for NOR and NAND flash; the “flash” command works with NOR flash, while the “nand” command works with NAND flash. This partially reflects different hardware technologies: NOR flash usually supports direct CPU instruction and data bus access, while data from a NAND flash must be copied to memory before it can be used. (SPI flash must also be copied to memory before use.) However, the documentation also uses “flash” as a generic term; for example, “Put flash configuration in board-specific files”.
openocd + gdb examples
- Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging : One of the best answer to provide clear introduction of GDB/OpenOCD programming the ROM.
- Using GDB Server Monitor Commands from Eclipse GDB Console

Using `arm-linux-gdb` commands to `openocd` for programming Flash ROM of the target hardware

The openocd + gdb examples above, Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging, provides a clear intruction on how to program the Flash ROM of the target STM32F107 board. However, need to modify some to use it on Olimex STM32-P103 board.

Below are the script from the article. First, launch openocd


# terminal for openocd
openocd -f /path to scripts/olimex-arm-usb-ocd-h.cfg -f /path to targets/stm32f107.cfg -c "init"

Then, launch gdb on another terminal window.





















# terminal for gdb
# launch arm-linux-gdb under shell
arm-linux-gdb
# under (gdb)
# connection to openocd
set remote hardware-breakpoint limit 6
set remote hardware-watchoint-limit 4
target remote localhost:3333
monitor halt
monitor poll

# Program the Flash ROM of the target
monitor flash probe 0
monitor flash protect 0 0 127 off
monitor reset halt
monitor stm32x mass_erase 0
monitor flash write_image STM3210CTest/test_rom.elf
monitor flash protect 0 0 127 on
disconnect
target remote localhost:3333
monitor soft_reset_halt

However, there are couples of changes need modification due to different hardware used.

Modification 1: Identify the hardware JTAG tap and Flash ROM memory adress. It can be found by gdb command monitor flash probe 0. From the system output, we identify the JTAG tap is stm32f1x and Flash ROM address at 0x08000000. Write it down, and we will use it later.
```
(gdb) monitor flash probe 0
device id = 0x20036410
flash size = 128 KiB
flash 'stm32f1x' found at 0x08000000
```
Modfication 2: Identify the numbers of blocks of Flash ROM by issuing gdb command monitor flash protect 0 0 127 off. The sytem responds with ERROR: last block must be <= 31. So we shall modify the command to (gdb) monitor flash protect 0 0 31 off. It is strange to me though. Olimex STM32-P103 use STM32F103RB CPU with 128KB Flash ROM. According to the datasheet, it is 2KB per block. That shall be 64 blocks, instead of 32 indicated by JTAG.
```
(gdb) monitor flash protect 0 0 127 off
ERROR: last block must be <= 31
(gdb) monitor flash protect 0 0 31 off
```
Modification 3: Using the right JTAG tap from stm32x to stm32f1x as identified in previous command monitor flash probe 0.
```
(gdb) monitor stm32f1x mass_erase 0
```
Modification 4: Adding the Flash ROM memory address in the command monitor flash write_image, and use the absolute file path. The Flash ROM memory address was identified in previous command monitor flash probe 0.
```
(gdb) monitor flash write_image /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf 0x08000000
```

Good to go, after those modifications. Below are the result of the execution.


# terminal for openocd
$ openocd -f interface/ftdi/olimex-arm-usb-tiny-h.cfg -f openocd/openocd_stm32_p103.cfg -c "init"











































# terminal for arm-linux-gdb
$ arm-linux-gdb -q  stm32_p103_demos/demos/blink_flash/main.elf
Reading symbols from stm32_p103_demos/demos/blink_flash/main.elf...done.
(gdb) tar ext:3333
Remote debugging using :3333
0xfffffffe in ?? ()
(gdb) monitor reset init
JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) monitor flash probe 0
device id = 0x20036410
flash size = 128 KiB
flash 'stm32f1x' found at 0x08000000
(gdb) monitor flash protect 0 0 127 off
ERROR: last block must be <= 31
(gdb) monitor flash protect 0 0 31 off
cleared protection for blocks 0 through 31 on flash bank 0
(gdb) monitor reset halt
JTAG tap: stm32f1x.cpu tap/device found: 0x3ba00477 (mfg: 0x23b (ARM Ltd), part: 0xba00, ver: 0x3)
JTAG tap: stm32f1x.bs tap/device found: 0x16410041 (mfg: 0x020 (STMicroelectronics), part: 0x6410, ver: 0x1)
[stm32f1x.cpu] halted due to debug-request, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) monitor stm32f1x mass_erase 0
stm32x mass erase complete
(gdb) monitor flash write_image /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf 0x08000000
wrote 23728 bytes from file /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf in 1.060053s (21.859 KiB/s)
(gdb) monitor flash protect 0 0 31 on
set protection for blocks 0 through 31 on flash bank 0
(gdb) disconnect
Ending remote debugging.
(gdb) tar ext:3333
Remote debugging using :3333
0xfffffffe in ?? ()
(gdb) monitor soft_reset_halt
[stm32f1x.cpu] requesting target halt and executing a soft reset
[stm32f1x.cpu] halted due to breakpoint, current mode: Thread 
xPSR: 0x01000000 pc: 0xfffffffe msp: 0xfffffffc
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/kernel-dev/myworks/qemu-stm32/stm32_p103_demos/demos/blink_flash/main.elf

You can see Olimex STM32-P103 LED blinking now.

QEMU for Hardware Example 2 : Embedded Linux from Scratch in 45 minutes, on RISC-V

This example follows the Youtube - Embedded Linux from Scratch in 45 minutes, on RISC-V, and Presentation pdf.

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

Below is the script of the whole process. That’s possible to compile and assemble in less than 45 minutes!

Things to build
▶ Cross-compiling toolchain: Buildroot 2020.11.1
▶ Firmware / first stage bootloader: OpenSBI
▶ Bootloader: U-Boot 2021.01
▶ Kernel: Linux 5.11-rc3
▶ Root filesystem and application: BusyBox 1.33.0

Starting OpenSBI (from u-boot with Buildroot) in QEMU

























































































































# Generating a RISC-V musl toolchain with Buildroot
cd
mkdir riscv -p
cd riscv
wget https://buildroot.org/downloads/buildroot-2020.11.1.tar.gz
tar xvf buildroot-2020.11.1.tar.gz
cd buildroot-2020.11.1
make menuconfig
# In ▶ Target options ▶ Target Architecture, choose ▶ RISCV
# In ▶ Toolchain ▶ C library, choose ▶ musl.
# ▶ Save your configuration
make sdk
# At the end, you have an toolchain archive
ls -l output/images/riscv64-buildroot-linux-musl_sdk-buildroot.tar.gz
# Extract the archive in a suitable directory, and in the extracted directory, run: ./relocate-sdk.sh
mkdir ~/toolchain
cd ~/toolchain
tar xvf ../riscv/buildroot-2020.11.1/output/images/riscv64-buildroot-linux-musl_sdk-buildroot.tar.gz
cd riscv64-buildroot-linux-musl_sdk-buildroot
# Check the riscv64-linux-gcc under bin/
ls -al bin
./relocate-sdk.sh
#> Relocating the buildroot SDK from /home/kernel-dev/riscv/buildroot-2020.11.1/output/host to /home/kernel-dev/toolchain/riscv64-buildroot-linux-musl_sdk-buildroot ...
cd ~/riscv
# set up path for riscv64-linux-gcc
export PATH=$HOME/toolchain/riscv64-buildroot-linux-musl_sdk-buildroot/bin:$PATH
# create hello.c
vim hello.c
riscv64-linux-gcc -static -o hello hello.c
file hello
sudo apt install -y qemu-user
qemu-riscv64 ./hello

# Hardware emulator section
# Install qemu-system-riscv64
sudo apt install qemu-system-misc
qemu-system-riscv64 -M ?

# U-Boot bootloader
# Set up environment for U-Boot cross-compiling
# Download U-Boot 2021.01 sources
cd ~/riscv
wget https://ftp.denx.de/pub/u-boot/u-boot-2021.01.tar.bz2
tar xvf u-boot-2021.01.tar.bz2
cd u-boot-2021.01
# Add an environment variable for cross-compiling
export CROSS_COMPILE=riscv64-linux-
# Find U-Boot ready-made configurations for RISC-V:
ls configs | grep riscv
# We will choose the configuration for QEMU and U-Boot running in S Mode:
make qemu-riscv64_smode_defconfig
# Now let’s compile U-Boot (-j$(nproc): Use the variable $(nproc) to compile jobs in parallel)
make -j$(nproc)

# Firmware - OpenSBI: Open Supervisor Binary Interface (based on u-boot)
# Required to start an OS (S mode) from the Supervisor/Firmware (M mode)
cd ~/riscv
git clone https://github.com/riscv/opensbi.git
cd opensbi
git checkout v0.8
make PLATFORM=generic FW_PAYLOAD_PATH=../u-boot-2021.01/u-boot.bin
# Run the above command every time you update U-Boot
# This generates the build/platform/generic/firmware/fw_payload.elf file which is a binary that QEMU can boot.
ls build/platform/generic/firmware/fw_payload.elf

# Starting OpenSBI (from u-boot) in QEMU
cd ~/riscv
qemu-system-riscv64 -m 2G \
-nographic \
-machine virt \
-smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf
#> System output until next #>
OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : riscv-virtio,qemu
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 0
Boot HART ISA       : rv64imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 148 KB
Runtime SBI Version : 0.2

MIDELEG : 0x0000000000000222
MEDELEG : 0x000000000000b109
PMP0    : 0x0000000080000000-0x000000008003ffff (A)
PMP1    : 0x0000000000000000-0xffffffffffffffff (A,R,W,X)


U-Boot 2021.01 (Oct 31 2022 - 21:01:02 +0800)

CPU:   rv64imafdcsu
Model: riscv-virtio,qemu
DRAM:  2 GiB
In:    uart@10000000
Out:   uart@10000000
Err:   uart@10000000
Net:   No ethernet found.
Hit any key to stop autoboot:  0 

Device 0: unknown device
scanning bus for devices...

Device 0: unknown device
No ethernet found.
No ethernet found.
=> 
#> end of system output

# Exit QEMU with [Ctrl][a] followed by [x]

Preparing Linux kernel for OpenSBI (from u-boot with Buildroot) in QEMU

Try and Error with Linux kernel

Download and install Linux kernel, but kernel panic, as expected.

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →






















































cd ~/riscv
wget https://git.kernel.org/torvalds/t/linux-5.11-rc3.tar.gz
tar xvf linux-5.11-rc3.tar.gz
cd linux-5.11-rc3
# Add two environment variables for kernel cross-compiling 
# ▶ CROSS_COMPILE is the cross-compiler prefix, as our cross-compiler is riscv64-linux-gcc.
# ▶ ARCH is the name of the subdirectory in arch/ corresponding to the target architecture.
export CROSS_COMPILE=riscv64-linux-
export ARCH=riscv

# Lets take the default Linux kernel configuration for RISCV:
make help | grep defconfig
make defconfig
# ▶ We can now further customize the onfiguration:
make menuconfig
make -j$(nproc)

# At the end, you have these files:
# vmlinux: raw kernel in ELF format (not bootable, for debugging)
# arch/riscv/boot/Image: uncompressed bootable kernel
# arch/riscv/boot/Image.gz: compressed kernel

# Firmware - OpenSBI: Open Supervisor Binary Interface (based on Linux kernel)

cd ../opensbi
make PLATFORM=generic FW_PAYLOAD_PATH=../linux-5.11-rc3/arch/riscv/boot/Image
cd ..
qemu-system-riscv64 -m 2G \
-nographic \
-machine virt \
-smp 8 \
-kernel opensbi/build/platform/generic/firmware/fw_payload.elf \
-append "console=ttyS0" 
#> system output below till next #>
......
[    2.319112] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[    2.324103] Please append a correct "root=" boot option; here are the available partitions:
[    2.327364] DEBUG_BLOCK_EXT_DEVT is enabled, you need to specify explicit textual name for "root=" boot option.
[    2.331364] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    2.334303] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.11.0-rc3 #1
[    2.339795] Call Trace:
[    2.341004] [<ffffffe000004614>] walk_stackframe+0x0/0xaa
[    2.343770] [<ffffffe0006aec96>] show_stack+0x32/0x3e
[    2.346145] [<ffffffe0006b14da>] dump_stack+0x74/0x8e
[    2.349506] [<ffffffe0006aee2c>] panic+0xfc/0x2b2
[    2.352511] [<ffffffe00080124c>] mount_block_root+0x1c6/0x262
[    2.354797] [<ffffffe0008013fc>] mount_root+0x114/0x13e
[    2.356841] [<ffffffe00080156a>] prepare_namespace+0x144/0x186
[    2.360358] [<ffffffe000800e22>] kernel_init_freeable+0x1ce/0x1ea
[    2.364176] [<ffffffe0006b8a3a>] kernel_init+0x12/0xf8
[    2.366129] [<ffffffe0000032be>] ret_from_exception+0x0/0xc
[    2.370044] SMP: stopping secondary CPUs
[    2.376382] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---
#>

Set up storage image to boot the Linux kernel from U-Boot

We want to show how to set the U-Boot environment to load the Linux kernel and to specify the Linux kernel command line
For this purpose, we will need some storage space to store the U-Boot environment, load the kernel binary, and also to contain the filesystem that Linux will boot on.


























































































































# Let’s create a 128 MB disk image:
dd if=/dev/zero of=disk.img bs=1M count=128
# Let’s create two partitions in this image 
cfdisk disk.img
# "Select label type" ▶ DOS
# ▶ "New" - "Partition Size" : 64M ▶ Primary ▶ Bootable ▶ "Type"  : c W95 FAT32 (LBA) 
# Move cursor down to "Free space"
# ▶ "New" : 63M (remaining space) ▶ Primary ▶ "Partition type" is aotumatically selected "Linux 83" ▶ "Write" partition table, and key in "yes" ▶ "Quit" 
# Now we have disk.img 
# ▶ A first 64 MB primary partition (type W95 FAT32 (LBA)), marked as bootable
# ▶ A second partition with remaining space (default type: Linux)
fdisk -lu disk.img
#> System output below until next #>
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xe340e0ae

Device     Boot  Start    End Sectors Size Id Type
disk.img1  *      2048 133119  131072  64M  c W95 FAT32 (LBA)
disk.img2       133120 262143  129024  63M 83 Linux
#> system output ends here

# Let’s access the partitions in this disk image:
sudo losetup -f --show --partscan disk.img
#> /dev/loop0        # check the loop(n) response from the system, n could be 0 or 1 or 2, any number
ls /dev/loop0*
# We can now format the partitions:
sudo mkfs.vfat -F 32 -n boot /dev/loop0p1
#> mkfs.fat 3.0.28 (2015-05-16)
#> mkfs.fat: warning - lowercase labels might not work properly with DOS or Windows
#> unable to get drive geometry, using default 255/63

sudo mkfs.ext4 -L rootfs /dev/loop0p2
#> mke2fs 1.42.13 (17-May-2015)
#> Discarding device blocks: done                            
#> Creating filesystem with 64512 1k blocks and 16128 inodes
#> Filesystem UUID: ff0bd56e-1dca-4ee8-9187-aa053daced34
#> Superblock backups stored on blocks: 
#> 	8193, 24577, 40961, 57345

#> Allocating group tables: done                            
#> Writing inode tables: done                            
#> Creating journal (4096 blocks): done
#> Writing superblocks and filesystem accounting information: done

# Let’s create a mount point for the FAT partition:
sudo mkdir -p /mnt/boot
# Let’s mount it:
sudo mount /dev/loop0p1 /mnt/boot
# Let’s copy the kernel image to it:
sudo cp linux-5.11-rc3/arch/riscv/boot/Image /mnt/boot
# And then unmount the filesystem to commit changes:
sudo umount /mnt/boot

# We want U-Boot be able to use an environment in a FAT partition on a virtio disk. 
# So, let’s reconfigure U-Boot with the following settings
cd ~/riscv/u-boot-2021.01
make menuconfig
# Go to Environment setting page
# ▶ CONFIG_ENV_IS_IN_FAT=y
# ▶ CONFIG_ENV_FAT_INTERFACE="virtio"
# ▶ CONFIG_ENV_FAT_DEVICE_AND_PART="0:1"
make -j$(nproc)
cd ../opensbi/
make PLATFORM=generic FW_PAYLOAD_PATH=../u-boot-2021.01/u-boot.bin
cd ..
qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0
#> System output below until next #>
OpenSBI v0.8
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : riscv-virtio,qemu
Platform Features   : timer,mfdeleg
Platform HART Count : 8
Boot HART ID        : 3
Boot HART ISA       : rv64imafdcsu
BOOT HART Features  : pmp,scounteren,mcounteren,time
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 148 KB
Runtime SBI Version : 0.2
......
U-Boot 2021.01 (Nov 01 2022 - 14:54:15 +0800)

CPU:   rv64imafdcsu
Model: riscv-virtio,qemu
DRAM:  2 GiB
Loading Environment from FAT... OK
In:    uart@10000000
Out:   uart@10000000
Err:   uart@10000000
Net:   No ethernet found.
Hit any key to stop autoboot:  0 

Device 0: QEMU VirtIO Block Device
            Type: Hard Disk
            Capacity: 128.0 MB = 0.1 GB (262144 x 512)
... is now current device
scanning bus for devices...

Device 0: unknown device
No ethernet found.
No ethernet found.
=>
#>
# Below commands are entered under OpenSBI
setenv foo bar
saveenv
# Exit QEMU with [Ctrl][a] followed by [x], and re-enter QEMU / OpenSBI
printenv foo

setenv bootargs and bootcmd to boot Linux















































































































































































cd ~/riscv
qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0

# below commands are entered under OpenSBI
setenv bootargs 'root=/dev/vda2 rootwait console=ttyS0 earlycon=sbi rw'
setenv bootcmd 'fatload virtio 0:1 84000000 Image; booti 0x84000000 - ${fdtcontroladdr}'
saveenv
boot
#> System output until next #> a long one
19804160 bytes read in 162 ms (116.6 MiB/s)
Moving Image from 0x84000000 to 0x80200000, end=81530000
## Flattened Device Tree blob at ff7475b0
   Booting using the fdt blob at 0xff7475b0
   Using Device Tree in place at 00000000ff7475b0, end 00000000ff74be1d

Starting kernel ...

[    0.000000] Linux version 5.11.0-rc3 (kernel-dev@ubuntu-vm) (riscv64-linux-gcc.br_real (Buildroot 2020.11.1) 9.3.0, GNU ld (GNU Binutils) 2.34) #1 SMP Mon Oct 31 22:19:30 CST 2022
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] software IO TLB: mapped [mem 0x00000000f9b47000-0x00000000fdb47000] (64MB)
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acdfimsu
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 17 pages/cpu s32488 r8192 d28952 u69632
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516615
[    0.000000] Kernel command line: root=/dev/vda2 rootwait console=ttyS0 earlycon=sbi rw
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 1977192K/2095104K available (6902K kernel code, 4904K rwdata, 4096K rodata, 2147K init, 305K bss, 117912K reserved, 0K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]       fixmap : 0xffffffcefee00000 - 0xffffffceff000000   (2048 kB)
[    0.000000]       pci io : 0xffffffceff000000 - 0xffffffcf00000000   (  16 MB)
[    0.000000]      vmemmap : 0xffffffcf00000000 - 0xffffffcfffffffff   (4095 MB)
[    0.000000]      vmalloc : 0xffffffd000000000 - 0xffffffdfffffffff   (65535 MB)
[    0.000000]       lowmem : 0xffffffe000000000 - 0xffffffe07fe00000   (2046 MB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU debug extended QS entry/exit.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] plic: interrupt-controller@c000000: mapped 53 interrupts with 8 handlers for 16 contexts.
[    0.000000] random: get_random_bytes called from start_kernel+0x3ae/0x580 with crng_init=0
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [3]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000107] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[    0.005357] Console: colour dummy device 80x25
[    0.010911] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[    0.013346] pid_max: default: 32768 minimum: 301
[    0.015643] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.016532] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.076973] rcu: Hierarchical SRCU implementation.
[    0.079447] EFI services will not be available.
[    0.084805] smp: Bringing up secondary CPUs ...
[    0.107909] smp: Brought up 1 node, 8 CPUs
[    0.178760] devtmpfs: initialized
[    0.193667] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.196196] futex hash table entries: 2048 (order: 5, 131072 bytes, linear)
[    0.204203] NET: Registered protocol family 16
[    0.351072] vgaarb: loaded
[    0.353865] SCSI subsystem initialized
[    0.358430] usbcore: registered new interface driver usbfs
[    0.359476] usbcore: registered new interface driver hub
[    0.360719] usbcore: registered new device driver usb
[    0.430208] clocksource: Switched to clocksource riscv_clocksource
[    0.504310] NET: Registered protocol family 2
[    0.511287] tcp_listen_portaddr_hash hash table entries: 1024 (order: 3, 40960 bytes, linear)
[    0.512721] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.514451] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    0.516448] TCP: Hash tables configured (established 16384 bind 16384)
[    0.521106] UDP hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.522906] UDP-Lite hash table entries: 1024 (order: 4, 98304 bytes, linear)
[    0.526330] NET: Registered protocol family 1
[    0.535205] RPC: Registered named UNIX socket transport module.
[    0.536965] RPC: Registered udp transport module.
[    0.539670] RPC: Registered tcp transport module.
[    0.540269] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.542282] PCI: CLS 0 bytes, default 64
[    0.554499] workingset: timestamp_bits=62 max_order=19 bucket_order=0
[    0.575618] NFS: Registering the id_resolver key type
[    0.577296] Key type id_resolver registered
[    0.577990] Key type id_legacy registered
[    0.579024] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.582410] 9p: Installing v9fs 9p2000 file system support
[    0.586787] NET: Registered protocol family 38
[    0.587686] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.588624] io scheduler mq-deadline registered
[    0.589768] io scheduler kyber registered
[    0.601836] pci-host-generic 30000000.pci: host bridge /soc/pci@30000000 ranges:
[    0.603496] pci-host-generic 30000000.pci:       IO 0x0003000000..0x000300ffff -> 0x0000000000
[    0.604630] pci-host-generic 30000000.pci:      MEM 0x0040000000..0x007fffffff -> 0x0040000000
[    0.607574] pci-host-generic 30000000.pci: ECAM at [mem 0x30000000-0x3fffffff] for [bus 00-ff]
[    0.610581] pci-host-generic 30000000.pci: PCI host bridge to bus 0000:00
[    0.611751] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.612362] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.613252] pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
[    0.617806] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000
[    1.002980] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    1.015206] printk: console [ttyS0] disabled
[    1.017049] 10000000.uart: ttyS0 at MMIO 0x10000000 (irq = 2, base_baud = 230400) is a 16550A
[    1.025299] printk: console [ttyS0] enabled
[    1.025299] printk: console [ttyS0] enabled
[    1.051860] printk: bootconsole [sbi0] disabled
[    1.051860] printk: bootconsole [sbi0] disabled
[    1.057663] [drm] radeon kernel modesetting enabled.
[    1.143809] loop: module loaded
[    1.224179] virtio_blk virtio0: [vda] 262144 512-byte logical blocks (134 MB/128 MiB)
[    1.270068]  vda: vda1 vda2
[    1.282726] libphy: Fixed MDIO Bus: probed
[    1.286115] e1000e: Intel(R) PRO/1000 Network Driver
[    1.286525] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.287430] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.287951] ehci-pci: EHCI PCI platform driver
[    1.289161] ehci-platform: EHCI generic platform driver
[    1.290024] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.316459] ohci-pci: OHCI PCI platform driver
[    1.317678] ohci-platform: OHCI generic platform driver
[    1.320310] usbcore: registered new interface driver uas
[    1.321076] usbcore: registered new interface driver usb-storage
[    1.323645] mousedev: PS/2 mouse device common for all mice
[    1.328237] goldfish_rtc 101000.rtc: registered as rtc0
[    1.334751] goldfish_rtc 101000.rtc: setting system clock to 2022-11-01T07:34:33 UTC (1667288073)
[    1.341077] syscon-poweroff poweroff: pm_power_off already claimed for sbi_shutdown
[    1.342770] syscon-poweroff: probe of poweroff failed with error -16
[    1.344875] usbcore: registered new interface driver usbhid
[    1.345528] usbhid: USB HID core driver
[    1.349055] NET: Registered protocol family 10
[    1.360726] Segment Routing with IPv6
[    1.362869] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    1.367194] NET: Registered protocol family 17
[    1.372807] 9pnet: Installing 9P2000 support
[    1.374920] Key type dns_resolver registered
[    1.376191] debug_vm_pgtable: [debug_vm_pgtable         ]: Validating architecture page table helpers
[    1.486949] EXT4-fs (vda2): mounted filesystem with ordered data mode. Opts: (null). Quota mode: disabled.
[    1.490060] ext4 filesystem being mounted at /root supports timestamps until 2038 (0x7fffffff)
[    1.492075] VFS: Mounted root (ext4 filesystem) on device 254:2.
[    1.496181] devtmpfs: error mounting -2
[    1.536872] Freeing unused kernel memory: 2144K
[    1.538627] Run /sbin/init as init process
[    1.539526] Run /etc/init as init process
[    1.539997] Run /bin/init as init process
[    1.540609] Run /bin/sh as init process
[    1.541191] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
[    1.542603] CPU: 1 PID: 1 Comm: swapper/0 Not tainted 5.11.0-rc3 #1
[    1.542994] Call Trace:
[    1.543260] [<ffffffe000004614>] walk_stackframe+0x0/0xaa
[    1.543704] [<ffffffe0006aec96>] show_stack+0x32/0x3e
[    1.543958] [<ffffffe0006b14da>] dump_stack+0x74/0x8e
[    1.544416] [<ffffffe0006aee2c>] panic+0xfc/0x2b2
[    1.544891] [<ffffffe0006b8b14>] kernel_init+0xec/0xf8
[    1.545291] [<ffffffe0000032be>] ret_from_exception+0x0/0xc
[    1.545860] SMP: stopping secondary CPUs
[    1.547075] ---[ end Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. ]---
#>

It runs, and fails. As expected, it mounts the root file system, but it fails in executing any process, because we haven't feeded the root file system yet.

Bulding the root filesystem (using Busybox)


















































































cd ~/riscv
wget https://busybox.net/downloads/busybox-1.33.0.tar.bz2
tar xvf busybox-1.33.0.tar.bz2
cd busybox-1.33.0
make allnoconfig
# Starts with no applet selected
make menuconfig
# ▶ In Settings →Build Options, enable Build static binary (no shared libs)
# ▶ In Settings →Build Options, set Cross compiler prefix to riscv64-linux-
# ▶ Then enable support for the following commands: ash, init, halt, mount, cat, mkdir, echo, ls, chmod, uptime, vi, ifconfig, httpd
# Check https://asciinema.org/a/384727 for playback on how to select those commands
make install
tree _install/
#> System output until next #>
_install/
├── bin
│   ├── ash -> busybox
│   ├── busybox
│   ├── cat -> busybox
│   ├── echo -> busybox
│   ├── ls -> busybox
│   ├── mkdir -> busybox
│   ├── mount -> busybox
│   ├── sh -> busybox
│   └── vi -> busybox
├── sbin
│   ├── halt -> ../bin/busybox
│   ├── ifconfig -> ../bin/busybox
│   └── init -> ../bin/busybox
└── usr
    ├── bin
    │   └── uptime -> ../../bin/busybox
    └── sbin
        └── httpd -> ../../bin/busybox

5 directories, 14 files
#>

# Installing to the root filesystem:
# We need to create a dev directory. The devtmpfs filesystem will automatically be mounted there (as CONFIG_DEVTMPFS_MOUNT=y)
sudo mkdir /mnt/rootfs
sudo mount /dev/loop0p2 /mnt/rootfs
sudo rsync -aH _install/ /mnt/rootfs/
sudo mkdir /mnt/rootfs/dev
sudo mkdir /mnt/rootfs/proc
sudo mkdir /mnt/rootfs/sys
sudo umount /mnt/rootfs

qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0

# Let’s also create /proc and /sys so that we can also mount the proc and sysfs virtual filesystems on the target:
# below commands are entered under QEMU
mount
#> mount: no /proc/mounts
mount -t proc nodev /proc
mount -t sysfs nodev /sys
mount
#> /dev/root on / type ext4 (rw,relatime)
#> devtmpfs on /dev type devtmpfs (rw,relatime,size=988596k,nr_inodes=247149,mode=755)
#> nodev on /proc type proc (rw,relatime)
#> nodev on /proc type proc (rw,relatime)
#> nodev on /sys type sysfs (rw,relatime)

mkdir /etc
vi /etc/inittab
# input below 4 lines in vi
# This is run first script:
::sysinit:/etc/init.d/rcS
# Start an "askfirst" shell on the console:
::askfirst:/bin/sh
# Press :wq! to save and quit vi
mkdir /etc/init.d
vi /etc/init.d/rcS
# input below 3 lines in vi
#!/bin/sh
mount -t proc nodev /proc
mount -t sysfs nodev /sys
# Press :wq! to save and quit vi
chmod +x /etc/init.d/rcS

Add support for networking



































# add networking in QEMU
sudo qemu-system-riscv64 -m 2G -nographic -machine virt -smp 8 \
-bios opensbi/build/platform/generic/firmware/fw_payload.elf \
-drive file=disk.img,format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-netdev tap,id=tapnet,ifname=tap2,script=no,downscript=no \
-device virtio-net-device,netdev=tapnet

# On the target machine:
ifconfig -a
ifconfig eth0 192.168.2.100
# On the host machine:
ifconfig -a
sudo ifconfig tap2 192.168.2.1
ping 192.168.2.100

# On the target machine:
# Store below contents in /www/cgi-bin/uptime and make it executable.
#!/bin/sh
echo "Content-type: text/html"
echo
echo "<html>"
echo "<meta http-equiv=\"refresh\" content=\"1\">"
echo "<header></header><body>"
echo "<h1>Uptime information</h1>"
echo "Your embedded device has been running for:<pre><font color=Blue>"
echo `uptime`
echo "</font></pre>"
echo "</body></html>"
# then execute in target machine:
/usr/sbin/httpd -h /www

# On the host machine, open in your browser: (make sure NOT https://)
http://192.168.2.100/cgi-bin/uptime

References

Check the References of QEMU for Beginner and Advanced, as the References listed here is more related to hardware.
QEMU related to hardware
Olimex STM32_P103 and ARM-USB-TINY-H hardware related
- STM32-P103 - Header board for STM32F103RBT6 CORTEX M3 ARM microcontroller
  - STM-P103 development boardUser's manual
  - ARM-USB-TINY-H OLIMEX OPENOCD ARM JTAG DEBUGGER
- OpenOCD official documents
  - OpenOCD Concept Index
  - OpenOCD User's Guide
  - GDB and OpenOCD : Specific instructions of gdb for openocd.
  - OpenOCD General Commands
  - GDB and OpenOCD : Specific instructions of gdb for openocd
  - gdb and openocd examples
    - Stackoverflow - STM32 GDB/OpenOCD Commands and Initialization for Flash and Ram Debugging : One of the best answer to provide clear introduction of GDB/OpenOCD programming the ROM.
    - Upload Code to STM32L4, Using Linux, GNU Make, and OpenOCD
    - GDB example on x86 - 透過 GDB 進行遠端除錯 by jserv
    - 使用 gdb+qemu 來執行/除錯 raspberry pi linux kernel
- GDB official documents and examples
  - Debugging with GDB - ftp.gnu.org : GDB manual Ninth Edition, for GDB version 5.1.1, January 2002
  - Debugging with GDB - sourceware.org : This is the Tenth Edition, of Debugging with GDB: the GNU Source-Level Debugger for GDB (GDB) Version 12.1.90.20221022-git. This document includes Section 20.3.3 Monitor Commands for gdbserver
  - gdb examples
    - 用Open Source工具開發軟體: 新軟體開發關念 - Chapter 6. 除錯工具 - GDB Basics (and Advanced) : Check link above for OpenOCD specific commands for GDB.
    - GDB實用教學：自動化你的debug : gdb -tui or cgdb for Text UI or GUI operation.
QEMU STM32 sample programs
- Sample program #1 - QEMU for STM32 and Olimex STM32 P103 Development Kit Demos are maintained by Andre Beckus
- Sample program #2 - qemu的STM32虛擬化環境 : Good entry for a beginning to know how to set up STM32 QEMU environment, a Chinese translation of QEMU for STM32. There is a minor mistake, though. The ./configure should be removed.
```
git clone git://github.com/beckus/stm32_p103_demos.git
cd stm32_p103_demos
./configure           # remove this line
make
```
- Sample program #3 - NCKU Jserv Embedded Courses, also based on QEMU for STM32
  - NCKU Jserv Embedded Course Lab 1
- Sample program #4 - Building a Linux system for the STM32MP1: basic system by bootlin
STM32 official documents
- STM32 hardware related documents
  - STM32F103 introductory page
  - STM32F103x8 and xB spec :
    - STM32F103's flash memory is NOR type. Check this article - Flash memory type for STM32F103C8T6.
    - About OpenOCD flash commands : OpenOCD has different commands for NOR and NAND flash; the “flash” command works with NOR flash, while the “nand” command works with NAND flash. This partially reflects different hardware technologies: NOR flash usually supports direct CPU instruction and data bus access, while data from a NAND flash must be copied to memory before it can be used. (SPI flash must also be copied to memory before use.) However, the documentation also uses “flash” as a generic term; for example, “Put flash configuration in board-specific files”.
- STM32 programming related documents
Flash related topics - hardware, firmware, and software
- 成大資工 Wiki - Flash memory: from IC, hardware, timing to programming flash
- STM32学习笔记：FLASH读写之一

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

Previous article - QEMU for Beginner and Advanced

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

Next article - gdb Introduction

Image Not Showing Possible Reasons

The image file may be corrupted
The server hosting the image is unavailable
The image path is incorrect
The image format is not supported

Learn More →

back to marconi's blog

QEMU simulation with hardware boards

tags: 2022/10 qemu qemu-architecture qemu-system-architecture hardware boards stm32 raspberrypi

QEMU for Hardware Example 1 : STM32

Running qemu-system-arm on STM32

Running qemu-system-arm and arm-linux-gdb on STM-32

Running openocd and arm-linux-gdb on hardware Olimex STM32_P103

Deep Dive into openocd and arm-linux-gdb commands

Using arm-linux-gdb commands to openocd for programming Flash ROM of the target hardware

QEMU for Hardware Example 2 : Embedded Linux from Scratch in 45 minutes, on RISC-V

Starting OpenSBI (from u-boot with Buildroot) in QEMU

Preparing Linux kernel for OpenSBI (from u-boot with Buildroot) in QEMU

Try and Error with Linux kernel

Set up storage image to boot the Linux kernel from U-Boot

setenv bootargs and bootcmd to boot Linux

Bulding the root filesystem (using Busybox)

Add support for networking

References

Read more

Q&A - Linux related

Midjourney

Q&A - Advanced Linux and C Language related

讀書心得 : 操作系統原型 - xv6 分析與實踐 羅秋明 著

tags: `2022/10` `qemu` `qemu-architecture` `qemu-system-architecture` `hardware` `boards` `stm32` `raspberrypi`

Running `qemu-system-arm` on STM32

Running `qemu-system-arm` and `arm-linux-gdb` on STM-32

Running `openocd` and `arm-linux-gdb` on hardware Olimex STM32_P103

Deep Dive into `openocd` and `arm-linux-gdb` commands

Using `arm-linux-gdb` commands to `openocd` for programming Flash ROM of the target hardware

讀書心得 : 操作系統原型 - xv6 分析與實踐羅秋明著