RVVM

contributed by <huang-me>, <OliveLake>

Expectation

• RVVM ( https://github.com/LekKit/RVVM ) 同時支援 RV64 和 RV32，本課程著重於 RV32，請針對 RV32 準備相關的開發工具 (可重用 GNU Toolchain)
• 留意 Linux 啟動流程: OpenSBI -> U-Boot -> Linux kernel -> userspace，請學習相關的背景知識，說明個別元件的作用，特別是 OpenSBI
• 準備 RV32 的 Linux 核心，你們可能會遭遇技術困難，記錄下來並嘗試排除
• 說明 Linux 核心在 RV32/RVVM 啟動過程中，除了 CPU，還有哪些週邊的互動 (如中斷控制器和 timer)
• RVVM 可通過 RISC-V compliance tests，請以最新的程式碼搭配新的測試，確認在 RV32 (及相關 extension) 得以通過，並記錄過程，若發現問題就回報給 RVVM 開發團隊.

Introduction

As shown in figure below

RVVM replace QEMU, and we need to prepare OS kernel & cross compiler tools & Distributions for it.

Prerequisites to build RVVM

Build riscv32 linux toolchain

There are two types riscv-gnu-toolchain,

1. riscv32-unknown-elf-gcc, cross compiler using newlib for 32bit RISC-V bare-metal.
2. riscv32-unknown-linux-gnu-gcc, corss compiler using glibc which provides the core libraries for GNU/Linux systems.

stupid try

Why wrong

Toolchain leading with riscv-none-embed- is toolchain for embedded system.
We're going to build risc-v linux, so we need toolchain leading with riscv64-unknown-linux-gnu-.

Build OpenSBI

export CROSS_COMPILE=riscv-none-embed-
git clone https://github.com/starfive-tech/opensbi.git
cd opensbi
make PLATFORM="generic"

• fw_jump.bin will be generated in
  opensbi/build/platform/generic/firmware

Build linux kernel

// generate config
make ARCH=riscv CROSS_COMPILE=riscv-none-embed- defconfig
// compile
make ARCH=riscv CROSS_COMPILE=riscv-none-embed- -j 4

• Image will be generated in
  linux/arch/riscv/boot

Build busy box

git clone https://git.busybox.net/busybox
cd busybox
CROSS_COMPILE=riscv-none-embed- make menuconfig
CROSS_COMPILE=riscv-none-embed- make // error

include/platform.h:168:11: fatal error: byteswap.h: No such file or directory
  168 | # include <byteswap.h>
      |           ^~~~~~~~~~~~

riscv-none-embed- has no byteswap.h

Prerequisite

For ubuntu, there are some needed packages to build toolchain.

$ sudo apt-get install autoconf automake autotools-dev curl python3 libmpc-dev libmpfr-dev libgmp-dev gawk build-essential bison flex texinfo gperf libtool patchutils bc zlib1g-dev libexpat-dev

Installation

git clone https://github.com/riscv-collab/riscv-gnu-toolchain.git --recursive
cd riscv-gnu-toolchain
./configure --prefix=/opt/riscv32-linux --with-arch=rv32imc --enable-linux
sudo make linux

Errors in building riscv linux toolchain

After sudo make linux

error : mips nios2 powerpc riscv glibc requires the A extension

Because RISC-V Linux require A-extension as minimal requirement, so at least we should use --wtih-arch=rv32ia or --with-arch=rv64ia for building linux toolchain. We sholud use

make clean
./configure --prefix=opt/riscv32-linux --with-arch=rv32imac --enable-linux
make linux

PICTURE

Setup toolchain & environment variable

export PATH=$PATH:/opt/riscv32-linux/bin
export CROSS_COMPILE=riscv32-unknown-linux-gnu-
export ARCH=riscv

Introduction for Boot flow

RISC-V Upstream Boot Flow

• ZSBL(Zeroth Stage Boot Loader): Initial platform-specific bootloader,the first program being run after power-on. Core starts fetching from ROM of SoC, copying FSBL image from SD into memory.
• FSBL(first Stage Boot Loader): Initial platform-specific bootloader. FSBL is SiFive specific and will be replaced by Coreboot/U-Boot SPL.

What is U-Boot SPL

Usually regular bootloader (e.g. U-Boot) binary is bigger than SRAM, So we need to create some additional bootloader, referred as first-stage bootloader (in two-stage bootloader scenario). Regularly the first-stage bootloader is U-Boot SPL, and the second-stage bootloader is U-Boot.

• OpenSBI: The RISC-V Supervisor Binary Interface (SBI) specifications. Aimed at providing RUNTIME services in M-mode, typically used in boot stage following ROM/LOADER.

RVVM allows boot the Linux kernel directly after SBI, without U-Boot. Therefore we need to execute fw_jump.bin which contain OpenSBI and initial firmware.

• fw_jump: Firmware provides by OpenSBI with static jump address to the next booting stage, so previous booting stage (i.e. LOADER) has to load next booting stage (i.e. BOOTLADER) at a fixed location.

Build linux kernel image

Create simple Hello world of booting Linux on RVVM

mkdir riscv32-linux
cd riscv32-linux
git clone https://github.com/torvalds/linux
git checkout v5.4.0
make ARCH=riscv CROSS_COMPILE=riscv32-unknown-linux-gnu- defconfig

Build U-Boot

git clone https://github.com/u-boot/u-boot.git
make qemu-riscv32_smode_defconfig
make

Build OpenSBI

git clone https://github.com/starfive-tech/opensbi.git
cd opensbi
make PLATFORM="generic" FW_PAYLOAD_PATH=../u-boot/u-boot.bin

error occurs

cc: error: unrecognized argument in option ‘-mabi=lp64’
cc: note: valid arguments to ‘-mabi=’ are: ms sysv
cc: error: unrecognized argument in option ‘-mcmodel=medany’
cc: note: valid arguments to ‘-mcmodel=’ are: 32 kernel large medium small; did you mean ‘medium’?

Caused by failing to use RISC-V compiler, reset path:

export CROSS_COMPILE=/opt/riscv32-unknown-linux-gnu-

fw_jump.bin will be generated in opensbi/build/platform/generic/firmware

Build RVVM

Build RVVM using GNU make on linux

git clone https://github.com/LekKit/RVVM
cd RVVM
make

There is an error about no such file or directory : X11/keysym.h & X11/extentions/XShm.h

Solution :

sudo apt install libx11-dev
sudo apt install x11proto-xext-dev

Then keep building RVVM

cd release.linux.riscv
./rvvm_riscv fw_jump.bin -kernel linux_Imag -mem 2G -smp 2 -res 1280x720

loop forever

After contact with RVVM team, it seems like we are running U-Boot as SBI payload, and U-Boot tries to access MMC flash - RVVM doesn't support flash (yet), so U-Boot keep resets the VM. One solution is we built U-Boot with IDE (ATA) hard-drives, and another way is we boot the Linux kernel directly after SBI, without U-Boot.

Rebuild without U-Boot

make PLATFORM="generic"

cd release.linux.riscv
./rvvm_riscv fw_jump.bin -kernel linux_Imag

Build Distributions

There is not that much pre-built distributions for 32-bit RISC-V, so we use Buildroot

./rvvm_riscv fw_jump.bin -kernel linux_image -image rootfs.img -mem 1G -res 1280x720

Complience Test

What complience test for?

Due to RISC-V is an open ISA, and the freedom of extending the ISA with custom instructions and extensions, the compliance tests are essential to confirm the basic operation is in accordance with the specification

Start Test

TODO