# WebAssembly based Linux/RISC-V System Emulation
> 蘇湘婷
## Project Description
This project involves improving the WebAssembly port of rv32emu, a RISC-V emulator capable of running the Linux kernel. The goal is to enable the emulated Linux/RISC-V system to access a designated browser directory via virtio, referencing concepts from previous implementations and standards
## What/Why virtio?
Virtio is a standardized interface for virtual devices, commonly used in virtualized and emulated environments. In the context of your project, using Virtio provides several key benefits:
1. Efficient Communication:
Virtio enables efficient communication between the emulated Linux/RISC-V system and the underlying host (in this case, the browser). It minimizes overhead, allowing faster data transfer and interaction with virtual devices.
2. Flexibility:
Virtio is designed to be flexible, supporting a variety of device types (e.g., block devices, network interfaces, etc.). For your project, it facilitates seamless access to browser-designated directories, enabling functionality similar to file sharing.
3. Standardization:
As a widely adopted standard, Virtio simplifies development by providing well-documented protocols. This reduces the complexity of integrating the emulated system with WebAssembly and the browser environment.
4. Compatibility:
By leveraging Virtio, your project can align with existing Linux drivers that natively support Virtio devices, reducing the need for custom driver development.
5. Cross-Platform Integration:
Virtio is not tied to a specific platform, making it ideal for environments like WebAssembly where portability and platform independence are critical.
## Problem encountered
1. Can't use
```bash
sudo apt-get install llvm-18
```
2. A compilation error is found. Reproducer:
```bahs!
$ make distclean
$ make ENABLE_SYSTEM=1 ENABLE_MOP_FUSION=0 ENABLE_JIT=1 ENABLE_T2C=0
```
Error messages:
```
src/riscv.c: In function ‘rv_delete’:
src/riscv.c:586:18: error: ‘attr’ undeclared (first use in this function)
586 | u8250_delete(attr->uart);
| ^~~~
src/riscv.c:586:18: note: each undeclared identifier is reported only once for each function it appears in
make: *** [Makefile:257: build/riscv.o] Error 1
```
[Clone this branch as the issue has already been resolved.](https://github.com/ChinYikMing/rv32emu/commit/b4bc65bf1f7eb7ab3e584619d6377cd57d916603)
## Reference
1. Previous Implementation: https://github.com/sysprog21/semu/blob/master/virtio-blk.c
2. Conceptual Illustration: https://projectacrn.github.io/latest/developer-guides/hld/virtio-blk.html
3. Specification Document: https://docs.oasis-open.org/virtio/virtio/v1.3/virtio-v1.3.html
:::danger
Always write in English.
:::
## Code
- [ ] `src/virtio-blk.c`
```c
/**
* virtio-blk.c - A minimal skeleton for virtio-blk device
*
* This file is for demonstration purpose only.
* It omits error handling, concurrency control,
* and other complexities involved in real projects.
*
* ...
*/
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <assert.h>
#include <inttypes.h>
/* ========== VirtIO Block Related Definitions ========== */
#define VIRTIO_BLK_F_RO 5
#define VIRTIO_BLK_F_SCSI 7
#define VIRTIO_BLK_F_CONFIG_WCE 11
#define VIRTIO_BLK_F_MQ 12
#define VIRTIO_BLK_F_DISCARD 13
#define VIRTIO_BLK_F_WRITE_ZEROES 14
#define VIRTIO_CONFIG_S_ACKNOWLEDGE 1
#define VIRTIO_CONFIG_S_DRIVER 2
#define VIRTIO_CONFIG_S_DRIVER_OK 4
#define VIRTIO_CONFIG_S_FEATURES_OK 8
#define VIRTIO_CONFIG_S_FAILED 128
#define VIRTIO_BLK_T_IN 0
#define VIRTIO_BLK_T_OUT 1
#define VIRTIO_BLK_T_FLUSH 4
#define VIRTIO_BLK_T_DISCARD 11
#define VIRTIO_BLK_T_WRITE_ZEROES 13
#pragma pack(push, 1)
struct virtio_blk_req {
uint32_t type;
uint32_t reserved;
uint64_t sector;
};
#pragma pack(pop)
#pragma pack(push, 1)
struct virtio_blk_config {
uint64_t capacity;
uint32_t size_max;
uint32_t seg_max;
uint16_t cylinders;
uint8_t heads;
uint8_t sectors;
uint32_t blk_size;
};
#pragma pack(pop)
enum {
VIRTIO_BLK_S_OK = 0,
VIRTIO_BLK_S_IOERR,
VIRTIO_BLK_S_UNSUPP,
};
#define QUEUE_SIZE 128
struct virtqueue_desc {
uint64_t addr;
uint32_t len;
uint16_t flags;
uint16_t next;
};
struct virtqueue_avail {
uint16_t flags;
uint16_t idx;
uint16_t ring[QUEUE_SIZE];
};
struct virtqueue_used_elem {
uint32_t id;
uint32_t len;
};
struct virtqueue_used {
uint16_t flags;
uint16_t idx;
struct virtqueue_used_elem ring[QUEUE_SIZE];
};
struct virtio_queue {
struct virtqueue_desc desc[QUEUE_SIZE];
struct virtqueue_avail avail;
struct virtqueue_used used;
};
struct virtio_blk_dev {
struct virtio_blk_config config;
struct virtio_queue vq;
uint8_t status;
uint64_t capacity;
uint8_t *backend;
};
/* ========== Helper Functions ========== */
static inline void set_device_status(struct virtio_blk_dev *dev, uint8_t status)
{
dev->status |= status;
printf("[virtio-blk] Device status updated to 0x%x\n", dev->status);
}
void virtio_blk_init(struct virtio_blk_dev *dev, uint64_t capacity_in_sectors)
{
dev->status = 0;
set_device_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE);
set_device_status(dev, VIRTIO_CONFIG_S_DRIVER);
set_device_status(dev, VIRTIO_CONFIG_S_FEATURES_OK);
dev->capacity = capacity_in_sectors;
dev->config.capacity = capacity_in_sectors;
dev->config.blk_size = 512;
dev->backend = (uint8_t*)malloc(dev->config.capacity * dev->config.blk_size);
memset(dev->backend, 0, dev->config.capacity * dev->config.blk_size);
memset(&dev->vq, 0, sizeof(dev->vq));
set_device_status(dev, VIRTIO_CONFIG_S_DRIVER_OK);
printf("[virtio-blk] Device initialized with capacity = %" PRIu64 " sectors\n",
capacity_in_sectors);
}
static void process_request(struct virtio_blk_dev *dev,
struct virtio_blk_req *req,
uint8_t *data_buf, size_t data_len,
uint8_t *status)
{
switch (req->type) {
case VIRTIO_BLK_T_IN:
if ((req->sector + (data_len / 512)) > dev->capacity) {
*status = VIRTIO_BLK_S_IOERR;
break;
}
memcpy(data_buf,
dev->backend + (req->sector * dev->config.blk_size),
data_len);
*status = VIRTIO_BLK_S_OK;
break;
case VIRTIO_BLK_T_OUT:
if ((req->sector + (data_len / 512)) > dev->capacity) {
*status = VIRTIO_BLK_S_IOERR;
break;
}
memcpy(dev->backend + (req->sector * dev->config.blk_size),
data_buf, data_len);
*status = VIRTIO_BLK_S_OK;
break;
case VIRTIO_BLK_T_FLUSH:
*status = VIRTIO_BLK_S_OK;
break;
default:
*status = VIRTIO_BLK_S_UNSUPP;
break;
}
}
void virtio_blk_handle_queue(struct virtio_blk_dev *dev)
{
struct virtio_queue *vq = &dev->vq;
/* Assume there is a request placed in desc[0] */
struct virtqueue_desc *desc = &vq->desc[0];
/* These pointers usually require address translation (MMU / host-guest mapping) */
struct virtio_blk_req *req_hdr = (struct virtio_blk_req*)(uintptr_t)desc->addr;
size_t data_len = 512;
uint8_t data_buf[512];
uint8_t status_byte = 0;
printf("[virtio-blk] Handling request: type=%d, sector=%" PRIu64 "\n",
req_hdr->type, req_hdr->sector);
process_request(dev, req_hdr, data_buf, data_len, &status_byte);
printf("[virtio-blk] Request done, status = %u\n", status_byte);
vq->used.ring[vq->used.idx].id = 0;
vq->used.ring[vq->used.idx].len = (uint32_t)data_len;
vq->used.idx++;
}
/*
* For use in main.c.
*/
void virtio_blk_demo(void)
{
struct virtio_blk_dev blk_dev;
/* Initialize and test a simple request */
virtio_blk_init(&blk_dev, 1024);
static struct virtio_blk_req req;
req.type = VIRTIO_BLK_T_IN; /* read */
req.reserved = 0;
req.sector = 10;
blk_dev.vq.desc[0].addr = (uintptr_t)&req;
blk_dev.vq.desc[0].len = sizeof(req);
virtio_blk_handle_queue(&blk_dev);
free(blk_dev.backend);
}
```
:::danger
Refine the comments. Always write in English!
:::
- [ ] `src/main.c`
```c
...
extern void virtio_blk_demo(void);
...
virtio_mmio_init();
virtio_blk_demo();
rv_run(rv);
...
```
`src/virtio_mmio.h`
```C
#ifndef VIRTIO_MMIO_H
#define VIRTIO_MMIO_H
#define VIRTIO_MMIO_DRIVER_FEATURES 0x20
#define VIRTIO_MMIO_BASE 0xf5000000
#define VIRTIO_MMIO_SIZE 0x1000
extern struct virtio_mmio_dev g_virtio_mmio0;
#ifdef __cplusplus
extern "C" {
#endif
struct virtio_mmio_dev {
uint32_t magic_value;
uint32_t version;
uint32_t device_id;
uint32_t vendor_id;
uint32_t device_features;
uint32_t driver_features;
uint32_t queue_sel;
uint32_t queue_num_max;
uint32_t queue_ready;
uint32_t status;
};
/**
* Initialize the virtio-mmio region, mapping the address range
* [0xf5000000, 0xf5000fff] to the emulator's memory/IO map.
* This allows Linux to detect virtio-mmio devices (e.g., block devices)
* through these addresses.
*/
void virtio_mmio_init(void);
#ifdef __cplusplus
}
#endif
#endif /* VIRTIO_MMIO_H */
```
`src/virtio_mmio.c`
```C
#include "virtio_mmio.h"
#include "io_map.h"
#include <stdio.h>
#include <string.h>
struct virtio_mmio_dev g_virtio_mmio0;
static uint8_t virtio_mmio_read8(uint32_t addr, void *opaque)
{
struct virtio_mmio_dev *dev = opaque;
uint32_t offset = addr - VIRTIO_MMIO_BASE;
switch (offset) {
case 0x00: // MAGIC: lower 8 bits
case 0x01:
case 0x02:
case 0x03: {
uint32_t val = dev->magic_value;
int shift = (offset - 0x00) * 8; // 0,8,16,24
return (val >> shift) & 0xFF;
}
case 0x04: // VERSION (lower 8 bits)
return (uint8_t)dev->version;
default:
return 0;
}
}
static uint16_t virtio_mmio_read16(uint32_t addr, void *opaque)
{
struct virtio_mmio_dev *dev = opaque;
uint32_t offset = addr - VIRTIO_MMIO_BASE;
switch (offset) {
case 0x00: // MAGIC: lower 16 bits
case 0x02: {
uint32_t val = dev->magic_value;
int shift = (offset & 0x2) * 8; // offset=0 => shift=0; offset=2 => shift=16
return (val >> shift) & 0xFFFF;
}
case 0x04: // VERSION (lower 16 bits)
return (uint16_t)dev->version;
default:
return 0;
}
}
static uint32_t virtio_mmio_read32(uint32_t addr, void *opaque)
{
struct virtio_mmio_dev *dev = opaque;
uint32_t offset = addr - VIRTIO_MMIO_BASE;
switch (offset) {
case 0x00: // MAGIC
return dev->magic_value;
case 0x04: // VERSION
return dev->version;
case 0x08: // DEVICE_ID
return dev->device_id;
default:
return 0;
}
}
/*
* Write callbacks must return `int`,
* matching `io_write8_fn_t`, `io_write16_fn_t`, `io_write32_fn_t`.
* Return 1 if handled, 0 if not handled.
*/
static int virtio_mmio_write8(uint32_t addr, uint8_t val, void *opaque)
{
// Example: Not actually handling offset except for demonstration
// Return 1 if handled, 0 if not
(void)addr;
(void)val;
(void)opaque;
return 0;
}
static int virtio_mmio_write16(uint32_t addr, uint16_t val, void *opaque)
{
// Example: Not actually handling offset except for demonstration
// Return 1 if handled, 0 if not
(void)addr;
(void)val;
(void)opaque;
return 0;
}
static int virtio_mmio_write32(uint32_t addr, uint32_t val, void *opaque)
{
struct virtio_mmio_dev *dev = opaque;
uint32_t offset = addr - VIRTIO_MMIO_BASE;
switch (offset) {
case 0x60: // STATUS
dev->status = val;
printf("virtio_mmio: STATUS <- 0x%x\n", val);
return 1;
default:
break;
}
return 0;
}
void virtio_mmio_init(void)
{
memset(&g_virtio_mmio0, 0, sizeof(g_virtio_mmio0));
g_virtio_mmio0.magic_value = 0x74726976; // 'triv'
g_virtio_mmio0.version = 2;
g_virtio_mmio0.device_id = 2; // block device
g_virtio_mmio0.vendor_id = 0x554D4552; // 'UMER'
g_virtio_mmio0.queue_num_max = 128;
map_io_register_ex(
VIRTIO_MMIO_BASE,
VIRTIO_MMIO_SIZE,
virtio_mmio_read8,
virtio_mmio_read16,
virtio_mmio_read32,
virtio_mmio_write8,
virtio_mmio_write16,
virtio_mmio_write32,
&g_virtio_mmio0
);
printf("virtio_mmio_init: successfully registered mmio dev\n");
}
```
`src/io_map.h`
```C
#ifndef IO_MAP_H
#define IO_MAP_H
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
/* Functions used by the CPU/bus to perform read/write */
uint8_t io_map_read8(uint32_t addr);
uint16_t io_map_read16(uint32_t addr);
uint32_t io_map_read32(uint32_t addr);
int io_map_write8(uint32_t addr, uint8_t val);
int io_map_write16(uint32_t addr, uint16_t val);
int io_map_write32(uint32_t addr, uint32_t val);
/* Callback types for read and write operations of various sizes */
typedef uint8_t (*io_read8_fn_t) (uint32_t addr, void *opaque);
typedef uint16_t (*io_read16_fn_t)(uint32_t addr, void *opaque);
typedef uint32_t (*io_read32_fn_t)(uint32_t addr, void *opaque);
/* Write callbacks must return int: 1 = handled, 0 = not handled */
typedef int (*io_write8_fn_t) (uint32_t addr, uint8_t val, void *opaque);
typedef int (*io_write16_fn_t)(uint32_t addr, uint16_t val, void *opaque);
typedef int (*io_write32_fn_t)(uint32_t addr, uint32_t val, void *opaque);
/**
* Registers the address range [base, base + size) into the emulator's I/O map,
* allowing 8/16/32-bit read and write operations within this range.
*/
void map_io_register_ex(uint32_t base,
uint32_t size,
io_read8_fn_t read8_cb,
io_read16_fn_t read16_cb,
io_read32_fn_t read32_cb,
io_write8_fn_t write8_cb,
io_write16_fn_t write16_cb,
io_write32_fn_t write32_cb,
void *opaque);
#ifdef __cplusplus
}
#endif
#endif /* IO_MAP_H */
```
`src/io_map.c`
```C
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "io_map.h"
#include "map.h" // Red-black tree or your chosen data structure
#include "io.h" // If fallback to memory_xxx is needed within these calls (optional)
typedef struct io_region_s {
uint32_t base;
uint32_t size;
io_read8_fn_t read8;
io_read16_fn_t read16;
io_read32_fn_t read32;
io_write8_fn_t write8;
io_write16_fn_t write16;
io_write32_fn_t write32;
void *opaque; // Device-specific pointer
} io_region_t;
/* Compare two base addresses as keys */
static map_cmp_t region_comparator(const void *key1, const void *key2)
{
const uint32_t *a = key1;
const uint32_t *b = key2;
if (*a < *b) return _CMP_LESS;
if (*a > *b) return _CMP_GREATER;
return _CMP_EQUAL;
}
static map_t g_io_map = NULL;
static void ensure_map_initialized(void)
{
if (!g_io_map) {
g_io_map = map_new(sizeof(uint32_t), sizeof(io_region_t),
region_comparator);
}
}
void map_io_register_ex(uint32_t base,
uint32_t size,
io_read8_fn_t read8_cb,
io_read16_fn_t read16_cb,
io_read32_fn_t read32_cb,
io_write8_fn_t write8_cb,
io_write16_fn_t write16_cb,
io_write32_fn_t write32_cb,
void *opaque)
{
ensure_map_initialized();
io_region_t region = {
.base = base,
.size = size,
.read8 = read8_cb,
.read16 = read16_cb,
.read32 = read32_cb,
.write8 = write8_cb,
.write16 = write16_cb,
.write32 = write32_cb,
.opaque = opaque
};
map_insert(g_io_map, &base, ®ion);
printf("map_io_register_ex: [0x%x .. 0x%x] registered\n",
base, base + size - 1);
}
uint8_t io_map_read8(uint32_t addr)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0xFF; // Not in any region
io_region_t *r = (io_region_t *)it.node->data;
/* Check range */
if (addr < r->base || addr >= r->base + r->size)
return 0xFF;
if (r->read8)
return r->read8(addr, r->opaque);
return 0xFF;
}
uint16_t io_map_read16(uint32_t addr)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0xFFFF;
io_region_t *r = (io_region_t *)it.node->data;
if (addr < r->base || addr >= r->base + r->size)
return 0xFFFF;
if (r->read16)
return r->read16(addr, r->opaque);
return 0xFFFF;
}
uint32_t io_map_read32(uint32_t addr)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0xFFFFFFFF;
io_region_t *r = (io_region_t *)it.node->data;
if (addr < r->base || addr >= r->base + r->size)
return 0xFFFFFFFF;
if (r->read32)
return r->read32(addr, r->opaque);
return 0xFFFFFFFF;
}
/* Write functions return int: 1 if handled, 0 if not. */
int io_map_write8(uint32_t addr, uint8_t val)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0;
io_region_t *r = (io_region_t *)it.node->data;
if (addr < r->base || addr >= r->base + r->size)
return 0;
if (r->write8)
return r->write8(addr, val, r->opaque);
return 0;
}
int io_map_write16(uint32_t addr, uint16_t val)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0;
io_region_t *r = (io_region_t *)it.node->data;
if (addr < r->base || addr >= r->base + r->size)
return 0;
if (r->write16)
return r->write16(addr, val, r->opaque);
return 0;
}
int io_map_write32(uint32_t addr, uint32_t val)
{
ensure_map_initialized();
map_iter_t it;
map_find(g_io_map, &it, &addr);
if (map_at_end(g_io_map, &it))
return 0;
io_region_t *r = (io_region_t *)it.node->data;
if (addr < r->base || addr >= r->base + r->size)
return 0;
if (r->write32)
return r->write32(addr, val, r->opaque);
return 0;
}
```
`src/devices.c`
```C
#include "io.h"
#include "io_map.h"
static inline uint8_t bus_read8(uint32_t addr)
{
uint8_t val = io_map_read8(addr);
if (val == 0xFF) { // Sentinel value indicates IO map did not intercept
return memory_read_b(addr); // Fallback to normal RAM read
}
return val;
}
static inline uint16_t bus_read16(uint32_t addr)
{
uint16_t val = io_map_read16(addr);
if (val == 0xFFFF) { // Sentinel value for 16-bit
return memory_read_s(addr); // Fallback to normal RAM read
}
return val;
}
static inline uint32_t bus_read32(uint32_t addr)
{
uint32_t val = io_map_read32(addr);
if (val == 0xFFFFFFFF) { // Sentinel value for 32-bit
return memory_read_w(addr); // Fallback to normal RAM read
}
return val;
}
static inline void bus_write8(uint32_t addr, uint8_t val)
{
if (!io_map_write8(addr, val)) { // If IO map didn't handle, write to RAM
memory_write_b(addr, &val);
}
}
static inline void bus_write16(uint32_t addr, uint16_t val)
{
if (!io_map_write16(addr, val)) {
memory_write_s(addr, (const uint8_t *)&val);
}
}
static inline void bus_write32(uint32_t addr, uint32_t val)
{
if (!io_map_write32(addr, val)) {
memory_write_w(addr, (const uint8_t *)&val);
}
}
/*
Make sure you call bus_read8/bus_read16/bus_read32 and bus_write8/bus_write16/bus_write32
in your CPU instruction decode (lb/lh/lw/sb/sh/sw) so that MMIO devices can be intercepted.
*/
```
`Makefile`
```c
$(OUT)/virtio-blk.o: src/virtio-blk.c
$(VECHO) " CC\t$@\n"
$(Q)$(CC) -o $@ $(CFLAGS) $(CFLAGS_emcc) -c -MMD -MF $@.d $<
$(OUT)/virtio-mmio.o: src/virtio-mmio.c
$(VECHO) " CC\t$@\n"
$(Q)$(CC) -o $@ $(CFLAGS) $(CFLAGS_emcc) -c -MMD -MF $@.d $<
$(OUT)/io_map.o: src/io_map.c
$(VECHO) " CC\t$@\n"
$(Q)$(CC) -o $@ $(CFLAGS) $(CFLAGS_emcc) -c -MMD -MF $@.d $<
$(OUT)/devices.o: src/devices.c
$(VECHO) " CC\t$@\n"
$(Q)$(CC) -o $@ $(CFLAGS) $(CFLAGS_emcc) -c -MMD -MF $@.d $<
...
OBJS := \
...
virtio_mmio.o \
io_map.o \
devices.o \
virtio-blk.o \
```
`assets/system/configs/linux.config`
```
CONFIG_CMDLINE="root=/dev/vda rw console=ttyS0"
```
`src/devices/minimal.dts`
```
soc: soc@F0000000 {
...
virtio_mmio@5000000 {
compatible = "virtio,mmio";
reg = <0x5000000 0x1000>;
interrupt-parent = <&plic0>;
interrupts = <2>;
status = "okay";
};
};
```
## Demo
Console on web
```
...
[ 0.000000] Kernel command line: root=/dev/vda rw console=ttyS0
...
[ 0.196825] virtio-mmio f5000000.virtio_mmio: Wrong magic value 0x00000000!
...
Welcome to Buildroot
buildroot login:
```
:::danger
Avoid using screenshots when you can simply include the text directly.
:::
## Progress
I have implemented a VirtIO block and virtio-mmio device simulation in my custom RISC-V emulator. I created map_io_register_ex(...) to handle 8/16/32-bit read/write callbacks, allowing the kernel to access the device’s registers. However, the kernel still reads 0x00000000 instead of the correct magic value. The problem is that the CPU core’s load/store instructions are directly accessing memory (memory_read_*), bypassing my bus_read*/bus_write* system. This prevents my virtio-mmio callbacks from being invoked. I need to modify the CPU’s instruction decode/execution to ensure all memory accesses, especially in IO regions, go through my bus_read*/bus_write* handlers. Once fixed, Linux should recognize the correct magic value and properly detect /dev/vda.
Sign in with Wallet
Connect another wallet