## Trebuchet Notes

This is a bunch of unstructured notes compiled for Trebuchet.

### Resources

- [MIPS64 Privileged Resource Architecture](https://s3-eu-west-1.amazonaws.com/downloads-mips/documents/MD00091-2B-MIPS64PRA-AFP-05.04.pdf)
- [MIPS Memory Map](https://training.mips.com/basic_mips/PDF/Memory_Map.pdf)
- [MIPS Exceptions Overview](https://training.mips.com/basic_mips/PDF/Exceptions.pdf)
- [MIPS R4000 Manual](https://course.ece.cmu.edu/~ece447/s15/lib/exe/fetch.php?media=mips_r4000_users_manual.pdf)

### Linux Kernel Source Notes

- see arch/mips/kernel/traps.c for the interrupt install logic
- main interrupt handler is implemented by [`except_vec3_generic`](https://livegrep.com/search/linux?q=except_vec3_generic&fold_case=auto&regex=false&context=true). And it's installed at 0x80 and 0x180.
- mips entry point is at [arch/mips/kernel/head.S](https://github.com/torvalds/linux/blob/v6.13/arch/mips/kernel/head.S#L85)
- Kernel quirk: See [arch/mips/mm/tlb-r4k.c](https://github.com/torvalds/linux/blob/v6.13/arch/mips/mm/tlb-r4k.c#L535)
- The ELPA bit is only set in `PageGrain` if the cpu "has rixi". So in order to take advantage of large addressing trebuchet may have to support rixi.

### Linux Kconfig

These are the settings you'll find in `make menuconfig`:
- System type - Malta board system
- Endianess - Big endian
- CPU Selection - MIPS64r2
- Kernel type / code model - 64-bit
- custom initramfs source (under General setup)
- (optional) Disable compressed initramfs (so `none` compression type) for faster emulation at linux boot.

### Building MIPS Kernel Image
Make sure you have the essential cross-compile packages. On Ubuntu:

sudo apt-get install crossbuild-essential-mips64 gcc-mips64-linux-gnuabi64

Then build the kernel:

export ARCH=mips
EXPORT CROSS_COMPILE=mips64-linux-gnuabi64-
git clone https://github.com/torvalds/linux
cd linux
# ensure you have the core kernel build tools
sudo apt-get install build-essential bc bison flex libssl-dev libelf-dev
cp /path/to/custom.kconfig .config
make defconfig
make -j8

### Building MIPS64 Go program

env GOOS=linux GOARCH=mips64 GOMIPS64=softfloat go build -o hello.mips64 ./main.go

## R4000 Notes

The R4000 processor supports the following interrupts: six hardware interrupts, one internal “timer interrupt,” two software interrupts, and one nonmaskable interrupt. The processor takes an exception on any interrupt.
### Exception Vector Locations
#### Exception Vector Base Addresses
The 64-bit-mode vector base address for all exceptions; the 32-bit mode address is the low-order 32 bits (for instance, the base address for NMI in 32-bit mode is 0xBFC0 0000):

| Exception | BEV 0 | BEV 1 |
| ---------------------- | --------------------- | --------------------- |
| Cache Error | 0xFFFF FFFF A000 0000 | 0xFFFF FFFF BFC0 0200 |
| Others | 0xFFFF FFFF 8000 0000 | 0xFFFF FFFF BFC0 0200 |
| Reset, NMI, Soft Reset | 0xFFFF FFFF BFC0 0000 | 0xFFFF FFFF BFC0 0000 |

#### Exception Vector Offsets

The vector offset added to the base address to create the exception address

| Exception | R4000 Processor Vector Offset |
| ------------------------------------- | ----------------------------- |
| TLB refill, EXL = 0 | 0x000 |
| XTLB refill, EXL = 0 (X = 64-bit TLB) | 0x080 |
| Cache Error | 0x100 |
| Others | 0x180 |
| Reset, Soft Reset, NMI | None |

### Exception Priority Order
- Reset
- Soft Reset
- NMI
- Address error - instruction fetch
- TLB refill - instruction fetch
- TLB invalid - instruction fetch
- Cache error - instruction fetch
- Virtual Coherency - instruction fetch
- Bus Error - instruction fetch
- Integer Overflow, Trap, System Call, Breakpoint, Reserved Instruction, Coprocessor Unusable
- Address Error - data access
- TLB refill - data access
- TLB invalid - data access
- TLB modified - data write
- Cache error - data access
- Watch
- Virtual Coherency – Data access
- Bus error – Data access
- Interrupt (lowest priority)

### Emulating using Qemu

Emulating the mips64 bzImage on qemu using a custom init:

 qemu-system-mips64 -singlestep -d in_asm,cpu -M malta -cpu MIPS64R2-generic -kernel vmlinux -nographic -append "console=ttyS0 rdinit=/bin/hello.mips64"

Useful to quickly check how qemu emulates certain instructions. The `-singlestep -d in_asm,cpu` flags will dump the CPU state of every executed instruction.

## TODOs

### Performance-related open questions.

- Cache line size (see the `Config` register). It's 8K for I and D cache on the r4k. But might be worth tweaking. But I wouldn't be surprised if Linux doesn't handle non-standard sizes correctly here.

### Hardware Page Table Walk?
Basically to assist in page fault handling by checking page tables in the emulator, rather than rely on Linux to do this. This should reduce the overhead of emualting page faults. But we have to ensure linux supports this before emualting. This can be dealt with later if we see performance issues in the VM.

## CP0 Notes

### LLAddr
Contains the phyiscal address read by the most recent load linked instruction. The ISA states it's for diagnostic purposes only. But for trebuchet, it might be required to properly emulate and use for RMW emulation. Similar to how its counterpart is used in cannon.

### Config5
No need to reset. Segmentation control is a MIPS64 release 3 feature and not supported by Trebuchet. Same deal with EVA.
However, we will need to support RI/XI due to a Linux quirk.

### PRId
On r4k, this should be set to 0x04.

### PageMask

Always zeroed for 4KB pages.