# Assignment 3: 黃偉宸

## Requirements
* Following the instructions of Lab3: Reindeer - RISCV RV32I[M] Soft CPU, you shall modify the assembly programs used/done with Assignment1 or Assignment2 as new test case(s) for Reindeer Simulation with Verilator
* The following reference may help you to get familiar what to do in this assigmaent:
* [RISC-V Compliance Tests](https://github.com/riscv/riscv-compliance/blob/master/doc/README.adoc)
* [riscv-tests](https://github.com/riscv/riscv-tests)
* [Reindeer](https://github.com/PulseRain/Reindeer)

A test program for RISC-V is written within a single assembly language file, which is passed through the C preprocessor, and all regular assembly directives can be used. An example test program is shown below. Each test program should first include the `riscv_test.h` header file, which defines the macros used by the TVM(Test Virtual Machine)

The header file will have different contents depending on the target environment for which the test will be built. One of the goals of the various TVMs is to allow the same test program to be compiled and run on very different target environments yet still produce the same results.

#include "compliance_test.h"
#include "compliance_io.h"
#include "test_macros.h"

* The three needed header file to write compliance code can be found in the following github:
* compliance_test.h : [riscv-compliance/riscv-target/riscvOVPsim/compliance_test.h](https://github.com/riscv/riscv-compliance/blob/master/riscv-target/riscvOVPsim/compliance_test.h)
* compliance_io.h : [riscv-compliance/riscv-target/riscvOVPsim/compliance_io.h](https://github.com/riscv/riscv-compliance/blob/master/riscv-target/riscvOVPsim/compliance_io.h)
* test_macros.h: [riscv-tests/isa/macros/scalar/test_macros.h](https://github.com/riscv/riscv-tests/blob/master/isa/macros/scalar/test_macros.h)
* riscv_test.h : [riscv-test-env/v/riscv_test.h](https://github.com/riscv/riscv-test-env/blob/master/v/riscv_test.h)

Use the RVTEST macros (defined in `compliance_io.h`) to make it easy to see the details of a Test’s execution. There are macros for assertions (`RVTEST_IO_ASSERT_GPR_EQ`) and tracing (`RVTEST_IO_WRITE_STR`) which are empty on targets that can not implement them.

Each test program must next specify for which TVM it is designed by including the appropriate TVM macro, `RVTEST_RV64U` in this example. This specification can change the way in which subsequent macros are interpreted, and supports a static check of the TVM functionality used by the program.

# Test Virtual Machine (TVM) used by program.
RV_COMPLIANCE_RV32M /*TVM macro*/

The test program will begin execution at the first instruction after `RVTEST_CODE_BEGIN`, and continue until execution reaches an `RVTEST_PASS` macro or the `RVTEST_CODE_END macro`, which is implicitly a success. A test can explicitly fail by invoking the `RVTEST_FAIL `macro.

# Test code region
RV_COMPLIANCE_CODE_BEGIN
/*test code */
 RV_COMPLIANCE_DATA_END

I analyzed the [I-ADD-01.S](https://github.com/riscv/riscv-compliance/blob/master/riscv-test-suite/rv32i/src/I-ADD-01.S) file to get a clue to write my own combined language file as a test case.
The following macros are frequently appered in I-ADD-01.S:
* RVTEST_IO_ASSERT_GPR_EQ(x31, x0, 0x00000000):
* `RVTEST_IO_ASSERT_GPR_EQ(_SP, _R, _I) `: The marco is used to check whther value in `_R` is the same as value specified in `_I`
*

## How to run make test
I put all my repository needed in my `$Home` folder, which is `/home/steven/` and I assume you have already installed the risc-v compiler.

$ cd $Home
$ git clone https://github.com/riscv/riscv-test-env.git
$ git clone https://github.com/riscv/riscv-tests
$ git clone https://github.com/riscv/riscv-compliance
$ git clone https://github.com/PulseRain/Reindeer.git
$ cd /home/steven/Reindeer/sim/verilator
$  make test I-ANDI-01                     

After typing the following command you will find `I-ADD-01.vcd` been generated in the current folder.
The vcd file will be used to do wave analysis for the verilator program, which we will mention later.

====> Testing ./obj_dir/VPulseRain_RV2T_MCU
TEST CASE: I-ANDI-01
testing ../compliance/I-ANDI-01.elf  -r ../compliance/references/I-ANDI-01.reference_output

=============================================================
=== PulseRain Technology, RISC-V RV32IM Test Bench
=============================================================

     elf file 	: ../compliance/I-ANDI-01.elf
     reference	: ../compliance/references/I-ANDI-01.reference_output

     start address 		= 0x80000000
     begin signature address 	= 0x80002030
     end signature address 	= 0x800020e0

=============> reset...
=============> init stack ...
=============> load elf file...
...

## How to add elf file into test suite
1. Add elf file (say `I-ADD-01.elf` ) to [Reindeer/sim/compliance/](https://github.com/PulseRain/Reindeer/tree/master/sim/compliance)
2. Write the corresponding reference_output file and add it to the /home/steven/Reindeer/sim/compliance/references folder（I-ADD-01.reference_output）
3. Modify the makefile in [Reindeer/sim/verilator/Makefile] (https://github.com/PulseRain/Reindeer/blob/master/sim/verilator/Makefile) to add the file to be used for test, but I have to understand the code below.

test_run_case = $(filter-out $@, $(MAKECMDGOALS))

count_test_run_case = $(words $(test_run_case))
count_test_all = $(words $(all_test_cases))

build : $(target)

test : build
	@echo "====> Testing $(target)"    
	@echo "TEST CASE: $(test_run_case)"
	@for t in $(test_run_case) ; do \
		elf_file=$(elf_folder)/$$t.elf ;\
		ref_file=$(ref_folder)/$$t.reference_output ;\
		echo "testing $$elf_file  -r $$ref_file" ; \
		./obj_dir/V$(src_top) $$elf_file  -r $$ref_file -t $$t.vcd || exit ; \
		sleep 1 ; \
	done
	@echo "====> Test PASSED, Total of $(count_test_run_case) case(s)"     

* Make will set the special variable `MAKECMDGOALS` to the list of goals you specified on the command line. If no goals were given on the command line, this variable is empty. Note that this variable should be used only in special circumstances.
* In this makefile，`MAKECMDGOALS` = `make test I-ADD-01`
* The actual call test command is `./obj_dir/V$(src_top) $$elf_file -r $$ref_file -t $$t.vcd` in the makefile above and this string is printed by me.
* `./obj_dir/VPulseRain_RV2T_MCU ../compliance/I-ADD-01.elf -r ../compliance/references/I-ADD-01.reference_output -t I-ADD-01.vcd`
* It can be seen that in order to successfully test the test code in the reindeer, compile the .elf file into `/home/steven/Reindeer/sim/compliance` and add the .reference_output file according to the expected output to `/home/steven/Reindeer/sim/compliance/references`
* After that, the corresponding vcd file (I-ADD-01.vcd) can be used for Wave trace..

## Reindeer ReadMe
* As mentioned early, the Reindeer soft CPU uses an OCD to load code/data. And for the verilator simulation, a C++ testbench will replace the OCD. The testbench will invoke the toolchain (objdump, readelf) to extract code/data from sections of the .elf file. The testbench will mimic the OCD bus to load the code/data into CPU's memory. Afterwards, the start-address of the .elf file ("`_start`" or "`__start`" symbol) will be passed onto the CPU, and turn the CPU into active state.
* The C++ testbench above been refered is [Reindeer/sim/verilator/tb_PulseRain_RV2T.cpp](Reindeer/sim/verilator/tb_PulseRain_RV2T.cpp)
* the test bench will automatically extract the address for begin_signature and end_signature symbol.
The compliance test will utilize the hold-and-load feature of the PulseRain Reindeer soft CPU, and do the following:

1. Reset the CPU, put it into hold state
2. Call upon toolchain to extract code/data from the .elf file for the test case
3. Start the CPU, run for 2000 clock cycles
4. Reset the CPU, put it into hold state for the second time
5. Read the data out of the memory, and compare them against the reference signature
6. 

## How to compile test-case.S
* Reference this issue : [How to compile the test code dot S #](https://github.com/riscv/riscv-tests/issues/168)
* Reference this makefile : https://github.com/riscv/riscv-tests/blob/master/isa/Makefile
* There are also makefiles in `/home/steven/riscv-compliance/riscv-test-suite/rv32i` that can compile .S files into .elf just need to set the two variables `RISCV_TARGET` and `RISCV_DEVICE`.
* Add in the makefile：
1. TARGETDIR := /home/steven/riscv-compliance/riscv-target
2. RISCV_TARGET := riscvOVPsim
3. RISCV_DEVICE := rv32i
* However, this will still result in an error in the `/home/steven/riscv-compliance/riscv-target/riscvOVPsim/device/rv32i/Makefile.include` because the corresponding riscvOVPsim.exe path won't be found. You have to do further to `Makefile.include` to deal with the error：
* change `$(ROOTDIR)/riscv-ovpsim/bin/$(ARCH)/riscvOVPsim.exe` in makefile to `TARGET_SIM ?= /home/steven/riscv-compliance/riscv-ovpsim/bin/Linux64/riscvOVPsim.exe`

Let's continue to explore `/home/steven/riscv-compliance/riscv-test-suite/rv32i/Makefile`
In order to be able to successfully use the .S file to compile the .elf file, what else do we need to do?
* `vpath %.S $(act_dir)` : will use the vpath function to ask make to find all .S files under `$(act_dir) ` (`act_dir = .` defined as the current directory).
* To solve the problem of #include `"test_macros.h" `compilation terminated, add `-I/home/steven/riscv-compliance/riscv-test-env \` to `makefile.include`

As a result, typing `make I-ADD-01.elf` in /home/steven/riscv-compliance/riscv-test-suite/rv32i will generate I-ADD-01.elf in /home/steven/work And I-ADD-01.elf.objdump file, then move I-ADD-01.elf to /home/steven/Reindeer/sim/compliance, then enter make test I-ADD-01 at the terminal Complete the test and output the I-ADD-01.vcd file.
The detailed compilation process is underneath:

 $ make I-ADD-01.elf    
 
 riscv64-unknown-elf-gcc -march=rv32i -mabi=ilp32 -static -mcmodel=medany -fvisibility=hidden -nostdlib -nostartfiles \
 -I/home/steven/riscv-compliance/riscv-test-env  \
 -I/home/steven/riscv-test-env/ -I/home/steven/riscv-test-env/p/  \
 -I/home/steven/riscv-compliance/riscv-target/riscvOVPsim/  \
 -T/home/steven/riscv-test-env/p/link.ld \
 /home/steven/riscv-compliance/riscv-test-suite/rv32i/src/I-ADD-01.S \
 -o /home/steven/work/I-ADD-01.elf;
 riscv64-unknown-elf-objdump -D /home/steven/work/I-ADD-01.elf > /home/steven/work/I-ADD-01.elf.objdump

In order to be able to successfully compile the test-case that you have added, you have to modify it at /home/steven/riscv-compliance/riscv-test-suite/rv32i/Makefrag, you have to add your own test-case under `rv32i_sc_tests = \` This way you can perform the above compilation (say `make reverse.elf`).

## bit-reverse
In the previous homework1 I wrote the following combined language program. First we have to change this code to the version of match test-case:

.data

.text
main:
reverse:
	li,a3,0x55000011
	li a5,0
	li a4,0
	li a2,32
loop:
	
	slli a5,a5,0x1 # r <<= 1
	andi a6,a3,0x1 # a6 = v & 0x01
	or a5,a5,a6    # r|= (v&0x1)
	srli a3,a3,1   # v >>= 1
	addi a4,a4,1
	bne a2,a4,loop
	mv a3,a5
	# Exit program

Rewrite the code above to the version that matches tets-suits:

#include "compliance_test.h"
#include "compliance_io.h"
#include "test_macros.h"

# Test Virtual Machine (TVM) used by program.
RV_COMPLIANCE_RV32M

# Test code region.
RV_COMPLIANCE_CODE_BEGIN
    RVTEST_IO_INIT
    RVTEST_IO_ASSERT_GPR_EQ(x31, x0, 0x00000000)
    RVTEST_IO_WRITE_STR(x31, "# Test Begin\n")
    # ---------------------------------------------------------------------------------------------
    RVTEST_IO_WRITE_STR(x31, "# Test part A1 - general test of value 0x55000011 register values\n");
    # Addresses for test data and results
    la      x1, test_A1_data
    la      x2, test_A1_res
    # Load testdata
    lw      x3, 0(x1)
    # Register initialization
    li      x4, 0
    li      x5, 0
    li      x6, 32
    li      x7, 0
    # Test
loop:
    slli    x4,x4,0x1 
	andi    x7,x3,0x1 
	or      x4,x4,x7    
	srli    x3,x3,1   
	addi    x5,x5,1
	bne     x6,x5,loop
	mv      x3,x4
    # Store results
    sw      x3, 0(x2)
    //
    // Assert
    //
    RVTEST_IO_CHECK()
    RVTEST_IO_ASSERT_GPR_EQ(x2, x3, 0x880000AA)
    RVTEST_IO_WRITE_STR(x31, "# Test part A1  - Complete\n");

# ---------------------------------------------------------------------------------------------
    RVTEST_IO_WRITE_STR(x31, "# Test part A2 - general test of value 0x880000AA register values\n");
     # Addresses for test data and results
    la      x1, test_A2_data
    la      x2, test_A2_res
    # Load testdata
    lw      x3, 0(x1)
    # Register initialization
    li      x4, 0
    li      x5, 0
    li      x6, 32
    li      x7, 0
    # Test
loop2:
    slli    x4,x4,0x1 
	andi    x7,x3,0x1 
	or      x4,x4,x7    
	srli    x3,x3,1   
	addi    x5,x5,1
	bne     x6,x5,loop2
	mv      x3,x4
    # Store results
    sw      x3, 0(x2)
     //
    // Assert
    //
    RVTEST_IO_CHECK()
    RVTEST_IO_ASSERT_GPR_EQ(x2, x3, 0x55000011)
    RVTEST_IO_WRITE_STR(x31, "# Test part A2  - Complete\n");
RV_COMPLIANCE_HALT
RV_COMPLIANCE_CODE_END

# Input data section.
    .data

test_A1_data:
    .word 0x55000011
test_A2_data:
    .word 0x880000AA
#Output data section.
RV_COMPLIANCE_DATA_BEGIN
    .align 4

test_A1_res:
    .fill 1, 4, -1
test_A2_res:
    .fill 1, 4, -1

RV_COMPLIANCE_DATA_END

Then modify the /home/steven/Reindeer/sim/verilator/Makefile to add Reverse to `all_test_cases = \ `.
Then go to `/home/steven/riscv-compliance/riscv-test-suite/rv32i ` and type `make Reverse.elf` and will find it `Reverse.elf.objdump` and `Reverse.elf` two files in `/home/steven/work`
Finally, throw these two files into `/home/steven/Reindeer/sim/compliance` and type `make test Reverse` in ` /home/steven/Reindeer/sim/verilator` to test according to the expected And output the `Reverse.vcd` file.
It's not all done here, we haven't produced the output data of the `Reverse.out32` file .
Continue to explore `/home/steven/riscv-compliance/riscv-test-suite/rv32i/Makefile ` to find the following paragraph from it:

%.out32: %.elf
	$(RUN_TARGET)

Executed under `/home/steven/riscv-compliance/riscv-test-suite/rv32i/`

$ make Reverse.out32     

You will get the `Reverse.out32` and `Reverse.signature.output` files in `/home/steven/work`, and you should do a make test again sooner.

## Waveform analysis via GtkWave
Check the generated VCD file and use GTKwave to view the waveform. Then, explain how your program is executed along with Reindeer Simulation.
在我們開始之前，得先了解在 assmebly 中各個 "section" 所代表的意義：
Open `Reverse.vcd` generated by Verilator:
可以由下圖看出：
1. 這是在上升邊緣將資料寫入 register 的設計。
2. mem_data_in_reg 這個 singal 為定義在 RV2T_data_access.v 中的 wire `output reg [XLEN - 1 : 0] mem_data_to_write` 利用先前對 `Reverse.elf`所產生的測試結果以及對照觀察下圖中可以看出

     ...
     start address 		= 0x80000000
     begin signature address 	= 0x80002010
     end signature address 	= 0x80002020
     ...

* 起始位址為 0x80000000 ，而觀察 `Reverse.elf` 的測試結果兩者相符，得經歷兩個 clk 才會更改 mem_data_in_reg 內的數值。

 =============> start running ...
	...
	0005 80000000  04c0006f  
	0006 80000000  04c0006f  
	0007 80000004  34202f73  
	0008 80000004  34202f73  
	0009 80000008  00800f93  
	0010 80000008  00800f93
  ...


## Explain how Reindeer works with Verilator.
The resulting design in Reindeer soft CPU is to be simulated using Verilator.
we can check the [Reindeer/build/synth/Microsemi/Reindeer.prj](https://github.com/PulseRain/Reindeer/blob/master/build/synth/Microsemi/Reindeer.prj) for more detail .

## What is “Hold and Load”? And, how the simulation does for bootstraping?
In Reindeer soft cpu OCD means on-chip debugger which is used to controll the mux connected to the memory.

In Reindeer soft cpu desing the soft CPU and the OCD can share the same UART port. The RX signal goes to both the soft CPU and OCD, while the TX signal has to go through a mux. And that mux is controlled by the OCD.

After reset, the soft CPU will be put into a hold state, and it will have access to the UART TX port by default. But a valid debug frame sending from the host PC can let OCD to reconfigure the mux and switch the UART TX to OCD side, for which the memory can be accessed, and the control frames can be exchanged. A new software image can be loaded into the memory during the CPU hold state, which gives rise to the name "hold-and-load".

## Can you show some signals/events inside Reindeer and describe?