FPGA Group6 Final Project - Systolic Array as an CNN Accelerator

文章網址

Team Members

姓名	學號	課程
劉永勝	F94089032	FPGA / AOC
蔡宗瑾	E14083137	FPGA / AOC
謝懷頡	M16124172	AOC
李宇洋	E24099025	FPGA

Introduction

The Convolutional Neural Network(CNN) is widly used both in image-processing domain and machine learning related field. CNN involves numerous MAC(multiply accumulate) operations, the amount of data a CNN model nowadays required to process is enormous, and the goal is to accelerate this process. Many ideas have been proposed, systolic array is one of them, the unique way of data flow in systolic array reduces the data movement from off-chip DRAM to on-chip buffer, it not only saves energy consumption but also increases the througput rate.
In our workshop, we implemented a systolic array of size 8 by 8, the main architecture is in reference of the Google TPU.

CNN Model

• CNN overview:
  Input feature map(ifmap) of size
  $N\ast H\ast W\ast C$ convolves with filter(kernel) size of
  $M\ast R\ast S\ast C$ will get a
  $N\ast E\ast F\ast M$ output feature map(ofmap). In the figure belowed,
  $N$ is considered as 1,
  $E$ and
  $F$ can be obtained form
  $(H-R+2P+U)/U$, where
  $P$ is the padding number, and
  $U$ is the stride number.
  

Shape Parameter	Description
N	Batch size of 3D ifmaps
M	# of 3D filters/ # of ofmap channels
C	# of ifmap/filter channels
H/W	ifmap plane hieght/width
R/S	filter plane height/width
E/F	ofmap plane height/width

• Model Description:
  Using Fashion Mnist as our targeted dataset and build a small model as followed:
  
• Model summary:

We only aim at accelerating the convolutional layers for now, fully connected layers are left for future work.

Layers	Input Activations $H\ast W\ast C$	Kernels $R\ast S\ast C\ast M$	Output Activations $E\ast F\ast M$
Layer1	$32\ast 32\ast 1$	$3\ast 3\ast 1\ast 8$	$32\ast 32\ast 8$
Layer2	$16\ast 16\ast 8$	$3\ast 3\ast 8\ast 16$	$16\ast 16\ast 16$
Layer3	$8\ast 8\ast 16$	$3\ast 3\ast 16\ast 32$	$8\ast 8\ast 32$
Layer4	$4\ast 4\ast 32$	$3\ast 3\ast 32\ast 64$	$4\ast 4\ast 64$
Layer5	$2\ast 2\ast 64$	$3\ast 3\ast 64\ast 128$	$2\ast 2\ast 128$

Data load and Quantization(python)

• data load
  

• Quantization
  

• weight value distributed
  

  
  
  
  
  
  
  
  
  
  
  
  

• bias value distributed
  

  
  
  
  
  
  
  
  
  
  
  
  

GEMM

• GEMM stands for GEneral Matrix to Matrix Multiplication. What is does is to flatten multi-dimensional data into 2-D matrices and do matrix multiplication.

• Convolution applies each filter across the whole image. The diagram belowed shows how input image can be packed into multiple patches to form a 2-D matrix by im2col algorithm. The diagram is a example of stride equals to kernel width, meaning no overlapping pixels accross patches, however, with our simple model, stride number is set to 1, hence, there are planty of overlapping pixels. However, the overhead of replicating pixels can be diminished by the advantage of GEMM brings.
  

  Reference: Why GEMM is at the heart of deep learning

• Flattening visualization
  Input flatten

  
  Kernel flatten
  
  Output flatten

• Layers after im2col:

Layers	Input Activations $(E\ast F)(R\ast S\ast C)$	Kernels $(R\ast S\ast C)(M)$	Output Activations $(E\ast F)(M)$
Layer1	$1024\ast 9$	$9\ast 8$	$1024\ast 8$
Layer2	$256\ast 72$	$72\ast 16$	$256\ast 16$
Layer3	$64\ast 144$	$144\ast 32$	$64\ast 32$
Layer4	$16\ast 288$	$288\ast 64$	$16\ast 64$
Layer5	$4\ast 576$	$576\ast 128$	$4\ast 128$

• Taking layer 2 as an example, this is how input and kernels are flatten to form 2-D matrices, and do matrix multiplication to obtain output. A row of input activations(IA) does dot product with a column of weights(W) to obtain an element in output activations(OA), shown in blue color.
  

• col2im: After obtaing the output activations from GEMM method, we still need to convert the matrix back to the original image format. Here, we operate col2im algorithm to get the result.

Tiling

• Tiling is a technique used in CNN accelerators to partition the computation and memory access into smaller tasks that can be executed in parallel. Considered our one 8 by 8 systolic array as a tile on the hardware, it can only compute 1 block of output activations at a time, which size is also 8 by 8, therefore, the input data needed by a tile of hardware is a row of IA, and a column of W.
• In layer 2, IA is partitioned into 256/8 = 32 rows, and every row contains nine 8 by 8 matrix blocks. As for W, it is partitioned into 16/8 = 2 columns, each column also contains nine matrix blocks.
• Every matrix block is sent to Systolic Array for computation, and every 9 blocks are accumulated to get one 8 by 8 output block.
  

Systolic Array Architecture

• We implemented the TPU (Tensor Processing Unit) weight stationary architecture shown in the diagram. Input activations are fed from the side, while weight parameters are input from the top. Each processing element (PE) has a weight buffer to temporarily store the weights. However, the main drawback of the TPU is that it requires waiting for weight preload to complete before the entire Systolic array can start operating.
  

• Therefore, we referred to another paper and implemented a double buffer, as shown in the diagram, in each processing element (PE) to store the current and next weight parameters. This allows the Systolic array to perform calculations without any delays.
  

  
  

• The control mechanism of the entire Systolic array is extremely simple, requiring only two control signals: "work" and "done." The "work" signal indicates the initiation of input activation and weight parameter inputs. It propagates from the side register, moving downward until it reaches the corner, where it serves as the starting signal for the output accumulator. Once the computation in the entire Systolic array is completed, with all values stored in the output accumulators, the last output accumulator emits a "done" signal.
  

• The diagram below illustrates an example of data flow in a tile. Each dot represents an 8-bit data, and the color of the dot corresponds to the tiling data on the left. It can be observed that the computation results are generated sequentially starting from the first column. Therefore, the completion signal of the entire Systolic array will be provided by the output accumulator of the last column.
  

Ideal Work Flow

PS is the ZYNQ processing system, and PL is the programmable logics. SA is systolic array, we can see that most of the computation is done on SA, some light tasks are distrubuted to PS done by CPU. The current work only covers quantization, im2col, SA output collecting and col2im. Other software computations are left for future work.

Communication Between PS & PL

CPU<->SA AXI interface definition

Input

CPU	SA
slv_reg0	cpu_enable
slv_reg1	bnumin
slv_reg2	tile_num
slv_reg3	cpu_received
slv_reg4	ps_rst

Output

CPU	SA
slv_reg5	valid
slv_reg6	outData0
slv_reg7	outData1
slv_reg8	outData2
slv_reg9	outData3
slv_reg10	outData4
slv_reg11	outData5
slv_reg12	outData6
slv_reg13	outData7

BRAM<->SA interface definition

Input

BRAM0	SA
ADDRBWRADDR	counter
ENBWREN	readenable

BRAM1	SA
ADDRBWRADDR	counter
ENBWREN	readenable

Output

BRAM0	SA
DOADO	ifeature

BRAM1	SA
DOADO	iweight

• Systolic Array interface definition

  wire [19:0] outData0, outData1, outData2, outData3, outData4, outData5, outData6, outData7;
  wire [11:0] counter;
  wire valid, readenable;
top_SA top_SA (
    .clk(S_AXI_ACLK),
    .rstn(S_AXI_ARESETN),
    //* from PS
    .cpu_enable(slv_reg0),
    .bnumin(slv_reg1),
    .tile_num(slv_reg2),
    .cpu_received(slv_reg3),
    .ps_rst(slv_reg4),
    //* from BRAM
    .ifeature(ifeature),
    .iweight(iweight),

    //* to PS
    .valid(valid),
    .outData0(outData0),
    .outData1(outData1),
    .outData2(outData2),
    .outData3(outData3),
    .outData4(outData4),
    .outData5(outData5),
    .outData6(outData6),
    .outData7(outData7),
    //* to BRAM
    .readenable(readenable),
    .counter(counter)
);

Program running on PS

• C code running in PS

#include <stdio.h>
#include <string.h>
#include "xil_printf.h"
#include "xil_io.h"
#include "xparameters.h"
#include "weight.h"
#include "ifeature.h"

int main()
{
//* Get data from file
printf("Program start\r\n");
for(int j=0; j<64;j++){
    int i;
    int counter = 0;
    int tile_counter = 0;
    u16 read = 0;
    u32 read_output = 0;
    //! test Ifeature write
    for (i = 0; i < 72; i++) {
        Xil_Out16(XPAR_AXI_BRAM_CTRL_0_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][0]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_1_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][1]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_2_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][2]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_3_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][3]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_4_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][4]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_5_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][5]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_6_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][6]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_7_S_AXI_BASEADDR + 4*i, ifeature[i+72*j][7]);
    }
    //! test Ifeature write
    for (i = 0; i < 72; i++) {
        Xil_Out16(XPAR_AXI_BRAM_CTRL_8_S_AXI_BASEADDR + 4*i, weight[i+72*j][0]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_9_S_AXI_BASEADDR + 4*i, weight[i+72*j][1]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_10_S_AXI_BASEADDR + 4*i, weight[i+72*j][2]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_11_S_AXI_BASEADDR + 4*i, weight[i+72*j][3]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_12_S_AXI_BASEADDR + 4*i, weight[i+72*j][4]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_13_S_AXI_BASEADDR + 4*i, weight[i+72*j][5]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_14_S_AXI_BASEADDR + 4*i, weight[i+72*j][6]);
        Xil_Out16(XPAR_AXI_BRAM_CTRL_15_S_AXI_BASEADDR + 4*i, weight[i+72*j][7]);
    }
    //! PS setup bnumin & tile_num
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4, 9);
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*2,1);
    //! PS send ps_rst signal
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*4, 1);
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*4, 0);
    //! PS send cpu_enable signal
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR, 1);
    Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR, 0);
    printf("Tile %d\n", j);
    while(1){
        //! Wait for valid signal
        read = Xil_In32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*5);
        if(read == 1){
            printf("Data = ");
            counter++;
            for (i = 6; i < 14; i++) {
                read_output = Xil_In32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*i);
                if(i!=13)
                    printf("%05x_", read_output);
                else
                    printf("%05x\r\n", read_output);
            }
            read_output = 0;
            //! PS send cpu_received signal
            Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*3, 1);
            Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*3, 0);
            if(counter == 8){
                //! PS send cpu_received signal
                Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*3, 1);
                Xil_Out32(XPAR_SA_TOP_0_S00_AXI_BASEADDR + 4*3, 0);
                printf("===================\n");
                counter = 0;
                break;
            }
        }
    }
}
return 0;
}

Step:

1. PS will write data into BRAM0 & BRAM1 through AXI bus, where BRAM0 consists of input feature, and BRAM1 consists of weight.

   Note: One thing to notice is that there are total 16 BRAMs, where each has I/O bandwith of 18-bits(2 parity bits + 16 data bits)

2. When data write is completed on both BRAM, PS will set cpu_enable and ps_rst to true through AXI bus, and then turn off the signal.
   This step will inform Systolic Array controller to start input weight and ifeature from BRAM, and also start the computing.
3. When the computing output are all push to FIFO and finished the accumulation, controller will set valid signal to true, and sent out the data out to PS through AXI bus. Everytime when PS receive the data out, PS will sent back the signal of received to complete the handshake.

   Note: This loop will repeat for the times of blocks number inside a tile in order to output the result of the matrix multiplication respectively in row.

4. When all the data outputs are received, PS will proceed to the next loop to send new weights and input features. This process will continue for the number of tiles in the computing convoultion layer.

• Since we only have one tile on PL, only data needed by one tile is sent to PL at a time. Therefore, whenever one tile of computation is done, PS sends ps_rst signal to PL, makes it jump back to WAIT_PS state, and wait for cpu_enable signal when all data is written into BRAM.
  
  

Overall system FSM:

Pseudo code for a layer operation, here in layer 2, total_tile_number is 64. When one block of output is collected, ps_rst is raised high again, and PS starts to send data to PL. When data is written, cpu_enable is raised high, SA begins its computation.

BRAM

• BRAM template

module bram0(
	input clk,
	input rst_a,
	input [10:0] ADDRARDADDR, // This is address from PS
	input ENARDEN,
	input ENBWREN,
	input [3:0] WEA,
	input [31:0] DIADI,
	input [11:0] ADDRBWRADDR,	 // This is counter from Controller inside top_SA
	output [31:0] DOADO,
	output [31:0] DOBDO
);

RAMB36E1 #(
// Address Collision Mode: "PERFORMANCE" or "DELAYED_WRITE"
.RDADDR_COLLISION_HWCONFIG("DELAYED_WRITE"),
// Collision check: Values ("ALL", "WARNING_ONLY", "GENERATE_X_ONLY" or "NONE")
.SIM_COLLISION_CHECK("ALL"),
// DOA_REG, DOB_REG: Optional output register (0 or 1)
.DOA_REG(0),
.DOB_REG(0),
.EN_ECC_READ("FALSE"), // Enable ECC decoder,
// FALSE, TRUE
.EN_ECC_WRITE("FALSE"), // Enable ECC encoder,
// FALSE, TRUE
// INITP_00 to INITP_0F: Initial contents of the parity memory array
.INITP_00(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_01(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_02(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_03(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_04(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_05(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_06(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_07(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_08(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_09(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INITP_0F(256'h0000000000000000000000000000000000000000000000000000000000000000),
// INIT_00 to INIT_7F: Initial contents of the data memory array
.INIT_00(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_01(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_02(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_03(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_04(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_05(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_06(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_07(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_08(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_09(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_0F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_10(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_11(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_12(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_13(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_14(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_15(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_16(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_17(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_18(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_19(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_1F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_20(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_21(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_22(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_23(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_24(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_25(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_26(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_27(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_28(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_29(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_2F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_30(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_31(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_32(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_33(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_34(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_35(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_36(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_37(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_38(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_39(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_3F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_40(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_41(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_42(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_43(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_44(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_45(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_46(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_47(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_48(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_49(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_4F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_50(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_51(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_52(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_53(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_54(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_55(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_56(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_57(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_58(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_59(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_5F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_60(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_61(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_62(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_63(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_64(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_65(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_66(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_67(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_68(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_69(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_6F(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_70(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_71(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_72(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_73(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_74(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_75(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_76(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_77(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_78(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_79(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7A(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7B(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7C(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7D(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7E(256'h0000000000000000000000000000000000000000000000000000000000000000),
.INIT_7F(256'h0000000000000000000000000000000000000000000000000000000000000000),
// INIT_A, INIT_B: Initial values on output ports
.INIT_A(36'h000000000),
.INIT_B(36'h000000000),
// Initialization File: RAM initialization file
.INIT_FILE("NONE"),
// RAM Mode: "SDP" or "TDP"
.RAM_MODE("TDP"),
// RAM_EXTENSION_A, RAM_EXTENSION_B: Selects cascade mode ("UPPER", "LOWER", or "NONE")
.RAM_EXTENSION_A("NONE"),
.RAM_EXTENSION_B("NONE"),
// READ_WIDTH_A/B, WRITE_WIDTH_A/B: Read/write width per port
.READ_WIDTH_A(18), // 0-72                                                                      
.READ_WIDTH_B(18), // 0-36                                                                      
.WRITE_WIDTH_A(18), // 0-36                                                                     
.WRITE_WIDTH_B(0), // 0-72                                                                     
// RSTREG_PRIORITY_A, RSTREG_PRIORITY_B: Reset or enable priority ("RSTREG" or "REGCE")
.RSTREG_PRIORITY_A("RSTREG"),
.RSTREG_PRIORITY_B("RSTREG"),
// SRVAL_A, SRVAL_B: Set/reset value for output
.SRVAL_A(36'h000000000),
.SRVAL_B(36'h000000000),
// Simulation Device: Must be set to "7SERIES" for simulation behavior
.SIM_DEVICE("7SERIES"),
// WriteMode: Value on output upon a write ("WRITE_FIRST", "READ_FIRST", or "NO_CHANGE")
.WRITE_MODE_A("WRITE_FIRST"),
.WRITE_MODE_B("WRITE_FIRST")
)
RAMB36E1_inst (
// Cascade Signals: 1-bit (each) output: BRAM cascade ports (to create 64kx1)
.CASCADEOUTA(), // 1-bit output: A port cascade
.CASCADEOUTB(), // 1-bit output: B port cascade

// Cascade Signals: 1-bit (each) input: BRAM cascade ports (to create 64kx1)
.CASCADEINA(), // 1-bit input: A port cascade
.CASCADEINB(), // 1-bit input: B port cascade

// ECC Signals: 1-bit (each) output: Error Correction Circuitry ports
.DBITERR(), // 1-bit output: Double bit error status
.ECCPARITY(), // 8-bit output: Generated error correction parity
.RDADDRECC(), // 9-bit output: ECC read address
.SBITERR(), // 1-bit output: Single bit error status

// ECC Signals: 1-bit (each) input: Error Correction Circuitry ports
.INJECTDBITERR(), // 1-bit input: Inject a double bit error
.INJECTSBITERR(), // 1-bit input: Inject a single bit error

// Port A Data: 32-bit (each) output: Port A data 
.DOADO(DOADO), // 32-bit output: A port data/LSB data                                         
.DOPADOP(), // 4-bit output: A port parity/LSB parity  
.ADDRARDADDR({3'b0, ADDRARDADDR[10:2], 4'b0000}), // 16-bit input: A port address/Read address
// Port A Address/Control Signals: 16-bit (each) input: Port A address and control signals (read port when RAM_MODE="SDP")
.CLKARDCLK(clk), // 1-bit input: A port clock/Read clock                                         
.ENARDEN(ENARDEN), // 1-bit input: A port enable/Read enable                                          
.REGCEAREGCE(1'b1), // 1-bit input: A port register enable/Register enable                        
.RSTRAMARSTRAM(), // 1-bit input: A port set/reset
.RSTREGARSTREG(), // 1-bit input: A port register set/reset
.WEA(WEA), // 4-bit input: A port write enable                           

// Port A Data: 32-bit (each) input: Port A data
.DIADI(DIADI), // 32-bit input: A port data/LSB data                                          
.DIPADIP(), // 4-bit input: A port parity/LSB parity

// Port B Data: 32-bit (each) output: Port B data
.DOBDO(DOBDO), // 32-bit output: B port data/MSB data                                         
.DOPBDOP(), // 4-bit output: B port parity/MSB parity

// Port B Address/Control Signals: 16-bit (each) input: Port B address and control signals (write port
// when RAM_MODE="SDP")
.ADDRBWRADDR({1'b0, ADDRBWRADDR[10:0], 4'b0000}), // 16-bit input: B port address/Write address       
.CLKBWRCLK(clk), // 1-bit input: B port clock/Write clock                                        
.ENBWREN(ENBWREN), // 1-bit input: B port enable/Write enable                                         
.REGCEB(1'b0), // 1-bit input: B port register enable                                             
.RSTRAMB(), // 1-bit input: B port set/reset
.RSTREGB(), // 1-bit input: B port register set/reset
.WEBWE(), // 8-bit input: B port write enable/Write enable       

// Port B Data: 32-bit (each) input: Port B data
.DIBDI(), // 32-bit input: B port data/MSB data                                          
.DIPBDIP() // 4-bit input: B port parity/MSB parity
);
// End of RAMB36E1_inst instantiation

endmodule

• Note: Data R/W bandwidth is 16-bits, beacuse the data input to BRAM is one block(9-bits).

Block diagram

Implementation

Usage of LUTs and DSP

From the analysis, it can be observed that approximately 138 lookup tables (LUTs) are required to synthesize a single processing element (PE). However, if we use DSP, only two DSPs are needed. Therefore, in terms of computation, it will rely on DSP synthesis to achieve better FPGA hardware utilization.

Utilization

DSP ver.

Correctness of our design

Only one layer has been implemented so far(layer2), the functionality of the SA is proven to be correct by Modelsim. The result on FPGA is also correct, hence, as long as PS provides correct input data to PL, the output result is always correct.

• Simulation:
  
  
• Implementation on FPGA:
  
  

Our design shortcoming

1. DSP timing violation due to the lack of weight registers.

To ensure the opsum calculation is performed within a single clock cycle, we removed the weight input register. However, this resulted in the weight signal being broadcasted, which contradicts the design on the FPGA that avoids broadcasting. Each input of the DSP requires an input register, which conflicts with our design and causes a timing violation. To resolve this issue, we have decided to reintroduce the input weight register and change the weight buffer to a latch.

2. The interface definition of the Systolic array cannot be modified independently.

In the design process, the input and output interface bit widths of the Systolic array are fixed and cannot be modified. However, for quick verification of correct implementation, we added an additional control signal from the internal multiplexer of the processing element (PE) to the most significant bit (MSB) of the 8-bit data for input activation and weight parameters, resulting in 9-bit data. Although the entire system is self-designed, this approach of modifying the interface bit widths on our own is not a good design practice. Since the internal operation of the Systolic array follows a regular pattern, the solution is to incorporate additional registers internally to allow the control signals to flow according to the pattern.

3. BRAM 16-bit R/W issue.

The design of our PE requires a parity bit to control the data flow based on different tiles. This is why our input features and weights are 9 bits instead of 8 bits. However, this design leads to wastage of resources in our implementation. To accommodate the 9-bit input requirement, the BRAM I/O bandwidth API call needs to be changed from 8-bit to 16-bit, resulting in the presence of all-zero 7 bits in every transmission that occurs in the BRAM.
The solution we propose is to avoid adding the parity control bit to the input features and weights before data transmission between the PS and BRAM, as well as between the BRAM and the Systolic Array. Instead, we can incorporate the control signal through a simple hardware toggle generator. This generator will add a control bit to the most significant bit (MSB) of the data and toggle between 0 and 1 when the tile changes. The optimal location to add this generator in our current design is within the truncate module, which concatenates all the data from BRAM0 and BRAM1 before sending it to the Systolic Array.

Reference

1.Google TPU
2 PE double buffer