# Work Note (2021-Sep-1 ~ 2021-Oct-5) ## TODO - [ ] Tacoma12 FTK R5/RV32(Danny/Bob) - [ ] #13578 #10780 Performance - [x] ramdisk, - [x] ramdisk + DDR - [x] programmer, - [x] rawdisk - [x] read decoding error issue. wip. - [ ] ramdisk +DDR - [ ] mpc rawdisk - [ ] performance evaludation - [ ] BTN, 3M IOPS 4k randr(Shengwei/Jun) - [ ] command group - works well with single cpu - code ready with multiple cpus, debugging - [ ] write buffer - two tickets filed to Dishi(Jun) - [ ] #13431 AIENG.(Mingzhi/Danny) - [ ] FTK from rainierB0 (Jun) - [ ] DDR scan #13911 - [ ] RainierS3 - [x] [Validation Report](https://innogrit.sharepoint.com/:x:/r/_layouts/15/doc2.aspx?sourcedoc=%7Bb98131a3-a6a8-4bee-a2e0-a8ef4f4fc0ea%7D&action=edit&activeCell=%27Statistics%27!B10&wdinitialsession=003e3952-d6f0-4136-9de3-343f198a9052&wdrldsc=6&wdrldc=1&wdrldr=AccessTokenExpiredWarning%2CRefreshingExpiredAccessT&cid=f178367f-af1a-45f6-bd93-111c387fa0e9) - [x] #12842, 4Byte PI(Shengwei) - [x] #12842 functional validation(Shengwei) - [x] failed in stress(Shengwei/Bob) - [x] #13137, 8K DU - [x] single IO test(Danny) - [x] [#13541](http://10.10.0.17/issues/13541) regression(Steve/Danny) - [ ] ROM development - [ ] [#13417](http://10.10.0.17/issues/13417) USB-2-SATA. wip(Danny/Mingzhi) - [ ] #12765 BTN(Mingzhi/Danny) - [ ] (T0)PECC[#13571](http://10.10.0.17/issues/13571) issue(Bob/Danny) - [x] programmer - [x] ramdisk + ddr - [ ] rawdisk, pending on programmer - [ ] #13692 RainierS3 Stress with final design(Bob/Danny) - [ ] SATA - [x] 4bpi + 4kdu + rawdisk + {hsec512/4k, aes-on/off, hcrc-on} - [ ] 4bpi + 8kdu + rawdisk + {hsec512/4k, aes-on/off, hcrc-on} - [ ] 4bpi + 4kdu + ramdisk + {hsec512/4k, aes-on/off, hcrc-on} - [ ] 4bpi + 8kdu + ramdisk + {hsec512/4k, aes-on/off, hcrc-on} - [ ] PCIE - [x] defined the configuration matrix for stress. - 3x PI - 2x 8B/16B PI(nvme), - [x] 4B PI(sata) - 2x 4KDU, 8KDU - 2x hsec4K, hsec512B - [x] AES/CRYPTO, default - [x] HCRC - [x] spc rawdisk - NAND - TSB - MU ```bash $sort /tmp/1 | uniq cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 -DUSE_512B_HOST_SECTOR_SIZE=1 cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_CRYPTO_HW=1 cmake ../ -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_TSB_NAND=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DTARGET=4 cmake ../ -DUSE_MU_NAND=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 cmake ../ -DUSE_TSB_NAND=1 -DRAWDISK=1 -DHMETA_SIZE=8 -DPI_SUPPORT=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 cmake ./ -DUSE_TSB_NAND=1 -DRAWDISK=1 -DHMETA_SIZE=8 -DPI_SUPPORT=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 cmake ../../rs3.fpga -DUSE_TSB_NAND=1 -DRAWDISK=1 -DHMETA_SIZE=8 -DPI_SUPPORT=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 ``` ## Machines | FPGA | IP | Host | Projects |Used/Shared| | -------- | -------- | -------- | -------- | -------- | | VU440 | 10.10.0.132 | pcie/sata(TSB) | rainierS3/Tacoma12 | TPE/US | | VU19P | 10.10.0.159 | pcie | Tacoma12/zns/rainierb0 | SH/TPE/US | | VU19P | 10.10.0.219 | pcie | Tacoma12/rainierb0 | SH/US | | VU440 | 10.10.0.78 (X570) | 10.10.0.187 (NVME) | rainierS3/Tacoma12/zns | TPE/US| | VU440 | 10.10.0.78 (X570) | 10.10.1.8 (sata) | rainierS3/Tacoma12/zns | TPE/US| | H2K | 10.10.0.120 | pcie | rainierS3/rainierb0 | SH/TPE/US| | VU440 | 10.10.1.21 | pcie(MU) | rainierS3/Tacoma12 | TPE/US| ## Shorterm Schedule ```mermaid gantt title Shorterm Schedule(Q4-2021)(Rolling), [!obsoleted] excludes weekdays 2021-09-01 section Tacoma12 Ramdisk : a1, 2021-09-01, 3d Rawdisk+DDR : a2, after a1, 15d MPC rawdisk : a3, after a2, 5d Performance Evaluation :a4, after a3, 25d AIENG : a5, 2021-09-08, 30d BTN : a6, after b6, 10d section RainierS3 RTL freeze : b3, 2021-09-01, 7d 12842 4B PI : b4, 2021-09-01, 7d 13137 8K DU : b5, 2021-09-01, 17d 8k DU regression : b7, after b5, 10d USB2SATA : b2, 2021-09-07, 20d ROM : b1, after b2, 20d BTN : b6, after b3, 25d ``` ## Tickets ### 13571 rainiers3 T0 pecc new dfi design - cmake options ``` # programmer cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_PARALLEL_ECC=1 -DUSE_TSB_NAND=1 -DU2_0802=1 -DENABLE_SOUT=1 -DPROGRAMMER=1 -DDDR=1 -DDDR_AUTO=1 -DSATA_MODE=1 -DUSE_512B_HOST_SECTOR_SIZE=1 # ramdisk cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_PARALLEL_ECC=1 -DRAMDISK=1 -DU2_0802=1 -DENABLE_SOUT=1 -DDDR=1 -DDDR_AUTO=1 -DSATA_MODE=1 -DUSE_512B_HOST_SECTOR_SIZE=1 # rawdisk cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_PARALLEL_ECC=1 -DUSE_TSB_NAND=1 -DU2_0802=1 -DENABLE_SOUT=1 -DRAWDISK=1 -DDDR=1 -DDDR_AUTO=1 -DSATA_MODE=1 -DUSE_512B_HOST_SECTOR_SIZE=1 ``` ### 12765 BTN validation [RainierS3][A0][FPGA][BTN] Rainier S3 BTN change list #### validation(from RainierB0) Basic validation environment. - validated bitfile: `rainier_s3_vu440_nvme_1ch_ddr_4xr5_0811_4537.bit` - fw: `main.elf.12765.btn.cmd.sample.ddr.enabled` - this is a mistake to apply -DSATA_MOD=1 in the build option, but has no impact in gdb operations. Just to enable DDR. - cmake option ``` cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DDDR=1 ``` - NOTE - need to enable DDR - sample operations ``` ig@ig-System-Product-Name:~/djiang$ cat 12680.btn.cmd mon mww 0xc0010294 0 mon mww 0xc0010298 0x20 mon mww 0x20000000 0x81400000 mon mww 0x20000004 0x101 mon mww 0x20000008 0x102 mon mww 0x2000000c 0x103 mon mww 0x20101000 0xdead8c37 mon mww 0x20102000 0xdead7cde x/4xw 0xc0010290 mon mww 0xc0010290 0x1 x/4xw 0x20000000 x/4xw 0x20103000 ``` - patch for `-DDDR=1` ``` --- a/btn/dtag/dtag.c +++ b/btn/dtag/dtag.c @@ -920,7 +920,7 @@ init_code void dtag_init(void) dtag_mgr_init(DTAG_T_DDR, sz, 4); dtag_add(DTAG_T_DDR, (void *) ddtag2mem(DDR_IO_DTAG_START), sz << DTAG_SHF); - #if defined(SATA_MODE) + #if defined(SATA_MODE) && defined(RDISK) extern volatile int *shr_dm_avail; shr_dm_avail = (int *) (btcm_to_dma(&_dtag_mgr[DTAG_T_DDR].dtag_avail.data)); #endif ``` ### 13431 AIENG validation [Tacoma12][RV32][FPGA][AIENG] AI engine validation ### 13417 USB2SATA Bridge issue [Rainier_S][ROM] recovery mode, the usb2sata bridge enumeration failed. ```graphviz digraph G { #sata_cmd -> read_dma dsata_thr -> dsata_thr [label="dsata_isr_pnd = 0" ] dsata_thr -> dsata_ctx_irq_run [label="dsata_isr_pnd = 1" ] dsata_ctx_irq_run -> dsata_ctx_run dsata_ctx_run -> fsm_ctx_run fsm_ctx_run -> _fsm_ctx_run _fsm_ctx_run -> fsm_cmd_rx_fis [label="S_CMD_RX_FIS"] fsm_cmd_rx_fis -> ata_cmd_rx ata_cmd_rx -> ncq_cmd ata_cmd_rx -> non_data_cmd ata_cmd_rx -> data_cmd data_cmd -> CMD_READ_DMA_C8h CMD_READ_DMA_C8h -> data_cmd ncq_cmd -> ata_cmd_rx [label="cmd_done"] non_data_cmd -> ata_cmd_rx [label="cmd_done"] data_cmd -> ata_cmd_rx [label="cmd_done"] ata_cmd_rx -> fsm_cmd_rx_fis fsm_cmd_rx_fis -> fsm_ctx_set fsm_ctx_set -> _fsm_ctx_run [label="set S_CMD_START"] _fsm_ctx_run -> fsm_cmd_start [label="S_CMD_START"] fsm_cmd_start -> fsm_cmd_setup [label="S_CMD_SETUP"] fsm_cmd_setup -> fsm_cmd_setup_done [label="S_CMD_SETUP_DONE"] fsm_cmd_setup_done -> fsm_cmd_issue [label="S_CMD_PRE_XFER"] fsm_cmd_issue -> dsata_cmd_issue } ``` ### [13642 Jun Liu, RV32, MPC ramdisk panic](http://10.10.0.17/issues/13642) [RV32][RAMDISK] Multi-CPU Ramdisk panic when using RV32 ### 13578 Tacoma12 R5/RV32 Performance From #10780 Tacoma12 R5/RV32 Performance - RISC-V, RV32 - ramdisk only, ``` cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAMDISK=1 -DCPU_RV32=1 ``` - ramdisk ddr, ``` cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAMDISK=1 -DDDR=1 -DCPU_RV32=1 ``` - rawdisk only, can't run now ``` cmake ../ -DUSE_MU_NAND=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAWDISK=1 -DCPU_RV32=1 ``` - ARM, R5 - ramdisk only, ``` cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAMDISK=1 -DCPU_R5=1 ``` - ramdisk ddr, ``` cmake ../ -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAMDISK=1 -DDDR=1 -DCPU_R5=1 ``` - rawdisk only, can't run now ``` cmake ../ -DUSE_MU_NAND=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DFPGA=1 -DRAWDISK=1 -DCPU_R5=1 ``` #### Toolchains - RV32, - `riscv64-unknown-elf-gcc (SiFive GCC-Metal 10.2.0-2020.12.8) 10.2.0` - bitops options, zba_zbb_zbc - `-march=rv32imac_zba_zbb_zbc` - bitops options, zbb only - `-march=rv32imac_zbb` - R5, - `arm-none-eabi-gcc (GNU Arm Embedded Toolchain 10-2020-q4-major) 10.2.1 20201103 (release)` #### r5/rv32 code size compare - Os sz compare ``` # rv32, Os text data bss dec hex filename 170648 66384 9455951 9692983 93e737 main # r5, Os text data bss dec hex filename 154239 66320 9511527 9732086 947ff6 main ``` - O3 sz compare ``` # rv32, O3 text data bss dec hex filename 185513 63248 9456927 9705688 9418d8 main # r5, O3 text data bss dec hex filename 179611 63120 9489355 9732086 947ff6 main # rv32, O3, bitops zba_zbb_zbc text data bss dec hex filename 184737 63248 9456991 9704976 941610 main # rv32, O3, bitops zbb only text data bss dec hex filename 185013 63248 9456959 9705220 941704 main ``` #### elf static statistics - static statistics from elf ``` riscv64-unknown-elf-objdump -d main | awk '{print $5}' | sort | uniq -cd | sort -g -k 1 > rv32.callee riscv64-unknown-elf-objdump -d main | awk '{print $3}' | sort | uniq -cd | sort -g -k 1 > rv32.insts ``` #### FIO test - /home/ig/pchen/tacoma12/bit/tacoma12_vu440_ddr_omc_sifive_0825_5191.bit - /home/ig/pchen/tacoma12/bit/tacoma12_vu440_ddr_omc_ai_0907_5423_dfi555.bit ``` # seq wr fio --filename=/dev/nvme0n1 --direct=1 -rw=write --bs=32k --iodepth=32 --name=seqwr --thread --ioengine=libaio --time_based --runtime=30 rv32(Os): write: IOPS=3724, BW=116MiB/s r5(Os): write: IOPS=5289, BW=165MiB/s (173MB/s) rv32(O3): write: IOPS=4115, BW=129MiB/s (135MB/s) rv32(O3.bitops): write: IOPS=4172, BW=130MiB/s (137MB/s) # seq rd fio --filename=/dev/nvme0n1 --direct=1 -rw=read --bs=32k --iodepth=32 --name=seqwr --thread --ioengine=libaio --time_based --runtime=30 rv32(Os): read: IOPS=6665, BW=208MiB/s (218MB/s) r5(Os): read: IOPS=7803, BW=244MiB/s (256MB/s) rv32(O3): read: IOPS=8161, BW=255MiB/s (267MB/s) rv32(O3.bitops): read: IOPS=8281, BW=259MiB/s (271MB/s) # randw fio --filename=/dev/nvme0n1 --direct=1 -rw=randwrite --bs=4k --iodepth=32 --name=seqwr --thread --ioengine=libaio --time_based --runtime=30 rv32(Os): write: IOPS=11.8k, BW=46.1MiB/s (48.3MB/s) r5(Os): write: IOPS=15.4k, BW=60.0MiB/s (62.9MB/s) rv32(O3): write: IOPS=13.4k, BW=52.3MiB/s (54.8MB/s) rv32(O3.bitops): write: IOPS=13.7k, BW=53.5MiB/s (56.1MB/s) # randr fio --filename=/dev/nvme0n1 --direct=1 -rw=randread --bs=4k --iodepth=32 --name=seqwr --thread --ioengine=libaio --time_based --runtime=30 rv32(Os): read: IOPS=16.6k, BW=64.7MiB/s (67.8MB/s) r5(Os): read: IOPS=20.1k, BW=78.4MiB/s (82.2MB/s) rv32(O3): read: IOPS=19.4k, BW=75.7MiB/s (79.4MB/s) rv32(O3.bitops): read: IOPS=20.0k, BW=78.2MiB/s (82.0MB/s) ``` write: IOPS=4172, BW=130MiB/s (137MB/s) read: IOPS=8281, BW=259MiB/s (271MB/s) write: IOPS=13.7k, BW=53.5MiB/s (56.1MB/s) read: IOPS=20.0k, BW=78.2MiB/s (82.0MB/s) ### #12842 ![](https://i.imgur.com/U9Dtb6s.png) - environment and config - bitfile - firmware - branch - commit - patch - operation flow - how to judge the result ### #13137 - FW build ``` # 8kdu, hsec512, crypto : build failed, comment out the failed lines is fine. cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_CRYPTO_HW=1 # 8kdu, hsec512 cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 -DUSE_512B_HOST_SECTOR_SIZE=1 # 8kdu, hsec4k cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_8K_DU=1 ``` - base - [Code Base](http://10.10.0.17/attachments/download/46125/rainier_a0_fpga_s3_a0.rainiers3_sata.patches.tgz) - patch ``` diff --git a/build/gcc/ldscript/sections.ld b/build/gcc/ldscript/sections.ld index 64cd1661f..6fa018d18 100644 --- a/build/gcc/ldscript/sections.ld +++ b/build/gcc/ldscript/sections.ld @@ -194,7 +194,7 @@ SECTIONS { { . = ALIGN(4096); __dtag_fwconfig_start = .; - . = . + 4096; + . = . + 2*4096; __dtag_fwconfig_end = .; } > SRAM ``` - tests ``` rmmod ahci modprobe ahci dmesg dd if=/dev/zero of=/dev/sda bs=512 count=17 dd if=/dev/urandom of=/dev/sda bs=512 count=17 sg_dd blk_sgio=1 if=/dev/zero of=/dev/sda bs=512 count=17 ``` ### Utils in "scripts" (tmp) - `code_analyze.sh output.map` ![code_analyze.sh output.map](https://i.imgur.com/nVvg6ku.png) - `./checkpatch.pl --no-tree -f sourcefile` - generate *.p file for cdump - update ./scripts/gdb/gef/tacoma_regs.p - `python ./scripts/gdb/gef/tacoma_regs.py -d ./scripts/specs/ -g GEN` - affect ./scripts/gdb/gef/plugins/cdump.py - generate header file from xlsx file - `./gen_header.sh uart` ``` bwang@sysarch11 ~/tacoma/scripts (rainiers3_nvme_bringup_a0)$ ./gen_header.sh uart Gen header for uart uart_share_registers.xlsm move to ../rtos/armv7r/ ``` - git - update submodule - `git submodule update` - check git log - `git log -p origin/<branch> <path-to-file>/<file>` - `git log -p origin/rainier_a0 <commit-id>` - `git cherry-pick <commit-id>`, alias cp -> cherry-pick - `git cp --continue` - `git cp --abort` - `git form-patch -n<#> -o <path-to-put-patches>` - test if your patch is ok - `cat s/3.35972.patch | patch -p1 --dry-run` - `patch -p1 <file> --dry-run` - rebase - `git rebase -i <commit-id>` - gdb - cpu can't halt - `mon mww 0xc0204000 0xf` reset cpus - `mon mww 0xc004000c 0xf` chip reset - dump memory content to a file - `(gdb) dump binary memory logg 0x40000000 0x40000040` - restore memory content from a file - `(gdb) restore ./logg1 binary 0x40000000` - convert images - [Convert images in command line](https://stackoverflow.com/questions/20075087/how-to-merge-images-in-command-line) ### SPEC - RainierS3 - [CPU private](https://innogrit.sharepoint.com/:x:/r/Shared%20Documents/Engineering/Rainier_S3/A0/SPEC/cpu_group_top/cpu_private_register.xlsx?d=w530623d89b9848a0931340f6c87f1993&csf=1&web=1&e=JwvhdO) - [CPU common](https://innogrit.sharepoint.com/:x:/r/Shared%20Documents/Engineering/Rainier_S3/A0/SPEC/cpu_group_top/common_cpu_cfg_register.xlsx?d=w99f7e44a13854bf8bdf47a58f253fd54&csf=1&web=1&e=XXqviL) - Tacoma12 - AIENG - [AIENG reg map](https://innogrit.sharepoint.com/:x:/r/_layouts/15/Doc.aspx?sourcedoc=%7BF78B4F0F-B3E1-44C6-9336-7C1A20F0EEB1%7D&file=AIENG_registers.xlsx&action=default&mobileredirect=true) - [AIENG Spec](https://innogrit.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx?RootFolder=%2FShared%20Documents%2FEngineering%2FTacoma12%2FSPEC%2FAIENG%2FQuantization%2F02%5FRTL%20SPEC%2F01%5FSW%20Interface) - S3 - [SD_XFER Spec](https://innogrit.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx?csf=1&web=1&e=0MTrRA&cid=ecc996f2%2D7d78%2D4ebc%2D81c2%2Db78edb495549&FolderCTID=0x012000F24789E8A6A913488CA8D10E7B304B74&viewid=0c1b468e%2D6e4e%2D486b%2D8644%2Da6c9704b601e&id=%2FShared%20Documents%2FEngineering%2FRainier%5FS3%2FA0%2FSPEC%2FSD%5FXFER) - [CMD_PROC Spec](https://innogrit.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx?csf=1&web=1&e=0MTrRA&cid=ecc996f2%2D7d78%2D4ebc%2D81c2%2Db78edb495549&FolderCTID=0x012000F24789E8A6A913488CA8D10E7B304B74&viewid=0c1b468e%2D6e4e%2D486b%2D8644%2Da6c9704b601e&id=%2FShared%20Documents%2FEngineering%2FRainier%5FS3%2FA0%2FSPEC%2FCMD%5FPROC) - [Address Map](https://innogrit.sharepoint.com/:x:/r/_layouts/15/Doc.aspx?sourcedoc=%7B1E17E7DB-95B2-4BFC-BA3D-F10F55BD8CAC%7D&file=RainierS3_cmd_proc_reg.xlsx&action=default&mobileredirect=true) ### FPGA - target: ig@10.10.1.21 - host: remote@10.10.1.20 ```graphviz digraph G { Bitfile -> Tacoma12 Bitfile -> RainierS3 Tacoma12 -> R5 Tacoma12 -> RV32 RainierS3 -> SATA RainierS3 -> PCIE ramdisk rawdisk ddr } ``` ```graphviz digraph G { Tacoma -> Shasta Tacoma -> Rainier Tacoma -> Tacoma12 Shasta -> ShastaPlus Rainier -> Rainier_A0 -> Rainier_B0 -> ZNS Rainier_A0 -> ZNS Rainier -> Rainier_S3_T0 -> Rainier_S3_A0 Tacoma12 -> Tacoma12_T0 -> ZNS } ``` ```graphviz digraph G { "commander" -> "ig(GDB)" "ig(GDB)" -> "remote(PCIE host)" "ig(GDB)" -> "vivado(dl bitfile)" "ig(GDB)" -> "uart" "ig(GDB)" -> "gdb-cli" "ig(GDB)" -> "jtag(gdbserver)" "ig(GDB)" -> "ps(power switch)" } ``` ```mermaid sequenceDiagram participant commander participant GDB Note right of GDB:10.10.1.21 participant host Note right of host: 10.10.1.20 commander ->> GDB: Get up loop Startup GDB ->> GDB: download bitfile GDB ->> GDB: launch uart GDB ->> GDB: launch gdbserver GDB ->> GDB: launch gdb-cli end commander ->> host: Get up loop identify host ->> host: identify target end ``` ### Work Scope ```graphviz digraph G { rankdir=LR; "Validation" -> "FPGA" -> "Block" "FPGA" -> "System" "Validation" -> "ASIC" -> "System" "HOST" -> "SATA" "HOST" -> "PCIE" -> "NVME" -> "ZNS" "FTL" -> "GC" "FTL" -> "Wear Leveling" "FTL" -> "Power Loss Protection" "FTL" -> "End-2-End Protection" "SSD Arch" "SSD perf" "SSD Driver" -> "Linux Device Driver" "SSD Driver" -> "Linux Kernel" "SSD Driver" -> "Virtual Functional IO" "SSD Driver" -> "Kernel based VM" "Automation" -> "Bash" "Automation" -> "Git" "Automation" -> "CI tools" "Automation" -> "Python" } ``` ### Trouble Shooting #### install openocd from source [Link to Install openocd from source](https://hackaday.io/page/4991-compiling-openocd-from-source-on-ubuntu-1604) #### SSH permission denied ``` pi@raspberrypi:~ $ sudo vi /etc/ssh/sshd_config # add your group in AllowGroup pi@raspberrypi:~ $ sudo systemctl restart sshd ``` ## Bob ### Hot-Cold Data Determination ### Prepare Phase (Boot) ```mermaid graph TB id1{Start} --> id2["Memory Initialize:<br/><br/> Input/CNN Buffer Init<br/>MODE=2'd2, SUB_MODE=ˋd0, FETCH_OUT=1'b0"] --> id3["Scheduler Load: <br/>-----------------------------------<br/> [1] Kmeans Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd2) <br/>-----------------------------------<br/> [2] CNN Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd3) <br/>-----------------------------------<br/> [3] FNN Ping Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd4) <br/>-----------------------------------<br/> [4] FNN Pong Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd5) <br/>-----------------------------------<br/> [5] Quant Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd6) <br/>-----------------------------------<br/> [6] LUT Scheduler<br/>(MODE=2'd0, SUB_MODE=4'd7) "] --> id4["Weight Load: <br/>-----------------------------------<br/> [1] CNN Weight<br/>(MODE=2'd0, SUB_MODE=4'd9) <br/>-----------------------------------<br/> [2] FNN Ping Weight<br/>(MODE=2'd0, SUB_MODE=4'd10) <br/>-----------------------------------<br/> [3] FNN Pong Weight<br/>(MODE=2'd0, SUB_MODE=4'd11) <br/>-----------------------------------<br/> [4] LUT Low Slope<br/>(MODE=2'd0, SUB_MODE=4'd12) <br/>-----------------------------------<br/> [5] LUT High Slope<br/>(MODE=2'd0, SUB_MODE=4'd13) "] --> id5["Quant Factor and Biast Load: <br/>-----------------------------------<br/> [1] CNN Bias<br/>(MODE=2'd1, SUB_MODE=4'd0) <br/>-----------------------------------<br/> [2] FNN Ping Bias<br/>(MODE=2'd1, SUB_MODE=4'd1) <br/>-----------------------------------<br/> [3] FNN Pong Bias<br/>(MODE=2'd1, SUB_MODE=4'd2) <br/>-----------------------------------<br/> [4] CNN Quant Factor<br/>(MODE=2'd1, SUB_MODE=4'd3) <br/>-----------------------------------<br/> [5] FNN Ping Quant Factor<br/>(MODE=2'd1, SUB_MODE=4'd4) <br/>-----------------------------------<br/> [5] PNN Pong Quant Factor<br/>(MODE=2'd1, SUB_MODE=4'd5) <br/>-----------------------------------<br/> [5] Partial Result Quant Factor<br/>(MODE=2'd0, SUB_MODE=4'd6) "] ``` ### Operation Phase ```mermaid graph TB id1{Start} --> id2["Data Load<br/><br/> Source Data Buffer Load:<br/> (MODE=2'd2, SUB_MODE=4'd1)"] --> id3["Hot-Cold Data Determination<br/><br/> Task Activation<br/> (MODE=2'd3, SUB_MODE=4'd0)"] ``` ### Abbreviation ```info - NCB NAND Control Block - BTN Buffer-Table-NCMD controrl - DPE Data Process Engine - LTP Logical to Physical - DU Data Unit - DPTR Data Pointer - PDA Physical DU Address - SPB Super Physical Block - SRB System Root Block http://10.20.27.59/zns/wiki/-/wikis/SRB - FRB FRB, FTL Root Blocks http://10.20.27.59/zns/wiki/-/wikis/FRB - MCTP Management Component Transport Protocol - TCM Tightly-Coupled Memory (ARM) Memory region very close to CPU, access in one cycle. Used at fixed dedicated memory. [https://stackoverflow.com/questions/30797106/whats-arm-tcm-memory] - SLBA Starting LBA - BTCM ? B tighly coupled memory - ATCM ? A tighly coupled memory - FDMA Flash DMA (跟BTN溝通搬運data的module) -ECCU Error Correction Control Unit - PI Protection Information [http://www.ifuun.com/a2017713448486/] - PRD “Physical Region Descriptor”. PRD entries describe the location and length of data to be transferred ``` ```bash # Replace <username> with your own name. # edit .git/config $vi .git/config [submodule "scripts/shared_tool"] active = true url = ssh://<username>@10.20.0.16:29418/shared_tool [submodule "srb_header"] active = true url = ssh://<username>@10.20.0.16:29418/srb_header $git submodule update --init --recursive # Build it ### for RISCV $cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_RV32=1 ### for ARM $cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_R5=1 $make -j16 # [Read Only] mode tmux attach, with "-r" $tmux a(ttach) -t [section_id] -r # pane swap, switch p1 and p0 (tmux cmd)swap-pane -s 1 -t 0 ## tmux in tmux, use double command prefix use ctrl^b^b # pda_rawread 0x4001b i_por_n : toogle from 1-> 0 -> 1 for FPGA reset u_vio_gpio_n_5 : rom boot u_vio_gpio_n_10 : ?? u_vio_gpio_n_11 : ?? (gdb) p/x l2p[0x1e14] $1 = 0x40292 (gdb) p/x l2p[0x141e] $2 = 0xffffffff # cdump, gdb session (gdb) so(oure) scripts/gdb/gef/gef.py gef> cdump nvme # startvm, target machine $cd linux $./startvm.sh $dd if=/dev/zero of=/dev/sda bs=4096 count=1 # processing a rar file $unrar e xxx.rar ### [0] Program bitstream ### [1] Open uart ### [2] start jlink ### [3]target remote ---------------------------------------------- #[0] Turbo VNC #[1] $cd sifve_jlink $./openocd/src/openocd -f ig.cfg (gdb) target rem:333 (gdb) file xxxx (gdb) lo (gdb) c host machine $./host.sh # sata case $./startvm.sh # print mem (gdb) x 0xc003201c 0xc003201c: 0x00000005 # upate it with '4' (gdb) mon mww 0xc003201c 4 # logging script # add below com to script, then so(urce) it (gdb) set pagination off (gdb) set logging file ./aieng.reg.log (gdb) set logging on (gdb) set trace-commands on (gdb) show logging (gdb) flush # set logging off (gdb) set logging off # issue nvme write traffic $sudo nvme write /dev/nvme01 -s 0 -c 23 -z 96k -d /dev/zero # issue nvme read and hexdump it $sudo nvme read /dev/nvme0n1 -s 0 -c 23 -z 96k | hexdump # git restore a modification $git restore modified.file ``` :::info DD tips https://blog.csdn.net/xiaoxiaozhu2010/article/details/17997811 - input : skip - output: seek ![](https://i.imgur.com/dNZqrMx.png) ::: ### FILE Guide - [Tacoma chip top architecture diagram](https://innogrit.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx?id=%2FShared%20Documents%2FEngineering%2FTacoma%2FArchitecture%2Fchip%5Ftop%2Ftop%5Fdiagram%5F062118%2Epdf&parent=%2FShared%20Documents%2FEngineering%2FTacoma%2FArchitecture%2Fchip%5Ftop) - [Engineering ](https://innogrit.sharepoint.com/Shared%20Documents/Forms/AllItems.aspx?csf=1&web=1&e=0MTrRA&cid=ecc996f2%2D7d78%2D4ebc%2D81c2%2Db78edb495549&FolderCTID=0x012000F24789E8A6A913488CA8D10E7B304B74&viewid=0c1b468e%2D6e4e%2D486b%2D8644%2Da6c9704b601e&id=%2FShared%20Documents%2FEngineering) - [L2P Engine AppNote](https://innogrit.sharepoint.com/:w:/r/_layouts/15/Doc.aspx?sourcedoc=%7BA5E2C1AC-C777-4447-8257-8C9D5BDFFD42%7D&file=L2P%20Engine%20AppNote.docx&action=default&mobileredirect=true) - [**Sector 512B/4096B** explanation](https://innogrit.sharepoint.com/:p:/r/_layouts/15/Doc.aspx?sourcedoc=%7BF8D4C24B-699D-4B3F-A994-1F20E7D7F2F0%7D&file=sata_logical_sector_vs_1k_4k_0.1.pptx&action=edit&mobileredirect=true) - [**RainierS EVB**](https://innogrit.sharepoint.com/:p:/r/_layouts/15/Doc.aspx?sourcedoc=%7B30b8f0cb-1512-4924-abb1-227879762f00%7D&action=edit&uid=%7B30B8F0CB-1512-4924-ABB1-227879762F00%7D&ListItemId=89837&ListId=%7B743EB1EC-2F40-41E4-899E-5A5F1295985B%7D&odsp=1&env=prod) :::info Hardware FTL accelerator helps to search and maintain L2P (Logical to Physical) address mapping relation, keep track of the valid DU (Data Unit) counter of each SPB (Super Physical Block), and create PDAs (Physical DU Address) of valid DUs for a GC (garbage collection) SPB. ::: ### AI Engine- :::info - SQ : submission queue (2 DW) - CQ : command queue (1 DW) - ![](https://i.imgur.com/ioESAKo.png) ::: ### Hot Issue List, (Ticket) --- - [10780# [RV32][R5] ramdisk performance compare](http://10.10.0.17/issues/10780) - [13137# [RainerS3 A0][FPGA][SATA] 8K DU validation](http://10.10.0.17/issues/13137) - [12842# [RainierS3][A0][FPGA][SATA] Support 4-byte internal PI](http://10.10.0.17/issues/12842) - [12765# [RainierS3][A0][FPGA][BTN] Rainier S3 BTN change list validation(from RainierB0)](http://10.10.0.17/issues/12765) - [13417# [Rainier_S][ROM] recovery mode, the usb2sata bridge enumeration failed.](http://10.10.0.17/issues/13417) - [13571# [Rainier_S][DDR] ddr patch sync to rainier_s base + pecc enable](http://10.10.0.17/issues/13571) ### Closed Ticket --- ### GDB --- ```bash= #dump memory content to a file (gdb) dump binary memory logg 0x40000000 0x40000040 #restore memory content from a file (gdB) restore ./logg1 binary 0x40000000 ``` ### Freq used Skill - Win+ALT+R 進行錄影功能，再按一次則結束錄影 - Win + Alt + Print Screen 進行螢幕截圖 - Win + Shift + S 選取區塊截圖 ### Debugging --- **[Enabling SCSI verbose logging and HBA verbose logging for analysis](https://support.microfocus.com/kb/doc.php?id=7007138)** **[How to Enable/Disable CPUs (Limiting CPU count) in CentOS / RHEL](https://www.thegeekdiary.com/how-to-enable-disable-cpus-limiting-cpu-count-in-centos-rhel/)** - [sg_inq](https://www.systutorials.com/docs/linux/man/8-sg_inq/) ```shell $ sysctl -q -w dev.scsi.logging_level=9 $ sg_inq /dev/sda $ fdisk -l /dev/sda $ echo 0x180000F1 > /proc/sys/dev/scsi/logging_level $ sg_inq /dev/sda $ dd if=/dev/zero of=/dev/sda bs=512 count=40 $ echo 0xFFFFFFFFFFFF > /proc/sys/dev/scsi/logging_level $ echo 0xFFFFFFF > /proc/sys/dev/scsi/logging_level ``` ### Scratch pad :::warning * DTAG sz = 4K DTAG index need to shift 4K to dereference DTAG ![](https://i.imgur.com/TYZZGD1.jpg) * TBD ::: ### Ticket related :::info * attached file reference ![](https://i.imgur.com/peuw169.png) ::: ### Tmux #### logging ```bash [1] $git clone https://github.com/tmux-plugins/tpm ~/.tmux/plugins/tpm [2] $vi .tmux.conf #add below line to .tmux.conf #------------------------------plugin manager -------------- set -g @plugin 'tmux-plugins/tpm' set -g @plugin 'tmux-plugins/tmux-sensible' set -g @plugin 'tmux-plugins/tmux-logging' run '~/.tmux/plugins/tpm/tpm' [3] # type this in terminal if tmux is already running $ tmux source ~/.tmux.conf [4] # call tpm to install the plugins listed in tmux.conf prefix + <I> [5] # now tmux logging is ready, use prefix + shfit + p to (start/stop) logging in the current pane. https://github.com/tmux-plugins/tmux-logging ``` #### restruct ```bash= set -g @plugin 'tmux-plugins/tmux-resurrect' prefix + Ctrl-s - save prefix + Ctrl-r - restore ``` ### Tools - [Graphviz doc](http://www.graphviz.org/doc/info/command.html) ### Raspberry ```bash= $sudo apt list --installed | grep openssh | cut -d'/' -f1 | xargs openssh-client openssh-server openssh-sftp-server $sudo systemctl enable ssh $sudo service ssh status $sudo service ssh restart ``` ### DDR/DFI debug - dfi_stress - 1st arg 0 ; 0=DDR4, 1=LPDDR4 - 2nd arg 2 ; loopCnt = 2 ```info Example: $ dfi_stress 0 2 ``` - ddr_chk Run it directly will get errors ```info 1[20211004_10:20:04.091892] cpu1: !!!ERR!!! addr 0x0000000000000004 expected 0x0E1C3870, but 0x5BDF46C2 1[20211004_10:20:04.111256] cpu1: !!!ERR!!! addr 0x0000000000000008 expected 0x152A54A8, but 0xB377E2AE 1[20211004_10:20:04.112004] cpu1: !!!ERR!!! addr 0x000000000000000C expected 0x1C3870E0, but 0x641F1B60 ``` Need to use **ddr_fill** before ddr_chk ```info Example: $ ddr_fill $ ddr_chk ``` - ddr_test_block 2 ```info Example: $ ddr_test_block 2 1[20211004_10:46:41.167014] cpu1: serial@rainier:# ddr_test_block 200003EE3 A utils/dfi/dfi_common.c +1223 ddr_test_pattern() - eng0 size 0x0000000020000000 loop 20 sys_clk 250000000 1[20211004_10:46:41.188329] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1229 ddr_test_pattern() - eng0 complete wr op, cnt: 64114432, 1904 MB/s 1[20211004_10:46:41.189826] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1242 ddr_test_pattern() - eng1 size 0x0000000020000000 loop 20 sys_clk 250000000 1[20211004_10:46:41.193624] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1248 ddr_test_pattern() - eng1 complete wr op, cnt: 64114432, 1904 MB/s 1[20211004_10:46:44.249237] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1223 ddr_test_pattern() - eng0 size 0x0000000020000000 loop 20 sys_clk 250000000 1[20211004_10:46:44.269517] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1237 ddr_test_pattern() - eng0 complete rd op, cnt: 36163872, 3332 MB/s 1[20211004_10:46:44.270851] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1242 ddr_test_pattern() - eng1 size 0x0000000020000000 loop 20 sys_clk 250000000 1[20211004_10:46:44.281873] cpu1: 00003EE3 A utils/dfi/dfi_common.c +1256 ddr_test_pattern() - eng1 complete rd op, cnt: 36041408, 3332 MB/s 1[20211004_10:46:44.283157] cpu1: Fixed pattern test0:00005A5A test1:00007E7E pass ``` - dfi_SRD ```info 1[20211004_10:54:47.488619] cpu1: serial@rainier:# dfi_SRDDFI Shadow Regsiter Dump: 1[20211004_10:54:47.489423] cpu1: DQ{ODDLY}: 1[20211004_10:54:47.490872] cpu1: {0000001F}, {0000001F}, {0000001F}, {0000001F}, {0000001F}, {0000001F}, {0000001F}, {0000001F}, 1[20211004_10:54:47.510886] cpu1: {00000056}, {00000056}, {00000056}, {00000056}, {00000056}, {00000056}, {00000056}, {00000056}, 1[20211004_10:54:47.511645] cpu1: {00000014}, {00000014}, {00000014}, {00000014}, {00000014}, {00000014}, {00000014}, {00000014}, 1[20211004_10:54:47.533230] cpu1: {0000004E}, {0000004E}, {0000004E}, {0000004E}, {0000004E}, {0000004E}, {0000004E}, {0000004E}, 1[20211004_10:54:47.534686] cpu1: {00000026}, {00000026}, {00000026}, {00000026}, {00000026}, {00000026}, {00000026}, {00000026}, 1[20211004_10:54:47.554803] cpu1: {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, 1[20211004_10:54:47.556039] cpu1: {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, 1[20211004_10:54:47.577905] cpu1: {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, 1[20211004_10:54:47.579099] cpu1: {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, {00000000}, 1[20211004_10:54:47.579672] cpu1: DBYTE{DLL, STRGT, QS_ODDLY, DM_ODDLY}: 1[20211004_10:54:47.599548] cpu1: {0061100F, 00000C31, 0000001E, 0000001F} 1[20211004_10:54:47.599631] cpu1: {0062110F, 00000D3C, 00000058, 00000056} 1[20211004_10:54:47.600178] cpu1: {005E1310, 00000B4C, 00000013, 00000014} 1[20211004_10:54:47.600819] cpu1: {005F100F, 00000D29, 0000004E, 0000004E} 1[20211004_10:54:47.601407] cpu1: {00631312, 00000C47, 00000027, 00000026} 1[20211004_10:54:47.617587] cpu1: {00003F3F, 00000E1E, 0000005E, 00000000} 1[20211004_10:54:47.618177] cpu1: {00003F3F, 00000E1E, 0000005E, 00000000} 1[20211004_10:54:47.618252] cpu1: {00003F3F, 00000E1E, 0000005E, 00000000} 1[20211004_10:54:47.618832] cpu1: {00003F3F, 00000E1E, 0000005E, 00000000} 1[20211004_10:54:47.620082] cpu1: dfiSRD done. ``` ## Mingzhi ### Tacoma12 - http://10.10.0.17/projects/tacoma12/wiki/SW_Validation_on_FPGA - Compile Option: ``` - cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_RV32=1 (for RISC-V) - cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_R5=1 (for ARM) ``` - AIENG - http://10.10.0.17/issues/13431 - Control Sequence (From designer) 1. VIP Queue Enable 0xC0100100 0x1 2. AIENG Sub-system Control 0xC0100000 0x1 3. VIP Queue SQ Lower 32-Bit 0xC0100108 [TCM Address] 4. VIP Queue Doorbell 0xC0100100 0x3 - Verify Steps: 1. $ openocd -f ~/djiang/openocd.cfg/ig.test.2c.cfg 2. $ riscv64-unknown-elf-gdb main.ramdiskddr.tacoma12.rv32 3. (gdb) target remote:3333 4. (gdb) load 5. (gdb) source aieng.2 6. (gdb) x/4xw 0xc0100100 7. (gdb) x/32w 0xc0100700 - BTN ### RainierS3 - Stable Code: - NVME 1. git checkout rainier3_fpga_a0 2. git branch rainier3_fpga_a0_develop 3. git am rainier3_fpga_a0.rainiers3_nvme.patches - SATA 1. git checkout rainier_a0_fpga_s3_a0 2. git checkout -b rainier_a0_fpga_s3_a0_develop 3. git am rainier_a0_fpga_s3_a0.rainiers3_sata.patches (or git am rainier_a0_fpga_s3_a0.rainiers3_sata_x570.patches) - Compile Options: ``` cmake ../ -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_TSB_NAND=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_CRYPTO_HW=1 cmake ../ -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_TSB_NAND=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_CRYPTO_HW=1 cmake ../ -DUSE_512B_HOST_SECTOR_SIZE=1 -DUSE_TSB_NAND=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 -DUSE_CRYPTO_HW=1 cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_R5=1 cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_R5=1 cmake ../ -DFPGA=1 -DDISABLE_HS_CRC_SUPPORT=1 -DENABLE_SOUT=1 -DRAMDISK=1 -DCPU_R5=1 cmake ../ -DUSE_TSB_NAND=1 -DRAWDISK=1 -DHMETA_SIZE=8 -DPI_SUPPORT=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 cmake ../ -DUSE_TSB_NAND=1 -DUSE_512B_HOST_SECTOR_SIZE=1 -DRAWDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 -DTARGET=4 ``` - BTN: - Ticket 12680: - Target: 10.10.1.21 - Bitfile: rainier_s3_vu440_sata_3xr5_1ch_ddr_aes_0908_4836 (CPU: R5) - Branch: rainier3_fpga_a0 - Patch: rainier3_fpga_a0.rainiers3_nvme.patches - Compile Option: ``` cmake ../ -DRAMDISK=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DMAX_CHANNEL=1 -DFPGA=1 -DDDR=1 ``` - Verification Steps: 1. openocd -f r5.cfg 2. arm-none-eabi-gdb t12/main.elf.ramdisk.r5 target remote:2331 3. ./hosh.sh - Others: - Ticket 12739: - Ticket 12842: - Ticket 13137: - Bitfile: - Branch: rainier_a0_fpga_s3_a0 - Patch: patch01-4byte_pi patch02-add-btn_rd_hcrc_err_dump - Compile Option: ``` cmake ../ -DUSE_TSB_NAND=1 -DRAWDISK=1 -DHMETA_SIZE=8 -DPI_SUPPORT=1 -DENABLE_SOUT=1 -DCPU_ID=1 -DCUSTOMER=1 -DSATA_MODE=1 -DMAX_CHANNEL=1 -DFPGA=1 ``` - Ticket 13417: - Bitfile: - Branch: - Patch: - Ticket 13571: ``` rainier_s3_vu440_sata_3xr5_1ch_ddr_aes_0908_4836.rar ``` ### Terminology - ZNS: **Z**oned **N**ame **S**paces (Compare to SMR) - BTN: Data-**B**uffer, L2P-**T**able, **N**AND-Coontrol-Layer Acceleration - BTN: **B**uffer **T**able **N**Cmd - NCB: **N**and **C**ontrol **B**lock - DPE: **D**ata **P**rocess **E**ngine - SQ: **S**ubmission **Q**ueue - CQ: **C**ompletion **Q**ueue - SEQ: **Seq**uence Command - DPE: **D**ata **P**rocess **E**ngine - VIC: **V**ector **I**nterrupt **C**ontroller - DWC: **D**esign**W**are **C**ores (Synopsys) - BIU: **B**us **I**nterface **U**int - PRD: **P**hysical **R**egion **D**escriptor - PDMA: **P**ort **DMA** - CBS: **C**ommand-**B**ased **S**witching - FIS: **F**rame **I**nformation **S**tructure - FBS: **F**IS-**B**ased **S**witching - SRB: **S**CSI **R**equest **B**locks - CDB: **C**ommand **D**escriptor **B**lock) ### Setup Environment - Set alias: git config --global user.name "xxxx" git config --global user.email "xxxx@trustteks.com" git config --global alias.co checkout git config --global alias.br branch git config --global alias.ci commit git config --global alias.st status git config --global alias.cp cherry-pick - Setup network drive: - List mapped drives: - net use - Map network drive: - net use [DRIVE]: \\IP\folder /user:username password - example: net use T: \\10.110.27.245\mlin /user:mlin password - Delete mapped drive: - net use [DRIVE]: /delete - example: net use T: /delete - Submodule: - git submodule --init --recursive ### Setup 10.10.1.21 from scratch 1. Enable VNC: $~/vnc.sh 2. Connect to 10.10.1.21 via VNC, Run Vivado 3. Run gdb server: ~/jltacoma16_mpc 1 4. Run gdb client: - $ gdb-multiarch - (gdb) cd mlin - (gdb) file main_rainiers3_nvme_bringup_a0 - (gdb) tar rem:2331 - (gdb) lo - (gdb) c 5. Output debug message via UART: - $ ~/uart_tacoma.sh 2000 7. At remote: (10.10.1.20) - $ sudo modprobe -r nvme - $ sudo modprobe nvme - $ sh fio1.sh ### Execute Firmware on 10.10.0.132 1. Run gdb server: ~/jltacoma16_mpc 1 2. Run gdb client: - $ gdb-multiarch - (gdb) cd mlin - (gdb) disc - (gdb) file main.13137.ramdisk.par.data.fill.disabled - (gdb) tar rem:2331 - (gdb) lo - (gdb) c 3. Output debug message via UART: - $ ~/uart_tacoma.sh 2000 4. Start VM: - $ ~/linux/start_vm.sh 5. At remote: (10.0.2.15) - $ dd bs=4096 count=50 with 8kdu fw 6. cdump: - (gdb) so ~/djiang/scripts/gdb/gef/gef.py - gef> cdump - The dumped file is saved at ~/djiang/bitfile/*.zip (ls -ltr) 7. Save ILA (Integrated Logic Analyzer) data as *.vcd. - The *.vcd is saved at ~/djiang/bitfile ### Exit VM 1. Ctrl+aa (Ctrl+a twice) 2. c 3. q [ENTER] ### GDB - Server: - openocd -f [config_file] - Config File: - R5: ~/djiang/r5.cfg (mlin/gdbserver_r5.cfg) - RV32: ~/djiang/ig.test.2c.cfg (mlin/gdbserver_rv32.cfg) - ~/jltacoma16_mpc 1 (R5 only) - Client: - R5: arm-none-eabi-gdb [elf_file] - RV32: riscv64-unknown-elf-gdb [elf_file] - Both: gdb-multiarch [elf_file] (support cdump) - Read register: - x/<n/f/u> [address] example: x 0xc0010054 example: x/12xw 0xc0010054 - Write register - mon mww [address] [value] - mon mww [address] [value] [count] - Dump log to file: 1. set pagination off 2. set logging file ~/gdb.132.r5.c0.log 3. set logging on 4. set trace-commands on 5. show logging 6. flush - Dump memory content to a file - (gdb) dump binary memory logg 0x40000000 0x40000040 - Restore memory content from a file - (gdB) restore ./logg binary 0x40000000 - cdump - (gdb) source ~/djiang/scripts/gdb/gef/gef.py - gef> cdump - The dumped file is saved at ~/djiang/bitfile/*.zip (ls -ltr) ### UART Debugging - Start: uart_tacoma.sh 2000 - Stop: q [Enter] - Read raw data: pda_rawread 0x40000 - Connect to Remote: - python muxterm.zip -g sout.tar -t localhost:$PORT ### Install/Remove Drive - SATA - $ rmmod ahci - $ modprobe ahci - NVME - $ modprobe -r nvme - $ modprobe nvme ### Test Command - sudo nvme write /dev/nvme0n1 -s 0 -c 23 -z 96k -d /dev/zero ### Multi CPU (RainierS3_A0) - Spec: - cpu_private_register.xlsx (Core 0) - common_cpu_cfg_register.xlsx (Core 1, 2, 3) - Firmware: - ~/djiang/fw.rs3.nvme.ftk.mu - Shared Memory Address: - 0x20000000 - Check Register: - 0xc0204000 should be 0x0000000f ### Team Arch #### US Arch - Steve Hu - Jun Liu - Shengwei Yu - Lily Rui #### Production FW - Ying Chu - Jasper Lai - Yuhung Chen - Kidd Hsieh #### ZNS FW - Stanley Miao - Zhongyong Lu - Jiapeng Ma - Hangchao Tsao - Junbiao Huang - Dahui Jiang - Shuai Ma #### Validation FW - Danny Jiang - Bob Wang - Mingzhi Lin #### Other - Tianren Li ### Knowledge Share - [大數運算](https://hackmd.io/@CLKO/r1KtmuMdf?type=view) - [upsidedown text](https://www.upsidedowntext.com/) - [FTL and GC](https://www.owlfox.org/blog/2019-11-27-coding-for-SSD-part-3/) -