---
# System prepended metadata

title: LogBook & Weekly Report

---

# LogBook & Weekly Report

##  :fleur_de_lis:Muhammad Sayyid Afif (23222314)
::: spoiler Kontak
:iphone:  : 0852-8257-7396

:email: : 23222314@std.stei.itb.ac.id
:::

## :bulb: Topik Riset
:::info
Accelerator for Transformer Model machine learning
:::


## :trident: Metoda Penelitian

:::warning
Hasil dari mata kuliah metoda penelitian

:Scroll: [Short Paper](https://drive.google.com/file/d/1tZUvt34SHp4flgsGw6w4n2yqQFZWRGg0/view?usp=sharing)

:Scroll: [Pra-Thesis](https://drive.google.com/file/d/1tsvBwuIN9_999X4WJ4HKrQMmg08kmgxY/view?usp=sharing)


:::

## :calendar: Weekly Roadmap

:::info
Goal setiap minggunya
:::

|       Week     |     Goals     | Status |
|--------------- |---------------|--------|
| **Week 1**     | - |- |
| **Week 2**     | <ul><li>~~Pembuatan page HackMD~~ </li><li> ~~10 study literature transformer pada hardware/accelerator~~</li><li>~~Lanjutkan pembuatan ke hardware setelah matrix mult *Baru belajar saja~~</li> </ul> |:o: |
| **Week 3**     | <ul><li>~~Revisi PPT~~</li><li>~~Menentukan paper utama dan membuat ppt~~</li><li>~~Menulis Introduction (Baru membuat page overleaf)~~</li></ul> |:o: |
| **Week 4**     | <ul><li>~~Arah input output~~</li><li>~~Mencoba bikin verilognya~~</li><li>~~Membedah matematikanya~~</li></ul> |:o: |
| **Week 5**     | <ul><li>~~Simulasi perbandingan ada akar K dan tidak~~</li><li>~~Mencoba ukuran dimensi 5x5~~</li><li>~~Verilog hardware~~</li></ul> |:x: |
| **Week 6**     | <ul><li>~~Pisahkan /dK dari QKt/(/dK) menjadi Q' = Q/(/dK).~~</li><li>~~Mulai arsitektur RTL-HW. Buat perblock. Bikin top level dulu.~~</li><li>~~Bikin model dari suku 3-5.~~</li></ul> |:o: |
| **Week 7**     | <ul><li>~~PRA UTS~~</li></ul> |:o: |
| **Week 8**     | <ul><li>~~Membuat paralel partisi~~</li><li>~~Membuat proses elementnya~~</li></ul> |:o: |
| **Week 9**     | <ul><li>~~Membuat Verilognya~~</li><li>~~Submit Paper(Di undur)~~</li></ul> |:o: |
| **Week 10**    |  ~~Diskusi dan review kembali pemrograman PS da PL~~|:o: |
| **Week 11**    | <ul><li>~~Melanjutkan Verilog~~</li><li>~~Revisi Paper~~</li></ul> |:o: |
| **Week 12**    | ~~Belajar menggunakan bram~~ |:o: |
| **Week 13**    | ~~Desai 2 bram input 1 output~~ |:o: |
| **Week 14**    | ~~Simulasi 2 input 1 output bram~~ |:o: |
| **Week 15**    | ~~Mempelajari axi control~~ |:o: |
| **Week 16**    | ~~Menggabungkan modul PE dan simulasi~~ |:o: |
| **Week 17**    | Implementasi pemanggilan modul, tetapi modul Plus |:x: |
| **Week 18**    | <ul><li>Pembuatan Matrix 4x4</li><li>Mengupgrade ke 8x8</li></ul> |:x: |


## :notebook_with_decorative_cover: Daily Report

:::success
Laporan pekerjaan harian
:::

### Week 1

:::spoiler
:musical_score: 

:notes:
:::

### Week 2
:::spoiler Selasa, 20/2
:musical_score: 

Pembuatan page HackMD dan mencari beberapa pdf untuk literature study.

:notes: Hasil

A. [Page HackMD](https://hackmd.io/@23222314/Afif_LogBook)

B. Paper 

[1. ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers](https://ieeexplore.ieee.org/document/10244348)

[2. Self-Attention and Transformers: Driving the Evolution of Large Language Models](https://ieeexplore.ieee.org/document/10245906/)
:::

:::spoiler Rabu, 21/2
:musical_score: 

Melanjutkan pencarian literature dan apa itu softmax

:notes: Hasil

A. [Simple exlanation softmax](https://youtu.be/8ah-qhvaQqU)

B. Paper 

[ 3. COSA:Co-Operative Systolic Arrays for Multi-head Attention Mechanism in Neural Network using Hybrid Data Reuse and Fusion Methodologies](https://ieeexplore.ieee.org/document/10247678)

[4. Towards A Reconfigurable Systolic Array with Multi-Level Packing for Transformers](https://www.semanticscholar.org/paper/Towards-A-Reconfigurable-Systolic-Array-with-for-Zhao-Miao/58b37e820234c82c91ce78f71cfafc1e0216a565)

[5. A Fully-Integrated Energy-Scalable Transformer Accelerator Supporting Adaptive Model Configuration and Word Elimination for Language Understanding on Edge Devices](https://ieeexplore.ieee.org/document/10244459)
:::

:::spoiler Kamis, 22/2
:musical_score: 

Melanjutkan pencarian literature

:notes: Hasil

[6. Deep Neural Network Model and FPGA Accelerator Co-Design: Opportunities and Challenges](https://ieeexplore.ieee.org/document/8564956)

[7. Deep Neural Network Model and FPGA Accelerator Co-Design: Opportunities and Challenges](https://ieeexplore.ieee.org/document/8564956)

[8. SKYNET: A HARDWARE-EFFICIENT METHOD FOR OBJECT DETECTION AND TRACKING ON EMBEDDED SYSTEMS ](https://arxiv.org/abs/1909.09709)

[9. An Algorithm–Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers ](https://ieeexplore.ieee.org/document/9857911)

[10. P3 ViT: A CIM-Based High-Utilization Architecture With Dynamic Pruning and Two-Way Ping-Pong Macro for Vision Transformer ](https://ieeexplore.ieee.org/document/10263621)

[Hasil PPT Study Literature ](https://docs.google.com/presentation/d/1RzHGng0s4x54riwtCSQ2i3SpcWm3mqBe/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)

:::



### Week 3

:::spoiler Senin, 26/2
:musical_score: Revisi ppt

:notes: Hasil

[Study Literature_Afif](https://docs.google.com/presentation/d/1RzHGng0s4x54riwtCSQ2i3SpcWm3mqBe/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::

:::spoiler Selasa, 27/2
:musical_score: Update HackMD

:::

:::spoiler Rabu, 28/2
:musical_score: Menentukan paper utama dan pelajari

:notes:
[Paper Utama](https://ieeexplore.ieee.org/document/10247678)
:::

:::spoiler Kamis, 29/2
:musical_score: Membuat presentasi & kelas

:notes:
[PPT paper utama](https://docs.google.com/presentation/d/1rQzRKDlAggEQWJYmZzOxFswPgFjBckub/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::
### Week 4

:::spoiler Senin, 4/3
:musical_score: 
Update HackMD & upluad assignment, membaca ulang paper, mencari flow input-output, belajar cara membuat aritmatika/algoritmanya

:notes:

[Paper Utama PDF](https://drive.google.com/file/d/1g3pZTAwK4qho3oIqbm21dpEFeGzIOTDk/view?usp=sharing)
:::

:::spoiler Selasa 5/3
:musical_score: 
Berdiskusi dengan teman bagaimana cara membuat aritmatikanya. Further reading tentang Output Stationary (WS) dan Weight Stationary (OS)

:notes:

[OS & WS di Chapter V](https://arxiv.org/abs/1703.09039)
:::

:::spoiler Rabu 6/3
:musical_score: 
Mengubah dari algoritma ke matematis, menggambar flowchart/alur input output

:notes: 
:::

:::spoiler Kamis 7/3
:musical_score: 
Kelas dan presentasi, serta goal minggu depan

:notes: 

[Proposed Method](https://docs.google.com/presentation/d/1VbKjsExMHx9IE8yCnuy-hbJAv60z7ZCY/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true) 
:::
### Week 5

:::spoiler Selasa, 12/3
:musical_score: Mencoba perbandingan antara adanya akar d_k dan tidak

:notes:

With dk
![with dk](https://hackmd.io/_uploads/r19MKsa6p.png)

Without dk
![without dk](https://hackmd.io/_uploads/SyfXtj6aT.png)

:::

:::spoiler Rabu, 13/3
:musical_score: 
Melanjutkan perbandingan akar dan mencoba verilog

:notes:

![Multihead pada head 2](https://hackmd.io/_uploads/ryF0OyeCT.png)

![Hasil Concatenate](https://hackmd.io/_uploads/SJvbKkg06.png)

:::

:::spoiler Kamis, 14/3
:musical_score: 
Merapihkan powerpoint dan menlanjutkan verilognya

:notes:

[Power point](https://docs.google.com/presentation/d/1S-JNO2hHf4jBAQkU_OPPl3YJW-Rgz6_C/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)

[Kode Verilog](https://drive.google.com/drive/folders/18DJxUMxZ8MRss8fIWqBBOx2P-mX_bt2x?usp=sharing)
:::
### Week 6

:::spoiler Selasa, 19/3
:musical_score: 
Membuat perbandingan tanpa Akar d_k, Q', dan konvensional (Error)
:notes:
:::

:::spoiler Rabu, 20/3
:musical_score: 
Memperbaiki simulasi perbadingan dan mencoba menambahkan suku kata, tetapi terdapat error, dan mencari solusinya (belum dapat). Mencoba membuat diagram bloknya.

:notes:
![head 0](https://hackmd.io/_uploads/ryv7Xzu0p.png)

![multihead](https://hackmd.io/_uploads/BJoX7GO0a.png)

Error code
![Error](https://hackmd.io/_uploads/ByGRnNd0T.png)


:::

:::spoiler Kamis, 21/3
:musical_score: 
Membuat 3-5 kata tetapi menggunakan nilai random, membuat pptnya.

:notes:
[Power point](https://docs.google.com/presentation/d/1DOwgaBIFlWllMSwi8mgq91l21Q0mCjQQ/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::
### Week 7

:::spoiler Rabu, 24/3
:musical_score: 
Mempelajari paralel partisi
:notes:
:::

:::spoiler Kamis, 25/3
:musical_score: 
Menulis proposed method pada overleaf
:notes:
:::

### Week 8

:::spoiler Selasa, 2/4
:musical_score: Melanjutkan partisi dari matriks 8x768 dan membuat pemodelannya

:notes:
:::

:::spoiler Rabu, 3/4
:musical_score: Membuat arsitektur processing element

:notes:
![image](https://hackmd.io/_uploads/Sy0ULt4lR.png)


:::

:::spoiler Kamis, 3/4
:musical_score: Merevisi processing elementnya dan membuat Verilognya

:notes:
![image](https://hackmd.io/_uploads/rJuf8FNxA.png)

:::
### Week 9

:::spoiler Rabu, 10/4
:musical_score: Riset dan mencoba membuat verilognya

:notes:
:::

:::spoiler Kamis, 11/4
:musical_score: Masih mencoba verilog dan melengkapi paper

:notes:
:::
### Week 10

:::spoiler Kamis, 18/4
:musical_score: Membuat verilog matriks multiplication dengan flow yang sesuai dengan PE yang didislusikan

:notes:
[Top module](https://drive.google.com/file/d/1wo7s8GwS96wC6ugnV7NdfGa0MLvQfjhU/view?usp=sharing)

[PE](https://drive.google.com/file/d/1ksLNYmWruKoSmM0N4mjUHfEXLgMYom8r/view?usp=sharing)

:::

### Week 11

:::spoiler Senin, 22/4
:musical_score: Diskusi dan mereview kembali pemrograman PL dan PS pada zybo

:notes:
Modul 5 & 6 mas Erwin
:::

:::spoiler Selasa, 23/4
:musical_score: Memperbaiki paper dari awal

:notes:
:::

:::spoiler Rabu, 24/4
:musical_score: Melanjutkan menulis paper dan membuat algoritma control datanya

:notes:
:::
### Week 12

:::spoiler Kamis 2/5
:musical_score: Belajar menggunakan bram 

:notes:
:::

### Week 13

:::spoiler Selasa 7/5
:musical_score: Mencoba mendesain 2 Bram input dan 1 Bram output

:notes:
:::
### Week 14

:::spoiler Selasa 14/5
:musical_score: Membuat 2 port bram sebagai input dan 1 bram sebagai output

:notes:
:::

:::spoiler Rabu 15/5
:musical_score: Melakukan simulasi 2 bram input dan 1 bram output

:notes:
:::
### Week 15

:::spoiler  Kamis 23/5
:musical_score: Mempelajari axi untuk menghubungkan bram ke PS

:notes:
:::
### Week 16

:::spoiler Selasa 28/5
:musical_score: Melakukan percobaan menggunakan/pemanggilan modul pe

:notes:
:::

:::spoiler Rabu 29/5
:musical_score: Bisa memanggil modul PE tetapi output tidak sesuai

:notes:
:::

:::spoiler Kamis 30/5
:musical_score: Mengganti modul PE menjadi modul Plus yang lebih sederhana, modu Plus berfungsi hanya menambahkan inputA dan inputB menjadi outputC

:notes:
:::

### Week 17

:::spoiler Senin 3/6
:musical_score: Melanjutkan hingga Implementasi tetapi menggunakan modul Plus. Tetapi belum sampai C

:notes:
:::
## :camera: Catatan

:::success
To be or not to be
:::