# LogBook & Weekly Report

## :fleur_de_lis:Muhammad Sayyid Afif (23222314)
::: spoiler Kontak
:iphone: : 0852-8257-7396

:email: : [e-mail]
:::

## :bulb: Topik Riset
:::info
Accelerator for Transformer Model machine learning
:::

## :trident: Metoda Penelitian

:::warning
Hasil dari mata kuliah metoda penelitian

:Scroll: [Short Paper](https://drive.google.com/file/d/1tZUvt34SHp4flgsGw6w4n2yqQFZWRGg0/view?usp=sharing)

:Scroll: [Pra-Thesis](https://drive.google.com/file/d/1tsvBwuIN9_999X4WJ4HKrQMmg08kmgxY/view?usp=sharing)

:::

## :calendar: Weekly Roadmap

:::info
Goal setiap minggunya
:::

| Week | Goals | Status |
|--------------- |---------------|--------|
| **Week 1** | - |- |
| **Week 2** | <ul><li>~~Pembuatan page HackMD~~ </li><li> ~~10 study literature transformer pada hardware/accelerator~~</li><li>~~Lanjutkan pembuatan ke hardware setelah matrix mult *Baru belajar saja~~</li> </ul> |:o: |
| **Week 3** | <ul><li>~~Revisi PPT~~</li><li>~~Menentukan paper utama dan membuat ppt~~</li><li>~~Menulis Introduction (Baru membuat page overleaf)~~</li></ul> |:o: |
| **Week 4** | <ul><li>~~Arah input output~~</li><li>~~Mencoba bikin verilognya~~</li><li>~~Membedah matematikanya~~</li></ul> |:o: |
| **Week 5** | <ul><li>~~Simulasi perbandingan ada akar K dan tidak~~</li><li>~~Mencoba ukuran dimensi 5x5~~</li><li>~~Verilog hardware~~</li></ul> |:x: |
| **Week 6** | <ul><li>~~Pisahkan /dK dari QKt/(/dK) menjadi Q' = Q/(/dK).~~</li><li>~~Mulai arsitektur RTL-HW. Buat perblock. Bikin top level dulu.~~</li><li>~~Bikin model dari suku 3-5.~~</li></ul> |:o: |
| **Week 7** | <ul><li>~~PRA UTS~~</li></ul> |:o: |
| **Week 8** | <ul><li>~~Membuat paralel partisi~~</li><li>~~Membuat proses elementnya~~</li></ul> |:o: |
| **Week 9** | <ul><li>~~Membuat Verilognya~~</li><li>~~Submit Paper(Di undur)~~</li></ul> |:o: |
| **Week 10** | ~~Diskusi dan review kembali pemrograman PS da PL~~|:o: |
| **Week 11** | <ul><li>~~Melanjutkan Verilog~~</li><li>~~Revisi Paper~~</li></ul> |:o: |
| **Week 12** | ~~Belajar menggunakan bram~~ |:o: |
| **Week 13** | ~~Desai 2 bram input 1 output~~ |:o: |
| **Week 14** | ~~Simulasi 2 input 1 output bram~~ |:o: |
| **Week 15** | ~~Mempelajari axi control~~ |:o: |
| **Week 16** | ~~Menggabungkan modul PE dan simulasi~~ |:o: |
| **Week 17** | Implementasi pemanggilan modul, tetapi modul Plus |:x: |
| **Week 18** | <ul><li>Pembuatan Matrix 4x4</li><li>Mengupgrade ke 8x8</li></ul> |:x: |

## :notebook_with_decorative_cover: Daily Report

:::success
Laporan pekerjaan harian
:::

### Week 1

:::spoiler
:musical_score:

:notes:
:::

### Week 2
:::spoiler Selasa, 20/2
:musical_score:

Pembuatan page HackMD dan mencari beberapa pdf untuk literature study.

:notes: Hasil

A. [Page HackMD](https://hackmd.io/@23222314/Afif_LogBook)

B. Paper

[1. ITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers](https://ieeexplore.ieee.org/document/10244348)

[2. Self-Attention and Transformers: Driving the Evolution of Large Language Models](https://ieeexplore.ieee.org/document/10245906/)
:::

:::spoiler Rabu, 21/2
:musical_score:

Melanjutkan pencarian literature dan apa itu softmax

:notes: Hasil

A. [Simple exlanation softmax](https://youtu.be/8ah-qhvaQqU)

B. Paper

[ 3. COSA:Co-Operative Systolic Arrays for Multi-head Attention Mechanism in Neural Network using Hybrid Data Reuse and Fusion Methodologies](https://ieeexplore.ieee.org/document/10247678)

[4. Towards A Reconfigurable Systolic Array with Multi-Level Packing for Transformers](https://www.semanticscholar.org/paper/Towards-A-Reconfigurable-Systolic-Array-with-for-Zhao-Miao/58b37e820234c82c91ce78f71cfafc1e0216a565)

[5. A Fully-Integrated Energy-Scalable Transformer Accelerator Supporting Adaptive Model Configuration and Word Elimination for Language Understanding on Edge Devices](https://ieeexplore.ieee.org/document/10244459)
:::

:::spoiler Kamis, 22/2
:musical_score:

Melanjutkan pencarian literature

:notes: Hasil

[6. Deep Neural Network Model and FPGA Accelerator Co-Design: Opportunities and Challenges](https://ieeexplore.ieee.org/document/8564956)

[7. Deep Neural Network Model and FPGA Accelerator Co-Design: Opportunities and Challenges](https://ieeexplore.ieee.org/document/8564956)

[8. SKYNET: A HARDWARE-EFFICIENT METHOD FOR OBJECT DETECTION AND TRACKING ON EMBEDDED SYSTEMS ](https://arxiv.org/abs/1909.09709)

[9. An Algorithm–Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers ](https://ieeexplore.ieee.org/document/9857911)

[10. P3 ViT: A CIM-Based High-Utilization Architecture With Dynamic Pruning and Two-Way Ping-Pong Macro for Vision Transformer ](https://ieeexplore.ieee.org/document/10263621)

[Hasil PPT Study Literature ](https://docs.google.com/presentation/d/1RzHGng0s4x54riwtCSQ2i3SpcWm3mqBe/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)

:::

### Week 3

:::spoiler Senin, 26/2
:musical_score: Revisi ppt

:notes: Hasil

[Study Literature_Afif](https://docs.google.com/presentation/d/1RzHGng0s4x54riwtCSQ2i3SpcWm3mqBe/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::

:::spoiler Selasa, 27/2
:musical_score: Update HackMD

:::

:::spoiler Rabu, 28/2
:musical_score: Menentukan paper utama dan pelajari

:notes:
[Paper Utama](https://ieeexplore.ieee.org/document/10247678)
:::

:::spoiler Kamis, 29/2
:musical_score: Membuat presentasi & kelas

:notes:
[PPT paper utama](https://docs.google.com/presentation/d/1rQzRKDlAggEQWJYmZzOxFswPgFjBckub/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::
### Week 4

:::spoiler Senin, 4/3
:musical_score:
Update HackMD & upluad assignment, membaca ulang paper, mencari flow input-output, belajar cara membuat aritmatika/algoritmanya

:notes:

[Paper Utama PDF](https://drive.google.com/file/d/1g3pZTAwK4qho3oIqbm21dpEFeGzIOTDk/view?usp=sharing)
:::

:::spoiler Selasa 5/3
:musical_score:
Berdiskusi dengan teman bagaimana cara membuat aritmatikanya. Further reading tentang Output Stationary (WS) dan Weight Stationary (OS)

:notes:

[OS & WS di Chapter V](https://arxiv.org/abs/1703.09039)
:::

:::spoiler Rabu 6/3
:musical_score:
Mengubah dari algoritma ke matematis, menggambar flowchart/alur input output

:notes:
:::

:::spoiler Kamis 7/3
:musical_score:
Kelas dan presentasi, serta goal minggu depan

:notes:

[Proposed Method](https://docs.google.com/presentation/d/1VbKjsExMHx9IE8yCnuy-hbJAv60z7ZCY/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::
### Week 5

:::spoiler Selasa, 12/3
:musical_score: Mencoba perbandingan antara adanya akar d_k dan tidak

:notes:

With dk

Without dk

:::

:::spoiler Rabu, 13/3
:musical_score:
Melanjutkan perbandingan akar dan mencoba verilog

:notes:



:::

:::spoiler Kamis, 14/3
:musical_score:
Merapihkan powerpoint dan menlanjutkan verilognya

:notes:

[Power point](https://docs.google.com/presentation/d/1S-JNO2hHf4jBAQkU_OPPl3YJW-Rgz6_C/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)

[Kode Verilog](https://drive.google.com/drive/folders/18DJxUMxZ8MRss8fIWqBBOx2P-mX_bt2x?usp=sharing)
:::
### Week 6

:::spoiler Selasa, 19/3
:musical_score:
Membuat perbandingan tanpa Akar d_k, Q', dan konvensional (Error)
:notes:
:::

:::spoiler Rabu, 20/3
:musical_score:
Memperbaiki simulasi perbadingan dan mencoba menambahkan suku kata, tetapi terdapat error, dan mencari solusinya (belum dapat). Mencoba membuat diagram bloknya.

:notes:


Error code

:::

:::spoiler Kamis, 21/3
:musical_score:
Membuat 3-5 kata tetapi menggunakan nilai random, membuat pptnya.

:notes:
[Power point](https://docs.google.com/presentation/d/1DOwgaBIFlWllMSwi8mgq91l21Q0mCjQQ/edit?usp=sharing&ouid=115748719573294335061&rtpof=true&sd=true)
:::
### Week 7

:::spoiler Rabu, 24/3
:musical_score:
Mempelajari paralel partisi
:notes:
:::

:::spoiler Kamis, 25/3
:musical_score:
Menulis proposed method pada overleaf
:notes:
:::

### Week 8

:::spoiler Selasa, 2/4
:musical_score: Melanjutkan partisi dari matriks 8x768 dan membuat pemodelannya

:notes:
:::

:::spoiler Rabu, 3/4
:musical_score: Membuat arsitektur processing element

:notes:

:::

:::spoiler Kamis, 3/4
:musical_score: Merevisi processing elementnya dan membuat Verilognya

:notes:

:::
### Week 9

:::spoiler Rabu, 10/4
:musical_score: Riset dan mencoba membuat verilognya

:notes:
:::

:::spoiler Kamis, 11/4
:musical_score: Masih mencoba verilog dan melengkapi paper

:notes:
:::
### Week 10

:::spoiler Kamis, 18/4
:musical_score: Membuat verilog matriks multiplication dengan flow yang sesuai dengan PE yang didislusikan

:notes:
[Top module](https://drive.google.com/file/d/1wo7s8GwS96wC6ugnV7NdfGa0MLvQfjhU/view?usp=sharing)

[PE](https://drive.google.com/file/d/1ksLNYmWruKoSmM0N4mjUHfEXLgMYom8r/view?usp=sharing)

:::

### Week 11

:::spoiler Senin, 22/4
:musical_score: Diskusi dan mereview kembali pemrograman PL dan PS pada zybo

:notes:
Modul 5 & 6 mas Erwin
:::

:::spoiler Selasa, 23/4
:musical_score: Memperbaiki paper dari awal

:notes:
:::

:::spoiler Rabu, 24/4
:musical_score: Melanjutkan menulis paper dan membuat algoritma control datanya

:notes:
:::
### Week 12

:::spoiler Kamis 2/5
:musical_score: Belajar menggunakan bram

:notes:
:::

### Week 13

:::spoiler Selasa 7/5
:musical_score: Mencoba mendesain 2 Bram input dan 1 Bram output

:notes:
:::
### Week 14

:::spoiler Selasa 14/5
:musical_score: Membuat 2 port bram sebagai input dan 1 bram sebagai output

:notes:
:::

:::spoiler Rabu 15/5
:musical_score: Melakukan simulasi 2 bram input dan 1 bram output

:notes:
:::
### Week 15

:::spoiler Kamis 23/5
:musical_score: Mempelajari axi untuk menghubungkan bram ke PS

:notes:
:::
### Week 16

:::spoiler Selasa 28/5
:musical_score: Melakukan percobaan menggunakan/pemanggilan modul pe

:notes:
:::

:::spoiler Rabu 29/5
:musical_score: Bisa memanggil modul PE tetapi output tidak sesuai

:notes:
:::

:::spoiler Kamis 30/5
:musical_score: Mengganti modul PE menjadi modul Plus yang lebih sederhana, modu Plus berfungsi hanya menambahkan inputA dan inputB menjadi outputC

:notes:
:::

### Week 17

:::spoiler Senin 3/6
:musical_score: Melanjutkan hingga Implementasi tetapi menggunakan modul Plus. Tetapi belum sampai C

:notes:
:::
## :camera: Catatan

:::success
To be or not to be
:::