# 9/24進度紀錄 GreenAI

## 簡筠方

### llama-3.2-1b-it測試 (fewshot)

| model | acc |
| -------- | ------ |
| baseline | 0.4503 |
| bnb-4bit | 0.2782 |
| awq-4bit | 0.3723 |

### 困難

現在只剩 `gptq` 跑 `llama-3.2-1b-it` 會 OOM 了
但是可以跑 `opt-350m` ，算是測試成功
🥲🥲🥲🥲🥲🥲

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 GiB. GPU 0 has a total capacity of 23.52 GiB of which 7.67 GiB is free. Process 1423663 has 14.27 GiB memory in use. Including non-PyTorch memory, this process has 1.55 GiB memory in use. Of the allocated memory 1.11 GiB is allocated by PyTorch, and 1.74 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

## 游婷安
進度主要是繼續檢查程式問題出在哪 :smiling_face_with_tear:
我發現上次準確率長很像是因為測資太短，所以大部分測資幾乎沒壓縮。硬要壓縮會造成反效果：花很多時間計算，accuracy也比較低。
所以我可能要換資料集?

| compress ratio (保留多少) | accuracy |
| -------- | -------- |
| 0.75 | 0.213798332069749 |
| 0.5 | 0.21304018195602728 |
| 0.3 | 0.21152388172858225 |

## 陳芊羽

- 找一些專門對 llm 做剪枝的工具
只有大概看能用在哪些 model，原理啥的都沒看 :cry:
~~數學跟英文是雙重 debuff~~
(?)：不確定 or 可能要實際跑跑看會不會出 bug
| 剪枝工具 | 適用模型 | 連結 |
| ------ | ------- | ---- |
| LLM-Pruner | Llama-3.1, Llama-3, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama | https://github.com/horseee/LLM-Pruner |
| sparseGPT | OPT, Llama, BLOOM(?) | https://github.com/IST-DASLab/sparsegpt |
| wanda | hugging_face大部分(AutoModelForCausalLM)(?) | https://github.com/locuslab/wanda |
| Z-Pruner | Llama, OPT | https://github.com/sazzadadib/Z-Pruner |
| llm-kick | Vicuna, LlaMa | https://github.com/VITA-Group/llm-kick |
| RIA | hugging_face大部分(AutoModelForCausalLM)(?)| https://github.com/biomedical-cybernetics/Relative-importance-and-activation-pruning |
- 補了一點點抓答案的判斷
gemma-3-270M 天賦異稟，他甚至會給我 5.500.000.000.000.000.000.000.000.000.000.000.000.0
- TMP
prune_standard 亂弄的，他現在就是一般 accuracy 而已
- git push error
我一小時前 push 好好的，他現在突然不給 pull 跟 push QQ