---
# System prepended metadata

title: DLHLP

---

# DLHLP

### Datasets
- TAT
    - 比較乾淨的dataset
- 台語連續劇
    - 比較noisy

### 方向
- model: xls-r
- baseline: 跟finetuning做比較

### Next
- xls-r 如何pretrain
- 怎麼做continual learning
- 試試看只train adapter
- adapter: [Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/abs/1902.00751)
- 下載xls-r checkpoint，把config load出來看用什麼pretrain task
- 先拿TAT(~600小時)試試看

### Meeting note
#### 9/26
- 去改fairseq的trainer
- strict: false

### Misc
[Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://arxiv.org/abs/2004.10964)
New SOTA model: WavLM
[XLS-R](https://github.com/facebookresearch/fairseq/tree/main/examples/wav2vec/xlsr)

### 10/15 meeting note
先拿少量的資料
小規模實驗
adapter -> 不用花那麼多時間pretrain
用monolingual wav2vec2(xlsr沒那麼必要)
dust magic
https://arxiv.org/abs/2110.03560
https://arxiv.org/abs/2210.06175

target: 
1. xlsr, wav2vec2, HuBERT (wavLM) finetune on adapters
2. adapters
    1. houslbee
    2. other 子晴姊試過的