---
# System prepended metadata

title: '2020-08-11'

---

# 2020-08-11
## 1
[李宏毅課程](http://speech.ee.ntu.edu.tw/~tlkagk/courses_ML20.html)
### ML intro
Regression
Classification
http://cs231n.stanford.edu/slides/
NN architecture & flow calculation
Network Compression
    - BERT
    - ALBERT

### NLP & Human language
- Token definition
- Token Embedding
- Speech intro
    - [speech ](http://ocw.aca.ntu.edu.tw/ntu-ocw/ocw/cou/104S204/7)
    - Speech2Vector
- 語音與是否出現高/低頻率有關係(尋找哪些音頻有出現)
- Voice Conversion
    - supervised
    - unsupervised
        - different people different language
        - one-shot learning 
    - seq2seq variants
    - seq2seq task
        - sentiment classification
        - stance detection
            - post、replies => denied/acept
        - vearcity prediction(fake article)
            - post、replies、wiki => true/false
        - natural language inference
            - premise、hypothesis
            - =>contradiction、entailment、neutral
        - search engine
            - keyword、article
            - output: relevant
        - part-of-speech tagging
            - 詞性
            - input length=output length
        - word segmentation
            - 斷詞，中文獨有
            - 找斷詞點
        - text style transfer
            - input lenght != output lenght
        - summarization
            - input lenght != output lenght
        - translation
            - input lenght != output lenght
        - unsupervised abstractive summarization
            - document -> model -> summary
            - GAN
        - Unsupervised translation
        - Question Answering
            - reading comprehension
        - task-oriented dialogue
            - =>> natural language generation
            - =>>> policy & state tracker
        - Natural Language Understanding
            - intent classification
            - slot filling
        - chatbot
            - 與人對聊
            - input multi-seq=>output a sequence
            - 有方向性的決定回話方式(正向、負向)
        - coreference resolution(指代消解)
            - 代名詞指向誰
        - parsing
    - self-supervised learning
        - 解釋supervised & self-supervised difference
    - self-supervised learning model intro
        - ELMo
        - BERT
        - ERNIE
        - Grover
        - Big Bird(Transformer for longer sequence)
    - General Language Understanding Evaluation(GLUE)
        - test model on different NLP task ability
        - extend: Super GLUE
- BERT
    - Contextualized Word Embedding
        - BERT series
        - Word Embedding
            - w2v
            - Glove
            - fasttext
        - Contextualized Word Embedding
    - How to Train BERT
        - Masked LM
        - NSP
            - RoBERTa說NSP不work，拿掉比較好
            - 改良: Sentence Order Predcition(SOP)
    - How to use BERT
        - sentiment analysis
        - NLI classification
        - Extraction QA
    - MASS/BART
        - seq2seq model
        - solve input length!=output length

## 2. torch
https://youtu.be/z0uOq2wEGc
- tensor calculation
- fc layer
- build model/module
- loss
- optimizer
    - gradient descent
- 