or
or
By clicking below, you agree to our terms of service.
Sign in with Wallet
New to HackMD? Sign up
| Syntax | Example | Reference | |
|---|---|---|---|
| # Header | Header | 基本排版 | |
| - Unordered List |
|
||
| 1. Ordered List |
|
||
| - [ ] Todo List |
|
||
| > Blockquote | Blockquote |
||
| **Bold font** | Bold font | ||
| *Italics font* | Italics font | ||
| ~~Strikethrough~~ | |||
| 19^th^ | 19th | ||
| H~2~O | H2O | ||
| ++Inserted text++ | Inserted text | ||
| ==Marked text== | Marked text | ||
| [link text](https:// "title") | Link | ||
|  | Image | ||
| `Code` | Code |
在筆記中貼入程式碼 | |
| ```javascript var i = 0; ``` |
|
||
| :smile: |  |
Emoji list | |
| {%youtube youtube_id %} | Externals | ||
| $L^aT_eX$ | LaTeX | ||
| :::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Syncing
xxxxxxxxxx-
Any changes
Be notified of any changes
-
Mention me
Be notified of mention me
-
Unsubscribe
SubscribeBeyond Word Embedding : Keyword and BERT - JerryWu
tags:
COSCUP2020入門TR212歡迎來到 https://hackmd.io/@coscup/2020 共筆
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →點擊本頁上方的 開始用 Markdown 一起寫筆記！
手機版請點選上方 按鈕展開議程列表。
今天主要分享BERT的演進。
Keyword and Bert
只用keyword判斷：會有誤差。只用句向量和詞向量時，句子和詞都會變成向量。認識字之後就可以直接回答，但是遇到不認識的字就無法回答。
BERT事先train超過3億筆data，比只用Keyword識別能力更強。
Example
懷孕可以吃藥嗎？/生小孩可以吃藥嗎？
人類定義：懷孕=生小孩
Keyword：不認為懷孕=生小孩，無法分辨
BERT
Model Architecture - Encoder from Transformer
一句話：具有權重的向量，訓練模型有更強能力。
Next Sentence Prediction(NSP) 後期被BERT拿掉囉