# Paper Note 08/15
---
---
## Basic Ideas
- Continual Pre-training
Word-aware -> Structure-aware -> Semantic-aware
- Task Embedding
---
# Tasks
- Word-aware
- Knowledge Masking
- Capitalization Prediction
- Token-Document Relation
- Structure-aware
- Sentence Reordering
- Sentence Distance
- Semantic-aware
- Discourse Relation
- IR Relevance
---
## Model Structure
{"metaMigratedAt":"2023-06-14T23:25:27.500Z","metaMigratedFrom":"Content","title":"Paper Note 08/15","breaks":true,"contributors":"[{\"id\":\"fa032ae2-ac9e-4754-9d6f-947b0ea9cb6e\",\"add\":488,\"del\":1}]"}