# [Paper筆記] : A Survey on Transfer Learning (2010)

### [Paper筆記] : A Survey on Transfer Learning (2010)

### **<Introduction>**

--------------------------------------

Common machine learning algorithms work under an assumption:

> **Training and Testing data are drawn from the same feature space and have the same distribution**

Therefore, when the distribution changes, models need to be rebuild from scratch using new data. **There's no knowledge transfer between tasks.**.

In reality, there are many scenarios where such assumption can be problematic when modeling.

> * **Limited data for our target task**
> * There may exist similar tasks with abundant dataset and more sophisticated models.
> * However, by traditional methodology, we can't harvest the knowledge from these similar tasks.
> * **The data may be easily outdated due to dynamic environments**
> * Models need to adapt quickly, while based on existing knowledge.
> * (ex) Wifi data to infer location.

---------------------------------------

Focus of this paper:

* Definition of terms and topics in transfer learning
* Survey transfer learning techniques for classification, regression, and clustering.

--------------------------------------------

### **<Overview>**

-------------------------------------------------

#### Brief History of Transfer Learning

------------------------------------------------

Many techniques have been proposed to address the problem of not enough labeled data.
For example, **semi-supervised learning** and **active learning** are two of them.

> - **Semi-Superised Learning:**
> - Large amount of unlabeled data + Small amount of labeled data
> - **Active Learning:**
> - Select the next candidate data to be labeled, for cost-effective concern.

However, most of these techniques assume the distributions between labeled and unlabeled data are the same.

Transfer Learning, in constrast, allows the domains/tasks/distributions in training and testing data to be different.

------------------------------------------------

A closely-related technique to transfer learning is **multi-task learning**.

The difference lies in the asymmetric role of source and target tasks in transfer learning:

- Transfer Learning:
- Extract knowledge from one or more source tasks, and apply the knowledge to a target task.
- Multi-Task Learning:
- Learn multiple tasks simultenously

Transfer Learning cares most about the target task.

-------------------------------------------------

#### Notations and Definitions

-------------------------------------------------