---
# System prepended metadata

title: '[Paper筆記] : A Survey on Transfer Learning (2010)'

---

# [Paper筆記] : A Survey on Transfer Learning (2010)

### [Paper筆記] : A Survey on Transfer Learning (2010)


### **<Introduction>**

--------------------------------------

Common machine learning algorithms work under an assumption:

> **Training and Testing data are drawn from the same feature space and have the same distribution**

Therefore, when the distribution changes, models need to be rebuild from scratch using new data. **There's no knowledge transfer between tasks.**.

In reality, there are many scenarios where such assumption can be problematic when modeling.

> * **Limited data for our target task** 
>     * There may exist similar tasks with abundant dataset and more sophisticated models. 
>     * However, by traditional methodology, we can't harvest the knowledge from these similar tasks.
> * **The data may be easily outdated due to dynamic environments**
>     * Models need to adapt quickly, while based on existing knowledge.
>     * (ex) Wifi data to infer location. 

---------------------------------------

Focus of this paper:

* Definition of terms and topics in transfer learning
* Survey transfer learning techniques for classification, regression, and clustering.

--------------------------------------------

### **<Overview>**

-------------------------------------------------

#### Brief History of Transfer Learning

------------------------------------------------

Many techniques have been proposed to address the problem of not enough labeled data. 
For example, **semi-supervised learning** and **active learning** are two of them.

> - **Semi-Superised Learning:** 
>     - Large amount of unlabeled data + Small amount of labeled data
> - **Active Learning:**
>     - Select the next candidate data to be labeled, for cost-effective concern.

However, most of these techniques assume the distributions between labeled and unlabeled data are the same.

Transfer Learning, in constrast, allows the domains/tasks/distributions in training and testing data to be different.

------------------------------------------------

A closely-related technique to transfer learning is **multi-task learning**.

The difference lies in the asymmetric role of source and target tasks in transfer learning:

- Transfer Learning: 
    - Extract knowledge from one or more source tasks, and apply the knowledge to a target task.
- Multi-Task Learning:
    - Learn multiple tasks simultenously

Transfer Learning cares most about the target task.

-------------------------------------------------

#### Notations and Definitions

-------------------------------------------------













