# Regression

###### tags: `MachineLearning`

> Written by @Chihung0522
> [Reference : Regression 李宏毅教授](https://www.youtube.com/watch?v=fegAeph9UaA&list=PLJV_el3uVTsPy9oCRY30oBPNLCo89yu49&index=3)
> > [time=Sun, Mar 29, 2020 10:22 PM]

## Model
>To define a function that could describe some situation
- Stock market forecast case: Input the history price data or flactuation and output the Dow Jones industrial tomorrow.
- Self-driving Car : Input the enviroment senesor data and get the angle of steering.
- Recommendation : Input the customer data and goods data and find the probability of buying.

$$
f(Input) = Output
$$

### Liner model

$$
y = b + \sum w_ix_i
$$

## Loss function

> The goodness of function
>
Estimated Error :
$$
L(f)
= L(w,b) \\
= \sum (y^n-(b+w \cdot x^n))^2
$$
## Gradient Descent

> To consider loss function L(w) with w which make the loss smallest.
- [ ] Pick an initial value randomly.
- [ ] Compute the slope of that point.

$$
\frac{dL}{dw}|w=w^0
$$
- [ ] Update the w with different learning rate.
$$
w^1 = w^0 -\eta \frac{dL}{dw}
$$

- [ ] After many iteration until the slope equal to 0 and that is local optimal solution.


:high_brightness: linear regression no local optimal cause the loss funciotn of MSE is **convex**

## Generalization
> Train a more complicated mode to fit more into the training data and get a better loss. While we input a testing data, but obtain a bigger loss.
> > A more complex model does not always lead to better performance on **testing data**
> > > ***This is Overfitting***

### hidden factors

- Plot all point together and found a relation in different cluster.
- The hidden factor is species.

> Redesign the model considering about the species


## Regularization
> Add a regularization to make the loss function more smoothy against the noise of input value.
> No need to take bias into Regularization.

$$
L(w,b) + \lambda \sum (w_i)^2
$$