{%hackmd 8nGPjMOiTy-0DWU2xy0q0Q %}

# Multiple Linear Regression

[Home Page](/_9v1g3C3TXmUfBbxkjnn0A)
[toc]

## Formula

| Name | Notation | Definition |
|:------------------------------------- |:--------- |:-------------------------------------------------------------------------------------- |
| Sum of square total | $SSTO$ | $\sum_{i = 1}^{n} (y_i - \ol y)^2$ |
| Sum of square errors | $SSE$ | $\sum_{i = 1}^{n} (y_i - \hat y_i)^2$ |
| Sum of square regression | $SSR$ | $\sum_{i = 1}^{n} (\hat y_i - \ol y)^2$ |
| Mean of squared error | $MSE$ | $\dps \frac{SSE}{n − p}$ |
| Mean of squared regression | $MSR$ | $\dps \frac{SSE}{p − 1}$ |
| Coefficient of Determination | $R^2$ | $\dps 1 - \frac{SSE}{SSTO} = \frac{SSR}{SSTO}$ |
| Adjusted Coefficient of Determination | $R_a^2$ | $\dps 1 - \frac{SSE / (n - p)}{SSTO / (n - 1)} = 1 - \frac{(n - 1) SSR}{(n - p) SSTO}$ |
| F-Statistic | $F^*$ | $\dps \frac{MSR}{MSE}$ |
| P-value | $P$-value | $P (F^* < F_{1, n - 2})$ |

## Multiple Linear Regression

Given data $(X_{i1}, X_{i2}, \cdots, X_{ik}, Y_i)_{i = 1}^{n}$, let $p = k + 1$

$$
\ut Y
= \begin{pmatrix}
Y_1 \\
Y_2 \\
\vdots \\
Y_n
\end{pmatrix}
$$

and

$$
\bs X
= \begin{pmatrix}
1 & X_{11} & X_{12} & \cdots & X_{1k} \\
1 & X_{21} & X_{22} & \cdots & X_{2k} \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
1 & X_{n1} & X_{n2} & \cdots & X_{nk} \\
\end{pmatrix}.
$$

$$
\ut \beta
= (\bs X^T \bs X)^{-1} \bs X^T \ut Y.
$$