---
# System prepended metadata

title: Multiple Linear Regression
tags: [Applied Linear Regression Models]

---

{%hackmd 8nGPjMOiTy-0DWU2xy0q0Q %}

# Multiple Linear Regression

[Home Page](/_9v1g3C3TXmUfBbxkjnn0A)
[toc]

## Formula

| Name                                  | Notation  | Definition                                                                             |
|:------------------------------------- |:--------- |:-------------------------------------------------------------------------------------- |
| Sum of square total                   | $SSTO$    | $\sum_{i = 1}^{n} (y_i - \ol y)^2$                                                     |
| Sum of square errors                  | $SSE$     | $\sum_{i = 1}^{n} (y_i - \hat y_i)^2$                                                  |
| Sum of square regression              | $SSR$     | $\sum_{i = 1}^{n} (\hat y_i - \ol y)^2$                                                |
| Mean of squared error                 | $MSE$     | $\dps \frac{SSE}{n − p}$                                                               |
| Mean of squared regression            | $MSR$     | $\dps \frac{SSE}{p − 1}$                                                               |
| Coefficient of Determination          | $R^2$     | $\dps 1 - \frac{SSE}{SSTO} = \frac{SSR}{SSTO}$                                         |
| Adjusted Coefficient of Determination | $R_a^2$   | $\dps 1 - \frac{SSE / (n - p)}{SSTO / (n - 1)} = 1 - \frac{(n - 1) SSR}{(n - p) SSTO}$ |
| F-Statistic                           | $F^*$     | $\dps \frac{MSR}{MSE}$                                                                 |
| P-value                               | $P$-value | $P (F^* < F_{1, n - 2})$                                                               |

## Multiple Linear Regression

Given data $(X_{i1}, X_{i2}, \cdots, X_{ik}, Y_i)_{i = 1}^{n}$, let $p = k + 1$ 

$$
\ut Y
= \begin{pmatrix}
    Y_1 \\
    Y_2 \\
    \vdots \\
    Y_n
\end{pmatrix}
$$

and 

$$
\bs X
= \begin{pmatrix}
    1 & X_{11} & X_{12} & \cdots & X_{1k} \\
    1 & X_{21} & X_{22} & \cdots & X_{2k} \\
    \vdots & \vdots & \vdots & \ddots & \vdots \\
    1 & X_{n1} & X_{n2} & \cdots & X_{nk} \\
\end{pmatrix}.
$$

$$
\ut  \beta
= (\bs X^T \bs X)^{-1} \bs X^T \ut Y. 
$$