# Introduction
這篇主要在講我們的期望值

# Expectation of a Random Variable
這些名詞所代表的東西是一樣的，都是表示期望值 :
* expected value
* mean
* first moment

直觀的來看，就是代表這個函式的中心

如果用數學表示 :
$\mu_x = \mathbb{E}(X) = \int x \ dF(x) \ \begin{cases} \Sigma_x \ xf(X),\ \text{X is discrete} \\ \\ \int \ xf(x) \ dx,\ \text{X is continuous} \end{cases}$

## 常見 distribution 的 $EX$
接下來就用常見的 distribution 當作例子 :
* $X \sim Bernoulli(p) \Rightarrow x \in \left\{ 0, 1 \right\}$
$EX = 0 \times (1-p) + 1 \times p = p$

* $X \sim BIN(n, p) \Rightarrow x \in \left\{ 0, 1, ......, n \right\}$
$EX = \Sigma_{x= 0}^{n}\ xC^{n}_{x}p^x(1-p)^{n-x} = np$

* $X \sim UNIF(p)$
$EX = \int_a^b xf(x)dx = \cfrac{a+b}{2}$

* $X \sim POI(\lambda)$
$EX =...... \ \text{by def} = \lambda$

* $X \sim EXP(\beta)$
$EX =...... \ \text{by def} = \beta$

* $X \sim Gamma(\alpha, \beta)$
$EX =...... \ \text{by def} = \alpha\beta$

## 基礎公式
先令 $Y = r(X),\ Z = r(X,\ Y)$
* $EY = E(r(x)) = \int r(x) dF_X(x)$
* $EZ = E(r(X,\ Y)) = \int \int r(x,\ y) f(x,\ y) \ dxdy$

證明過程如下 :
:::spoiler
$EY$ :
我們在已知 $X$ 的分布的情況下要求出 $EY$
$\text{Let } Y = r(X)$
$\Rightarrow \underbrace{E(Y)}_{=\int y dF_Y(y) \Rightarrow 找\ y \ 的分佈} = E(r(X)) = \int r(x) d\underbrace{F_X(x)}_{直接使用\ x \ 的分佈}$
:::

## Examples
### **Question 1**
**question** : $X \sim UNIF(0,\ 1),\ Y =r(X) = e^X$, 求出 $EY$

**solution** :
首先把題目做解析
從 $X$ 可以看出 : $f_X = \begin{cases} 1,\ 0 \le x \le 1 \\ \\ 0 ,\ \text{otherwise} \end{cases}$
從 $Y$ 可以看出 : $x = r^{-1}(y) = ln(y)$

然後有 2 種解題方式 :
* method 1 : 找出 $y$ 的分佈
$f_Y(y) = f_X(ln(y))\ \cfrac{d \ ln(y)}{dy} = 1 \times \cfrac{1}{y} = \cfrac{1}{y},\ 1 \le y \le e$
$EY = \int_{1}^{e} y f_Y(y)dy = \int_1^e y \cdot \cfrac{1}{y} dy = \left[ \ y \ \right]^{e}_{1} = e - 1$

* method 2 : lazy, by fomula
$\underbrace{EY}_{對\ y\ 的分佈做} = \underbrace{Er(X)}_{對\ x\ 的分佈做} = \int_{\color{red}{0}}^{\color{red}{1}} r(x)f_{\color{red}{x}} \color{red}{(x)dx} = \int^1_0 e^x \times 1 = \left[ \ e^x \ \right]^{1}_{0} = e - 1$

### **Question 2**
**question** : $X \sim UNIF(0,\ 1),\ Y = max(x,\ 1-x)$, 求出 $EY$

**solution** :
有 2 種解題方式 :
* method 1 : 找出 $y$ 的分佈
$F_Y(y) = P(Y \le y) = P(max(x,\ 1-x) \le y) = P(x \le y \ \And\ (1-x) \le y) = P((1-y) \le x \le y)$
$= \int^y_{1-y}f_X(x)dx = \left[ \ x \ \right]^{y}_{1-y} = 2y-1,\ \cfrac{1}{2} \le y \le 1$
因為上面做出來的是 $cdf$，我們要得知 $EY$ 是要用 $pdf$，因此要進行微分
$f_Y(y) = \cfrac{d}{dy}F_Y(y) = y \Rightarrow EY = \int^1_{\frac{1}{2}} y \cdot f_Y(y)dy = \int^1_{\frac{1}{2}}2y \ dy = \cfrac{3}{4}$

* method 2 : lazy, by fomula
從題目可以先得知可能會需要拆開來看，因此我們先做表格確認一下
| $x$ | $y = r(x)$ |
|:-------------- |:------------------------------------------- |
| $\cfrac{1}{3}$ | $\cfrac{2}{3}\Rightarrow\color{green}{1-x}$ |
| $\cfrac{2}{3}$ | $\cfrac{2}{3}\Rightarrow\color{green}{x}$ |

從表格可以看出我們會需要拆成 $2$ 段來計算
$\begin{cases} 0 < x < \cfrac{1}{2} : 1-x \\ \cfrac{1}{2} < x < 1 : x \end{cases}$

$EY = Er(X) = \int^{\color{purple}{1}}_{\color{purple}{0}} \color{purple}{r(x)} f_X(x)dx$
$= \int^{\frac{1}{2}}_{0} (1-x)f_X(x)dx + \int^{1}_{\frac{1}{2}} (x)f_X(x)dx$
$= \int^{\frac{1}{2}}_{0} (1-x)dx + \int^{1}_{\frac{1}{2}} (x)dx = \cfrac{3}{4}$

### **Question 3**
**question** : $(X,\ Y) \sim \underbrace{\text{uniform distribution}}_{\color{purple}{f(x,\ y) = 1}} \underbrace{\text{ on the unit square}}_{\color{blue}{[0,\ 1] \times [0,\ 1]}},\ Z = r(x,\ y) = X^2 + Y^2,$ 求 $EZ$

**solution** :
有 2 種解題方式 :
* method 1 : 找出 $z$ 的分佈
$...同上,\ 省略...$

* method 2 : lazy, by fomula
$EZ = Er(X,\ Y) = \int^1_0 \int^1_0 (x^2 + y^2) f(x,\ y) \ dxdy$
$= \int^1_0 \int^1_0 (x^2 + y^2) \ dxdy = \cfrac{2}{3}$

# Properties of Expectation
## 基礎公式
$X_1,\ ...,\ X_n$ 是 random variables，且 $a_1,\ ...,\ a_n$ 是常數
這時 $E(\Sigma_i a_i X_i) = \Sigma_i a_i E(X_i)$

證明過程如下 :
:::spoiler
$E(a_1 x_1 + a_2 x_2) = \int \int (a_1 x_1 + a_2 x_2) f(x_1,\ x_2) dx_1 dx_2$
$= \int \int \color{orange}{(a_1 x_1)} \cdot f(x_1,\ x_2) \ dx_2 \color{orange}{dx_1} + \int \int \color{orange}{(a_2 x_2)} \cdot f(x_1,\ x_2) \ dx_1 \color{orange}{dx_2}$
$= \int (a_1 x_1) \cdot \int f(x_1,\ x_2) \ dx_2 \ dx_1 + \int (a_2 x_2) \cdot \int f(x_1,\ x_2) \ dx_1 \ dx_2$
$= \int a_1 \left[ \ \underline{x_1 \cdot f_1(x_1) \ dx_1} \right]_{=EX_1} + \int a_2 \left[ \ \underline{x_2 \cdot f_2(x_2) \ dx_2} \right]_{=EX_2}$
$= a_1 EX_1 + a_2 EX_2$
:::

---

$X_1,\ ...,\ X_n$ 是 independent random variables
這時 $E(\Pi_i^n X_i) = \Pi_i^n E(X_i)$

證明過程如下 :
:::spoiler
$E(x_1 x_2) = \int \int ( x_1 x_2) \ \underbrace{f(x_1,\ x_2)}_{\color{red}{\text{independent}}} \ dx_1 dx_2$
$= \int \int x_1 \color{orange}{\underline{x_2}} f_1(x_1) \color{orange}{\underline{f_2(x_2)}} \ dx_1 dx_2 \stackrel{\text{把橘色部分丟到外面}}{\Longrightarrow}$
$= \int x_2 f_2(x_2) \cdot \left[ \ \underline{\int x_1 f_1(x_1) \ dx_1} \right]_{=EX_1} dx_2$
$= \int x_2 f_2(x_2) \cdot EX_1 \ dx_2 = EX_1 \left[ \ \underline{\int x_2 \cdot f_2(x_2) \ dx_2} \right]_{=EX_2}$
$= EX_1 \cdot EX_2$
:::

# Variances and Covariance
## 基礎公式
先令 $X$ 是一個 random variable，並且它的 mean 跟 variance 分別為 $\mu,\ \sigma^2(也可用 \sigma^2_X,\ \mathbb{V}(X),\ \mathbb{V}X 代稱)$
並且 $\sigma^2$ 的定義為 :
$\sigma^2 = \mathbb{E}(X - \mu)^{\color{red}{2}} = \int (x-\mu)^2 dF(x)$

另外 variance 會遵循以下特性 :
1. $\mathbb{V}(X) = \mathbb{E}(X^2) - \mu^2$
2. $\mathbb{V}(aX - b) = a^2\mathbb{V}(X),\ \text{for } a,b \in \text{constant}$
3. $\mathbb{V}(\Sigma_{i=1}^n \ a_i X_i) = \Sigma_{i=1}^n \ \color{red}{a_i^2} \mathbb{V}(X_i),\ \text{for } X_1,\ ...,\ X_n \text{ are independent, and }a \in \text{constant}$

證明過程如下 :
:::spoiler
1. $VX = E(x - \mu)^2 = E(x^2 -2x\mu + \mu^2) = ...... = EX^2-\mu^2$
2. $V(ax+b) = E \left[ \ (ax+b)-E(ax+b) \ \right]^2 = ...... = a^2VX \longrightarrow$ 數學上
$V(ax+b) = V(aX) = a^2VX \longrightarrow$ 直觀上
3. $V(a_1x_1 + a_2x_2) = E \left[ \ a_1x_1 + a_2x_2 - E(a_1x_1 + a_2x_2) \ \right]^2 = ...(\text{related to } EY_1Y_2=EY_1EY_2)...$
:::

---

上述說的都是母體的情況，接下來這部分要講 sample mean & sample variance
先令 $X_i,\ ...,\ X_n$ 都是相同且相互獨立的分佈
並且令 $\mu = \mathbb{E}(X_i),\ \sigma^2 = \mathbb{V}(X_i)$
這時 :
$\mathbb{E}(\overline{X_n}) = \mu,\ \mathbb{V}(X) = \cfrac{\sigma^2}{n},\ \mathbb{E}(S_n^2) = \sigma^2$

證明過程如下 :
:::spoiler
$E \overline{X_n} = E(\cfrac{1}{n}\Sigma_{i=1}^{n} \ X_i) = ...... = \mu$
$V \overline{X_n} = E\left[ \ (\cfrac{1}{n}\Sigma_{i=1}^{n} \ X_i)-E(\overline{X_n}) \ \right]^2 = \cfrac{\sigma^2}{n}$
**important :**
$ES_n^2 = E\left[ \ \cfrac{1}{n-1} \Sigma(X_i - \overline{X_n})^2 \ \right] =E\left[ \ \cfrac{1}{n-1} \Sigma(X_i^2 - 2X_i \overline{X_n} + \overline{X_n}^2) \ \right] = ...... = \sigma^2$
:::

---

接著我們來看 covariance & correlation
先令 $X_i,\ ...,\ X_n$ 和 $Y_i,\ ...,\ Y_n$ 都是 random variables，並且 means 跟 standard deviation 分別為 $\mu_X,\ \sigma_X,\ \mu_Y,\ \sigma_Y$
這時 :
* **covariance :**
$Cov(X,\ Y) = \mathbb{E}((X - \mu_X)(Y - \mu_Y)) = \underbrace{E(XY) - E(X)E(Y)}_{= E \left[ \ (X - EX) (Y - EY) \ \right]}$
小補充 : $Cov$ 的大小代表線性關係的強度，$Cov$ 的正負代表線性關係的方向

* **correlation :**
$\rho = \rho_{X,\ Y} = \rho(X,\ Y) = \cfrac{Cov(X,\ Y)}{\color{red}{\rho_X \ \rho_Y}},\ -1 \le \rho(X,\ Y) \le 1$
假設 $Y = aX + b,\ a,\ b \in \text{constant}$
這時 : $\begin{cases} \rho (X,\ Y) = 1,\ a > 0 \\ \rho (X,\ Y) = -1,\ a < 0 \end{cases}$ (下面有附上證明)
另外，如果 $X$ 跟 $Y$ 是獨立的(independent) $\Rightarrow Cov(X,\ Y) = \rho = 0$
證明過程如下 :
:::spoiler
已知 $Y = aX +b ,\ VY = a^2VX$
$\Rightarrow cov(X,\ Y) = cov(X,\ aX +b) = E\left[ \ X(aX+b) \ \right] - (EX)(aEX + b)$
$= E(aX^2+bX) - (EX)(aEX + b) = ...... = \underbrace{a\sigma^2}_{corr(X,\ Y) = 1 \text{ or } -1 \text{ depending on }a} = \underbrace{(a\sigma)}_{sd(Y)} \underbrace{\sigma}_{sd(X)}$
:::

----

接著我們來看 $2$ 個變數的 variance
$\begin{cases}
V(X+Y) = V(X) + V(Y) + 2 \cdot Cov(X,\ Y) \\
V(X-Y) = V(X) + V(Y) - 2 \cdot Cov(X,\ Y)
\end{cases}$
廣義上來說，也可以轉換成這樣 :
$V(\Sigma_i \ a_i X_i) = \Sigma_i \ a_i^2 V(X_i) + 2 \Sigma\Sigma_{i<j} \ a_i a_j \cdot Cov(X_i,\ X_j)$

## Examples
### **Question 1**
**question** : $X \sim BIN(n,\ p),\ 求出\ EX,\ VX$

**solution** :
然後有 2 種解題方式 :
* method 1 : by basic def
$EX = \Sigma_X \ xf(x)$
$VX = \Sigma_X \ (x-\mu)^2 = EX^2 - \mu$

* method 2 : by scratch
$X = \Sigma^n_{i=1} \ Z_i,\ Z_i \overset{iid}{\sim} Bernoulli(p) \Rightarrow EZ_i = p,\ VZ_i = p(1-p)$
$EX = E\ \Sigma^n_{i=1} \ Z_i = \Sigma^n_{i=1} \ EZ_i = np \longleftarrow \color{purple}{不需要 \ Z_i \ 的條件}$
$VX = V\ \Sigma^n_{i=1} \ Z_i = \Sigma^n_{i=1} \ VZ_i = np(1-p) \longleftarrow \color{purple}{需要 \ Z_i \ 的條件}$

### **Question 2**
**question** : $X \sim Gamma(\alpha,\ \beta),\ 求出\ EX,\ VX$

**solution** :
然後有 2 種解題方式 :
* method 1 : by basic def
$......$

* method 2 : by scratch
$X = \Sigma^n_{i=1} \ Z_i,\ Z_i \overset{iid}{\sim} exp(\beta) \Rightarrow EZ_i = \beta,\ VZ_i = \beta^2$
$EX = E\ \Sigma^{\alpha}_{i=1} \ Z_i = \Sigma^{\alpha}_{i=1} \ EZ_i = \alpha \beta$
$VX = V\ \Sigma^{\alpha}_{i=1} \ Z_i = \Sigma^{\alpha}_{i=1} \ VZ_i = \alpha \beta^2$

# Expectation and Variances of Important Random Variables
**粗體的部分是一定要記住的!!!**
| Distribution | Mean | Variance |
|:------------------------------------ |:-------------------------- |:--------------------------------------------------- |
| **Point mass at $a$** | $a$ | $0$ |
| **Bernoulli $(p)$** | $p$ | $p(1-p)$ |
| **Binomial $(n,\ p)$** | $np$ | $np(1-p)$ |
| Geometric $(p)$ | $1/p$ | $(1-p)/p^2$ |
| **Poisson $(\lambda)$** | $\lambda$ | $\lambda$ |
| **Uniform $(a,\ b)$** | $(a+b)/2$ | $(b-a)^2/12$ |
| **Normal $(\mu,\ \sigma^2)$** | $\mu$ | $\sigma^2$ |
| **Exponential $(\beta)$** | $\beta$ | $\beta^2$ |
| **Gamma $(\alpha,\ \beta)$** | $\alpha \beta$ | $\alpha \beta^2$ |
| Beta $(\alpha,\ \beta)$ | $\alpha/ (\alpha + \beta)$ | $\alpha \beta / ((\alpha+\beta)^2(\alpha+\beta+1))$ |
| $t_{\nu}$ | $0,\ \text{if } \nu>1$ | $\nu /(\nu-2),\ \text{if }\nu>2$ |
| $\chi^2_{p}$ | $p$ | $2p$ |
| Multinomial $(n,\ p)$ | $np$ | $\text{see below}$ |
| Multivariate Normal $(\mu,\ \Sigma)$ | $\mu$ | $\Sigma$ |