# Introduction
這篇主要接續上一篇所說的
接著介紹 conditional 以及 MGF 的部分

# Conditional Expectation
## 基礎公式
**conditional expectation** :
$E(Y|X=x) = \begin{cases} \Sigma \ y \cdot f_{Y|X}(y|x) \longrightarrow \text{discrete} \\ \\ \int \color{red}{y} \cdot f_{Y|X}(y|x) \ \color{red}{dy} \longrightarrow \text{continuous} \end{cases}$
用定義來看就是代表在給定 $X=x$ 之下
小補充 : $Y$ 的 mean 會隨著 $x$ 的值而改變
並且如果做成圖形，可以發現 $E(Y|X= x)$ 是決定不同分佈之間分得多開

**conditional variance** :
$V(Y|X=x) = \begin{cases} \Sigma \ (y-\mu(x))^2 \cdot f_{Y|X}(y|x) \longrightarrow \text{discrete} \\ \\ \int \color{red}{(y-\mu(x))^2} \cdot f_{Y|X}(y|x) \ \color{red}{dy} \longrightarrow \text{continuous} \end{cases},\ \text{for } \mu(x) = E(Y|X = x)$
小補充 : $Y$ 的 var **不**會隨著 $x$ 的值而改變
並且如果做成圖形，可以發現 $V(Y|X= x)$ 是決定同分佈之中有多聚集

補充 : (用回歸分析的角度來看)
:::spoiler
假設 : $\begin{cases} Y_i = \beta_0 + \beta_1 + \epsilon_i \\ \epsilon_i \overset{iid}{\sim} N(0,\ \sigma^2) \end{cases}$
$\Rightarrow Y_t = (Y_i|X_i = x_i) \overset{indep}{\sim} N(\beta_0 + \beta_1x_i,\ \sigma^2)$
$\Rightarrow E(Y_i | X_i = x_i) = \beta_0 + \beta_1x_i \Longrightarrow Y$ 的 mean 會受到 $x$ 值所改變
$\Rightarrow V(Y_i | X_i = x_i) = \sigma^2 \Longrightarrow Y$ 的 var 不會受到 $x$ 值所改變
:::

---

接著我們來看 $2$ 個變數 :
**conditional expectation**
if $r(x,\ y)$ is a function of $x,\ y$
$E(r(X,\ Y) | X=x) = \begin{cases} \Sigma \ r(x,\ y) f_{Y|X}(y|x) \longrightarrow \text{discrete} \\ \int \ r(x,\ y) f_{Y|X}(y|x) dx \longrightarrow \text{continuous} \end{cases}$

**variance** :
$V(Y) = E \left[ \ V(Y|X) \ \right] + V \left[ \ E(Y|X) \ \right]$
直觀一點的來看，把 $V$ 跟 $E$ 都加入下標 :
$V(Y) = E_{\color{red}{X}} \left[ \ \underbrace{V_{\color{red}{Y}}(Y|X)}_{= \ h(x) \ \Rightarrow \text{ conditional variance}} \ \right] + V_{\color{red}{X}} \left[ \ \underbrace{E_{\color{red}{Y}}(Y|X)}_{= \ g(x) \ \Rightarrow \text{ conditional mean}} \ \right]$

證明過程如下 :
:::spoiler
$V(Y) = E(Y - EY)^2 = \color{green}{E(Y^2)} - \color{orange}{(EY)^2}$
$= \color{green}{\underbrace{E \left[ \ E(Y^2|X) \ \right] - E \left[ \ E(Y|X) \ \right]^2}_{(1)}} + \color{orange}{\underbrace{E \left[ \ E(Y|X)^2 \ \right] - E \left[ \ E(Y|X) \ \right]^2}_{(2)}}$
$\color{green}{(1)} : \ = E \left[ \ \left[ \ \color{purple}{E}(\color{purple}{Y^2}|X) \ \right] - \left[ \ \color{purple}{E}(\color{purple}{Y}|X) \ \right]^{\color{purple}{2}} \ \right] = E \left[ \ V(Y|X=x) \right] = \color{green}{E \left[ \ V(Y|X) \right]}$
$\color{orange}{(2)，先令 E(Y|X)=g(x)} : \ = Eg(x)^2 - \left[ \ Eg(x) \ \right]^2 = V \left[ \ g(x) \right] = \color{orange}{V \left[ \ E(Y|X) \right]}$

$\therefore V(Y) =\color{green}{E \left[ \ V(Y|X) \right]} + \color{orange}{V \left[ \ E(Y|X) \right]}$
:::

---

然後我們來看最困難的部分 -- Iterated Expectation :
假設 $X,\ Y$ 都是 random variable，並且期望值都存在
$\Rightarrow \begin{cases} E\left[ \ E \left[ \ Y|X \ \right] \ \right] = E(Y) \\ \\ E\left[ \ E \left[ \ X|Y \ \right] \ \right] = E(X) \end{cases}$
更直觀一點的來看，我們可以把 $E$ 都加上下標就會知道要怎麼化簡 :
$E_{\color{red}{X}}\left[ \ \underbrace{E_{\color{red}{Y}} \left[ \ Y|X \ \right]}_{= \ r(x)} \ \right] = E(Y)$
廣義上來說，也可以轉換成這樣 :
$E\left[ \ E \left[ \ r(X,\ Y)|X \ \right] \ \right] = E(r(X,\ Y))$
$E_{\color{red}{X}} \left[ \ \underbrace{E_{\color{red}{Y}} \left[ \ r(X,\ Y)|X \ \right]}_{= \ r(x)} \ \right] = E(r(X,\ Y))$

證明過程如下 :
:::spoiler
首先從 $E_Y(Y|X=x)$ 開始證明 :
$E_Y(Y|X=x) = \int y \cdot f_{Y|X}(y|x)dy = g(x) \Rightarrow E_Y(Y|X) = g(x)$

接著就能得到 $E\left[ \ E \left[ \ Y|X \ \right] \ \right] = E(Y)$ :
$E\left[ \ E \left[ \ Y|X \ \right] \ \right] = E(Y) = E_XE_Y(Y|X)$
$= E\left[ \ g(x) \ \right] = \int g(x)f_X(x)dx$
$= \int\int y \cdot f_{Y|X}(y|x)f_X(x) \ dy \ dx = \int\int y \cdot f(x,\ y) \ dx \ dy = \int y \color{purple}{\underline{\int f(x\ y) \ dx}}\ dy = \int y \color{purple}{\underline{f_Y(y)}}\ dy = EY$
:::

## Examples
### **Question 1**
**question** : $X \sim UNIF(0,\ 1),\ Y|X =x \sim UNIF(x,\ 1)$, 求出 $E(Y|X=x)$

**solution** :
可以先從題目得知 $f_{Y|X}(y|x) = \cfrac{1}{1-x}$
然後有 2 種解題方式 :
* method 1 : 在數學上來看
$E(Y|X=x) = \int_x^1 \ yf_{Y|X}(y|x) dy = \int^1_x \ y\cfrac{1}{1-x}dy = \underbrace{\cfrac{1+x}{2}}_{\text{固定會和 } x \text{有關}}$

* method 2 : 從直觀上來看
$E(Y|X=x) = \cfrac{1+x}{2}$

### **Question 2**
**question** : $X \sim UNIF(0,\ 1),\ Y|X =x \sim UNIF(x,\ 1)$, 求出 $EY$

**solution** :
可以先從上一題得知 $E(Y|X=x) = \cfrac{1+x}{2}$
然後有 2 種解題方式 :
* method 1 : 列出所有可能性
$Y$ 的所有可能值 : $\cfrac{1}{2}(1+x),\ x \in [0,\ 1]$
$可能性 : f_X(x) = 1$
$\Rightarrow EY = \int^1_0 \ \cfrac{1}{2}(1+x)f_X(x)dx = \cfrac{3}{4}$

* method 2 : 直接用公式
$EY = E_{\color{red}{X}}E_{\color{red}{Y}}(Y|X) = E_X \left[ \ \cfrac{1}{2}(1+x) \ \right] = \cfrac{1}{2} + \cfrac{1}{2} EX = \cfrac{3}{4}$

### **Question 3**
**question** : $Q \sim UNIF(0,\ 1),\ X|Q =q \sim BIN(n,\ q)$, 求出 $EX,\ VX$

**solution** :
先解析題目 :
$Q \sim UNIF(0,\ 1) \Rightarrow \begin{cases} EQ = \cfrac{1}{2} \\ VQ = \cfrac{1}{12} \end{cases},\ \text{by formula in chapter 3.4}$
$X|Q =q \sim BIN(n,\ q) \Rightarrow \begin{cases} E(X|Q =q) = nq \\ V(X|Q =q) = nq(1-q) \end{cases},\ \text{by formula in chapter 3.4}$
接著就可以計算了 :
$\underbrace{EX = E_{\color{purple}{Q}} \left[ \ E_{\color{purple}{X}}(X|Q) \ \right]}_{\color{red}{\text{important}}} = E_Q(nQ) = nEQ = \cfrac{2}{n}$
$\underbrace{VX = V_{\color{purple}{Q}} \left[ \ E_{\color{purple}{X}}(X|Q) \ \right] + E_{\color{purple}{Q}} \left[ \ V_{\color{purple}{X}}(X|Q) \ \right]}_{\color{red}{\text{important}}} = V_Q(nQ) + E_Q(nQ \cdot (1-Q)) = \cfrac{n^2}{12} + n(EQ - EQ^2)$
$= \cfrac{n^2}{12} + \cfrac{n}{2} - n \cdot \underbrace{(\cfrac{1}{12} + \cfrac{1}{4})}_{\color{red}{EQ^2 \ = \ VQ - (EQ)^2}}$

# Moment Generating Function(MGF)
又被稱作 MGF 或是 Laplace transform

## 基礎公式
$\psi_X(t) = \mathbb{E}(e^{tX}) = \int e^{tx} \ dF(x)$
並且 $\begin{cases} EX = \psi_X^{\prime}(t) \\ \\ EX^2 = \psi_X^{\prime \prime}(t) = VX + (EX)^2 \end{cases}$

證明過程如下 :
:::spoiler
* $\psi_X^{\prime}(t) = \cfrac{d}{dt} \int e^{tx}f_X(x) \ dx = \int \color{red}{x} \cdot e^{tx} f_X(x) \ dx \overset{t=0}{\Longrightarrow} \psi_X^{\prime}(0) = \int x f_X(x)dx = EX$

* $\psi_X^{\prime \prime}(t) = \cfrac{d}{dt} \psi_X^{\prime}(t) = \cfrac{d}{dt} \int xe^{tx}f_X(x) \ dx = \int \color{red}{x^2} \cdot e^{tx} f_X(x) \ dx \overset{t=0}{\Longrightarrow} \psi_X^{\prime \prime}(0) = \int x^2 f_X(x)dx = EX^2$
:::

----

接下來這邊介紹 $MGF$ 的一些重要特性
* if $Y = aX + b \Rightarrow \psi_Y(t) = e^{bt} \cdot \psi_X(\alpha t)$
* if $X_1,\ ... ,\ X_n$ are $\color{red}{\text{independent}}$, and $Y = \Sigma_i X_i \Rightarrow \psi_Y(t) = \Pi_i \ \psi_i(t),\ \phi_i = MGF \text{ of } X_i$
* $X,\ Y$ are random variables, if $\psi_X(t) = \psi_Y(t)$ for all $t \Rightarrow X \overset{d}{=} Y$

## Important MGF of Commom Distribution
| Distribution | MGF ($\psi(t)$) |
|:------------------------ |:---------------------------------------------------------- |
| Bernoulli ($p$) | $pe^t + (1-p)$ |
| Binomial ($n,\ p$) | $(pe^t + (1-p))^n$ |
| Poisson ($\lambda$) | $e^{\lambda (e^t-1)}$ |
| Normal ($\mu,\ \sigma$) | $e^{\mu t + \frac{\sigma^2 t^2}{2}}$ |
| Gamma ($\alpha,\ \beta$) | $(\cfrac{1}{1 - \beta t})^{\alpha},\ t < \cfrac{1}{\beta}$ |

## Examples
### **Question 1**
**question** : $X \sim Exp(\theta),\ f(x) = \cfrac{1}{\theta}e^{-\frac{x}{\theta}},\ x>0$, 求出 $EX,\ VX$

**solution** :
$\begin{split}
\psi(t) &= Ee^{tx} = \int^{\infty}_0 e^{tx}f(x)dx = \int^{\infty}_0 \cfrac{1}{\theta}e^{x(t-\frac{1}{\theta})}dx \\
&= \left[ \ \cfrac{1}{\theta} \cdot \cfrac{\theta}{t \theta - 1} \cdot e^{x(t-\frac{1}{\theta})} \ \right]^{\infty}_{0} = \cfrac{1}{1-\theta t} = \left[ \ 1 - \theta t \ \right]^{-1},\ t \approx 0,\ t - \cfrac{1}{\theta} < 0
\end{split}$
$\Rightarrow EX = \left[ \ \psi^{\prime}(t) \ \right]_{t=0} = \left[ \ -(1-\theta t)^{-2} \cdot (-\theta) \ \right]_{t=0} = \theta$
$\Rightarrow EX^2 = \left[ \ \psi^{\prime \prime}(t) \ \right]_{t=0} = \left[ \ 2(1-\theta t)^{-3} \cdot \theta^2 \ \right]_{t=0} = 2 \theta^2$
$\Rightarrow VX = EX^2 - (EX)^2 = 2 \theta^2 - \theta^2 = \theta^2$

### **Question 2**
**question** : $X_i \overset{iid}{\sim} Bernoulli(p),\ f(x) = \begin{cases} 1,\ \text{with probability } = p \\ 0,\ \text{with probability } = 1-p \end{cases},\ Y = \Sigma_{i=1}^{n}x_i \sim ?$ 求出 $Y$ 的分佈

**solution** :
$\psi_{x_i}(t) = Ee^{tx_i} = e^{t \cdot 0} \cdot f(0) + e^{t \cdot 1} \cdot f(1) = 1-p + pe^t$
$\psi_Y(t) = \Pi_{i=1}^{n}\psi_{x_i}(t) = (1-p + pe^t)^n \longleftarrow$ 確實是 $Bin(n,\ p)$ 的 $MGF$

### **Question 3**
**question** : $\begin{cases} X_1 \sim Bin(n_1,\ p) \\ X_2 \sim Bin(n_2,\ p) \\ X_1 \perp X_2 \end{cases} \Rightarrow 求 \ X_1 + X_2 \ 的分佈$

**solution** :
在開始計算之前，我們可以知道 :
$P(X_1 + X_2 = t) = \Sigma_{(X_1 + X_2 = t)}P(X_1 = x_1,\ X_2 = x_2)$
$\psi_{X_1 + X_2}(t) = Ee^t = \ ... \ = \psi_{X_1}(t) \cdot \psi_{X_2}(t) = (1-p + e^t p)^{n_1} \cdot (1-p + e^t p)^{n_2} = (1-p + e^t p)^{n_1+n_2}$
$\Rightarrow X_1 + X_2 \sim Bin(n_1 + n_2 ,\ p)$

### **Question 4**
**question** : $X_i \sim Poi(\lambda_i),\ i = 1,\ 2,\ ...,\ n,\ X_i \text{ is independent},\ 求 \Sigma_{i=1}^{n} X_i \sim ?$

**solution** :
在開始計算之前，我們可以知道 :
$P(X_1 + X_2 + ... + X_n = t) = \Sigma_{(X_1 + X_2 + ... + X_n = t)}P(X_1 = x_1,\ X_2 = x_2,\ ...,\ X_n = x_n)$
$\psi_{X_1 + ... + x_n}(t) = \Pi_{i=1}^{n} \psi_{X_i}(t) = \Pi_{i=1}^{n} \ \underbrace{\color{green}{e^{\lambda_i(e^t - 1)}}}_{Poi \text{ 的 mgf}} = e^{\color{green}{(\Sigma_{i=1}^n \lambda_i)}(e^t -1)}$
$\Rightarrow X_1 + X_2 + ... + X_n = \Sigma_{i=1}^{n} X_i \sim Poi(\Sigma_{i=1}^n \lambda_i)$