mathstat_ch2_random variables_2.5-2.8

# Introduction 這篇主要是接續前面章節並且把前面的 $1$ 個變數擴展到 $2$ 個變數 # Discrete Random Variables **joint pmf** ($一起看$) 給定一對 discrete random variables $X,\ Y$ 我們可以定義他們的 **joint pmf** $f(x,\ y) = P(X = x,\ Y = y)$ 因此也可以得到 **joint cdf** $F(x,\ y) = P(X \le x,\ Y \le y)$ 舉例來說 : $\begin{cases} X = \{0, 1 \} \\ Y = \{0, 1 \} \end{cases} \Rightarrow \left( X,\ Y \right) = \begin{Bmatrix} (0, 0),\ (0,1), \\ (1,0),\ (1,1) \end{Bmatrix}$ 因此在計算 $\underbrace{P ((X,\ Y) = (0,0))}_{ =\ f(0,0)} = P(X = 0,\ Y = 0) = \text{joint pmf at (0, 0)}$ --- **marginal pmf** ($分別看$) 給定一對 discrete random variables $X,\ Y$ 我們可以定義他們的 **marginal pmf** $f_X(x) = P(X=x) = \Sigma_y\ P(X = x,\ Y = y) = \Sigma_y\ f(x, y)$ $f_Y(y) = P(Y=y) = \Sigma_x\ P(X = x,\ Y = y) = \Sigma_x\ f(x, y)$ 舉例來說 : $\begin{cases} X = \{0, 1 \} \\ Y = \{0, 1 \} \end{cases} \Rightarrow \left( X,\ Y \right) = \begin{Bmatrix} (0, 0),\ (0,1) \\ (1,0),\ (1,1) \end{Bmatrix}$ 因此在計算 $f_X(0) = P(X = 0) = P(X = 0,\ Y = 0) + P(X = 0,\ Y = 1) = \Sigma_y\ \underbrace{P(X = 0,\ Y = y)}_{f(0,\ y)}$ --- **conditional pmf** ($\text{給定 y or x 值看 x or y}$) 給定一對 discrete random variables $X,\ Y$ 並且假設 $f_Y(y) > 0$ 我們可以定義他們的 **conditional pmf** $f_{X|Y}(x|y) = P(X = x| Y =y) = \cfrac{P(X = x,\ Y = y)}{P(Y=y)} = \cfrac{f(x,\ y)}{f_Y(y)}$ 舉例來說，我們想看 $y=1$ 時，看 $x$ 的值 $i. \text{所有可能看到的 } x: \{ 0,1 \}$ $ii. \text{每種值有多少機率看到} :$ $\Rightarrow x = 0, f_{X|Y}(0|1) = P(X = 0 | Y = 0) = \cfrac{P(X = 0,\ Y = 1)}{P(Y = 1)} = \cfrac{f(0,1)}{f_Y(1)}$ $\Rightarrow x = 1, f_{X|Y}(1|1) = \ ...... \ = \cfrac{f(1,1)}{f_Y(1)}$ --- ::: warning if $f(x)$ is a pmf : 1. $P(x_1,\ x_2) > 0$ 2. $\Sigma_{x_1,\ x_2}P(x_1,\ x_2) = 1$ ::: # Continuous Random Variables **joint pdf** ($一起看$) 需要符合以下條件 : * $f(x,\ y) \ge 0, \text{ for all } (x,\ y)$ * $\int_{-\infty}^{\infty}\int_{-\infty}^{\infty} \ f(x,\ y) \ dxdy = 1$ * $\text{for any set } A \subset \mathbb{R} \times \mathbb{R},\ P((X,\ Y) \in A) = \int \int_A f(x,\ y) \ dxdy$ 舉例來說 : $F(x,\ y) = P(X \le a,\ Y \le b) = \int_{-\infty}^{a}\int_{-\infty}^{b} \ f(x,\ y) \ dydx$ 證明如下 $\Rightarrow$ :::spoiler $f(x,\ y) \approx \cfrac{P(x \le X \le x + \Delta x,\ y \le Y \le y + \Delta y)}{\Delta x \ \Delta y}$ $\text{又 } P((X,Y) \in A) = \Sigma_x \Sigma_y f(x,\ y) \Delta x \Delta y \stackrel{\Delta x \rightarrow 0 \\ \Delta y \rightarrow 0 }{\approx} \int \int f(x,\ y) \ dxdy$ ::: --- **marginal pdf** ($分別看$) $f_X(x) = \underbrace{\int}_{\text{range of } y} f(x,\ y) dy$ $f_Y(x) = \underbrace{\int}_{\text{range of } x} f(x,\ y) dx$ 證明如下 $\Rightarrow$ :::spoiler $f_Y(y) \approx \cfrac{P(y \le Y \le y + \Delta y)}{\Delta y} \approx \Sigma_{k=0}^{n-1} \underbrace{\cfrac{P(y \le Y \le y + \Delta y,\ x_k \le X \le x_{k+1})}{\Delta y \ \Delta x}}_{= f(x_k,\ y)} \times \Delta x$ $= \Sigma_{k=0}^{n-1}\ f(x_k,\ y) \Delta x \stackrel{\Delta x \rightarrow 0}{\approx} \int_{-\infty}^{\infty} f(x,\ y)dx$ (最後的積分區間可以替換成在給定 $y$ 下, $x$ 可能出現的範圍) ::: **marginal cdf** ($分別看$) $F_X(x) = P(X \le x) = \int^x_{-\infty} f_X(t)dt = \int^x_{-\infty} \underbrace{\int^{\infty}_{-\infty}}_{\text{range of } y} f(t,\ y) \ dydt$ $F_Y(y) = P(Y \le y) = \int^y_{-\infty} f_Y(t)dt = \int^y_{-\infty} \underbrace{\int^{\infty}_{-\infty}}_{\text{range of } x} f(x,\ t) \ dxdt$ --- **conditional pdf** ($\text{給定 y or x 值看 x or y}$) $f_{X|Y}(x|y) = \cfrac{f(x,\ y)}{f_Y(y)}$ $\text{if } f_Y(y) > 0 \Rightarrow \underbrace{P(X \in A | Y = y)}_{\Downarrow} = \int_A f_{X|Y}(x|y)dx$ $= \cfrac{\int_A f(x,\ y)dx}{f_Y(y)} = \int_A \cfrac{f(x,\ y)}{f_Y(y)}dx = \int_A f_{X|Y}(x|y)dx$ --- :::warning * **A condition pdf is a pdf** : $f_{X|Y}(x|y)$ 在給定 $Y=y$ 的情況下，看 $x$ 1. $f_{X|Y}(x|y) \ge 0$ 2. $\int_{-\infty}^{\infty} f_{X|Y}(x|y)dx = 1$ * **A condition pmf is a pmf** : $f_{X|Y}(x|y)$ 在給定 $Y=y$ 的情況下，看 $x$ 1. $f_{X|Y}(x|y) \ge 0$ 2. $\Sigma_{x} f_{X|Y}(x|y) = 1$ ::: # Examples ## Question 1 **question** : Let $(X,\ Y)$ be uniform on the unit square **solution** : * 我們可以發現所有可能看到的值 : $(x,\ y) \in [0,\ 1] \times [0,\ 1]$ * 每個值的可能性 : $\underbrace{f(x,\ y)}_{\text{joint pdf}} = P(X = x,\ Y=y) = \begin{cases} c,\ 0 \le x \le 1,\ 0 \le y \le 1 \\ 0,\ \text{otherwise} \end{cases}$ $\Rightarrow 1 = P((x,\ y) \in \text{unit square}) = \int_0^1 \int_0^1 f(x,\ y) \ dxdy = \int_0^1 \int_0^1 c \ dxdy = c$ * $F(a,\ b) = \underbrace{P(X \le a,\ Y \le b)}_{\begin{cases} 0 \le x \le 1 \\ 0 \le y \le 1 \end{cases}} = \int^b_0 \int^a_0 f(x,\ y) \ dx dy = \int^b_0 \int^a_0 1 \ dx dy = ab$ $\Rightarrow F(x,\ y) = xy,\ \begin{cases} 0 \le x \le 1 \\ 0 \le y \le 1 \end{cases}$ * $P \left( X < \cfrac{1}{3},\ Y > \cfrac{1}{2} \right)$ $\text{第一種解法}= \int_{1/2}^{0} \int_{0}^{1/3} \ f(x,\ y)\ dxdy = \int_{1/2}^{0} \int_{0}^{1/3} \ 1\ dxdy = \cfrac{1}{6}$ $\text{第二種解法}= P \left( X < \cfrac{1}{3},\ Y < 1 \right) - P \left( X < \cfrac{1}{3},\ Y < \cfrac{1}{2} \right) = F \left( \cfrac{1}{3},\ 1 \right) - F \left( \cfrac{1}{3},\ \cfrac{1}{2} \right) = \cfrac{1}{3} - \cfrac{1}{6} = \cfrac{1}{6}$ ## Question 2 **question** : Let $(X,\ Y)$ have density $f(x,\ y) = \begin{cases} x + y,\ \text{if } 0 \le x \le 1,\ 0 \le y \le 1 \\ 0 ,\ \text{otherwise} \end{cases}$ **solution** : * confirm $f$ is a pdf (i) $f(x,\ y) = x + y \ge 0$ (ii) $\underbrace{P((x,\ y) \in [0,\ 1] \times [0,\ 1])}_{\color{red}{\text{probability}}} \overset{\text{?}}{=} 1$ $= \int^{1}_{0} \int^{1}_{0} \underbrace{f(x,\ y)}_{\color{red}{\text{joint density}}} dxdy = \int^{1}_{0} \int^{1}_{0} (x+y) \ dxdy = 1$ * $P(X < \cfrac{1}{4} | Y = \cfrac{1}{3})$ $\color{red}{=} \int^{\frac{1}{4}}_{0} f_{X|Y}(x \ | \ 1/3) \ dx \color{red}{=} \int^{\frac{1}{4}}_{0} \cfrac{f(x,\ 1/3)}{\color{orange}{f_Y(1/3)}} dx = \int^{\frac{1}{4}}_{0} \cfrac{x + 1/3}{1/3 + 1/2} dx = 1/80$ $\because \color{orange}{f_Y(1/3)} = \int_0^1 f(x,\ y) \ dx = \int^1_0 (x+y) \ dx = y + \cfrac{1}{2},\ 0 \le y \le 1$ ## Question 3 **question** : Let $(X,\ Y)$ have density $f(x,\ y) = \begin{cases} cx^2y,\ \text{if }\color{red}{x^2 \le y \le 1} \\ 0 ,\ \text{otherwise} \end{cases}$ **solution** : 我們可以先做出圖形以方便我們觀察 ![](https://hackmd.io/_uploads/SJ4OjoUGT.png) * **Joint Density** : $P(Y \ge \cfrac{3}{4} \ | \ X = \cfrac{1}{2})$ $= \underbrace{\int^{1}_{\frac{3}{4}}}_{\text{需要先 check, 像這裡是 } \\ x = \frac{1}{2} \Rightarrow \frac{1}{4} \le y \le 1} f_{Y|X} \ (y \ | \ \cfrac{1}{2}) \ dy$ $= \int^{1}_{\frac{3}{4}} \cfrac{f(1/2,\ y)}{\color{orange}{f_X(1/2)}} = \int^{1}_{\frac{3}{4}} \ [\cfrac{\frac{21}{4}x^2y}{\frac{21}{8}x^2(1-x^4)}]_{x\text{代入} 1/2} \ dy = \cfrac{7}{15}$ * **Marginal Density** : $\because \color{orange}{f_X(1/2)} = \int_{x^2}^1 \ f(x,\ y) dy = \int_{x^2}^1 \ cx^2y \ dy = \cfrac{c}{2}x^2(1-x^4),\ -1 \le x \le 1$ $(c \text{ 的值下方會算出來})$ * **Joint Probability** : $P\left( (x,\ y) \in \underbrace{A}_{\color{purple}{\text{紫色區域}}} \right) = 1$ $\text{第一種解法}= \underbrace{\int^1_0}_{\text{已知的 } y \text{ range}} \underbrace{\int_{-\sqrt{y}}^{\sqrt{y}}}_{\because y = x^2 \ \Rightarrow \ x = \pm \sqrt{y}} \ f(x,\ y) \ dxdy \ (\color{green}{\text{先切 }y})$ $= \int^1_0 \int_{-\sqrt{y}}^{\sqrt{y}} \ cx^2y \ dxdy = c \cdot \cfrac{4}{21} = 1 \Rightarrow c = \cfrac{21}{4}$ $\text{第二種解法}= \underbrace{\int^1_{-1}}_{\text{已知的 } x \text{ range}} \underbrace{\int_{x^2}^{1}}_{\because y = x^2} \ f(x,\ y) \ dxdy \ (\color{green}{\text{先切 }x})$ * **Probability** : $P(Y < X)$ 可以藉由原圖得知我們想要求得的 : ![](https://hackmd.io/_uploads/SywJz3IGT.png) (i) $= \int_0^1 \int_{y}^{\sqrt{y}} \ f(x,\ y) \ dxdy = \int_0^1 \int_{y}^{\sqrt{y}} \cfrac{21}{4}x^2 y \ dxdy \ (\color{green}{\text{先切 }y})$ (ii) $= \int_0^1 \int_{x^2}^{x} \ f(x,\ y) \ dxdy \ (\color{green}{\text{先切 }y})$ ## Question 4 **question** : $X \sim UNIF(0,\ 1),\ \text{and } Y|X = x \sim UNIF(x,\ 1)$ **solution** : * $\text{求 } f_Y(y)$ $\color{red}{=} \int_0^y \ f(x,\ y) \ dx \color{red}{=} \int_0^y f_X(x) \cdot f_{Y|X =x}(y|x) \ dx = \int_0^y 1 \times \cfrac{1}{1-x} \ dx = -ln(1-y),\ 0 \le y \le 1$ * $\because f_X(x) = X \sim UNIF(0,\ 1),\therefore f_X(x) = 1$ * $\because f_{Y|X=x}(y|x) = Y|X = x \sim UNIF(x,\ 1)$ $\Rightarrow f_{Y|X=x}(y|x) = \cfrac{f(x,\ y)}{f_X(x)} = \begin{cases} f_X(x) = 1,\ 0 \le x \le 1 \\ f_{Y|X=x}(y|x) = \cfrac{1}{1-x},\ x \le y \le 1 \end{cases}$ # Important Def or Theorem * If 2 random variables $X$ and $Y$ are independent $(X \perp\kern-5pt \perp Y)$ $\Rightarrow P(X \in A,\ Y \in B) = P(X \in A) \cdot P(X \in B),\ \text{for every }A,\ B$ 簡單來說就是 $\text{joint probability = marginal probabilities 相乘}$ 證明如下 $\Rightarrow$ :::spoiler $P(X \in A | Y \in B_1) = P(X \in A | Y \in B_2) = P(X \in A | Y \in B_3) = P(X \in A)$ $\cfrac{P(X \in A \text{ and } Y \in B)}{P(Y \in B)} = P(X \in A) \Rightarrow P(X \in A \text{ and } Y \in B) = P(X \in A) \times P(Y \in B)$ ::: * Let $(X,\ Y)$ have joint pdf $f_{X,\ Y}(x,\ y) \Rightarrow X \perp\kern-5pt \perp Y$ if and only if $f_{X,\ Y}(x,\ y) = f_X(x)f_Y(y),\ \text{for all x, y}$ 證明如下 $\Rightarrow$ :::spoiler $\color{red}{\underbrace{F(x,\ y)}_{\text{joint CDF}}} = \color{green}{P(X \le x,\ Y \le y)} \overset{\text{ independent 定義 }}{=} P(X \le x)P(Y \le y) = \color{red}{\underbrace{F_X(x)F_Y(y)}_{\text{marginal CDF 相乘}}}$ $\Rightarrow \color{green}{\int_{-\infty}^{y} \int_{-\infty}^{x} f(t,\ s)\ dtds} = \left( \int_{-\infty}^{x} \ f_X(t)dt \right) \left( \int_{-\infty}^{y} \ f_Y(s)ds \right)$ $\overset{\text{ 左右同微分 }}{\Rightarrow} \cfrac{d}{dy}\cfrac{d}{dx} \int_{-\infty}^{y} \int_{-\infty}^{x} f(t,\ s)\ dtds = \cfrac{d}{dy}\cfrac{d}{dx}\left( \int_{-\infty}^{x} \ f_X(t)dt \right) \left( \int_{-\infty}^{y} \ f_Y(s)ds \right)$ $\Rightarrow \color{red}{\underbrace{f(x,\ y)}_{\text{joint density}}} = \color{red}{\underbrace{f_X(x)f_Y(y)}_{\text{marginal densities 相乘}}}$ ::: * if the range of $X$, $Y$ is rectangle, and if $f(x,\ y) =g(x)h(y) \Rightarrow X \perp\kern-5pt \perp Y$ 證明如下 $\Rightarrow$ :::spoiler Let $ag(x)$ be density : $\int \int g(x)h(x) \ dxdy = \int \int f(x,\ y) \ dxdy = 1$ $\Rightarrow \int \int ag(x) \times \frac{1}{a} h(y) \ dydx = 1$ $\Rightarrow \int ag(x) \times [\int \frac{1}{a} h(y) \ dy] \ dx = 1$ $\Rightarrow [\int \frac{1}{a} \ h(y) \ dy]\underbrace{[\int ag(x) \ dx]}_{\because density \ \therefore =1} = 1$ $\Rightarrow \int \frac{1}{a} h(y) \ dy = 1 \Rightarrow \cfrac{1}{a}h(y) \text{ is a density}$ ::: ## Example let $(X,\ Y)$ have density $f(x,\ y) = \begin{cases} 2e^{-(x+2y)}, \ \text{if } x>0 \text{ and } y>0 \\ 0,\ \text{otherwise} \end{cases} \text{, prove or disprove } X \perp\kern-5pt \perp Y$ 標準程序 : 求出 1. $f_X(x)$ 2. $f_Y(y)$ 3. $f_X(x)f_Y(y) = f(x,\ y)$ 但 by 上述的 def 可以不需要那麼麻煩 $\Rightarrow f(x,\ y) = 2e^{-x}e^{-2y} \Rightarrow X \perp\kern-5pt \perp Y$