# Basic formulas for random variables

###### tags: `Mathematics`

### 基本數學
Mean and variance for random variables $X$ and $Y$
* Definition
* Mean: $\mu_X = E(X)$
* Variance: $\sigma_X^2 = V(X) \triangleq E((X-\mu_X)^2) = E(X^2)-\mu_X^2$
* Covariance: $\sigma_{XY}=\sigma_{YX}=E((X-\mu_X)(Y-\mu_Y)) = E(XY)-\mu_X\mu_Y$
* Basic formulas for a single variable
* $E(aX)=aE(X) \Rightarrow \mu_{aX}=a\mu_X$
* $V(aX)=a^2V(X) \Rightarrow \sigma_{aX}^2=a^2\sigma_{X}^2$
* Extension to two variables (not necessarily independent)
* $E(aX+bY)=aE(X)+bE(Y) \\
\Rightarrow \mu_{aX+bY}=a\mu_X+b\mu_Y$
* $V(aX+bY)=a^2V(X)+b^2V(Y)+2abE((X-\mu_X)(Y-\mu_Y)) \\
\Rightarrow \sigma_{aX+bY}^2=a^2\sigma_X^2+b^2\sigma_Y^2+2ab\sigma_{XY}=a^2\sigma_X^2 + b^2\sigma_Y^2+2ab\sigma_X\sigma_Y\rho_{XY}$
Derivation of the above expression:
$$
\begin{array}{rcl}
V(aX+bY)&=&E((aX+bY-E(aX+bY))^2)\\
&=& E((aX+bY-a\mu_X-b\mu_Y))^2)\\
&=& E((a(X-\mu_X)+b(Y-\mu_Y))^2)\\
&=& E(a^2(X-\mu_X)^2+b^2(Y-\mu_Y)^2+2ab(X-\mu_X)(Y-\mu_Y))\\
&=& a^2E((X-\mu_X)^2)+b^2E((Y-\mu_Y)^2)+2abE((X-\mu_X)(Y-\mu_Y))\\
&=& a^2\sigma_X^2 + b^2\sigma_Y^2 + 2ab\sigma_{XY}\\
&=& a^2\sigma_X^2 + b^2\sigma_Y^2 + 2ab\sigma_X\sigma_Y\rho_{XY}\\
\end{array}
$$
where $\rho_{XY}=\sigma_{XY}/(\sigma_X\sigma_Y)$ is the correlation coefficient between $X$ and $Y$, with $-1 \leq \rho_{XY} \leq 1$.
<!--
$\rho_{XY}=\frac{\sigma_{XY}}{\sigma_X\sigma_Y}$
-->

Exercises
1. Given a fair dice with its face value as a discrete random variable X, what are $E(X)$ and $V(X)$? **Answer**: $E(X)=7/2$, $V(X)=35/12$.
1. Given two fair dices with their face values as two discrete random variables X and Y, what are $E(Z)$ and $V(Z)$ if $Z=X+3Y$? **Answer**: $E(X)=14$, $V(X)=175/6$.
1. Define two random variables as follows:
* X: face values $[1,2,3,4,5,6]$ of a dice
* Y: face values $[0 for tails, 1 for heads]$ of an unfair coin, with $P(tails)=1/3$ and $P(heads)=2/3$.
What are $E(Z)$ and $V(Z)$ if $Z=X+2Y$?
1. Define two random variables as follows:
* X: face values $[1,2,3,4,5,6]$ of a dice
* Y: face values $[0 for tails, 1 for heads]$ of an unfair coin, with $P(tails)=1/3$ and $P(heads)=2/3$.
What are $E(Z)$ and $V(Z)$ if $Z$ is defined as follows?
$$
Z=\left\{
\begin{array}{rl}
2, & \text{if } X \geq 3,\\
Y, & \text{otherwise}.
\end{array}
\right.
$$
1. Define two random variables as follows:
* X: Gaussian PDF with $[\mu_X, \sigma_X^2]=[0, 4]$.
* Y: Uniform PDF with $[l_X, u_X]=[1, 3]$.
What are $E(Z)$ and $V(Z)$ if $Z=3X+4Y$?
1. Define two random variables as follows:
* X: Gaussian PDF with $[\mu_X, \sigma_X^2]=[0, 4]$.
* Y: Uniform PDF with $[l_X, u_X]=[1, 3]$.
What are $E(Z)$ and $V(Z)$ if $Z$ is defined as follows?
$$
Z=\left\{
\begin{array}{rl}
2Y, & \text{if } X \leq \mu_X,\\
Y, & \text{otherwise}.
\end{array}
\right.
$$