---
tags: Probability
---

# Further topics on RVs: Covariance and Corelation

by AWfulsome

---

## Covariance

* The covariance of two random variables $X$ and $Y$ is defined by
$$
\text{cov}(X, Y) = E[(X - E[X])(Y - E[Y])]
$$
* An alternative formula is
$$
\text{cov}(X, Y) = E[XY] - E[X]E[Y]
$$
* A positive or negative covariance indicates that the values of $X - E[X]$ and $Y - E[Y]$ tend to have the same or opposite sign, respectively.
* A few other properties
* $\text{cov}(X, X) = \text{var}(X)$
* $\text{cov}(X, aY + b) = a\cdot\text{cov}(X, Y)$
* $\text{cov}(X, Y + Z) = \text{cov}(X, Y) + \text{cov}(X, Z)$

* Note that if $X$ and $Y$ are *independent*, $E[XY] = E[X]E[Y]$. Therefore $\text{cov}(X, Y) = 0$.
* $X$ and $Y$ are independent $\implies$ $X$ and $Y$ are uncorrelated
* "$\impliedby$" is generally not true (see Example 4.13)

:::spoiler Example 4.13
:::

## Correlation

* Also denoted as "correlation coefficient"
* The correlation coefficient of two random variables $X$ and $Y$ is defined as
$$
\rho(X, Y) = \dfrac{\text{cov}(X, Y)}{\sqrt{\text{var}(X)\text{var}(Y)}}
$$
* It can be shown that $-1\leq\rho\leq 1$
* $\rho > 0$: positively correlated
* $\rho < 0$: negatively correlated
* $\rho = 0$: uncorrelated ( $\implies \mathrm{cov}(X, Y) = 0$ )
* It can be shown that
$$
\rho = \pm 1 \iff \exists c \not= 0 \text{, such that }Y - E[Y] = c(X - E[X]) \text{ and } c\rho > 0
$$


:::spoiler Example
:::

## Vairance of the Sum of RVs

* If $X_1, X_2, ..., X_n$ are random variables with finite variance, we have
$$
\mathrm{var}(X_1 + X_2) = \mathrm{var}(X_1) + \mathrm{var}(X_2) + 2\mathrm{cov}(X_1, X_2)
$$
* More generally,
$$
\mathrm{var}(\sum_{i = 1}^{n}X_i) = \sum_{i = 1}^{n}\mathrm{var}(X_i) + \sum_{\{(i, j) | i \not= j\}}\mathrm{cov}(X_i, X_j)
$$
> See the textbook for the proof of the above formula and see also Example 4.15 for the illustration of this formula

:::spoiler Example 4.15
:::