---
# System prepended metadata

title: 'Further topics on RVs: Covariance and Corelation'
tags: [Probability]

---

---
tags: Probability
---

# Further topics on RVs: Covariance and Corelation

by AWfulsome

---

## Covariance

* The covariance of two random variables $X$ and $Y$ is defined by
$$
\text{cov}(X, Y) = E[(X - E[X])(Y - E[Y])]
$$
    * An alternative formula is
$$
\text{cov}(X, Y) = E[XY] - E[X]E[Y]
$$
* A positive or negative covariance indicates that the values of $X - E[X]$ and $Y - E[Y]$ tend to have the same or opposite sign, respectively.
* A few other properties
    * $\text{cov}(X, X) = \text{var}(X)$
    * $\text{cov}(X, aY + b) = a\cdot\text{cov}(X, Y)$
    * $\text{cov}(X, Y + Z) = \text{cov}(X, Y) + \text{cov}(X, Z)$

* Note that if $X$ and $Y$ are *independent*, $E[XY] = E[X]E[Y]$. Therefore $\text{cov}(X, Y) = 0$.
    * $X$ and $Y$ are independent $\implies$ $X$ and $Y$ are uncorrelated
        * "$\impliedby$" is generally not true (see Example 4.13)

:::spoiler Example 4.13
![](https://i.imgur.com/57JVQiI.png)
:::

## Correlation

* Also denoted as "correlation coefficient"
* The correlation coefficient of two random variables $X$ and $Y$ is defined as
$$
\rho(X, Y) = \dfrac{\text{cov}(X, Y)}{\sqrt{\text{var}(X)\text{var}(Y)}}
$$
    * It can be shown that $-1\leq\rho\leq 1$
        * $\rho > 0$: positively correlated
        * $\rho < 0$: negatively correlated
        * $\rho = 0$: uncorrelated ( $\implies \mathrm{cov}(X, Y) = 0$ )
* It can be shown that
$$
\rho = \pm 1 \iff \exists c \not= 0 \text{, such that }Y - E[Y] = c(X - E[X]) \text{ and } c\rho > 0
$$

![](https://i.imgur.com/CTqeEd7.png)

:::spoiler Example
![](https://i.imgur.com/DfdvYfD.png)
:::

## Vairance of the Sum of RVs

* If $X_1, X_2, ..., X_n$ are random variables with finite variance, we have
$$
\mathrm{var}(X_1 + X_2) = \mathrm{var}(X_1) + \mathrm{var}(X_2) + 2\mathrm{cov}(X_1, X_2)
$$
* More generally,
$$
\mathrm{var}(\sum_{i = 1}^{n}X_i) = \sum_{i = 1}^{n}\mathrm{var}(X_i) + \sum_{\{(i, j) | i \not= j\}}\mathrm{cov}(X_i, X_j)
$$
> See the textbook for the proof of the above formula and see also Example 4.15 for the illustration of this formula

:::spoiler Example 4.15
![](https://i.imgur.com/QykLDm2.png)
:::