# Week 2 Lecture 1

## Conditional Probability
Most probabilities that we encounter are usually conditional probabilities. We must check for the underlying conditions when given a probability.

> In this diagram, Pr(A|B) is the ratio of the red and blue regions.

The **conditional probability** of an event A given that an event B has occurred is denoted by $Pr(A|B)$, where B is the conditioning event.
$$
Pr(A|B)=
\frac{Pr(A \cap B)}{Pr(B)}
$$

Given that B has occurred:
* The relevant outcomes are no longer **all** possible outcomes in the sample space, but only those outcomes that are contained in B.
* A has occurred iff one of the outcomes in A ∩ B has occurred.

∴ the conditional probability of A given B is proportional to Pr(A ∩ B).
### Examples
1.
$$
Pr(F_2|D)=
\frac{Pr(F_2 \cap D)}{Pr(D)}=
\frac{2/200}{5/200}=
\frac{2}{5}
$$
2. According to the table, we could also calculate:
$Pr(D)=\frac{5}{200}$
$Pr(F_2 \cap D)=\frac{2}{200}$

$\therefore Pr(F_2|D)= \frac{Pr(F_2 \cap D)}{Pr(D)}$

1.
$$
\begin{aligned}
\operatorname{Pr}(\text { sick }) &=\operatorname{Pr}(\text { sick and negative })+\operatorname{Pr}(\text { sick and positive }) \\
&=0.005+0.095=0.1
\end{aligned}
$$
2.
$$
\begin{aligned}
\operatorname{Pr}(\text { negative } \mid \text { sick }) &=\operatorname{Pr}(\text { negative and sick }) / \operatorname{Pr}(\text { sick }) \\
&=0.005 / 0.1=0.05
\end{aligned}
$$
3.
$$
\begin{aligned}
\operatorname{Pr}(\text { positive } \mid \text { healthy }) &=\operatorname{Pr}(\text { positive and healthy }) / \operatorname{Pr}(\text { healthy }) \\
&=0.18 / 0.9=0.2
\end{aligned}
$$

## Independent Events
### Intuition (Coin Toss)
Suppose that a fair coin is tossed twice. There are 4 possible outcomes: HH, HT, TH, TT (with probability of $\frac{1}{4}$ each).
Let A be the event A= {H on second toss}. What is Pr(A)?
A = {HH, TH}, so $Pr(A) = \frac{2}{4} = \frac{1}{2}$
Suppose we know that the first toss was T (Event B). What is Pr(A|B)?

$Pr(A|B) = \frac{Pr({TH})}{Pr({TH,TT})} \frac{1/4}{2/4}=\frac{1}{2}$

The probability of A occurring does not change, even after we learned that B has occurred.
∴ The outcome of the first toss does not affect the outcome of the second toss.
∴ A and B are independent events.
### Formal Definition
Two events $A$ and $B$ are called **independent** if
$$
\operatorname{Pr}(A \cap B)=\operatorname{Pr}(A) \operatorname{Pr}(B)
$$
Two events $A$ and $B$ are called **dependent** if
$$
\operatorname{Pr}(A \cap B) \neq \operatorname{Pr}(A) \operatorname{Pr}(B)
$$
:::warning
Disjoint doesn't imply independence
:::

:::info
If A and B are independent events, and Pr(B) >0, then Pr(A|B) = Pr(A)
:::

### Independence of Several Events
Events A~1~,A~2~,...,A~n~ are **mutually independent** if for every subset of indices {i~1~,i~2~,...,i~k~}(for k=2,3,...,n),
$$
\operatorname{Pr}\left(A_{i_{1}} \cap A_{i_{2}} \cap \cdots \cap A_{i_{k}}\right)=\operatorname{Pr}\left(A_{i_{1}}\right) \operatorname{Pr}\left(A_{i_{2}}\right) \cdots \operatorname{Pr}\left(A_{i_{k}}\right)
$$
In other words, the events A~1~,...A~n~ are mutually independent if the probability of the intersection of any subset of the n events is equal to the product of the individual probabilities.

:::info
If events A1, A2, A3 are pairwise independent, they are not necessarily mutually independent.
:::

### The Law of Total Probability
Let A~1~,...A~k~ be events in some sample space Ω.
* The events A~1~,...A~n~ are called **exhaustive** if at least one A~i~ must occur, i.e. A~1~∪A~2~∪...∪A~k~=Ω
* The events A~1~,...A~k~ are called **mutually exclusive** if no two distinct A~i~,A~j~ can occur at the same time, i.e. A~i~ ∩ A~j~ = ∅ for all i≠j.
<br/>
* Let A~1~,...A~k~ be **mutually exclusive** and **exhaustive** events. Then for any event B, the law of total probability states that
$$
\operatorname{Pr}(B)=\sum_{i=1}^{k} \operatorname{Pr}\left(B \mid A_{i}\right) P\left(A_{i}\right)
$$

## Bayes' Theorem

### Example

## Prior and Posterior Probabilities

### Intuition (Fair vs Biased Coin)
Friend A has two coins, a fair coin and a biased coin that always gives head. He randomly selects one of the coins, and asks if the selected coin is fair.

Let A be the event "selected coin is fair".

**Prior guess**: we guess that Pr(A) = 0.5.
We toss the coin 10 times and record all 10 outcomes.

Suppose the event B = "all heads for 10 tosses" occurs.
Event B strongly suggests that the selected coin is not fair. Knowing that, should we update the guess, given that B has occurred (i.e. what should Pr(A|B) be?)
Pr(A|B) is called the **posterior probability**.

### Bayes' Theorem (Reinterpreted)

$$
\operatorname{Pr}\left(A_{j} \mid B\right)=\frac{\operatorname{Pr}\left(B \mid A_{j}\right) \operatorname{Pr}\left(A_{j}\right)}{\operatorname{Pr}(B)}=\frac{\operatorname{Pr}\left(B \mid A_{j}\right) \operatorname{Pr}\left(A_{j}\right)}{\sum_{i=1}^{k} \operatorname{Pr}\left(B \mid A_{i}\right) \operatorname{Pr}\left(A_{i}\right)}
$$

Remarks:
* Typically the conditional probabilities Pr(B|A~1~),...,Pr(B|A~k~) are easy to determine directly.
* Since the prior probabilities Pr(A~1~),...,Pr(A~k~) are assumed to be known, we can use Bayes' theorem to compute the posterior probability Pr(A~j~|B) for each j=1,...,k.

### Bayesian Philosophy

The main idea of this philosophy is that the probability of a random event can be updated with new evidence:
* Assign a prior probability to the particular event of interest.
* Gather experimental evidence andc update our guess on whether the hypotehsis is true with the posterior probability.
* If A is the event of interest and B is the event representing experimental evidence, then Pr(A) is the prior probability, and PrA|B) is the posterior probability.
* The posterior probability can be calculated through the Bayes' theorem.