---
tags: Math, Probability/Stats, Courses
---

<h1> Berteskas & Tsikilis Chapter 1 </h1>

### Sets and Models
If a set has uncountably infinite elements, each can have 0 probability and we can still satisfy sum of probabilities = 1 (Normalization)

However if it has countably infinite elements, each element having 0 probability means the sum will be 0 violating the normalization principle.

Not so popular property:
$P(A \cup B \cup C) = P(A) + P(A^c \cup B ) + P(A^c \cup B^c \cup C)$

Example of poorly specified/ambiguous probabilistic models:
#### Bertrand's 'Paradox'
> Consider a circle and an equilateral triangle inscribed in the circle. What is the probability that the length of a randomly chosen chord of the circle is greater than the side of the triangle?

:::spoiler Solution(s)
In (a), we 'randomly choose' by dropping a radius AB, and drawing a perpendicular chord at a random point C on AB. As the triangle base bisects AB (rsin30), the probability the chord being longer is 1/2.

In (b), we randomly choose an angle from the tangent drawn on the tip of the triangle. The probability of the chord being longer is now 1/3.

So clearly, the problem is ambiguous and there is a need to define the distribution from which the chord was randomly chosen.
:::

### Conditional Probability and Total Probability Theorem
$P(A|B) = \frac{P(A \cap B)}{P(B)}$ - It can be considered as a probability law on a new universe B.



Bayes rule helps in 'inference'. There are a number of causes that lead to an effect with known probability. When the effect takes place, we want to know the cause, and use Bayes rule for it.
Numerator gives $A \cap B$. The denominator can be expanded using total probability theorem.

### Independent Events
**Disjoint events:** $P(A \cap B) = 0$
**Independent events:**
Definition: $P(A|B) = P(A)$, i.e. B provides no new information
Corollary: $P(A \cap B) = P(A)P(B)$

Though often confused, disjoint events are not necessarily independent. Infact if an event is both disjoint and independent, $P(A) = 0$ or $P(B) = 0$. Here, independence can sort of be seen as orthogonality, a 3rd dimension when Venn diagrams are visualized. It can be put on a venn diagram using $P(A \cap B) = P(A)P(B)$ but that isn't really insightful.

:::spoiler A Visualization
 - Independent events
 - Dependent events

https://bransingle.wixsite.com/home/post/visualizing-independent-events-for-probabilities-no-more-venn-diagrams
:::

It essentially puts independent events on a square of side 1, event A being cut off by a vertical line at x = P(A) and event B being a horizontal line at Y = P(B). Conveys independence well, stretches to non-independent events. But I think it'll get flimsy on adding more than 2 events. You can probably take a cube instead of square for 3 events (use volume instead of area obviously) but we can't see >=4 so that's where Venn diagrams win)

#### Conditional Independence
Definition: $P(A \cap B|C) = P(A|C)P(B|C)$
Corollary: $P(A |B \cap C) = P(A|C)$
:::spoiler Proof of corollary
:::
In words: If C is known to have occured, The occurance of B doesn't tell us anything about the occurance of A.

Note that this can be visualized using the trick mentioned above. In a 3D cube, in the area cutoff by $Z = P(C)$, the events A and B behave as independent, with clear $X = P(A)$ and $Y = P(B)$ demarcations. However they don't do it at $z > P(C)$ and there have a dependent nature.

Example 1.21 page 37 is particularly nice.

Problem 24 Page 68 is particularly nice.
Problem 32-33, Page 71 are particularly nice.

#### Problem 47: Laplace Law of Succession
If an event has happened n (large) times consecutively, the probability of it happening again is almost 1. Gets closer to 1 as n increases.

Problem 48 is nice.