# Introduction to Mathematical Statistics

###### tags: `수리통계학`
**Chapter01 Probability and Distribution**
=
1.1 Introduction
--
Random experiment : experiment can be repeated under the same condition
Sample space($\mathcal {C}$) : the collection of every possible outcome

\
1.2.2 Set function
--
**Function**
$\mathrm {f}$: $\ X$ $\mapsto$ $\ Y$
Each element of $\ X$ assigns to exactlty one element of $\ Y$

The set $\ X$ is called *domain* of the function
The set $\ Y$ is called *codomain* of the function
\
\
**Set Function**
A set function is a function whose domain is a family of subsets of some given set and that takes its values in the extended real number line.
\
\
Ex1.2.6
Let $\mathcal {C}$ = $\mathbb {R}^2$ For a subset $\mathsf {A}$ of $\mathcal {C}$, let $\mathsf {Q}$$\left( \mathsf {A} \right)$ be the area of $\mathsf {A}$
If A has a finite area; otherwise $\mathsf {Q}$$\left( \mathsf {A} \right)$ be undefined.
Thus if $\mathsf {A}$=$\left\{ \left( x, y\right): x^{2}+y^{2}\le1 \right\}$, then $\mathsf {Q}$$\left( \mathsf {A} \right)$ $=\pi$
$\quad$$\quad$ if $\mathsf {A}$=$\left\{ \left( 0, 0\right), \left( 1, 1\right), \left( 0, 1\right)\right\}$, then $\mathsf {Q}$$\left( \mathsf {A} \right)$ $=0$
$\quad$$\quad$ if $\mathsf {A}$=$\left\{ \left( x, y\right)| 0\le x, 0\le y, x+y\le 1\right\}$, then $\mathsf {Q}$$\left( \mathsf {A} \right)=$${1 \over 2}$

Often our set function defined in terms of sums or integrals.
$\int_{\mathsf {A}}$$f(n)dx$ or $\sum_{\mathsf {A}}f(x)$

\
1.3 The Probability Set Function
--
**Definition1.3.1 (Probability)**
Let $\mathcal {C}$ be a sample space and let $\mathcal {B}$ be the set of events.
Let $\ P$ be a real-valued function defined on $\mathcal {B}$ then $\ P$ is a probability set function.
If $\ P$ satisfies the following three conditions.
1. $\ P$$\left( \mathsf {A} \right)$$\ge$ 0
2. $\ P$$\left( \mathcal {C} \right)$ $=1$
3. If $\left\{ \mathsf {A}_{n} \right\}$ is a sequence of events in $\mathcal {B}$ and $\mathsf {A}_{m}$$\cap$$\mathsf {A}_{n}$=$\varnothing$ $\forall$m$\neq$n then P$\left( \bigcup_{n=1}^{\infty}\mathsf {{A}_n}\right)$$=$$\sum_{n=1}^{\infty}P$$\left( \mathsf {A}_{n} \right)$

A collection of events whose members are pairwise disjoint, as in 3., is said to be a mutually exclusive collection.

**Theorem 1.3.1**
For each event $\mathsf {A}$$\in$$\mathsf {B}$, $P\left( \mathsf {A} \right)$ $=1-$$\ P$$\left( \mathsf {A^{c}} \right)$

**Theorem 1.3.2**
The probability of the null set is zero: that is $\ P$$\left( \mathsf {\varnothing} \right)$ $=0$

**Theorem 1.3.3**
For each event $\mathsf {A}$ and $\mathsf {B}$ are events s.t. $\mathsf {A}$$\subset$$\mathsf {B}$, then $P\left( \mathsf {A} \right)$$\le$$P\left( \mathsf {B} \right)$

**Theorem 1.3.4**
For each $\mathsf {A}$$\in$$\mathcal {B}$, $0\le$ $P\left( \mathsf {A} \right)$$\le 1$

**Theorem 1.3.5**
If $\mathsf {A}$ and $\mathsf {B}$ are events in $\mathcal {C}$, then $P\left( \mathsf {A}\cup\mathsf {B} \right)$$=P\left( \mathsf {A} \right) +P\left( \mathsf {B} \right)$-$P\left( \mathsf {A}\cap\mathsf {B} \right)$

\
1.4 Conditional Probability and Independence
--
Let the probability set function $P\left( \mathsf {A} \right)$ be defined on the sample space $\mathcal {C}$ and let $\mathsf {A}$ be a subset of $\mathcal {C}$ s.t. $P\left( \mathsf {A} \right)$$\ge0$
Let $\mathsf {B}$ be another subset of $\mathcal {C}$ which relative to the new sample space $\mathsf {A}$.
The probability of the event $\mathsf {B}$ is called the conditional probability of the event $\mathsf {B}$, relative to the hypothesis of the event $\mathsf {A}$ denoted by symbol $P\left( \mathsf {B} \mid\mathsf {A} \right)$.

어떤 $\mathsf {A}$사건이 일어날 조건 하에서 다른 사건 $\mathsf {B}$가 일어날 확률이다.
즉, $\mathsf {A}$가 $\mathsf {B}$의 sample space가 된다.

**Definition 1.4.1**
Let $\mathsf {B}$ and $\mathsf {A}$ be events with $P\left( \mathsf {A} \right)\ge0$ then we defined the conditional probability of $\mathsf {B}$ given $\mathsf {A}$ as $P\left( \mathsf {B} \mid\mathsf {A} \right)=$${P\left( \mathsf {A}\cap\mathsf {B} \right) \over P\left( \mathsf {A} \right)}$
Moreover, we have
1. $P\left( \mathsf {B} \mid\mathsf {A} \right) \ge 0$
2. $P\left( \mathsf {A} \mid\mathsf {A} \right) = 1$
3. $P\left( \mathsf {\bigcup_{n=1}^{\infty}\ B_n} \mid\mathsf {A} \right)=$${P\left( \mathsf {\bigcup_{n=1}^{\infty}\ B_n}\cap\mathsf {A} \right) \over P\left( \mathsf {A} \right)}=\mathsf {\sum_{n=1}^{\infty}}{P\left[\mathsf {B_{n}}\cap\mathsf {A}\right] \over P\left( \mathsf {A} \right)}=\mathsf {\sum_{n=1}^{\infty}} P\left[\mathsf {B_{n}}\mid\mathsf {A}\right]$

*(Multiplication rule for probability set function)*
From the definition of the conditional probability set function we can get $P\left(\mathsf {A}\cap\mathsf {B} \right)=$$P\left( \mathsf {A} \right)$$P\left( \mathsf {B} \mid\mathsf {A} \right)$

Also, we can consider $\mathsf {k}$ mutually exclusive events $\mathsf {A_{1}, A_{2}, \cdots, A_{k}}$ such that $P\left(\mathsf {A_{i}}\right)\ge 0$, $i=\mathsf {1, 2, \cdots, k}$ ( The events form a partition of $\mathcal {C}$ ) here the events $\mathsf {A_{1}, A_{2}, \cdots, A_{k}}$ do not need to be equally likely.
Let $\mathsf {B}$ be another event such that $P\left(\mathsf {B}\right) \ge 0$. Thus $\mathsf {B}$ occurs with one and only one of the events $\mathsf {A_{1}, A_{2}, \cdots, A_{k}}$; that is
$\mathsf {B = B \cap\left(A_{1}\cup A_{2}\cup \cdots \cup A_{k}\right)}$
$\quad$$=\left(\mathsf {B\cap A_{1}}\right)\cup\left(\mathsf {B\cap A_{2}}\right)\cup\cdots\cup\left(\mathsf {B\cap A_{k}}\right)$
Since $\mathsf {B\cap A_{i}, i=1, 2, \cdots, k}$ are mutually exclusive,
We have $P\mathsf {\left(B\right)}=$$P\left(\mathsf {B}\cap\mathsf {A_{1}} \right)+ P\left(\mathsf {B}\cap\mathsf {A_{2}} \right)+\cdots + P\left(\mathsf {B}\cap\mathsf {A_{k}}\right)$
However, $P\left(\mathsf {B}\cap\mathsf {A_{i}} \right)=P\left( \mathsf {A_{i}} \right)P\left( \mathsf {B} \mid\mathsf {A_{i}} \right)\quad i=\mathsf {1, 2, \cdots k}$
$P\left(\mathsf {B}\right)=P\left( \mathsf {A_{1}} \right)P\left( \mathsf {B} \mid\mathsf {A_{1}} \right)+P\left( \mathsf {A_{2}} \right)P\left( \mathsf {B} \mid\mathsf {A_{2}} \right)+\cdots + P\left( \mathsf {A_{k}} \right)P\left( \mathsf {B} \mid\mathsf {A_{k}} \right)=\sum_{i=1}^{k}P\left( \mathsf {A_{i}} \right)P\left( \mathsf {B} \mid\mathsf {A_{i}} \right)$

\
\
**Theorem 1.4.1 (Bayes)**
Let $\mathsf {A_{1}, A_{2}, \cdots, A_{k}}$ be events such that $P\left(\mathsf {A_{i}}\right)\ge 0$, $\quad i=\mathsf {1, 2, \cdots, k}$
Assume that the events form a partition of $\mathcal {C}$
Let $\mathsf {B}$ be any event then
$P\left( \mathsf {A_{j}} \mid\mathsf {B} \right)={P\left( \mathsf {A_{j}} \right)P\left( \mathsf {B} \mid\mathsf {A_{j}}\right) \over \sum_{i=1}^{k}P\left( \mathsf {A_{i}} \right)P\left( \mathsf {B} \mid\mathsf {A_{i}} \right) }$

By Bayes' theorem, we can calculate the posterior probability.

\
즉, $\mathsf {A_{i}}$가 주어졌을 때 ($i=\mathsf {1, 2, \cdots k}$) $\mathsf {B}$의 확률들의 합은 $P\left(\mathsf {B}\right)$이다. $P\left(\mathsf {A_{i}}\right)$는 사전확률(prior probability)이라고 한다. 이들을 이용해서 $\mathsf {B}$가 주어졌을 때 $\mathsf {A_{i}}$의 확률을 구할 수 있다. $P\left( \mathsf {A_{i}} \mid\mathsf {B} \right)$는 사후확률(posterior probability)라고 한다.

예를 들어 여자와 남자의 비율(PA1, PA2)이 있는데, 여자일 때 x라는 질병에 걸릴 확률이 P(B|A1)이고 남자일 때 x라는 질병에 걸릴 확률이 P(B|A2)라면, 질병에 걸렸을 때 성별이 여자일 확률 혹은 질병에 걸렸을 때 성별이 남자인 확률을 구할 수 있다.

\
1.4.1 Independence
-
The occurence of event $\mathsf {A}$ does not change the probability of event $\mathsf {B}$; that is, when $P\left(\mathsf {A} \right)>0, P\left(\mathsf {B}\mid\mathsf {A} \right)=P\left(\mathsf {B}\right)$
In this case, we say that the events $\mathsf {A}$ and $\mathsf {B}$ are independent.
$P\left(\mathsf {A}\mid\mathsf {B} \right)=$${P\left(\mathsf {A}\cap\mathsf {B}\right)\over P\left(\mathsf {B}\right)}={P\left(\mathsf {A}\right)P\left(\mathsf {B}\right)\over P\left(\mathsf {B}\right)}=P\left(\mathsf {A}\right)$

**Definition 1.4.2**
Let $\mathsf {A}$ and $\mathsf {B}$ be two events.
We say that $\mathsf {A}$ and $\mathsf {B}$ are independent if $P\left(\mathsf {A}\cap\mathsf {B}\right)=P\left(\mathsf {A}\right)P\left(\mathsf {B}\right)$.
\
\
More generally, the $\mathsf {n}$ events $\mathsf {A_{1}, A_{2}, \cdots, A_{n}}$ are mutually independent iff for every collection of $\mathsf {k}$ of these events, $\mathsf{ 2\le k \le n}$, and for every permutation $\mathsf {d_{1}, d_{2}, d_{3}, \cdots, d_{k}}$ of $\mathsf {1, 2, \cdots, k}$
$P\left(\mathsf {A_{d_{1}}\cap A_{d_{2}}\cap\cdots\cap A_{d_{k}}} \right)=P\left(\mathsf {A_{d_{1}}}\right)P\left(\mathsf {A_{d_{2}}}\right)\cdots P\left(\mathsf {A_{d_{k}}}\right)$

\
mutually independent 라면 각각의 확률들의 곱으로 나타낼 수 있다.

\
1.5 Random Variables
--
**Definition 1.5.1**
Consider a random experiment with a sample space $\mathcal {C}$
A function $X$, which assigns to each element $c\in \mathcal {C}$ one and only one number $X\left(c\right)=x$, is call a random variable. The space or range of $X$ is the set of real numbers $\mathcal {D}=\{x\mid x=X\left(c\right), c \in \mathcal{C}\}$

\
\
표본 공간 $\mathcal {C}$ 에서 확률시행을 고려할 때, 함수 $X$에 표본 공간의 원소를 대입한다. 이때 나오는 결과가 확률 변수라고 하고, 그 확률 변수를 모아놓은 집합이 space 혹은 $X$의 치역이라고 한다.

쉽게 예를 들자면, 표본 공간(인구)에 각각의 원소인 철수, 유리, 짱구, $\cdots$ 이 있다고 하자.
이때 함수 $X$는 키라고 하자. 그렇다면 확률 변수는 철수의 키인 170, 유리의 키인 165, 짱구의 키인 178. 즉, space는 $\mathcal {D}=\{170, 165, 178, \cdots\}$가 될 수 있다.
\
\
Given a random variable $X$, its range $\mathcal {D}$ becomes the sample space of interest. $X$ also induces a probability which we call the distribution of $X$.

Consider, $X$ is a discrete random variable with a finite space $\mathcal {D}=\{d_{1}, d_{2}, \cdots, d_{m}\}$
Define the function $p_{\mathsf {x}}\left(d_{i}\right)$ on $\mathcal {D}$ by $p_{\mathsf {x}}\left(d_{i}\right)=P\left[\{c\mid X\left(c\right)=d_{i}\}\right]$ for $i=1, 2, \cdots , m$ Also $P_{X}\left(d_{i}\right)$ is defined as probability mass function of $X$ then the induced probability distribution, $P_{X}\left(\cdot\right)$, of $X$ is $P_{X}\left(D\right)=\sum_{d_{i}\in D}p_{X}\left(d_{i}\right), D\subset \mathcal {D}$
\
\

1.6 Discrete Random Variables
--

1.7 Continuous Random Variables
--

1.8 Expectation of a Random Variables
--

1.9 Some Special Expectation
--

1.10 Important Inequality
--

**Chapter02 Multivariate Distribution**
=
2.1 Distributions of Two Random Variables
--