---
tags: note
---

# Statistics Final Review Note

[toc]

## Summaries

**Tests**

**Confident Intervals**

**Confidence Intervals**

**z-test**
\begin{align}
\bar x-z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\leq \mu\leq \bar x+z_{\alpha/2}\frac{\sigma}{\sqrt{n}}
\end{align}
**t-test**
\begin{align}
\bar x-t_{\alpha/2,n-1}\frac{s}{\sqrt{n}}\leq \mu \leq \bar x+t_{\alpha/2,n-1}\frac{s}{\sqrt{n}}
\end{align}
**chi-sqare test**
\begin{align}
\frac{(n-1)s^2}{\chi^2_{\alpha/2,n-1}}\leq \chi^2\leq \frac{(n-1)s^2}{\chi^2_{1-\alpha/2,n-1}}
\end{align}
**binominal z-test**
\begin{align}
\hat p-z_{\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}\leq p\leq \hat p+z_{\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}
\end{align}
**2 samples z-test**
\begin{align}
\bar x_1-\bar x_2-z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}\leq \mu_1-\mu_2\leq \bar x_1 -\bar x_2 + z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}
\end{align}
**pooled t-test**
\begin{align}
\bar x_1-\bar x_2 -t_{\alpha/2,n_1+n_2-2}s_p\sqrt{\frac1{n_1}+\frac1{n_2}}\leq \mu_1-\mu_2\leq \bar x_1-\bar x_2 +t_{\alpha/2,n_1+n_2-2}s_p\sqrt{\frac1{n_1}+\frac1{n_2}}
\end{align}
**Welch's t-test**
\begin{align}
\bar x_1-\bar x_2 -t_{\alpha/2,v}\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}\leq \mu_1-\mu_2\leq \bar x_1-\bar x_2 +t_{\alpha/2,v}\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}
\end{align}
**paired t-test**
\begin{align}
\bar d-t_{\alpha/2,n-1}\frac{s_d}{\sqrt{n}}\leq \mu_D\leq \bar d+t_{\alpha/2,n-1}\frac{s_d}{\sqrt{n}}
\end{align}
**F chi-square test**
\begin{align}
\frac{s_1^2}{s_2^2}f_{1-\alpha/2,n_2-1,n_1-1}\leq \frac{\sigma_1^2}{\sigma_2^2}\leq \frac{s_1^2}{s_2^2}f_{\alpha/2,n_2-1,n_1-1}
\end{align}
**2 binominal z-test**
\begin{align}
\hat p_1-\hat p_2-z_{\alpha/2}\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}\leq p_1-p_2\leq \hat p_1-\hat p_2+z_{\alpha/2}\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}
\end{align}

**Others**

**Prediction Interval**
\begin{align}
\bar x-t_{\alpha/2,n-1}s\sqrt{1+\frac1n}\leq X_{n+1}\leq \bar x + t_{\alpha/2,n-1}s\sqrt{1+\frac1n}
\end{align}

**ANOVA F-test**
$\begin{align}
\sum_j^k\sum^{n_j}_i(X_{ij}-\overline{\overline X})^2=\sum^k_jn_j(\overline X_j-\overline{\overline X})^2+\sum_j^k\sum_i^{n_j}(X_{ij}-\overline X_j)^2
\end{align}$

$\Rightarrow SS(Total)=SST+SSE$

$SST=\text{TotalVar}\times (n-1)$

$SSE=\text{sum of Treatment Var}\times (n-1)$

$\begin{align}MS_{Treatments}=\frac{SS_{Treatments}}{a-1}\end{align}$

$\begin{align}E(MS_{Treatments})=\sigma^2+\frac{n\sum^a_{i=1}\tau_i^2}{a-1}\end{align}$

$\begin{align}MS_E=\frac{SS_E}{a(n-1)}\end{align}$

$\begin{align}E(MS_E)=\sigma^2\end{align}$

$\begin{align}F_0=\frac{MS_{Treatments}}{MS_E}\end{align}$

Rejection criterion for a fixed-level test: $f_0>f_{\alpha,a-1,a(n-1)}$

**Bonferroni method**

New $\begin{align}\alpha=\frac{\alpha}{n}\end{align}$

$\begin{align}n=\text{number of comparisons}=c=\frac{k(k-1)}{2}\end{align}$

## Point Estimation

| Unknown Parameter $\theta$ | Statistic $\hat \Theta$ | Point Estimate $\hat \theta$ |
|:--------------------------:|:---------------------------------------------------:|:----------------------------:|
| $\mu$ | $\begin{align}\bar X=\frac{\sum X_i}{n}\end{align}$ | $\bar x$ |
| $\sigma^2$ | $\begin{align}S^2=\frac{\sum (X_i-\bar X)^2}{n-1}\end{align}$ | $s^2$ |
| $p$ | $\begin{align}\hat P=\frac{X}{n}\end{align}$ | $\hat p$ |
| $\mu_1-\mu_2$ | $\begin{align}\bar X_1-\bar X_2=\frac{\sum X_{1i}}{n_1}-\frac{\sum X_{2i}}{n_2}\end{align}$ | $\bar x_i-\bar x_2$ |
| $p_1-p_2$ | $\begin{align}\hat P_1-\hat P_2=\frac{X_1}{n_1}-\frac{X_2}{n_2}\end{align}$ | $\hat p_1-\hat p_2$ |

### Unbiased estimator

The point estimator $\hat \Theta$ is an unbiased estimator for the paramter $\theta$ if $E(\hat \Theta)=\theta$.

If the estimator is not unbiased, then the difference $E(\hat \Theta)-\theta$ is called the **bias** of the estimator $\hat\Theta$.

If there are two unbiased estimators of a parameter, the one whose **variance is smaller** is said to be **relatively efficient**.

### Mean square error
The mean square error of an estimator $\hat \Theta$ of the parameter $\theta$ is define as
\begin{align}
MSE(\hat \Theta)&=E((\hat\Theta-\theta)^2)\\
&=Var(\hat \theta)+(Bias(\hat \theta))^2
\end{align}

### Standard error
The **standard error** of a statistic is the standard deviation of its sampling distribution. If the standard error **involves unknown parameters** whose values can be estimated, substitution of these estimates into the standard error results in an **estimated standard error**.

\begin{align}
\sigma_{\bar X}=\frac{\sigma}{\sqrt{n}}&&\hat \sigma_{\bar X}=\frac{S}{\sqrt{n}}
\end{align}


## Hypothesis Testing

### Statistical hypothesis

We like to think of statistical hypothesis testing as the data analysis stage of **comparative experiment**.

A **statistical hypothesis** is a statement about the **numerical value of a population parameter**.

**Two-sided hypothesis**
\begin{align}
H_0:\mu=50 \text{cm}/\text{s}\\
H_1:\mu\neq50 \text{cm}/\text{s}
\end{align}

**One-sided hypothesis**
\begin{align}
H_0:\mu\geq50 \text{cm}/\text{s}&&H_1:\mu<50 \text{cm}/\text{s}&&\text{or}\\
H_0:\mu\leq50 \text{cm}/\text{s}&&H_1:\mu>50 \text{cm}/\text{s}
\end{align}

**Null hypothesis $H_0$**

Will be accepted unless the data **provide convincing evidence that it is false**.

**Alternative hypothesis $H_a$**
Opposite of null hypothesis. The hypothesis that will be accepted only if the data provide convincing evidence of its **truth**.

It is **easier** to provide convincing evidence that it is **false**.
$\rightarrow$ can or can not reject $H_0$

## Hypothesis Testing on the Mean (z-Test)

Null hypothesis $H_0:\mu=\mu_0$

Test statistic: $\begin{align}Z_0=\frac{\bar X-\mu_0}{\sigma/\sqrt{n}}\end{align}$

| Alternative Hypotheses | P-Value | Rejection Criterion |
| ---------------------- | --- | ---------------------------------------------------------------------- |
| $H_1:\mu\neq \mu_0$ | $P=2(1-\Phi(\|z_0\|))$ | $z_0>z_\alpha$ or $z_0<-z_\alpha$ |
| $H_1:\mu>\mu_0$ | $P=1-\Phi(\|z_0\|)$ | $z_0>z_\alpha$ |
| $H_1:\mu<\mu_0$ | $P=\Phi(\|z_0\|)$ | $z_0<-z_\alpha$ |
### Confidence Interval

If $\bar x$ is the sample mean of a random sample of size $n$ from a population with known variance $\sigma^2$, a $100(1-\alpha)\%$ confidence interval on $\mu$ is given by

\begin{align}
\bar x-z_{\alpha/2}\frac{\sigma}{\sqrt{n}}\leq \mu\leq \bar x+z_{\alpha/2}\frac{\sigma}{\sqrt{n}}
\end{align}

where $z_{\alpha/2}$ is the upper $100\alpha/2$ percentage point and $-z_{\alpha/2}$ is the lower $100\alpha/2$ percentage point of the standard normal distribution.

### Relationship between Tests of Hypotheses and Confidence Intervals

If $[l,u]$ is a $100(1-\alpha)$ percent confidence interval for the parameter, then the test of significance level $\alpha$ of the hypothesis

\begin{align}
H_0: \theta=\theta_0\\
H_1: \theta\neq\theta_0
\end{align}

will lead to **rejection of $H_0$ if and only if the hypothesized value is not in the $100(1-\alpha)$ percent confidence interval $[l,u]$.**

### Choice of Sample Size

$\begin{align}z_{\alpha/2}(\frac{\sigma}{\sqrt{n}})=E\end{align}$

If $\bar x$ is used as an estimate of $\mu$, we can be $100(1-\alpha)\%$ confident that the error $|\bar x - \mu|$ will not exceed a specified amount $E$ when the sample size is

\begin{align}
n=(z_{\alpha/2}\frac{\sigma}{E})^2
\end{align}

## T Distribution (Variance Unknown & Small Sample Size)

For a **normal distribution** with unknown mean $\mu$ and **unknown** variance $\sigma^2$. The quantity

\begin{align}
T=\frac{\bar X -\mu}{S/\sqrt{n}}
\end{align}

has a $t$ distribution with **$n-1$ degrees of freedom**.

### Calculating the P-value

Find $f_{\alpha,v}$ on the table, where $v=$ degrees of freedom.


One-sided, two-sided are just like Z-test.

### Confidence Interval on the Mean

If $\bar x$ and $s$ are the mean and standard deviation of a random sample from a normal distribution with unknown variance $\sigma^2$, a $100(1-\alpha)\%$ CI on $\mu$ is given by
\begin{align}
\bar x-t_{\alpha/2,n-1}\frac{s}{\sqrt{n}}\leq \mu \leq \bar x+t_{\alpha/2,n-1}\frac{s}{\sqrt{n}}
\end{align}

where $t_{\alpha,n-1}$ is the upper $100\alpha/2$ percentage point of the $t$ distribution with $n-1$ degrees of freedom.

## Chi-square Distribution

$H_0: \sigma^2=\sigma_0^2,\ H_1: \sigma^2\neq\sigma_0^2$

Let $X_1,X_2...,X_n$ be a random sample from a normal distribution with unknown mean $\mu$ and unknown variance $\sigma^2$. The quantity

\begin{align}
X_0^2=\frac{(n-1)S^2}{\sigma_0^2}
\end{align}

has a **chi-square distribution with $n-1$** degrees of freedom, abbreviated as $\chi^2_{n-1}$.


\begin{align}
P(X^2>\chi^2_{\alpha,k})=\int^\infty_{\chi_{\alpha,k}^2}f(u)du=\alpha
\end{align}

Null hypothesis $H_0:\sigma^2=\sigma^2_0$
Test statistic: $\chi_0^2=(n-1)S^2/\sigma_0^2$

| Alternative Hypotheses | Rejection Criterion |
| ---------------------------- | ---------------------------------------------------------------------- |
| $H_1:\sigma^2\neq\sigma^2_0$ | $\chi^2_0>\chi^2_{\alpha/2,n-1}$ or $\chi^2_0<\chi^2_{1-\alpha/2,n-1}$ |
| $H_1:\sigma^2>\sigma^2_0$ | $\chi^2_0>\chi^2_{\alpha,n-1}$ |
| $H_1:\sigma^2<\sigma^2_0$ | $\chi^2_0<\chi^2_{\alpha,n-1}$ |

### Confidence Interval on the Variance

If $s^2$ is the sample variance from a random sample of $n$ observations from a **normal distribution** with **unknown variance** $\sigma^2$, a $100(1-\alpha)\%$ CI on $\sigma^2$ is

\begin{align}
\frac{(n-1)s^2}{\chi^2_{\alpha/2,n-1}}\leq \chi^2\leq \frac{(n-1)s^2}{\chi^2_{1-\alpha/2,n-1}}
\end{align}

where $\chi^2_{\alpha/2,n-1}$ and $\chi^2_{1-\alpha/2,n-1}$ are the upper and lower $100\alpha/2$ percentage points of the chi-square distribution with $n-1$ degrees of freedom, respectively.

## Binominal Distribution

### Large sample size

$\begin{align}H_0:p=p_0,\ H_1:p\neq p_0\end{align}$

$\begin{align}z=\frac{x-\mu}{\sigma}\end{align}$, $\begin{align}\hat p=\frac Xn=\frac{\texttt{#obs}}{\texttt{#samples}}\end{align}$

Let $X$ be the number of observations in a random sample of size $n$ that belongs to the class associated with $p$. Then the quantity

\begin{align}
Z=\frac{X-np}{\sqrt{np(1-p)}}
\end{align}

has approximately a standard normal distribution, N(0,1).

\begin{align}
z=\frac{\hat p-p_0}{\sigma_{\hat p}}=\frac{\hat p - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}
\end{align}

The sample size $n$ is considered large enough if both $np_0\geq 15$ and $nq_0\geq 15$

#### Testing Hypotheses

Null hypotheses $H_0:p=p_0$

Test statistic: $z_0=\frac{X-np_0}{\sqrt{np_0(1-p_0)}}$

| Alternative Hypotheses | P-Value | Rejection Criterion |
| ---------------------- | --- | ---------------------------------------------------------------------- |
| $H_1:p\neq p_0$ | $P=2(1-\Phi(\|z_0\|))$ | $z_0>z_\alpha$ or $z_0<-z_\alpha$ |
| $H_1:p>p_0$ | $P=1-\Phi(\|z_0\|)$ | $z_0>z_\alpha$ |
| $H_1:p<p_0$ | $P=\Phi(\|z_0\|)$ | $z_0<-z_\alpha$ |

#### Confidence Interval on a Binomial Proportion

If $\hat p$ is the proportion of observations in a random sample of size $n$ that belong to a class of interest, an approximate $100(1-\alpha)\%$ CI on the proportion $p$ of the population that belongs to this class is

\begin{align}
\hat p-z_{\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}\leq p\leq \hat p+z_{\alpha/2}\sqrt{\frac{\hat p(1-\hat p)}{n}}
\end{align}

where $z_{\alpha/2}$ is the upper $100\alpha/2$ percentage point of the standard normal distribution.

#### Choice of Sample Size

If $\hat p$ is used as an estimate of $p$, we can be $100(1-\alpha)\%$ confident that the error $|\hat p-p|$ will not exceed a specified amount $E$ when the sample size is

\begin{align}
n=(\frac{z_{\alpha/2}}{E})^2p(1-p)
\end{align}
\*usually use $\hat p$ here

For a specified error $E$, an upper bound on the sample size for estimating $p$ is
\begin{align}
n=(\frac{z_{\alpha/2}}{E})^2\frac14
\end{align}

In other words, that's the needed size for any $p$ for the specified $E$.

## Prediction Interval
- In some situations, we are interested in **predicting** a future observation of a random variable.
- We may also want to find a range of likely values for the variable, so a CI on the mean is not really appropriate.

A $100(1-\alpha)\%$ PI on a single future observation from a normal distribution is given by

\begin{align}
\bar x-t_{\alpha/2,n-1}s\sqrt{1+\frac1n}\leq X_{n+1}\leq \bar x + t_{\alpha/2,n-1}s\sqrt{1+\frac1n}
\end{align}

## Testing for Goodness of Fit
- There are many instances where the underlying distribution is not known, and we wish to test a particular distribution.
- Use a goodness-of-fit test procedure based on the chi-square distribution.

$n$ observations are arranged in a histogram, having $k$ bins or class intervals.
$O_i=$ observed frequency in the $i^{\text{th}}$ class interval
$E_i=$ the expected frequency in the $i^{\text{th}}$ class interval, denoted

\begin{align}
X^2_0=\sum^k_{i=1}\frac{(O_i-E_i)^2}{E_i}
\end{align}

with $k-p-1$ degrees of freedom

$p=$ the number of parameters of the hypothesized distribution estimated by sample statistics.

We would reject the hypothesis that the distribution of the population is the hypothesized distribution if the calculated value of the test statistic is too large.

- Calculate with chi-chquare test.
- The value 3,4, and 5 are widely used as minimal value of expected frequencies.

## Two Samples

### Determining the Target Parameter

| Parameter | Key Words or Phrases | Type of Data |
| --------------------------- | ----------------------------------------------------------------------------------------- | ------------ |
| $\mu_1-\mu_2$ | Mean difference;<br>differences in averages | Quantitative |
| $p_1-p_2$ | Differences between proportions, percentages, fractions, or rates;<br>compare proportions | Qualitative |
| $(\sigma_1)^2/(\sigma_2)^2$ | Ratio of variances;<br>differences in variability or spread;<br>compare variation | Quantitative |

## 2 Samples z-Test (Difference in Means, Variances Known)

### Assumptions

1. $X_{11},X_{12},...,X_{1n_1}$ is a random sample of size $n_1$ from population 1.
2. $X_{21},X_{22},...,X_{2n_2}$ is a random sample of size $n_2$ from population 2.
3. The two populations represented by $X_1$ and $X_2$ are independent.
4. Both populations are normal, or if they are not normal, the conditions of the central limit theorem apply.

Under the previous assumptions, the quantity

\begin{align}
Z=\frac{\bar X_1-\bar X_2-(\mu_1-\mu_2)}{\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}}
\end{align}

### Testing Hypotheses

Null hypothesis $H_0: \mu_1-\mu_2=\Delta_0$ (usually 0)

Test statistic: $\begin{align}
Z=\frac{\bar X_1-\bar X_2-(\mu_1-\mu_2)}{\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}}
\end{align}$

| Alternative Hypotheses | P-Value | Rejection Criterion for Fixed-Level Tests |
| ------------------------------ | ---------------------- | ----------------------------------------- |
| $H_1:\mu_1-\mu_2\neq \Delta_0$ | $P=2(1-\Phi(\|z_0\|))$ | $z_0>z_{\alpha/2}$ or $z_0<-z_{\alpha/2}$ |
| $H_1:\mu_1-\mu_2> \Delta_0$ | $P=1-\Phi(\|z_0\|)$ | $z_0>z_{\alpha/2}$ |
| $H_1:\mu_1-\mu_2< \Delta_0$ | $P=\Phi(\|z_0\|)$ | $z_0<-z_{\alpha/2}$ |

### Type II Error and Choice of Sample Size

For a **two-sided** alternative hypothesis significance level $\alpha$, the sample size $n_1=n_2=n$ required to detect a true difference in means of $\Delta$ with power at least $1-\beta$ is

\begin{align}
n\approx \frac{(z_{\alpha/2}+z_\beta)^2(\sigma_1^2+\sigma_2^2)}{(\Delta-\Delta_\sigma)^2}
\end{align}

If $n$ is not an integer, round the sample size up to the next integer.


For a one-sided alternative hypothesis significance level $\alpha$, the sample size $n_1=n_2=n$ required to detect a true difference in means of $\Delta(\neq \Delta_0)$ with power at least $1-\beta$ is

\begin{align}
n=\frac{(z_\alpha+z_\beta)^2(\sigma_1^2+\sigma_2^2)}{(\Delta-\Delta_0)^2}
\end{align}

### Confidence Interval

If $\bar x_1$ and $\bar x_2$ are the means of independenet random samples of size $n_1$ and $n_2$ from populations with known variances $\sigma_1^2$ and $\sigma_2^2$, respectively, a $100(1-\alpha)\%$ CI for $\mu_1-\mu_2$ is

\begin{align}
\bar x_1-\bar x_2-z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}\leq \mu_1-\mu_2\leq \bar x_1 -\bar x_2 + z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1}+\frac{\sigma_2^2}{n_2}}
\end{align}

where $z_{\alpha/2}$ is the upper $100\alpha/2$ percentage point and $-z_{\alpha/2}$ is the lower $100\alpha/2$ percentage point of the standard normal distribution.

### Choice of Sample Size

If $\bar x_1$ and $\bar x_2$ are used as estimates of $\mu_1$ and $\mu_2$, respectively, we can be $100(1-\alpha)\%$ confident that the error $|(\bar x_1-\bar x_2)-(\mu_1-\mu_2)|$ will not exceed a specified amount $E$ when the sample size $n_1=n_2=n$ is

\begin{align}
n=(\frac{z_{\alpha/2}}{E})^2(\sigma_1^2+\sigma_2^2)
\end{align}

## Inference on the Means of Two Populations (Variances Unknown)

### Case 1: Equal Variances (Pooled t-Test)

#### Hypothesis Testing

The **pooled estimator** of $\sigma^2$, denoted by $S^2_p$, is defined by

\begin{align}
S^2_p=\frac{(n_1-1)S_1^2+(n_2-1)S_2^2}{n_1+n_2-2}
\end{align}

Given the assumptions of this section, the quantity

\begin{align}
T=\frac{\bar X_1-\bar X_2-(\mu_1-\mu_2)}{S_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}
\end{align}

has a $t$ distribution with $n_1+n_2-2$ degrees of freedom.

Null hypothesis $H_0: \mu_1-\mu_2=\Delta_0$

Test statistic: $\begin{align}T_0=\frac{\bar X_1-\bar X_2-\Delta_0}{S_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\end{align}$

| Alternative Hypotheses | P-Value | Rejection Criterion for Fixed-Level Tests |
| ------------------------------ | ---------------------- | ----------------------------------------- |
| $H_1:\mu_1-\mu_2\neq \Delta_0$ | Sum of the probability above $\|t_0\|$ and below $-\|t_0\|$ | $t_0>t_{\alpha/2,n_1+n_2-2}$ or $t_0<-t_{\alpha/2,n_1+n_2-2}$ |
| $H_1:\mu_1-\mu_2> \Delta_0$ | Probability above $t_0$ | $t_0>t_{\alpha/2,n_1+n_2-2}$|
| $H_1:\mu_1-\mu_2< \Delta_0$ | Probability below $t_0$ | $t_0<-t_{\alpha/2,n_1+n_2-2}$ |

#### Confidence Interval

If $\bar x_1$, $\bar x_2$, $s_1^2$ and $s_2^2$ are the means and variances of two random samples of sizes $n_1$ and $n_2$, respectively, from two independent normal populations with unknown but equal variances, a $100(1-\alpha)\%$ CI on the difference in means $\mu_1-\mu_2$ is

\begin{align}
\bar x_1-\bar x_2 -t_{\alpha/2,n_1+n_2-2}s_p\sqrt{\frac1{n_1}+\frac1{n_2}}\leq \mu_1-\mu_2\leq \bar x_1-\bar x_2 +t_{\alpha/2,n_1+n_2-2}s_p\sqrt{\frac1{n_1}+\frac1{n_2}}
\end{align}

where $\begin{align}s_p=\sqrt{\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}}\end{align}$ is the pooled estimate of the common population standard deviation, and $t_{\alpha/2,n_1+n_2-2}$ is the upper $100\alpha/2$ percentage point of the distribution with $n_1+n_2-2$ degrees of freedom.

### Case 2: Unequal Variances (Welch's t-Test)

\begin{align}
T^\star_0=\frac{\bar X_1-\bar X_2-\Delta_0}{\sqrt{\frac{S_1^2}{n_1}+\frac{S_2^2}{n_2}}}
\end{align}

is distributed approximately as $t$ with degrees of freedom given by

\begin{align}
v=\frac{(\frac{S_1^2}{n_1}+\frac{S_2^2}{n_2})^2}{\frac{(S_1^2/n_1)^2}{n_1-1}+\frac{(S_2^2/n_2)^2}{n_2-1}}
\end{align}

if the null hypothesis $H_0: \mu_1-\mu_2=\Delta_0$ is true. If $v$ is not an integer, **round down** to the nearest integer.

**Welch-Satterthwaite equation**

Approximation to the effective degrees of freedom of a linear combination of independent sample variances.

\begin{align}
v=\frac{(\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2})^2}{\frac{s_1^4}{n_1^2(n_1-1)}+\frac{s_2^4}{n_2^2(n_2-1)}}
\end{align}

#### Confidence Interval

If $\bar x_1$, $\bar x_2$, $s_1^2$ and $s_2^2$ are the means and variances of two random samples of sizes $n_1$ and $n_2$, respectively, from two independent normal populations with unknown and unequal variances, then an approximate $100(1-\alpha)\%$ CI on the difference in means $\mu_1-\mu_2$ is

\begin{align}
\bar x_1-\bar x_2 -t_{\alpha/2,v}\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}\leq \mu_1-\mu_2\leq \bar x_1-\bar x_2 +t_{\alpha/2,v}\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}
\end{align}

where $v$ is given by equation above and $t_{\alpha/2,v}$ is the upper $100\alpha/2$ percentage point of the $t$ distribution with $v$ degrees of freedom.

## The Paired t-Test

### Hypothesis Testing

Null hypothesis $H_0:\mu_D=\Delta_0$

Test statistic: $\begin{align}T_0=\frac{\bar D-\Delta_0}{S_D/\sqrt{n}}\end{align}$

| Alternative Hypothesis | P-Value | Rejection Region for Fixed-Level Tests |
| ------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------- |
| $H_1:\mu_D\neq \Delta_0$ | Sum of the probability above $\|t_0\|$ and the probability below $-\|t_0\|$ | $t_0>t_{\alpha/2,n-1}$ or $t_0<-t_{\alpha/2,n-1}$ |
| $H_1:\mu_D> \Delta_0$ | Probability above $t_0$ | $t_0>t_{\alpha/2,n-1}$ |
| $H_1:\mu_D< \Delta_0$ | Probability below $t_0$ | $t_0<-t_{\alpha/2,n-1}$ |

### Paired v.s. Unpaired Comparisons

1. If the experimental units are relatively **homogeneous (small $\sigma$)** and the correlation within pairs is small, the gain in precision attributable to pairing will be offset by the loss of degrees of freedom, so an independent-sample experiment should be used.
2. If the experimental units are relatively **heterogeneous (large $\sigma$)** and there is large positive correlation within pairs, the paired experiment should be used. Typically, this case occurs when the experimental units are the same for both treatments.

### Confidence Interval

If $\bar d$ and $s_d$ are the sample mean and standard deviation, respectively, of the normally distributed difference of $n$ random pairs of measurements, a $100(1-\alpha\%)$ CI on the difference in means $\mu_D=\mu_1-\mu_2$ is

\begin{align}
\bar d-t_{\alpha/2,n-1}\frac{s_d}{\sqrt{n}}\leq \mu_D\leq \bar d+t_{\alpha/2,n-1}\frac{s_d}{\sqrt{n}}
\end{align}

where $t_{\alpha/2,n-1}$ is the upper $100(\alpha/2)$ percentage point of the $t$-distribution with $n-1$ degrees of freedom.

## The F Distribution

We wish to test the hypotheses:

\begin{align}
H_0:\sigma_1^2=\sigma_2^2\\
H_1:\sigma_1^2\neq\sigma_2^2
\end{align}

The development of a test procedure for these hypotheses requires a new probability distribution, the **F** distribution.

Let $W$ and $Y$ be **independent chi-square** random variables with $u$ and $v$ degrees of freedom, respectively. Then the ratio

\begin{align}
F&=\frac{W/u}{Y/v}=\frac{\chi_1^2/df_1}{\chi_2^2/df_2}\\
&=\frac{(\frac{(n_1-1)S_1^2}{\sigma_1^2})/(n_1-1)}{(\frac{(n_2-1)S_2^2}{\sigma_2^2})/(n_2-1)}=\frac{\sigma_2^2}{\sigma_1^2}\cdot\frac{S_1^2}{S_2^2}
\end{align}

has the probability density function

\begin{align}
f(x)=\frac{\Gamma(\frac{u+v}{2})(\frac uv)^{u/2}x^{(u/2)-1}}{\Gamma(\frac u2)\Gamma(\frac v2)((\frac uv)x+1)^{(u+v/2)}}
\end{align}

and is said to follow the $F$ distribution with $u$ degrees of freedom in the numerator and $v$ degrees of freedom in the denominator. It is usually abbreviated as $F_{u,v}$.


\begin{align}
f_{1-\alpha,u,v}=\frac{1}
{f_{\alpha,v,u}}\end{align}


### Hypothesis Testing

Let $X_{11},X_{12},...,X_{1n_1}$ be a random sample from a normal population with mean $\mu_1$ and variance $\sigma_1^2$, and let $X_{21},X_{22},...,X_{2n_2}$ be a random sample from a second normal population with mean $\mu_2$ and variance $\sigma_2^2$. Assume that both normal populations are independent. Let $S_1^2$ and $S_2^2$ be the sample variances. Then the ratio

\begin{align}
F=\frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2}
\end{align}

has an $F$ distribution with $n_1-1$ numerator degrees of freedom and $n_2-1$ denominator degrees of freedom.

Null hypothesis $H_0:\sigma_1^2=\sigma_2^2$

Test statistic: $\begin{align}F_0=\frac{S_1^2}{S_2^2}\end{align}$

| Alternative Hypotheses | Rejection Criterion |
| ------------------------------- | ------------------------------------------------------------------- |
| $H_1:\sigma_1^2\neq \sigma_2^2$ | $f_0>f_{\alpha/2,n_1-1,n_2-1}$ or $f_0<f_{1-\alpha/2,n_1-1,n_2-1}$ |
| $H_1:\sigma_1^2> \sigma_2^2$ | $f_0>f_{\alpha/2,n_1-1,n_2-1}$ |
| $H_1:\sigma_1^2< \sigma_2^2$ | $f_0<f_{1-\alpha/2,n_1-1,n_2-1}$ |


### Confidence Interval

If $s_1^2$ and $s_2^2$ are the sample variances of random samples of sizes $n_1$ and $n_2$ respectively, from two independent normal populations with unknown variances $\sigma_1^2$ and $\sigma_2^2$, a $100(1-\alpha)\%$ CI on the ratio $\sigma_1^2/\sigma_2^2$ is

\begin{align}
\frac{s_1^2}{s_2^2}f_{1-\alpha/2,n_2-1,n_1-1}\leq \frac{\sigma_1^2}{\sigma_2^2}\leq \frac{s_1^2}{s_2^2}f_{\alpha/2,n_2-1,n_1-1}
\end{align}

where $f_{\alpha/2,n_2-1,n_1-1}$ and $f_{1-\alpha/2,n_2-1,n_1-1}$ are the upper and lower $100\alpha/2$ percentage points of the $F$ distribution with $n_2-1$ numerator and $n_1-1$ denominator degrees of freedom, respectively.

## Two Binomial Proportions

\begin{align}H_0: p_1=p_2\\
H_1: p_1\neq p_2\end{align}

The quantity

\begin{align}
Z=\frac{\hat P_1-\hat P_2-(p_1-p_2)}{\sqrt{\frac{p_1(1-p_1)}{n_1}+\frac{p_2(1-p_2)}{n_2}}}
\end{align}

has approximately a standard normal distribution, $N(0,1)$.

$n_1\bar p_1\geq 15,\ n_1\bar q_1\geq 15,\ n_2\bar p_2\geq 15,\ n_2\bar q_2\geq 15$

$\begin{align}\hat p=\frac{x_1+x_2}{n_1+n_2}\end{align}$

\begin{align}
\sigma_{(\hat p_1-\hat p_2)}=\sqrt{\frac{p_1q_1}{n_1}+\frac{p_2q_2}{n_2}}\approx\sqrt{\hat p\hat q(\frac{1}{n_1}+\frac 1{n_2})}
\end{align}

### Hypothesis Testing

Null hypothesis $H_0: p_1=p_2$

Test statistic: $\begin{align}Z_0=\frac{\hat P_1-\hat P_2}{\sqrt{\hat P(1-\hat P)(\frac1{n_1}+\frac1{n_2})}}\end{align}$

| Alternative Hypotheses | P-Value | Rejection Criterion for Fixed-Level Tests |
| ---------------------- | ---------------------- | ----------------------------------------- |
| $H_1:p_1\neq p_2$ | $P=2(1-\Phi(\|z_0\|))$ | $z_0>z_{\alpha/2}$ or $z_0>-z_{\alpha/2}$ |
| $H_1:p_1>p_2$ | $P=1-\Phi(\|z_0\|)$ | $z_0>z_{\alpha/2}$ |
| $H_1:p_1<p_2$ | $P=\Phi(\|z_0\|)$ | $z_0>-z_{\alpha/2}$ |

### Confidence Interval

If $\hat p_1$ and $\hat p_2$ are the sample proportions of observation in two independent random samples of sizes $n_1$ and $n_2$ that belong to a class of interest, an approximate $100(1-\alpha\%)$ CI on the difference in the true proportions $p_1-p_2$ is

\begin{align}
\hat p_1-\hat p_2-z_{\alpha/2}\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}\leq p_1-p_2\leq \hat p_1-\hat p_2+z_{\alpha/2}\sqrt{\frac{\hat p_1(1-\hat p_1)}{n_1}+\frac{\hat p_2(1-\hat p_2)}{n_2}}
\end{align}

where $z_{\alpha/2}$ is thje upper $\alpha/2$ percentage point of the standard normal distribution.

## Completely Randomized Experiment


The levels of the factor are sometimes called **treatments**.
> 5% 10% 15% 20%

Each treatment has six observations or **replicates**.
> 6 observations per treatment, in total 30 observations

The runs are run in **random** order.

The experimental units (e.g. paper materials) are randomly assigned to the $k$ treatments, usually double blinding.
> Subjects are assumed homogeneous, with same variances.

**One factor** or independent variable (e.g. Hardwood concentration).
> Two or more treatment levels (e.g. 5%, 10%, 15%, 20%).

Analyzed by **one-way ANOVA**.

If the number of levels are two $\rightarrow$ two-sample hypothesis test (pooled t-test with equal variances)

## ANalysis Of VAriance

- Compares multiple types of variation to **test equality of means**
- Comparison basis is **ratio of variances**
- If **treatment variation** is significantly greater than **random variation** then **means are not equal**
$\begin{align}\frac{\text{Treatment variation}}{\text{Random variation}}>1\Rightarrow\end{align}$ at least one $u$ is different
- Variation measures are obtained by "partitioning" total variation

$Y_{ij}$ is a random variable denoting the $ij^{\text{th}}$ observation
$\mu$ is a parameter common to all treatments called the **overall mean**
$\tau_i$ is a parameter associated with the $i^{\text{th}}$ treatment called the $i^{\text{th}}$ treatment effect
$\epsilon_{ij}$ is a random error component

\begin{align}
&Y_{ij}=\mu+\tau_i+\epsilon_{ij}\begin{cases}
i=1,2,...,a\\
j=1,2,...,n
\end{cases}\\
&H_0: \tau_1=\tau_2=\cdot\cdot\cdot=\tau_a=0\\
&H_1:\exists\tau_i\neq 0
\end{align}


$\begin{align}
\sum_j^k\sum^{n_j}_i(X_{ij}-\overline{\overline X})^2=\sum^k_jn_j(\overline X_j-\overline{\overline X})^2+\sum_j^k\sum_i^{n_j}(X_{ij}-\overline X_j)^2
\end{align}$

$\Rightarrow SS(Total)=SST+SSE$

$SST=\text{TotalVar}\times (n-1)$

$SSE=\text{sum of Treatment Var}\times (n-1)$


$MST\geq MSE$

### Conditions for a Valid ANOVA F-Test

**Completely Randomized Design**

1. The samples are **randomly selected** in an **independent** manner from the **k treatment populations**.
- This can be accomplished by randomly assigning the experimental units to the treatments.
- Randomly assigned paper materials to different hardwood concentrations.
2. All k sampled populations have distributions that are approximately normal (reponse value).
- For each hardwood concentration groups, the tensile strength should be normally distributed.
3. **The k population variances are equal**

### Hypothesis Testing

$\begin{align}MS_{Treatments}=\frac{SS_{Treatments}}{a-1}\end{align}$

$\begin{align}E(MS_{Treatments})=\sigma^2+\frac{n\sum^a_{i=1}\tau_i^2}{a-1}\end{align}$

$\begin{align}MS_E=\frac{SS_E}{a(n-1)}\end{align}$

$\begin{align}E(MS_E)=\sigma^2\end{align}$

Null hypothesis $H_0: \tau_1=\tau_2=\cdot\cdot\cdot=\tau_a=0$
Alternative hypothesis $H_1:\exists \tau_i\neq 0$

Test statistic: $\begin{align}F_0=\frac{MS_{Treatments}}{MS_E}\end{align}$

P-value: Probability beyond $f_0$ in the $F_{a-1,a(n-1)}$ distribution

Rejection criterion for a fixed-level test: $f_0>f_{\alpha,a-1,a(n-1)}$



### Bonferroni method

New $\begin{align}\alpha=\frac{\alpha}{n}\end{align}$

$\begin{align}n=\text{number of comparisons}=c=\frac{k(k-1)}{2}\end{align}$

Adjusted critical value based on New $\alpha$

$S_P=\sqrt{MSE}$

Do Pooled t-test.

\begin{align}
(\bar X_i-\bar X_j)\pm t_{\alpha/2}\cdot S_p\sqrt{\frac1{n_i}+\frac1{n_j}}
\end{align}