# Nonparametric Testing

> @article{kallenberg1982chernoff,
title={Chernoff efficiency and deficiency},
author={Kallenberg, Wilbert CM},
journal={The Annals of Statistics},
pages={583--594},
year={1982},
publisher={JSTOR}}

Let $X_{1}, X_{2}, \ldots$ be a sequence of i.i.d. random variables (r.v.'s), each defined on $\mathscr{X}$ and distributed according to $P_{\theta}, \theta \in \Theta .$

The probability distribution of $S=\left(X_{1}, X_{2}, \cdots\right)$ is denoted by $\mathbb{P}_{\theta}$.

Suppose the hypothesis $H_{o}: \theta \in \Theta_{0}$ has to be tested against $H_{1}: \theta \in \Theta_{1}=\Theta-\Theta_{0}$ on the basis of the observations $X_{1}, \ldots, X_{n}, n$ $\in \mathbb{N},$ where $\Theta_{0} \subset \Theta .$

### 這裡開始testing rule.
Let $\left\{\varphi_{n ; \alpha} ; n \in \mathbb{N}, 0 \leq \alpha \leq 1\right\}$ be a family of (randomized) tests based on $X_{1}, \ldots, X_{n} ;$ i.e. for each $n \in \mathbb{N}, \alpha \in[0,1] \varphi_{n ; \alpha}$ is a measurable function of $X_{1}, \cdots, X_{n}$ with values in [0,1] such that
(2.1)
$$
\sup _{\theta_{0} \in \Theta_{0}} E_{\theta_{0}} \varphi_{n ; \alpha}(S) \leq \alpha
$$
In many cases the test $\varphi_{n ; \alpha}$ will have exactly size $\alpha .$ For $\alpha \in[0,1]$ and $\theta \in \Theta_{1}$ define
(2.2)
$$
\rho_{n}^{\varphi}(\alpha, \theta)=\max \left\{\alpha, 1-E_{\theta} \varphi_{n ; \alpha}(S)\right\}
$$
(2.3)
$$
\rho_{n}^{\varphi}(\theta)=\inf _{\alpha \in[0,1]} \rho_{n}^{\varphi}(\alpha, \theta)
$$
For many families of tests the limit
(2.4)
$$
\rho^{\varphi}(\theta)=-\lim _{n \rightarrow \infty} n^{-1} \log \rho_{n}^{\varphi}(\theta)
$$
exists for all $\theta \in \Theta_{1} ; \rho^{\varphi}(\theta)$ is called the Chernoff index of the family. For the likelihood ratio (LR) and the most powerful (MP) test we use the notations $\rho_{n}^{\mathrm{LR}}(\alpha, \theta), \rho^{\mathrm{LR}}(\theta),$ etc.

If $\left\{\tilde{\varphi}_{n ; \alpha}\right\}$ is another family of tests, the Chernoff efficiency of $\left\{\varphi_{n ; \alpha}\right\}$ with respect to $\left\{\tilde{\varphi}_{n ; \alpha}\right\}$ is defined by
(2.5)
$$
e_{\varphi, \tilde{\varphi}}^{C}(\theta)=\rho^{\varphi}(\theta) / \rho^{\tilde{\varphi}}(\theta)
$$
If $e_{\varphi, \tilde{\varphi}}^{C}(\theta) \geq 1$ for all families $\left\{\tilde{\varphi}_{n ; \alpha}\right\},$ then the family $\left\{\varphi_{n ; \alpha}\right\}$ is called efficient in the sense of Chernoff or simply Chernoff efficient at $\theta$. For a given family $\left\{\varphi_{n ; \alpha}\right\}$ we define
(2.6)
$$
N^{\varphi}(\alpha, \theta)=\min \left\{n ; 1-E_{\theta} \varphi_{m ; \alpha}(S) \leq \alpha \text { for all } \quad m \geq n\right\}
$$
i.e. the minimal required sample size of a level- $\alpha$ test with probability of error of the second kind at most $\alpha$ at $\theta$. An immediate consequence of (2.3) and (2.6) is
(2.7)
$$
\rho_{N^{\varphi}(\alpha, \theta)}^{\varphi}(\theta) \leq \alpha
$$
Moreover, if the family of tests satisfies
(2.8)
$$
\alpha<\alpha^{\prime} \Rightarrow E_{\theta} \varphi_{n ; \alpha}(S) \leq E_{\theta} \varphi_{n ; \alpha^{\prime}}(S), \quad n=1,2, \cdots,
$$
then
(2.9)
$$
\alpha \leq \rho_{N^{\varphi}(\alpha, \theta)-1}(\theta)
$$

### h3

In the rest of this section we assume that the observations are distributed according to a $k$ -parameter exponential family. Hence the distribution of $X_{i}$ is given by
(2.15)
$$
d P_{\theta}(x)=\exp \left\{\theta^{\prime} x-\psi(\theta)\right\} d \mu(x), \quad \theta \in \Theta \subset \mathbb{R}^{k}, \quad x \in \mathbb{R}^{k}
$$
where $\mu$ is a $\sigma$ -finite non-degenerate measure, $\Theta$ denotes the natural parameter space, i.e. $\theta=\left\{\dot{\theta} \in \mathbb{R}^{k} ; \int \exp \left(\theta^{\prime} x\right) d \mu(x)<\infty\right\},$ and
$$
\psi(\theta)=\log \int \exp \left(\theta^{\prime} x\right) d \mu(x), \quad \theta \in \Theta
$$
Here $\theta^{\prime} x$ denotes the inner product of $\theta$ and $x$. It is well known that $\Theta$ is a convex set in $\mathbb{R}^{k}$ and we assume that it has a non-empty interior. Without loss of generality assume that $\mu$ is not supported on a flat and that $0 \in \Theta .$ Let $\Theta^{*}=\left\{\theta \in \Theta ; E_{\theta}\left\|X_{i}\right\|<\infty\right\} .$ Note that int $\Theta \subset \Theta^{*} \subset \Theta .$ For $\theta \in \Theta^{*}$ define
$$
\lambda(\theta)=E_{\theta} X_{i}
$$
The mapping $\lambda$ is $1-1$ on $\Theta^{*}$. Defining $\Lambda=\lambda\left(\theta^{*}\right)=\left\{\lambda(\theta) ; \theta \in \Theta^{*}\right\},$ the inverse mapping $\lambda^{-1}$ exists on $\Lambda .$ Note that $\lambda(\theta)=\operatorname{grad} \psi(\theta)$ if $\theta \in$ int $\Theta .$ Moreover, for $\theta \in$ int $\Theta,$ the convariance matrix $\Sigma_{\theta}$ of $X_{i}$ is the Hessian of $\psi$. The Kullback-Leibler information number of $P_{\theta}$ with respect to $P_{\theta_{0}}$ is defined by
$$
I\left(\theta, \theta_{0}\right)=E_{\theta} \log d P_{\theta} / d P_{\theta_{0}}\left(X_{i}\right)=\psi\left(\theta_{0}\right)-\psi(\theta)+\left(\theta-\theta_{0}\right)^{\prime} \lambda(\theta)
$$
where $\theta \in \Theta^{*}$ and $\theta_{0} \in \Theta .$ There is an intimate relationship between the functions $M$ and $I$

## 直接算sample mean的Legendra transform?

Defining
$$
L(x)=\sup _{\theta \in \Theta}\left\{\theta^{\prime} x-\psi(\theta)\right\}
$$
the size- $\alpha$ LR test of $H_{0}: \theta=0$ against $H_{1}: \theta \neq 0$ is given by
$$
\varphi_{n ; \alpha}^{\mathrm{LR}}(S)=\left\{\begin{array}{ll}
1 & L\left(\bar{X}_{n}\right)>d_{n ; \alpha}, \\
\delta_{n ; \alpha} & L\left(\bar{X}_{n}\right)=d_{n ; \alpha} \\
0 & L\left(\bar{X}_{n}\right)<d_{n ; \alpha}
\end{array}\right.
$$
where $\bar{X}_{n}=n^{-1} \sum_{i=1}^{n} X_{i}$
$$
d_{n ; \alpha}=\inf \left\{d ; \sup _{\theta_{\theta} \in \Theta_{0}} \mathbb{P}_{\theta_{0}}\left(L\left(\bar{X}_{n}\right)>d\right) \leq \alpha\right\}
$$
and
$$
\delta_{n ; \alpha}=\sup \left\{\delta \in[0,1] ; \sup _{\theta_{0} \in \Theta_{0}} E_{\theta_{0}} \varphi_{n ; \alpha}^{\mathrm{LR}}(S) \leq \alpha\right\}
$$
The distribution of $\bar{X}_{n}$ is denoted by $\bar{P}_{\theta}^{n}$