# Maximum likelihood for GLM
###### tags: `Categorical data analysis`

## Fisher Information

:::success
<font size = 5.5>

Let $L(\theta) = lnf(x;\theta)$ be a log likelihood function, then

$I(\theta) = E((\frac{\partial L(\theta)}{\partial \theta})^2) = Var(\frac{\partial lnf(x;\theta)}{\partial \theta})$
is called the fisher imformation

</font>
:::

It can be shown that $E(\frac{\partial L(\theta)}{\partial \theta}) = 0$

## Likelihood for GLM
通常GLM假設observer來自exponential (dispersion) family, 即分配可以被表成以下形式

:::success
<font size = 5.5>

$f(x, \theta, \phi) = e^{\frac{x\theta - b(\theta)}{a(\phi)} + c(x, \phi)}$

</font>
:::

可得出對於一個來自exponential family的random variable $X$ 的平均跟變異數

:::success
<font size = 5.5>

Let $L(\theta) = lnf(x, \theta, \phi)$
Then $E(\frac{\partial L(\theta)}{\partial \theta}) = 0$
$E(\frac{\partial L(\theta)}{\partial \theta}) = E(\frac{X - b^{'}(\theta)}{a(\phi)}) = 0$
$\therefore E(X) = b^{'}(\theta)$

$Var(\frac{\partial L(\theta)}{\partial \theta}) = Var(\frac{X - b^{'}(\theta)}{a(\phi)}) = b^{''}(\theta)$
$\therefore Var(X) = (a(\phi))^2b^{''}(\theta)$

</font>
:::

---

對於n個observer的GLM的likelihood function可被表成

其中定義linear predictor為

要 maximize $L(\beta)$ 在每個 $\beta_j$ 下，即對其偏微


由前面的運算得知

:::success
<font size = 5.5>

$\frac{\partial L}{\partial \theta} = \frac{x - b^{'}(\theta)}{a(\phi)}$

---

$\mu = b^{'}(\theta)$
$\therefore \frac{\partial \theta}{\partial \mu} = b^{''}(\theta) = (a(\phi))^2Var(X)$

---
$\frac{\partial \eta}{\partial \beta} = x$

</font>
:::

因此上式可寫為