---
# System prepended metadata

title: Maximum likelihood for GLM
tags: [Categorical data analysis]

---

# Maximum likelihood for GLM
###### tags: `Categorical data analysis`

## Fisher Information

:::success
<font size = 5.5>
    
Let $L(\theta) = lnf(x;\theta)$ be a log likelihood function, then

$I(\theta) = E((\frac{\partial L(\theta)}{\partial \theta})^2) = Var(\frac{\partial lnf(x;\theta)}{\partial \theta})$
is called the fisher imformation    
    
</font>
:::

It can be shown that $E(\frac{\partial L(\theta)}{\partial \theta}) = 0$
![](https://i.imgur.com/NUOxNpZ.jpg)

## Likelihood for GLM
通常GLM假設observer來自exponential (dispersion) family, 即分配可以被表成以下形式

:::success
<font size = 5.5>
    
$f(x, \theta, \phi) = e^{\frac{x\theta - b(\theta)}{a(\phi)} + c(x, \phi)}$    
    
    
</font>
:::

可得出對於一個來自exponential family的random variable $X$ 的平均跟變異數

:::success
<font size = 5.5>
    
Let $L(\theta) = lnf(x, \theta, \phi)$
Then $E(\frac{\partial L(\theta)}{\partial \theta}) = 0$
$E(\frac{\partial L(\theta)}{\partial \theta}) = E(\frac{X - b^{'}(\theta)}{a(\phi)}) = 0$
$\therefore E(X) = b^{'}(\theta)$
    
$Var(\frac{\partial L(\theta)}{\partial \theta}) = Var(\frac{X - b^{'}(\theta)}{a(\phi)}) = b^{''}(\theta)$
$\therefore Var(X) = (a(\phi))^2b^{''}(\theta)$
    
</font>
:::


---

對於n個observer的GLM的likelihood function可被表成
![](https://i.imgur.com/7oxFAKb.jpg)

其中定義linear predictor為
![](https://i.imgur.com/qqW21CN.jpg)

要 maximize $L(\beta)$ 在每個 $\beta_j$ 下，即對其偏微
![](https://i.imgur.com/SZG3y11.jpg)

![](https://i.imgur.com/kjoNidN.jpg)

由前面的運算得知

:::success
<font size = 5.5>
    
$\frac{\partial L}{\partial \theta} = \frac{x - b^{'}(\theta)}{a(\phi)}$    


---
    
$\mu = b^{'}(\theta)$
$\therefore \frac{\partial \theta}{\partial \mu} = b^{''}(\theta) = (a(\phi))^2Var(X)$

    

---
$\frac{\partial \eta}{\partial \beta} = x$
    

    
</font>
:::

因此上式可寫為
![](https://i.imgur.com/to8aw06.jpg)









