Journal.06.22.21
===

###### tags: `research` `point processes` `crime modeling`

:::info
- **Paper references:** _REU and PPP notes and diary_ (Gluck); _Point process modeling with spatiotemporal covariates for predicting crime_ (Reinhart)
- **Date:** June 22, 2021
:::

## Details
Understanding updates to triggering parameters.

Suppose have parameters and probability distribution from $\beta$ step.
Let $p_{ij} = P_{\beta}(u_i=j)$, $S_\beta = \sum_{i=2}^n\sum_{j=1}^{i-1} p_{ij}$, $T_i = T-t_i$. For this discussion, $t_1\le t_2\le \ldots \le t_n$.

Then, as Gluck notes, $c_{\beta+1} = \left(n - \sum_{i=1}^n e^{-T_i/{\omega_{\beta+1}}}\right)^{-1}S_\beta$.
The other equation (involving $c$ and $\omega$) is: $$\sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}(t_i-t_j-\omega_{\beta+1}) + c_{\beta+1}\sum_{i=1}^nT_ie^{-T_i/{\omega_{\beta+1}}} = 0.$$

If we also want to think about $\sigma_{\beta+1}^2$, the update to this is given by: $$\sigma_{\beta+1}^2 = \frac{\sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}|x_i - x_j|^2}{2S_\beta}$$

Let $D^2 = \min_{i\ne j}|x_i-x_j|^2$. We know how to bound $\sigma_{\beta+1}^2$ from below: $$\frac{\sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}|x_i - x_j|^2}{2S_\beta} \ge \frac{D^2S_\beta}{2S_\beta} = \dfrac{D^2}{2}.$$

And so, if $D^2 > 0$ then $\sigma_\beta^2$ does not tend to $0$.
:question: **Does D=0 imply that $\sigma_{\beta}^2 \to 0$ as $\beta\to\infty$?**

In similar fashion, let $\check\tau = \min_{i > j}(t_i - t_j)$. Now, we have $$c_{\beta+1}\sum_{i=1}^nT_ie^{-T_i/{\omega_{\beta+1}}} = \sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}(\omega_{\beta+1} - (t_i-t_j)) \le \sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}(\omega_{\beta+1} - \check\tau)$$

and, further, $$\sum_{i=2}^n\sum_{j=1}^{i-1}p_{ij}(\omega_{\beta+1} - \check\tau) = (\omega_{\beta+1}-\check\tau)S_\beta.$$

This tells us the following (with the very right hand side not being at all surprising). $$\omega_{\beta+1} \ge \frac{1}{S_\beta}\left(\check\tau S_\beta + c_{\beta+1}\sum_{i=1}^nT_ie^{-T_i/{\omega_{\beta+1}}}\right) \ge \check\tau$$

:question: **Is there any reason to believe that $\check\tau = 0$ would imply that $\omega_\beta\to 0$?**

:question: Suppose that, when $D = 0$, events $(x_a,t_a), (x_b,t_b)$ are such that $|x_b - x_a| = 0$ and $t_b > t_a$. Also, suppose that $(x_c,t_c)$ and $(x_d,t_d)$ are such that $|x_d-x_c|>0$ and $t_d > t_c$. Is it true that (other things being equal)

* decreasing $\sigma^2$ necessarily increases $P(u_b = a)$?
* decreasing $\sigma^2$ necessarily decreases $P(u_d=c)$?
* decreasing $\sigma^2$ necessarily increases the log likelihood function?

**First bullet:** Let $\delta = t_b - t_a$ and let $C = \frac{c}{2\pi\sigma^2\omega}$. Then $$P(u_b = a) = \frac{g(\vec{0}, \delta)}{\lambda(x_b,t_b)} = \frac{e^{-\delta/\omega}}{\frac{1}{C}\mu(x_b)+\frac{1}{C}\sum_{\mathcal H_{t_b}}g(x_b-x,t_b-t)}.$$

Since $\frac{1}{C} = \frac{2\pi\sigma^2\omega}{c}$ decreases when $\sigma^2$ decreases (and all else stays constant), the term $\frac{1}{C}\mu(x_b)$ decreases.

Now, let $(x,t)\in\mathcal H_{t_b}$.
* Since $\frac{1}{C}g(x_b-x, t_b-t) = e^{-\frac{t_b-t}{\omega}}e^{-\frac{|x_b-x|^2}{2\sigma^2}}$, and decreasing $\sigma^2$ will either decrease $e^{-\frac{|x_b-x|^2}{2\sigma^2}}$, if $|x_b-x|>0$, or will leave it unchanged if $|x_b-x|=0$.

This means that $\frac{1}{C}\sum_{\mathcal H_{t_b}}g(x_b-x,t_b-t)$ is a sum of positive terms, each of which decreases or stays the same on decreasing $\sigma^2$. As a consequence, the full denominator $$\frac{1}{C}\mu(x_b)+\frac{1}{C}\sum_{\mathcal H_{t_b}}g(x_b-x,t_b-t)$$
must decrease upon decreasing $\sigma^2$, and so $P(u_b=a)$ must increase.

**Second bullet:** This statement is not true generally. For example, try running the following code:

import numpy as np
# squared distances between focus event and events in its history
d_sqs = np.array([0.001,0.002, 0.3, 0.35, 0.4, 1.2])

# time differences to events in its history
t_diffs = np.array([0.1, 0.113,0.096,0.121, 0.105,0.093])

# background intensity, and starting triggering params
a = 1
sig_sq = 0.05
omega=1
c=0.5

# If the focus event is event i, function that computes P(u_i = j)
def p(j):
    denom = ((2*np.pi*sig_sq*omega)/c)*a + np.sum(np.exp(-t_diffs/omega)*np.exp(-d_sqs/(2*sig_sq)))
    return np.exp(-t_diffs[j]/omega)*np.exp(-d_sqs[j]/(2*sig_sq))/denom

print([p(j) for j in range(6)])
# print successive probabilities with sig_sq being halved each time
for i in range(5):
    sig_sq *= 0.5
    print([p(j) for j in range(6)])

However: Note, a way to think about change from $\sigma^2$ to $\sigma_{new}^2$: let $r = \sigma^2/\sigma_{new}^2 > 1$, then $$e^{\frac{-(t_i-t_j)}{\omega}}\left(e^{\frac{-|x_i-x_j|^2}{2\sigma^2}}\right)^{r} = e^{\frac{-(t_i-t_j)}{\omega}}e^{\frac{-|x_i-x_j|^2}{2\sigma_{new}^2}}$$
and so, by setting $c_{ij} = e^{\frac{-(t_i-t_j)}{\omega}}$ and $p_{ij}=e^{\frac{-|x_i-x_j|^2}{2\sigma^2}}$, and noting that the new value of $C$ will be $rC$, you have that the new value of $P(u_i=j)$ after changing to $\sigma_{new}^2$ is
$$\dfrac{c_{ij}p_{ij}^{r}}{\frac{1}{rC}\mu(x_i) + {\displaystyle\sum_{k:(x_k,t_k)\in\mathcal H_{t_i}}}c_{ik}p_{ik}^r}$$

> Lemma. Let $p_1,p_2,\ldots,p_n$ and $c_1,c_2,\ldots,c_n$ be positive numbers, and $m$ a non-negative number. Also, suppose that $r > 1$. Then there exists a unique $x>0$, with $\min\{p_k\} < x$, so that $$\frac{x}{m+\sum_kc_kp_k} = \frac{x^r}{\frac{1}{r}m + \sum_kc_kp_k^r}.$$
Furthermore if, for some $j$, $x < p_j$, then you have the following inequality (and if $x > p_j$, then the inequality is reversed) $$\frac{p_j}{m+\sum_kc_kp_k} < \frac{p_j^r}{\frac{1}{r}m + \sum_kc_kp_k^r}.$$

This applies by thinking (for fixed $i$) of each $p_{ik}$ as one of the $p_k$, each $c_{ik}$ as one of the $c_k$, and $m = \frac{1}{C}\mu(x_i)$ (note you can multiply both sides of the inequality by $c_j$). Then, the Lemma says that some of the values of $P(u_i=j)$ will decrease when $\sigma^2$ decreases, and if some of the values of $p_{ij}$ are larger than $x$, then for such $j$ the value of $P(u_i=j)$ will increase.
> Remark: the value of $x$ is dependent on values of $p_1,p_2,\ldots,p_n$, among other things.
> Remark: $x$ satisfies $$x^{r-1} = \frac{\frac{1}{r}m + \sum_kc_kp_k^r}{m+\sum_kc_kp_k}$$
and when every $p_k\le 1$, it is easy to see that $x < 1$.

Since $p_{ij} \le 1$ for every $i,j$, we see a repeat of the first bullet being true. Also, note that as $\sigma^2\to 0$, the only way for $p_{ij}$ not to go to zero with it is if $|x_i - x_j| = 0$, in which case $p_{ij} = 1$.

:star2: An analogous argument can be made to understand how the $P(u_i=j)$ change as $\omega \to 0$, but other parameters are fixed.

**Third bullet:** The log-likelihood function is given by $$\sum_{i}\log(\lambda(x_i,t_i)) - \int_0^T\int_{\mathcal R}\lambda(x,t)\ dxdt.$$

For each $i$, the term $\lambda(x_i,t_i)$ is given by $$\mu(x_i) + C\sum_{k:(x_k,t_k)\in\mathcal H_{t_i}}c_{ik}p_{ik}.$$

For any $(x_k,t_k)\in\mathcal H_{t_i}$ so that $|x_i-x_k|=0$, we have $p_{ik}=1$. Let $N$ be the number of such points in $\mathcal H_{t_i}$, and let $c_{min}$ be the minimum $c_{ik}$ from among _those_ $N$ points. Then $$\lambda(x_i,t_i) \ge \mu(x_i) + C(Nc_{min}).$$

When $\sigma^2$ is changed to $\sigma_{new}^2$ then (using $r$ as defined above) this changes $C$ to $rC$. Hence, a decreasing infinite sequence of values for $\sigma^2$, which makes $r > 1$ at each change, will cause $\lambda(x_i,t_i) \to \infty$ if $N > 0$.
> Based on the types of values we get for these from our data, it is probably possible, but rare, that something like $\sigma^2 \mapsto \sigma_{new}^2 = 0.5\sigma^2$ would cause $\lambda(x_i,t_i)$ to decrease.

From the integral term, we integrate over $\mathcal R = \mathbb R^2$ as an approximation, and this causes the integral term to be independent of $\sigma^2$. Even if we did not approximate this way, the integral term would have rather insignificant change (and that integral is bounded above, so once $\sigma^2$ is small enough, this would not keep log-likelihood from increasing).

## Aside: things that can go wrong with solving for $c$ and $\omega$
**Extreme case: Two points $x_1, x_2$ with $t_2-t_1 = 1$ and $D=0$.**

_Assume $T=t_2$_.
Start with $\sigma_0^2=1$, $\omega_0 = 1$, and $c_0 = 0.5$. Now: $$P_0(u_2=1) = \frac{g(x_2-x_1,t_2-t_1)}{\lambda(x_2,t_2 | \mathcal H_{t_2})} = \dfrac{g(0,1)}{\mu(x_2)+g(0,1)} = S_0.$$

We get $\sigma_1^2 = 0$, and need $c_1, \omega_1$ to satisfy:

$$c_{1} = \left(2 - \sum_{i=1}^2 e^{-T_i/{\omega_{1}}}\right)^{-1}S_0$$
$$p_{21}(t_2-t_1-\omega_{1}) + c_{1}\sum_{i=1}^2T_ie^{-T_i/{\omega_{1}}} = 0$$

Now, $\sum_{i=1}^2 e^{-T_i/{\omega_{1}}} = e^{-1/\omega_1} + 1$. So the first equation is $c_1 = \left(1-e^{-1/\omega_1}\right)^{-1}S_0$. The second equation is $S_0(1-\omega_1)+c_1e^{-1/\omega_1}=0$. Rearranging:

$$S_0 = c_1 - c_1e^{-1/\omega_1} = c_1 + S_0(1-\omega_1)$$

And so, $c_1 = S_0\omega_1$, and using this, you get: $e^{-1/\omega_1} = 1-\frac{1}{\omega_1}$. This has no solution (the difference $e^{-1/\omega_1} - (1-\frac{1}{\omega_1})$ monotonically approaches zero as $\omega_1\to\infty$).