# [5/10/2022] Comparing Simulation Environment Base Models
## Base Models
1. Hurdle Model with Square Root Transform:
$$Z \sim Bern(1 - \sigma(x^Tw_b))$$
$$Y \sim \mathcal{N}(x^Tw_{\mu}, \sigma_u^2)$$
$$R = Z * Y^2$$

We're fitting parameters $\{w_b, w_{\mu}, \sigma_u\}$.

2. Hurdle Model with Log Transform:
$$Z \sim Bern(1 - \sigma(x^Tw_b))$$
$$Y \sim \mathcal{N}(x^Tw_{\mu}, \sigma_u^2)$$
$$R = Z * \exp(Y)$$

We're fitting parameters $\{w_b, w_{\mu}, \sigma_u\}$.

3. 0-Inflated Poisson Model:
$$Z \sim Bern(1 - \sigma(x^Tw_b))$$
$$Y \sim \text{Poisson}(\exp(x^Tw_p))$$
$$R = Z * Y$$

We're fitting parameters $\{w_b, w_{\mu}, \sigma_u\}$.

## State Space
### Stationary State Space
* 0 - Time of Day
* 1 - Prior Day Total Brush Time
* 2 - Prop. Non-Zero Brushing In Past 7 Days
* 3 - Weekday vs. Weekend
* 4 - Bias

### Non-Stationary State Space
* 0 - Time of Day
* 1 - Prior Day Total Brush Time
* 2 - Day In Study
* 3 - Prop. Non-Zero Brushing In Past 7 Days
* 4 - Weekday vs. Weekend
* 5 - Bias

## Fitting the Hurdle Model
We fit one model per user. To fit the hurdle model, we fit the Bernoulli component and the Normal component separately. To fit the Bernoulli component, we take all user's states $X$ and transform that user's rewards to be an indicator $\mathbb{I}[R > 0]$ and fit the Bernoulli component.

We then take all user's states $X$ and transform that user's rewards to be $Y = \sqrt{R}$ and fit the Normal component. We put $\mathcal{N}(0, 1)$ priors on all parameters and fit optimize the MAP with random restarts.

## Quality Checks For Stationary Env.
### 1. Histogram
ROBAS 2 Brush Times | Square Root
:-------------------------:|:-------------------------:
 | 
**Log Transform** | **0-Inflated Poisson**
 | 

### 2. Metrics
Metric | ROBAS 2 | Stationary Env.
| :-------------------------:|:-------------------------:|:-------------------------:
Average # of Zeros:| 21.094 | 22.625
Variance of # of Zeros:| 164.522 | 151.234
Proportion of # of Sessions with Zero Values: | 0.376|0.404
Mean of Total Non Zero Rewards: | 137.768 |113.634
Variance of Total Non Zero Rewards: | 2326.518 |2428.599
Skewness of Total Non Zero Rewards: | 1.399 |0.791
Kurtosis of Total Non Zero Rewards: | 3.879 |2.885
Variance of Means For Each User: | 1415.920 | 853.778
Mean of Variances of Rewards For Each User: | 1160.723 |2135.073
Mean of Skewness of Rewards For Each User: | -0.692 |0.511
Mean of Kurtosis of Rewards For Each User: | 3.857 |0.133

### 3. QQ Plots
Base Model | QQ Plots # of 0s | Rewards (Including 0)
| :-------------------------:|:-------------------------:|:-------------------------:
|Square Root |  | 
|Log Transform | |
|0-Inflated Poisson | |

### 4. Capturing Variance In Non_Zero Rewards of the data
1. Calculate $\mathbb{E}[R_t | X_t], std[R_t | X_t]$ of the model. Then measure:

$$
S_i := \frac{1}{T} \sum_{t = 1}^T \bigg(\frac{R_t - \mathbb{E}[R_t | X_t]}{std[R_t | X_t]}\bigg)^2
$$
for each user model $i$.

2. Then calculate the mean and variance of the statistic $S_i$ across all $i$ users.

**Kelly said if this statistic is close to 1, then we captured the variance well. Problem: I have one or two users with ridiculous values such as user P2_033. Do you know what this statistic is called and if so, is it a problem that we get such unreasonable values for one or two users but everyone else is reasonable?**

***************
I don't know a name.
Suppose you got a value of S_i=s* for a user and s* seems unreasonably large. To determine if a value is unreasonable, you could do the following:
Do 1000 times: 1. Generate trajectory of rewards for the N users assuming the hurdle model and the model for E[R_t|X_t], std[R_t|X_t] is correct (use the same X_t's as in the real data for the N users. )
2. form S_i for i=1,...,N
3) document how many users get an S_i>s*.
end do
document fraction of the 1000 runs in which you get one or more S_i>s*
*****************

FOR USER: P2_012
Statistic: 1.5474141158000643
FOR USER: P2_013
Statistic: 1.2035121725202518
FOR USER: P2_014
Statistic: 2.010419605987785
FOR USER: P2_017
Statistic: 1.519787049282806
FOR USER: P2_019
Statistic: 1.2576562400598
FOR USER: P2_020
Statistic: 1.0318542367167127
FOR USER: P2_021
Statistic: 1.9295477499470486
FOR USER: P2_022
Statistic: 1.0285546118881828
FOR USER: P2_024
Statistic: 1.1570957476388875
FOR USER: P2_025
Statistic: 1.183322754227782
FOR USER: P2_027
Statistic: 1.2375494088547698
FOR USER: P2_028
Statistic: 2.5585084740670307
FOR USER: P2_029
Statistic: 3.9032806228203962
FOR USER: P2_030
Statistic: 4.6369498052050115
FOR USER: P2_031
Statistic: 2.043906272126524
FOR USER: P2_033
Statistic: 56923.35425189367
FOR USER: P2_034
Statistic: 1.618722858003212
FOR USER: P2_035
Statistic: 0.8981011167854823
FOR USER: P2_040
Statistic: 0.7442899794933642
FOR USER: P2_041
Statistic: 2.3848942790055494
FOR USER: P2_042
Statistic: 256.12704393551206
FOR USER: P2_043
Statistic: 1.291315529722448
FOR USER: P2_044
Statistic: 4.2304592174830535
FOR USER: P2_045
Statistic: 2.0980911343377784
FOR USER: P2_046
Statistic: 0.6995063512749301
FOR USER: P2_047
Statistic: 1.0582442900522062
FOR USER: P2_048
Statistic: 2.781786606616837
FOR USER: P2_050
Statistic: 2.495337349742374
FOR USER: P2_051
Statistic: 1.7202751037151691
FOR USER: P2_052
Statistic: 3.3647712956338056
FOR USER: P2_053
Statistic: 3.7743978674385925
FOR USER: P2_056
Statistic: 15.528532448052324