---
# System prepended metadata

title: 'Experiment 1: Recommendation Level Comparison'
tags: [Finished Experiments]

---

---
tags: Finished Experiments
---

# Experiment 1: Recommendation Level Comparison

## Description
This experiment investigates how our choice of risk to recommendation mapping affects the performance of the transformer model.  When quantized, there are 16 possible risk levels (we number them 0 to 15, where 0 has the least estimated infectiousness and 15 the greatest). Recommendation levels are treated similarly, with the least restriction applied to rec level 0 and the most to rec level 3.

We select three maps: 
- [0, 0, 0] where any person with a non-zero risk is recommended to quarantine (the greatest restriction available)
- [0, 1, 2] where people who have a risk of 1 have a rec level of 1, risk level of 2 have a rec level of 2, and risk level 3 or above are recommended to quarantine.
- [1, 3, 5] is the default mapping. People with risks in bins 0 and 1 are mapped to rec level, risk levels 2 and 3 are mapped to rec level 1, risk levels 4 and 5 are mapped to rec level 2, and risk levels 6 and above are told to quarantine.

## Plots
### Pareto Adoption

![](https://i.imgur.com/Ww4mVu2.png)


### JellyBeans

![](https://i.imgur.com/shwlRUb.png)

### Pre-symptomatic curves

|  |  |  |
| -------- | -------- | -------- |
|   ![](https://i.imgur.com/DH6WyHu.png) |   ![](https://i.imgur.com/yoRn1be.png)|   ![](https://i.imgur.com/P0rfskM.png)  |


|  |  |  |
| -------- | -------- | -------- |
|  ![](https://i.imgur.com/R0f7fkS.png) | ![](https://i.imgur.com/3e74Th3.png) |  ![](https://i.imgur.com/er52Ssb.png)   |


## Analysis

[0,0,0] reduces the fraction of populate infected the most, but has the highest fraction of falsely put into quarantine (max 40%). If we look at the JellyBean plot we see that this model puts basically everyone in jail.

[0, 1, 2] has the next highest amount of false quarantines, and has (very roughly) comparable infection rate to heuristic and oracle. The transformer with [1, 3, 5] does worse at managing the disease and is roughly comparable to BDT1.



## Outcomes / Next Experiments
- we merged the refactor branch!
- Use Twilight Rain to generate another dataset for training. Which rec level? (we did this too)
- auto-generate MD when running plot scripts
    - upload images to imgur 
    - Create a "wiki" description for each plot that is linked to
- Squash columns to have presymptomatic, susceptible and delta as different colored bars