---
# System prepended metadata

title: Heuristic Debug Rev 4
tags: [Debug]

---

---
tags: Debug
---

# Heuristic Debug Rev 4

### Description
This experiment is run on simulator [version 9009e1310](https://github.com/mila-iqia/covi-simulator/commit/9009e13102d360e800bfadde583cc983519046c8). This contains Prateek's mobility changes and updates to the heuristic (adding heuristicv3). We run debugging plots (infection chains, baseball card plots, star plots) on 3 runs, and do aggregate statistics including a pareto front calculation with varying Global Mobility.

#### Useful links
- [googledoc for a high-level description of the heuristic (v1)](https://docs.google.com/document/d/1_J1jcn5b0h7h5_bzjH-qx4LerWNFRJOzBI2vCm1n3AA/edit?usp=sharing)
- [plot wiki](https://hackmd.io/N-g61aZLQaSB_cz5SttM6Q) for information on how to read the plots.


## todo:
- Run rev 5
	-  merge prateek's changes
	-  add bdt2 
	-  run pareto front ranging GLOBAL_MOBILITY_SCALING_FACTOR from 0.01 to 0.3
-  re-tune heuristic v3
-  fix tests

## Part 1: Aggregate Statistics

### Pareto Front
Observations:
these metrics don't make it clear that any method is better than any of the other methods. One thing to try would be to re-add bdt2 and the transformer to see how an aggressive method looks. 

![](https://i.imgur.com/53Kzlk0.png)


### Pareto Adoption
#### y-axis = Proxy Rt
![](https://i.imgur.com/PU1cCck.png)
#### y-axis = estimated Rt
![](https://i.imgur.com/Q39YjTX.png)



### JellyBeans


![](https://i.imgur.com/bBv2fce.png)
![](https://i.imgur.com/KtQATCi.png)
![](https://i.imgur.com/DYxcD13.png)

### Pre-symptomatic curves
![](https://i.imgur.com/V8ZutJT.png)
![](https://i.imgur.com/Q8Qeb0K.png)
![](https://i.imgur.com/NChxgzJ.png)

### Generation Times
![](https://i.imgur.com/icXF0qF.png)

## Part 2: HeuristicV1 Debug Plots

### Star plot
![](https://i.imgur.com/Kc6TJGI.png)


### Infection Chains
![](https://i.imgur.com/Py1AZyd.png)
![](https://i.imgur.com/EPayn7e.png)
![](https://i.imgur.com/O9mwpM9.png)
![](https://i.imgur.com/r9aSXiB.png)
![](https://i.imgur.com/5hmhgiu.png)
![](https://i.imgur.com/FNRxUd8.png)
![](https://i.imgur.com/lmm6lLX.png)
![](https://i.imgur.com/yplC5Py.png)
![](https://i.imgur.com/HNKb77E.png)
![](https://i.imgur.com/YJMTMxQ.png)



### Baseball Cards
![](https://i.imgur.com/5VSaU7U.png)
![](https://i.imgur.com/BjMzYUT.png)
![](https://i.imgur.com/CDsP02O.png)
![](https://i.imgur.com/mnp5I60.png)
![](https://i.imgur.com/lusHx1a.png)
![](https://i.imgur.com/ephCWXo.png)
![](https://i.imgur.com/srj1mRA.png)
![](https://i.imgur.com/G20f5oy.png)
![](https://i.imgur.com/HSSy4aa.png)
![](https://i.imgur.com/Rlkqmni.png)
![](https://i.imgur.com/RMLSUph.png)
![](https://i.imgur.com/RtbGQNX.png)
![](https://i.imgur.com/utcYsxH.png)
![](https://i.imgur.com/7aXi3W7.png)
![](https://i.imgur.com/Daejesb.png)
![](https://i.imgur.com/F0l8PX3.png)
![](https://i.imgur.com/hhFvNUE.png)
![](https://i.imgur.com/ahnVV2L.png)
![](https://i.imgur.com/9CrUpL8.png)
![](https://i.imgur.com/dVnUJnZ.png)
![](https://i.imgur.com/077c4pX.png)
![](https://i.imgur.com/zZSFjvy.png)
![](https://i.imgur.com/lm84F22.png)
![](https://i.imgur.com/o2Etlwu.png)
![](https://i.imgur.com/5aMQSBP.png)
![](https://i.imgur.com/V7PKN2p.png)
![](https://i.imgur.com/Xcw2UOv.png)
![](https://i.imgur.com/wXGinjy.png)
![](https://i.imgur.com/kMsB0X4.png)
![](https://i.imgur.com/qPU3RE2.png)
![](https://i.imgur.com/31gATLz.png)


## BDT1 Debug Plots

### Star Plot

![](https://i.imgur.com/AzEpcxN.png)

### Infection Chains
Weird behaviors - 
1. Message received is 15 but rec level is still green for human:85 and many others
2. human:93 is Asymptomatic and is still showing symptoms. Cold or allergies?
3. CAUTION / WARNING - Quarantining for BDT was bugged. It was not quarantining people. This could only put more people in jail but it will also improve percent infections and R for BDT. It doesn't affect rec levels. 
4. However, 1 above is still unexplained because the color in the chains are dependent upon rec levels.
![](https://i.imgur.com/Me6F0LU.png)
![](https://i.imgur.com/VaTwD6p.png)
![](https://i.imgur.com/eJ2lwRR.png)
![](https://i.imgur.com/MHfBJdQ.png)
![](https://i.imgur.com/2h6i3LY.png)
![](https://i.imgur.com/uYpHi3z.png)
![](https://i.imgur.com/NZByH3S.png)
![](https://i.imgur.com/IV4mdsj.png)
![](https://i.imgur.com/btdbXeq.png)
![](https://i.imgur.com/eXppVBH.png)

### Baseball Cards

![](https://i.imgur.com/1Xwr1hS.png)
![](https://i.imgur.com/IA7HFLb.png)
![](https://i.imgur.com/t7ljGQW.png)
![](https://i.imgur.com/1bFevfv.png)
![](https://i.imgur.com/hZf55AE.png)
![](https://i.imgur.com/gaus6Yt.png)
![](https://i.imgur.com/d19o53c.png)
![](https://i.imgur.com/58TiNaD.png)
![](https://i.imgur.com/rjp9fBx.png)
![](https://i.imgur.com/kSnKtYj.png)
![](https://i.imgur.com/NWNhqHZ.png)
![](https://i.imgur.com/C15kIQj.png)
![](https://i.imgur.com/Js2YUDp.png)
![](https://i.imgur.com/EWHtcgU.png)
![](https://i.imgur.com/RghhLcC.png)
![Uploading file..._fcldenf8z]()
![](https://i.imgur.com/nZweAJs.png)
![](https://i.imgur.com/h7APD9h.png)
![](https://i.imgur.com/w8kkVC0.png)
![](https://i.imgur.com/fWtw5fv.png)
![](https://i.imgur.com/6CMXVxa.png)
![](https://i.imgur.com/jMwwUP5.png)
![](https://i.imgur.com/kRcgGy3.png)
![](https://i.imgur.com/A12cH2y.png)
![](https://i.imgur.com/fijafR4.png)
![](https://i.imgur.com/OAqmftK.png)
![](https://i.imgur.com/GfYQC95.png)
![](https://i.imgur.com/XdVoUgd.png)
![](https://i.imgur.com/UHWJSva.png)
![](https://i.imgur.com/gsAw80E.png)
![](https://i.imgur.com/4L36jBN.png)
![](https://i.imgur.com/neZxV2Q.png)
![](https://i.imgur.com/BvoC9MY.png)
![](https://i.imgur.com/bR97WUr.png)
![](https://i.imgur.com/FryxoGA.png)
![](https://i.imgur.com/Xna7j66.png)
![](https://i.imgur.com/V4Z4VW4.png)
![](https://i.imgur.com/9QGnje9.png)
![](https://i.imgur.com/ffDfUv6.png)
![Uploading file..._omnpo32el]()
![Uploading file..._axr1mk8rd]()
![Uploading file..._wh6g84dd6]()

### Heuristic V3

#### Starplot
![](https://i.imgur.com/Fy0JCbV.png)


#### Infection Chains![](https://i.imgur.com/JdIpuIN.png)
![](https://i.imgur.com/MF71Fgl.png)
![](https://i.imgur.com/b5G6xRh.png)
![](https://i.imgur.com/bcTasDU.png)
![](https://i.imgur.com/XtBo1Qk.png)
![](https://i.imgur.com/WUoe1Ux.png)
![](https://i.imgur.com/gaRPgT4.png)
![](https://i.imgur.com/qlIamAk.png)
![](https://i.imgur.com/2IjfuBE.png)
![](https://i.imgur.com/UXFxhTu.png)



#### Baseball Cards

![](https://i.imgur.com/mvVtDYb.png)
![](https://i.imgur.com/J1j64oM.png)
![](https://i.imgur.com/WEL2iki.png)
![](https://i.imgur.com/SUpbcPO.png)
![](https://i.imgur.com/FEcDJbS.png)
![](https://i.imgur.com/ZUTRIAz.png)
![](https://i.imgur.com/L6dOrs6.png)
![](https://i.imgur.com/etbSujP.png)
![](https://i.imgur.com/jlsDstf.png)
![](https://i.imgur.com/UIMscdU.png)
![](https://i.imgur.com/lGKkgYy.png)
![](https://i.imgur.com/AQN5LiV.png)
![](https://i.imgur.com/XXaY2HZ.png)
![](https://i.imgur.com/jyYzPVu.png)
![](https://i.imgur.com/nUPAyUM.png)
![](https://i.imgur.com/1p8PV3z.png)
![](https://i.imgur.com/MHaWHMw.png)
![](https://i.imgur.com/jpBo0mj.png)
![](https://i.imgur.com/1XmxFsE.png)
![](https://i.imgur.com/9yDNCOy.png)
![](https://i.imgur.com/1mxiIZH.png)
![](https://i.imgur.com/DwbVYd6.png)
![](https://i.imgur.com/JKHPWvQ.png)
![](https://i.imgur.com/1T6J7hl.png)
![](https://i.imgur.com/BDENbys.png)
![](https://i.imgur.com/EgtqPCr.png)
![](https://i.imgur.com/WEea9vv.png)
![](https://i.imgur.com/7nsJojd.png)
![](https://i.imgur.com/zH8wTB4.png)
![](https://i.imgur.com/4zHOise.png)
