---
tags: Debug
---

# Debugging HeuristicV1 Revision 2

### Description

I made changes to the codebase as described below, and ran two batches of simulations. Described first is the "debugging" run, where we analyze one heuristic simulation. Second, we compare several runs of the heuristic method, BDT, and no intervention.

This code revision differs from the previous in several ways:
- plotting changes
- infection chains display reported and ground truth symptoms as S:3/8 where 3 is observed symptoms and 8 is ground truth symptoms.
- Infection chains also display the maximal disease severity as Sev: 3, which can take on a value from 0 to 4 where 1: mild, 2: moderate, 3: severe, 4: extremely severe, and 0 is "none of the above". The heuristic seems to have been written under the impression that there is always a severity symptom, which is not the case. I tried to address this below.
- In infection chains we now see a "more random" distribution of infection chains, instead of only the first time an infectious person infects another person. To achieve this, I re-implemented the infection chain logic as a tree, which is kinda cool. The center node is a fake "root" node that I added because we start off multiple people as infected (in this sim, 10 people). We then prune the shorter half of the simple paths to leaves, then sample from the remaining longer paths and display them as infection chains. I'd like to make the edge lengths proportional to the amount of time between infections, too.

Caption: The infection chain in the debug run.

- Heuristic Algorithm
- Changed the algorithm to resolve the severity issue by adding a rule which checks the length of the set of reported symptoms, and if it's greater than twice the rec level determined by the symptom severity, then it sets a new risk level equal to the length of the set of reported symptoms and a new rec level = min(new_risk_level // 2, 3)
- I also made a much more broad set of changes that address the issue we had where some high-risk signals were being crushed by lower signals later in the algorithm. Now, we check 3 primary signals in the same way as the algorithm (update messages, test results, and symptoms) and return the max rec level and risk history with the maximum sum (these are often different length arrays). N.b. The heuristic method should be further refactored and unit tests should be added to it to enforce correctness.

Please refer to this googledoc for a high-level description of the heuristic (v1): https://docs.google.com/document/d/1_J1jcn5b0h7h5_bzjH-qx4LerWNFRJOzBI2vCm1n3AA/edit?usp=sharing

Please refer to [the wiki](https://hackmd.io/N-g61aZLQaSB_cz5SttM6Q) for information on how to read the plots.

### Infection Chains
Questions:
- H51 at 2020-03-13 has moderate (sev=2) symptoms, which should put him in R>=10 and Rec=3. Why?
- Also, YB fixed the pseudo-code on the next line (mild symptoms) so RecLevel is set to L=max(2,L) and not to just 2. Should be done in the code too.
- We need to monitor and probably change the behavior at home, to have less infections at home, especially when one or both are at high rec levels.

Questions:
- I wonder why H64 is going to higher rec levels when he does not have any symptoms nor is receiving messages with R greater than 10. 10-5 puts him at R=5 ok but that normally should not trigger an increase in reclevel according to the pseudo-code (although I think it maybe should and we could play with that, I am concerned that the behavior does not match the pseudo-code). Maybe because H1 got infected? but why did H64 increase their R before H1 got infected (2020-03-05 and 2020-03-06)?

Questions:
- H93 gets in red level in spite of only mild symptoms 2020-03-13 and no high R message, yet his risk level is only at 9 while being in red. Weird!
- H55 lives with H93 but when H93 goes to red, H55 does not go to red.
Questions:
- H41 gets infected by H32 only ONE DAY after H32 got infected. SEEMS WRONG!
- H32 gets Rec=orange when he has severe symptoms which should bring him to red and R=12, on 2020-03-02. Then on the next day he gets extremely severe symptoms which should bring him to R=12, not 10. Something wrong!
- The heuristic currently does not use the number of symptoms, but it seems correlated with having the disease. If someone could do a histogram of the average and sdev of contagiousness for each value of the number of symptoms and symptom severeity, it would be appreciated. Then we could probably find a reasonably to add #symptoms as a feature in the heuristic by matching it with the severity.
Questions:
- similar to above the R and RecLevel of H32 is not explained by their received messages and symptoms severity in early days. Sev=2 should have led to red level and R=12 on 03-02.
- similar to above i don't understand the RecLevel of H30 in 03-02 and 03-03.
- it looks like H30 lives with H32 yet when H32 gets red, it seems to have no effect on H30.
Questions:
- more similar unexplained changes in color of H82 in early days (both yellow and orange are unexpected).
- When H32 gets red (while his R is still pretty low!) it does not seem to affect H67 who seems to live at the same place.
- the change in color from orange to yellow of H100 is weird. Why? Rec level should only increase only 7 days have passed with no symptoms, but it has been more than that, and H100 should have gone to green.
Questions:
- too short delay for infecting someone else for H21
- colours don't make sense in many places (e.g. H91 should have gone orange on 03-02, not one day later; why yellow on 03-02 for H21? should still be green.)
Questions:
- H40 should go to red on 03-02, not one day later
- after his negative test (03-04), H40 should go back to green and R=0 (with current pseudo-code at least).
- why is H78 becoming orange on 03-03? No good reason visible.

### Baseball Cards

#### Interesting Asymptomatic Cases
This one is interesting because they are asymptomatic and get notified early enough to theoretically prevent additional infections. They stay at home, but still have a fair amount of encounters. Also, they were infected by human 93, who lives with them. Additional interesting fact, they work in a senior residency and _quit going because of the app right before they became infectious_.

Here's their housemate who infected them:

#### All cases


### Simulation Logs
######## DEMOGRAPHICS #########
age distribution
0
count 100.000000
mean 42.110000
std 23.796229
min 1.000000
25% 25.750000
50% 42.000000
75% 59.250000
max 101.000000
house age distribution
0
count 44.000000
mean 44.567045
std 17.897037
min 15.333333
25% 31.500000
50% 42.250000
75% 58.125000
max 86.000000
house size distribution
0
count 44.000000
mean 2.272727
std 1.318273
min 1.000000
25% 1.000000
50% 2.000000
75% 3.000000
max 5.000000
Fraction of asymptomatic 0.21
######## COVID PROPERTIES #########
Avg. incubation days 5.45
Avg. recovery days 18.91
Avg. infectiousnes onset days 3.34
######## COVID SPREAD #########
human-human transmissions 47
environment-human transmissions 14
environmental transmission ratio 0.298
Ro 1.4716666666666667
Generation times 5.318455387205388
Cumulative Incidence [0.0, 0.025, 0.10256410256410256, 0.08571428571428572, 0.140625, 0.07272727272727272, 0.0784313725490196, 0.0, 0.02127659574468085, 0.043478260869565216, 0.06818181818181818, 0.0, 0.024390243902439025, 0.05, 0.0, 0.05263157894736842, 0.0, 0.0, 0.0, 0.0, 0.027777777777777776, 0.02857142857142857, 0.029411764705882353, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
R : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3.0, 1.3333333333333333, 1.0, 0.9, 1.125, 1.263157894736842, 1.2, 1.0, 0.9032258064516129, 0.8285714285714286, 0.7428571428571429, 0.65, 0.5897435897435898, 0.6052631578947368, 0.3888888888888889]
******** R0 *********
Asymptomatic R0 1.2
Presymptomatic R0 1.3636363636363635
Symptomatic R0 1.2
******** Transmission Ratios *********
% asymptomatic transmission 12.77%
% presymptomatic transmission 31.91%
% symptomatic transmission 25.53%
******** R0 LOCATIONS *********
household R0 1.2941176470588236
store R0 1.0
school R0 1.0
workplace R0 1.6
######## SYMPTOMS #########
P(symptoms = x | covid patient), where x is:
light_trouble_breathing : 0.4118
heavy_trouble_breathing : 0.4118
chills : 0.3725
moderate_trouble_breathing : 0.3137
sneezing : 0.2941
runny_nose : 0.1961
confused : 0.1765
unusual : 0.0392
extremely-severe : 0.0196
lost_consciousness : 0.0196
P(symptoms = x | human had some sickness e.g. cold, flu, allergies, covid), where x is:
light_trouble_breathing : 0.3443
heavy_trouble_breathing : 0.3443
chills : 0.3115
moderate_trouble_breathing : 0.2623
sneezing : 0.2459
runny_nose : 0.1639
confused : 0.1475
unusual : 0.0328
extremely-severe : 0.0164
lost_consciousness : 0.0164
######## MOBILITY #########
Day -
Mon #avg: 3.635 %:13.88
Tue #avg: 3.4525 %:13.18
Wed #avg: 3.2175 %:12.29
Thurs #avg: 2.8625 %:10.93
Fri #avg: 4.044 %:15.44
Sat #avg: 4.4975 %:17.17
Sun #avg: 4.48 %:17.11
Average Daily Contacts
(1, 10) #avg: 9.33103448275862 %:12.44
(11, 20) #avg: 10.338557993730406 %:15.16
(21, 30) #avg: 8.253694581280788 %:15.40
(31, 40) #avg: 7.428571428571429 %:13.86
(41, 50) #avg: 7.045092838196287 %:12.21
(51, 60) #avg: 6.6453201970443345 %:12.40
(61, 70) #avg: 7.453580901856764 %:12.91
(71, 80) #avg: 4.044334975369458 %: 3.77
(81, 101) #avg: 3.4827586206896552 %: 1.86
######## COVID Testing Statistics #########
Proportion infected : 67.000%
Positivity rate: 66.667%
Total Tests: 27 Total positive tests: 18 Total negative tests: 9
Maximum tests given to an individual: 2
Proportion of population tested until end: 27.000%
Proportion of population tested daily Avg: 0.844%
Proportion of population tested daily Max: 1.000%
Proportion of population tested daily Min: 0.000%
P(tested | symptoms = x), where x is
extremely-severe 1.000
severe 0.818
moderate 0.139
mild 0.078
P(symptoms = x | tested), where x is:
headache : 0.3333
sneezing : 0.2593
moderate : 0.1852
mild : 0.1481
runny_nose : 0.1111
confused : 0.0741
unusual : 0.0741
extremely-severe : 0.0370
moderate_trouble_breathing : 0.0370
light_trouble_breathing : 0.0370
######## Effective Contacts & % infected #########
Eff. contacts: 3.333 % infected: 47.000%

## Part 2: Aggregate Statistics

### Pareto Adoption


### JellyBeans


### Pre-symptomatic curves

### Generation Times
BDT1 Adoption 100%

HeuristicV1 Adoption 100%

### Epi Tables

binary_digital_tracing_order_1_-1
| | Average | Std Err |
|:----------------------------|----------:|----------:|
| Incubation | 5.48 | 0.16 |
| Infectiousness | 3.3 | 0.16 |
| Recovery | 19.05 | 0.26 |
| Generation Time | 5.08 | 0.19 |
| Daily Contact | 21.66 | 2.34 |
| Presymptomatic Transmission | 0.42 | 0.01 |
| Asymptomatic Transmission | 0.12 | 0.01 |

no_intervention_-1
| | Average | Std Err |
|:----------------------------|----------:|----------:|
| Incubation | 5.49 | 0.09 |
| Infectiousness | 3.34 | 0.07 |
| Recovery | 19.09 | 0.09 |
| Generation Time | 4.24 | 0.14 |
| Daily Contact | 27.99 | 2.05 |
| Presymptomatic Transmission | 0.6 | 0.01 |
| Asymptomatic Transmission | 0.18 | 0.03 |

heuristicv1_-1
| | Average | Std Err |
|:----------------------------|----------:|----------:|
| Incubation | 5.44 | 0.28 |
| Infectiousness | 3.26 | 0.26 |
| Recovery | 19.12 | 0.25 |
| Generation Time | 5.4 | 0.22 |
| Daily Contact | 15.81 | 1.65 |
| Presymptomatic Transmission | 0.31 | 0.01 |
| Asymptomatic Transmission | 0.09 | 0.01 |

### Analysis

### Todo