# Pandemia Notes - initial assessment of "doneness"

## Week w/c 07/11/2022

### Actions

- [ ] [name=Andy] Check the process for getting James T a guest account on Slack. If that proves too difficult, then we will use Teams chat.
- [ ] [name=Aoife and Andy] Review and attempt to run the code in https://github.com/PandemiaProject/pandemia based solely on the existing documentation.
- [ ] Review the current state of the repo against the REG Handbook "definition of done". (https://alan-turing-institute.github.io/REG-handbook/docs/how_we_work/defining-done/)
(Make notes in this HackMD)
- [ ] [name=James T] Review the current state of the repo against the REG Handbook "definition of done" based on your own knowledge.
- [ ] [name=James T] Evaluate the `ideas for reg tasks` list and in particular provide feedback on priority
- [ ] [name=Aoife and Andy] Review `ideas for reg tasks`
- [ ] [name=James T, Andy and Aoife] Come to agreement on initial tasks and document in the pandemia repositiory
- [ ] Discuss if continuous evaluation and documentation of project progress via github issues (and of course meetings) is the best way for us to move this collaboration forward.

## Aoife's notes

- James liked the idea of tutorials and things
- I wonder if we should consider binder related things

### Tasks
- [ ] Update this list WRT James' thoughts below and the 'doneness' ideas
- Q: Where does the data come from for using this, and do others' have the ability to create in the same way?
- [GH #21](https://github.com/PandemiaProject/pandemia/issues/21) Are there instructions to create these data in this form?
- There should be a way to make new scenarios
- Template?
- [ ] [GH # 11](https://github.com/PandemiaProject/pandemia/issues/11) Need a defination of each of the config variables
- [ ] Minimum of *most* important variables to tweak / typical changes
- [ ] Make clear which params are vital
- [ ] [GH #9](https://github.com/PandemiaProject/pandemia/issues/9) Need to think about cdll loading on different OS
- [ ] [name=Aoife] Spend a day testing how easy it is to get running on non-Windows
### Initial ideas for REG tasks

- [ ] Testing
- [ ] [GH #9](https://github.com/PandemiaProject/pandemia/issues/9) Compile C code ([name=James] This would be a preference)
- [ ] windows
- [ ] linux
- [ ] mac
- [ ] [GH #34](https://github.com/PandemiaProject/pandemia/issues/34) and [GH #12](https://github.com/PandemiaProject/pandemia/issues/12) Consider if a CI pipeline is useful and implement if so
- [ ] Check that all critical python code is included in `pytest`
- [ ] Unit test for C code
- [ ] Run C code against [valgrid](https://valgrind.org/) or similar memory leak and error problems
- [ ] [name=Andy] End-to-end testing using the scenarios
- [ ] Documentation
- [ ] Update documentation for different systems
- [ ] windows
- [ ] linux
- [ ] mac
- [ ] Evaluate python code for in-line documentation
- [ ] Evaluate C code for in-line documentation
- [ ] Check documentaion can be generated
- [ ] Check there is sufficent documentation
- [ ] Update where required
- [ ] Check consistentcy
- [ ] Consider if tex/pdf is most accessible format
- [ ] Create a user guide
- [ ] Define "target" user
- [ ] Consider a visual guide showing how usage should look
- [ ] Provide descriptions for scenarios
- [ ] Describe expected behaviour and results
- [ ] Add `further reading` or similar to documentation e.g., linking to explanations for `homogeneous mixing models` may be useful for clarity?
- [ ] Add contribution guidelines
- [ ] Performance ([name=James] Possibly one of the most useful areas for REG to contrib)
- [ ] Evaluate time and resource cost of simulations
- [ ] Scope possible optimisations
- [ ] Migrating some functionality to Cython
- [ ] Parallel running
- [ ] Maximise memory efficency e.g., using appropriate datatypes, array shapes
- [ ] Eliminate redundancy e.g., loop complexity
- [ ] Additional features and functionality ([name=James] low priority)
- [ ] rigorous statistical validation methods?
- [ ] make travel matrix represent proportions as opposed to absolute numbers?
- [ ] impact of hospitalization and capacity?
- [ ] testing and contact tracing delays?
- [ ] improved immunity subsampling approach?
- [ ] fast forward remaining sim if total infected is zero?
- [ ] can use malariaAtlas friction surface data and pathfinding to calculate weighting of locations, as opposed to gravity model?
- [ ] Misc
- [ ] Consider distributing binaries for C code to make sharing as a python package easier
- [ ] House keeping
- [ ] Remove clutter files
- [ ] e.g., `.vscode-launch.py`
- [ ] Keep naming conventions and folder structure consistent

## Andy's Notes:

I split "Reach" and "Robustness" into two seperate categories

## Reach

To get beyound Level-01
- What at a conceptual level is described in the senario files and the country data?
- [GH #33](https://github.com/PandemiaProject/pandemia/issues/33) What is an `.abm` file and where/how do I get one? (Is the answer in here https://github.com/abm-covid-lux/abmlux or here https://github.com/abm-covid-lux/output - I've not read either yet)

[GH #15](https://github.com/PandemiaProject/pandemia/issues/15) For level -03
* Who is the target audenice?
* Other epidemiologist?
* Public Health professionals
* Political Decision Makers?
* Those wishing to hold political decision makers to account

- What are simular tools?
- What is distinct about pandemia?

## Robustness
### Level-01
- How important is multi platform support? To users? To developers?
- Python version should be enforced in the package/environment definition.
- [GH #32](https://github.com/PandemiaProject/pandemia/issues/32) Apply Linting/black formating rules/pre-commits

## Functionality
### Level ?
I was unable to get the tool running on macOS so had difficulty exvaluating this (though I appricate that it was not the target OS).

- [GH #28](https://github.com/PandemiaProject/pandemia/issues/28) Add Standard CLI options eg `--help/-h`, `'--verison/-v,` etc.
- [GH #27](https://github.com/PandemiaProject/pandemia/issues/27) What is the different between one and two commandline params?
- [GH #29](https://github.com/PandemiaProject/pandemia/issues/29) How do you configure where the output file goes?
- What output is expected? In what form? How might a user use this? (ie would it be helpful to how the outputs in a format that is ideally suited to import into pandas? excel? some other visualisation tool etc?)
- [GH #30](https://github.com/PandemiaProject/pandemia/issues/30) Input files validation / sanity checking
- [GH #31](https://github.com/PandemiaProject/pandemia/issues/31) Use of absolute (*NIX) filenames for tempory files in config files.

## Documentation
### Level 2
- [GH #18](https://github.com/PandemiaProject/pandemia/issues/18) Badges in Readme.md are not rendering
- Add a conceptual level description of the senario files and the country data?
- [GH #26](https://github.com/PandemiaProject/pandemia/issues/26) Seperate User Docs and Contributing docs.
- [GH #27](https://github.com/PandemiaProject/pandemia/issues/27) There are a number of use cases described giving comand options. It is unclear what these difference use cases achieve.
- Should there be a difference in the commandline for "run the heterogeneous mixing example and save the world file" and "run the heterogeneous mixing example using the saved world file, thereby skipping the build phase". They both have the same commandline.
- Is "save the world" a tounge-in-cheek joke, or a specific bit of functionality?

## Testing
### Level 1
[GH #12](https://github.com/PandemiaProject/pandemia/issues/12) - A test senario exists which can be used end-to-end testing.
- No automated tests found
- Add minimal tests to justify the instructions in the readme.md
- [GH #17](https://github.com/PandemiaProject/pandemia/issues/17) Install `pdoc` with other dev tools (ie as part of the "test" group)

## Open Release
### Level 1-2
- No obvious institutional barriers to open release
- [GH #16](https://github.com/PandemiaProject/pandemia/issues/16) Remove and exclude .dll (and other complied files) from repo.
- [GH #14](https://github.com/PandemiaProject/pandemia/issues/14) Distriute binaries through PyPI / Conda or other platform

## James' Thoughts on current state

### Reach and Robustness

Who is currently using it
Who might use it in the future?

[GH #15](https://github.com/PandemiaProject/pandemia/issues/15)
2020 Previously working on [abm-covid-lux/abmlux](https://github.com/abm-covid-lux/abmlux)
(as part of the Covid-19 taskforce in Lux)

2021 In Turing - thinking about known limitation to LUX model.

2022 Started working on Pandemia.

Hoping to introduce it collegues in Imperial. Other models are inaccessible and not easy to use.

Near-term get more people using in Imperial.

Would like to get it used to a wider audenice - impossibly users outside academia

### Testing

[GH #12](https://github.com/PandemiaProject/pandemia/issues/12) No automated testing

There is a set of senarios that cover most of the functionality. Have been manually using these to test

### Functionality

Reasonibly complete

### Documentation

Documents in Latex/PDF which gives an overview of the workings of the model.
[GH #25](https://github.com/PandemiaProject/pandemia/issues/25) Would like to produce a tutorial video

### Open release

already public

* [GH #14](https://github.com/PandemiaProject/pandemia/issues/14) Would need to publish on PyPI or simular platformr
Also would like to aim for JOSS or simular

### [GH #21](https://github.com/PandemiaProject/pandemia/issues/21) How a new user might get started

- tweak params in an existing senario
- Write new config file from scratch
- Write a new health model component

Is there an explict schema for the config files?

# Priorities

(See "THEMES" column on project board)

Functionality - low priority

1. [GH #12](https://github.com/PandemiaProject/pandemia/issues/12) Create end-to-end testing using the existing senarios
2. [GH #9](https://github.com/PandemiaProject/pandemia/issues/9) (timebound) assess the work required for multi-platform support.

3. [GH #13](https://github.com/PandemiaProject/pandemia/issues/13) Performance - Potential for REG help
- All params passed to C as 1D NP arrays. Requires manual calculation of array indices.
- Would there be a better way to do this.

4. More testing
5. then Documention

# Review 2023/02/02

## Reach
Level-01

## Robustness
Level-01

## Functionality
Level ?

## Documentation
Level 2

## Testing

## Open Release