# AI Arsenal Hack Week Notes

## Monday

### Slides

https://thealanturininstitute-my.sharepoint.com/:p:/g/personal/jroberts_turing_ac_uk/EcTYTWo1MF5NqvdsLEXzneIB9OmdLqfxg8ltLMhjV7O7yg?e=54CqKW

### Zoom link

https://turing-uk.zoom.us/j/91940918296?pwd=a3huV1lRUDRaMVJCNmluVVhOamlUQT09

### Instalation instructions

- Docker is not expected to work

- Using conda, following instructions and setup the database in the following way

airsenal_env set -k FPL_TEAM_ID -v 2779516

airsenal_setup_initial_db --n_previous 7 --clean

(curreny README instructions fail if you havent setup configuration IDs)

Next do a sanity check on the data

airsenal_check_data

Currently there are 19 errors in players play per team in each fixture and 1 in Checking sum of player goals equals match results

This gets the latest data for this season:

airsenal_update_db

### Install in development mode

From AIrsenal directory:

git pull
git checkout develop
pip install -e .

### Pipeline

Running

airsenal_run_prediction --weeks_ahead 3

will create an error (or two) - `ValueError: No gameweeks in specified range` seems to be the expected error

airsenal_run_prediction --season 2122 --gameweek_start 1 --weeks_ahead 3

Pick the whole team of 15 for the start of the season
This relies on you having results and predictions in the database for the game weeks you are looking at

airsenal_make_squad --season 2122 --gameweek_start 1 --num_gameweeks 3 --verbose --is_replay

Setting `--is_replay` will save the results to the database to let you continue a season simulation as if you had made those selections in gameweek 1.

At this point we have finished the AIrsenal pipeline for the start of the season. For subsequent weeks we recommend changes to make to our existing squad only.

Compute points predictions for the next 3 weeks from week 2:

airsenal_run_prediction --season 2122 --gameweek_start 2 --weeks_ahead 3

Recommend transfers:

airsenal_run_optimization --season 2122 --gameweek_start 2  --gameweek_end 5 --is_replay

BUG: combination of `--gameweek_start` and `--weeks_ahead` arguments doesn't work. And a login error.

### What we're going to try

Team model:

- Time-weighting (team model)
- https://github.com/anguswilliams91/bpl-next/pull/21
- Try a different model with more data types (that we already have and other data types)

Project structure/repo:

bpl next repo (for team model):

- https://github.com/anguswilliams91/bpl-next
- https://github.com/alan-turing-institute/WorldCupPrediction
- https://github.com/anguswilliams91/bpl-next/tree/feature/neutral-dixon-coles

**Task 1:** Data structure + AIrsenal codebase changes (add a time since current date column, call new model class(es), model hyperparameters - time decay rate). Plus consider other covariates that may be helpful as an improved prior.

**Task 2:** Create a "NeutralDixonColesMatchPredictorWC"-like class for Premier League matches (we don't care about neutral venues only the time weighting, so may work best to create a new ExtendedDixonColes adding only the time weighting part)

**Task 3:** Model evaluation
- 3.a Team Model-specific evaluation (we care about getting the goal probabilities right, not about the match outcomes so much) - see WorldCupPrediction repo. **Camila**, **Helen**
- 3.b Whole season simulations and hyperparameter tuning (i.e. thrash a load of CPUs on Azure) `airsenal_replay_season`.

**Task 4:** Incorporate other data in a full model (either a probablistic one or try a different sklearn/similar prediction)

### playing with a VM

install micromamba:

curl micro.mamba.pm/install.sh | bash

restart shell (e.g. just exit and re-connect to VM)

install AIrsenal:

git clone https://github.com/alan-turing-institute/AIrsenal.git
cd AIrsenal
git checkout develop
micromamba env create -f environment.yml
micromamba activate airsenalenv

## Tuesday

- [name=Jack] I fixed the bug with simulating the most recent season - you should now be able to run all the commands from yesterday with `--season 2223` if you pull the latest changes on develop and create a database with:

  airsenal_setup_initial_db --clean --no_current_season

and don't run `airsenal_update_db` after (that _should_ fail now)

## Wednesday

- [name=Jack] We have 5 Azure VMs replaying seasons on an infinite loop now 🔥 storing results files here: https://thealanturininstitute-my.sharepoint.com/:f:/g/personal/jroberts_turing_ac_uk/EnXA9Yj4QYBOheawsKSeShwBjDELwl4mbdfIGKV2JhHNjQ?e=jG9CEv
- or they will be there once I remove colons in filenames so sharepoint can deal with them...

- Team model diagnosis:
- Problem: Parameters seem to shift a lot after week 1 and then not change at all (for teams we haven't seen before and teams we had a lot of past data for)
- Ideas to investigate:
- Plot change in model parameters for teams during/between seasons (Helen's notebook: https://github.com/alan-turing-institute/AIrsenal/blob/attack_param/notebooks/progression_of_attack_parameter.ipynb)
- Compare including or not including covariates in the model (used to get a prior for new teams) - I think Helen was also starting to think about this
- See how sensitive the model fit is to Pyro hyperparameters (e.g. no. MCMC samples)
- Make a simplified version of the model as a baseline (e.g. no correlation adjustment term for low scoring matches, one home advantage parameter rather than one per team, no correlation between attack and defence, ...)
- Make some dummy data to see the impact on the parameters if e.g. a team is winning all the time or losing all the time.