# Eurovision story notes/ideas/meetings

Issue: https://github.com/alan-turing-institute/TuringDataStories/issues/183
Paper: https://www.tandfonline.com/doi/full/10.1080/02664763.2014.909792

History of voting rules:https://eurovisionworld.com/esc/voting-systems-in-eurovision-history

Data OneDrive: https://thealanturininstitute.sharepoint.com/sites/EurovisionStory/Shared%20Documents/Forms/AllItems.aspx

Github: https://github.com/KatrionaGoldmann/Eurovision_TDS

Outline of Eurovision blog post for comms: https://hackmd.io/eM9TJCYHTC-6x093FutqTA?both

## Data wrangling colabs:

Five datasets.
- lyrics data: https://colab.research.google.com/drive/1Okkpt4eEQva_xXLmHqJpVr8u_teqKQZe
- voting scores: https://colab.research.google.com/drive/1UY5ayUa3HnFb13W_AnefWbUAPi0lQjjG?usp=sharing
- border data: https://colab.research.google.com/drive/1rOwE4Wkninqkq8Z0F450OkwGw_ah5_Pj?usp=sharing
- migration data:
https://colab.research.google.com/drive/1sV734BCm0mhTIRV6VpGemDlzlUpyCZqh#scrollTo=IX6LsycNXtWe
- last win data: https://colab.research.google.com/drive/1reaf0j-2aOerIB8yfy079XjAnv0n0Liu#scrollTo=9E9CI4iVrjeI
- merged data:
- [Camila 27/1 Colab](https://colab.research.google.com/drive/1tlw4UKWM_G58ywZeLgKXvqPCji1I-bTl#scrollTo=fhFdWAlJ60FT)
- [Joe 3/2 SharePoint](https://thealanturininstitute.sharepoint.com/:x:/s/EurovisionStory/EaR5sDFXmJJLpChEQyUZWAMBLI49q1P3-g5kA_tIzhg77g?e=YwQdSL) (+gender, +migration)
- [Jon 3/2 SharePoint](https://thealanturininstitute.sharepoint.com/:x:/s/EurovisionStory/EUhHx2QPlB9Mhlo3EApSpJAB2_Vm4QSo99dFZDC9_y6KqQ?e=MWlZrT) (+time since last win)

Model notebooks (article re-implementations)
- baseline: https://colab.research.google.com/drive/1i-2RchgpBRaztojwuJ0luawGXgl7rpjk?usp=sharing
- [WIP] model with groups: https://colab.research.google.com/drive/1d9R0CyNkoIw1Lq9AtaHIYFw16rdZa8Gw?usp=sharing
- [WIP] model with covariates: https://colab.research.google.com/drive/1VyPhnFIWuXrfIZGQr-7oKy6CYwJNdIZO?usp=sharing

Ideas for other features:
- Countries borders (KG explored: country [alpha 2 codes](https://en.wikipedia.org/wiki/ISO_3166-1) for reference)
- Migration patterns (MS exploring)
- Time since last win for each country (JY)
- Sentiment for each country in the news?

## 29 Mar 2023

### Randome notes

The voting will change in 2023 to allow anyone in the world to vote, with a new ["Rest of the World"](https://eurovision.tv/voting-changes-2023-faq) giving votes as if they were a separate country. We think this shouldn't have a big effect on the outcomes, but technically adds another voting country to the list.

Eurovision has its own algorithm for calculating predicted votes in case a country isn't able to submit votes, see [Wikipedia](https://en.wikipedia.org/wiki/Eurovision_Song_Contest_2016#New_voting_system):

*"if a country could not deliver a valid televoting/jury result, a substitute result would be calculated by the jury/televoting result of a pre-selected group of countries approved by the contest's Reference Group"*

This method [was used](https://www.euronews.com/culture/2022/05/16/eurovision-jury-results-of-six-countries-removed-after-voting-irregularities-identified) for six countries in the 2022 contest.

## 3 Mar 2023

Radka has followed up on the ranking models and found the [Prefmod R package](https://cran.r-project.org/web/packages/prefmod/index.html) for ranked, paired data.

Similar python packages:
- [BTM methods](https://datascience.oneoffcoder.com/btl-model.html)
- [choix](https://pypi.org/project/choix/) is a Python library that provides inference algorithms for models based on Luce’s choice axiom. These probabilistic models can be used to explain and predict outcomes of comparisons between items.

## 24 Feb 2023

- [ ] explore alternative models :
- [ ] XGBoost
- [ ] hurdle model
- [ ] Bayesian ranking
- [ ] Paired comparisons (prefmod)
- [ ] explore other visualisation ideas :
- [ ] ....
- [ ] create draft for blog/data story
- collect references etc.
- add covariate visualisations

## 23 Feb 2023

In the last couple of meetings we discussed whether ordinal regression is an appropriate model for the data given the large number of 0s in the dataset and given how those arise in the data. Some modelling notes following a bit of googling:
- The key assumption of ordered logistic regression is the **proportional odds assumption** which states the covariates effect the odds of moving from one category to another equally across all outcome categories (i.e., in the paper notation, $\mu$ shifts all the cut-offs $\lambda$ equally, see e.g., [wiki](https://en.wikipedia.org/wiki/Ordered_logit) or [this chapter](https://peopleanalytics-regression-book.org/ord-reg.html) for discussion of this assumption).
- It seems likely that the Eurovision data breaks this assumption. But is it a simplifying assumption that is nevertheless fairly safe to make as ordinal models seem to be pretty robust? 🤷🏻‍♀️
- Tullia suggested we could try to build a model where you first predict whether you get a 0 or not and then model the non-zero cases as we do now. I think this is in essence a hurdle model (there is a related class of models called zero-inflated).
- Another term for the data we have is **partially ranked** or **top-k ranked** data and this might be a more appropriate way to think about what is being modelled (although this seems to be getting into active areas of research).
- We found a [paper](https://arxiv.org/pdf/1607.06051.pdf) with some [R code](https://github.com/li-xinran/BayesRankAnalysis) for Bayesian rank analysis that seems relevant. Tullia and Radka plan to investigate whether it might be useful. It's in R but it could perhaps be reimplemented in Python (the code doesn't seem too involved although doing this might be a bit of an overkill for a TDS).
- According to the paper they propose a model "to deal with incomplete ranking lists, to accommodate rankers with different qualities or opinions, and to incorporate covariate information of the ranked entities".
- It's an extension of Thurstone models, which are commonly used for ranked data (the Plackett-Luce model which comes up a lot in this domain is a special case of the Thurstone model).
- The code implements a Gibbs MCMC sampler for 4 models of increasing complexity, if we were to choose one of those it really is not that much code to rewrite.
- A simple baseline that might be worth investigating is XGBoost with a rank objective function ( see e.g., [XGBRanker](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBRanker)). Perhaps we could also explore hyperparameters that can help account for imlabanced classes. This might be more accessible for a TDS.

## 3 Feb 2023
- Camila has merged the data into a covariate table
- Ed has had a go at using Radka's model with the merged covaraites data set: https://colab.research.google.com/drive/1VyPhnFIWuXrfIZGQr-7oKy6CYwJNdIZO?usp=sharing
- Created a github repo, not on TDS, for pre-production collaboration: https://github.com/KatrionaGoldmann/Eurovision_TDS to create a data cleaning and data modeling pipeline.
- Is the imbalance of zeroes a problem? Ed's model predicts a score of 0 the majority of the time. This is due to the 10:16 split of non-zeroe to zeroe votes.

### To Do
- [x] merge gender data and migration data into covariates data set [name=Joe]
- [x] create a covariate for time since last win for each country and merge into covariates data set [name=Jon]
- [ ] exploratory visualisations
- [ ] migration
- [x] voting [name=Martin]
- [x] lyrics/language [name=Kat] - to see if countries vote in favour of songs in their own language (the voting country's language)
- [ ] gender
- [x] create working [github repo](https://github.com/KatrionaGoldmann/Eurovision_TDS)
- [x] create a branch with data cleaning notebook [name=Joe].
- [x] add source code to the cleaning
- [ ] branch with modelling/visualisation notebook
- [ ] fit model with all covariates [name=Ed]
- [ ] visualisations and predictions for final model [name=Ed]
- [ ] outline the story
- [ ] move the notebooks onto TDS github.
- [ ] Load to binder (if possible - may be difficult with Stan, make sure it can be reproduced locally at least)
- [ ] Send to Jo for feedback (deadline: aim for start-mid April)
- [ ] Review (deadline: aim for start-mid April)
- [ ] Publish (deadline: End of April)

## 27 Jan 2023:

We now have 2 model notebooks:
- the simplest baseline model (which includes an example of how to generate predictions)
- a work in progress one which also tries to estimate groups (not quite the full model yet either but adding the rest should be straightforward)

## 20 Jan 2023:

- discussed how to deal with the different voting rules (see history link above).
- for the first pass we can treat all votes as ordered data. In the case of multiple votes (e.g. post 2016), we can just sum them together and assign a rank.
- An alternative approaches could be fitted a model per voting system then combine the predictions (the ones with less data have more uncertain posterior predictions).
-
-

## 13 Jan 2023:

Initial idea a simple ordinal regression with a few covariates:

- Year
- Language of the performer
- Own
- English
- Other
- Gender?

Data:
- Finals results:
- This [data](https://www.kaggle.com/datasets/orianao/eurovision-finals-voting-results-19572021) only considers voting from countries who also had performers. But the scraping code is [here](https://www.kaggle.com/code/orianao/eurovision-data-crawler) so we could update.
- This [data](https://www.kaggle.com/datasets/datagraver/eurovision-song-contest-scores-19752019) is only up until 2019 but contains the votes from all countries.

- Language of performer per country and year from this dataset:
- https://www.kaggle.com/datasets/minitree/eurovision-song-lyrics?select=eurovision-lyrics-2022.json
- Data wranggling happening here: https://colab.research.google.com/drive/1Okkpt4eEQva_xXLmHqJpVr8u_teqKQZe

-
Modelling:
- The paper we are following uses an ordinal regression model with a logit link function on the cumulative probabilities of being awarded one of the score categories (see e.g., this [blog post](https://betanalpha.github.io/assets/case_studies/ordinal_regression.html))
- Stan MCMC comes with an implementation for ordered logistic model as described [here](https://mc-stan.org/docs/2_23/stan-users-guide/ordered-logistic-section.html)
- In the linked Stan example N=number of data points, K=number of categories (scores), D=number of predictors, y are the observed scores and x are the covariates, beta are the coefficients and c the cut off points (lambdas in the paper)
- The Stan example model is expressed as ` y ~ ordered_logistic(x * beta, c);` which in the paper notation corresponds to (roughly)
- ```
mu = x * beta;
y ~ ordered_logistic(mu, c);

- we should be able to use the linked template and expand the mu expression with additional covariates (and eventually also the bias terms captured by `alpha`)
- This [post](https://medium.com/@liza_p_semenova/ordered-logistic-regression-and-probabilistic-programming-502d8235ad3f) includes an example of how to implement + generate predictions from ordered logistic models in both Stan and PyMC3 
- Next session we can start writing the model for dummy dataset whilst the data wrangling finishes.  

## 11 Jan 2023: Kick-off meeting

## Participants


## Agenda

- Initial idea from Comms
- Timeline for publication
- Analysis idea
- Discuss people's avalaibility-commitments, organise regular catch-ups


## Notes

- Mid April for a final version
- Some interesting data https://escincontext.com/
- Meet on Fridays for hack sessions.