Reviewing "An Analysis of Data Propagation Latencies in the Ethereum 2.0 Medalla Testnet"

--- tags: explainers description: Reviewing "An Analysis of Data Propagation Latencies in the Ethereum 2.0 Medalla Testnet" image: https://benjaminion.xyz/f/favicon-96x96.png --- # Reviewing "An Analysis of Data Propagation Latencies in the Ethereum 2.0 Medalla Testnet" This is a brief review of the [Medalla data challenge](https://ethereum.org/en/eth2/get-involved/medalla-data-challenge/) submission, [An Analysis of Data Propogation Latencies in the Ethereum 2.0 Medalla Testnet](https://blockblockdata.github.io/medalla-data-challenge/a002/an_analysis_of_data_propogation_latencies_in_ethereum2_medalla_testnet.html), undertaken for [challenge 10](https://www.reddit.com/r/ethstaker/comments/jieyg0/ethereum_studymaster_quiz_10_medalla_data/) of the [Ethereum StudyMaster](https://ethereumstudymaster.com/) programme. ## Introductory The paper begins by correctly identifying that data propagation latency is of high importance in the practical implementation of the Eth2 beacon chain. This includes the activities of transmitting data around the network and writing it immutably to the beacon chain. The authors plan to study average inclusion distances for attestations - we'll come back to this. The stated goals are two-fold: 1. Presenting a way to analyse such latency data, and 1. Providing insights that inform areas of performance that might be improved. There follows a brief, but useful and accurate, overview of Ethereum 2.0 and a summary of its mechanisms. The authors have done their homework and link out to a variety of good quality resources. ## The basis The article considers a quantity, _inclusion distance_, that the authors define in a different way to how it is normally understood. Attestations (votes) by validators are aggregated together by the network, and inserted into beacon blocks by the block proposers. It is good for the stability of the network, via the LMD GHOST fork choice, for these attestations to be included quickly: ideally in the first slot after they were generated. Swift inclusion of attestations is actually incentivised in the protocol with a micro-reward that is inversely proportional to the delay in including them. As far as the protocol is concerned, it is the _first_ appearance of an attestation in a block that matters. Any subsequent appearances of the same attestation are ignored. Duplicate time-separated attestations might be included for various reasons. Perhaps they appear in two overlapping aggregations on the network. Perhaps a proposer is lazy or faulty and fails to deduplicate its incoming attestation pool. In view of this, I find the authors' decision to redefine _inclusion distance_ as the _maximum_ inclusion distance for attestations quite puzzling. This metric has no basis in the protocol, and is strongly affected by arbitrary features of client behaviour that are not very relevant. To be fair, the authors recognise this, and justify the decision by saying that, "it instead measures the time-wise inefficiency with respect to attestations and inclusions." However, I don't know what this means, or why it is thought to be significant. An attestation included a second time, because it is part of a different aggregate, say, is in no way inefficient: it doesn't take any extra space; it requires trivial extra processing power; it is effectively costless; and it's not clear that it tells you anything useful about the network conditions. In short, I am not convinced that this is a useful basis on which to study the network, and would be much more interested in conclusions drawn from the _minimum_ inclusion distance. To summarise the summary of the summary: perhaps I am missing something :man-shrugging: ## Data extraction and preparation These sections clearly explain in a nicely reproducible way the how the data was obtained and processed. ## Analysis ### Univariate First the authors curve-fit to the distribution of average (max) inclusion distance per epoch and number of skipped slots per epoch. They choose the [beta-distribution](https://en.wikipedia.org/wiki/Beta_distribution), which seems a reasonable choice. They observe (in a kind of roundabout way) that 20% of slots are empty - this is due to participation by validators being only around 80% during the analysis period. ### Correlations Next, the authors look at correlations between attributes. They identify correlations between: - InclusionDistanceAvg and Epoch - InclusionDistanceAvg and NoOfCommittees - InclusionDistanceAvg and InclusionDistanceStdDev Interestingly, the matrix demonstrates a weakness of correlation analysis. The correlation matrix shows near zero (0.007) correlation between Epoch and Number of committees, while in the next section they show that the two are in fact strongly related in the data. This relationship makes the first two items in the list above essentially the same phenomenon. ### Patterns Some graphs of aspects of the data are presented without drawing many conclusions. The authors observe the pattern of voluntary exits and ask "What could be the underlying reason for certain brief time intervals with many voluntary exits? What happened in each of those time intervals to cause such high number of voluntary exits?" Voluntary exits are not straightforward with present tooling - I expect the observed behaviour is due to a small number experienced node operators, each running a relatively large number of validators, choosing to exit all of their own validators in one batch. Six such individuals would explain their data nicely. ### Relationships There follows a analysis, using statistical techniques that I'm not familiar with, such as the non-parametric Dunn test with Bonferroni correction, and the graph-based methodology of Ertek et al. The conclusion of all this is that there is a slight advantage to having 11 committees per slot in order to minimise `InclusionDistanceAvg`. However, as explained above, there seems to be no practical reason to try to minimise this value, since it is based on the protocol-irrelevant maximum inclusion distance rather than the critical minimum inclusion distance. So it is hard to evaluate the significance of this finding. Another sophisticated argument concludes that voluntary exits are positively associated with average inclusion distance. It is an interesting finding, but, again, hard to evaluate the significance of. As ever, statistics measure but don't explain. Perhaps high (maximum) inclusion distances occur at times of particular network turbulence, encouraging operators of struggling nodes to exit their validators. We can't tell from the data alone. ## Conclusion There is definitely huge scope for applying advanced statistical methods to this data set, and the authors give us a nice glimpse into this world. What was missing for me was coherent rationale around the article's metric of choice: the _maximum_ inclusion distance of attestations - it just seems to be a weird and not particularly insightful choice. Also missing was any reasoning or insight attempting to explain the observed behaviours. This is left as an exercise for the reader; I've tried to add some thoughts above, but there is much more that could be gleaned. Ultimately, it is the reasons that matter, not merely the observations. As such, for me, the authors accomplished the first of their stated goals nicely, but failed to provide enough clarity of insight to achieve the second.