## Linear mixed-effects models in R and a new tool for data simulation <br> **Pablo Bernabeu** <br> Lancaster University, 26 Nov 2020 --- ## Rationale - Earlier, weaker computers - Hence, aggregation (or downsampling) across trials, items, measurement channels (e.g., EEG), etc - Now, computational power - Multi-core laptops, High-Performance (or High-End) Computing clusters <div style='margin-bottom:25px'></div> :::info <div style="text-align:left; margin-top:5px; padding-bottom:11px; padding-left:45px;"> <span style="font-size:70%"> Singmann and Kellen (2019) </span> </div> ::: --- ## *Rationale* --- independence assumption - Accounting for variation within factors with multiple observations, such as participants, trials, items, channels, etc - Earlier: F1 and F2 ANOVAs (Clark, 1973) <div style='margin-bottom:35px'></div> :::info <div style="text-align:left; margin-top:5px; padding-bottom:11px; padding-left:45px;"> <span style="font-size:70%"> Brauer and Curtin (2018), Brown (2020), Meteyard and Davies (2020), Singmann and Kellen (2019) </span> </div> ::: --- ## *Rationale* --- statistical precision - Less unexplained variance = more precision --> more power - Lower Type II error rate (false negatives) without increasing Type I error rate (false positives) <div style='margin-bottom:35px'></div> :::info <div style="text-align:left; margin-top:5px; padding-bottom:11px; padding-left:45px;"> <span style="font-size:70%"> Brauer and Curtin (2018), Brown (2020), Meteyard and Davies (2020), Singmann and Kellen (2019) </span> </div> ::: --- ## *Rationale* --- theoretical accuracy In addition to statistical assumptions, linear mixed-effects models help uphold theoretical assumptions about the generalisability of findings (Yarkoni, 2020). --- ## Robustness - Quite robust to assumption violations (Schielzeth et al., 2020) --- ### Random effects (intercepts and slopes) Visualise fixed regression and the three variations of random effects. [](https://bookdown.org/steve_midway/DAR/random-effects.html) <div style="font-size:60%; font-family:courier;"> Image source: Midway (2021), https://bookdown.org/steve_midway/DAR/random-effects.html </div> --- - R code structure: <div style="font-size:60%; font-family:courier;"> + (effect | highest_level / lower_level / lowest_level) </div> - Minimum five or six levels in nesting factor (Bolker, 2015; Singmann & Kellen, 2019) --- ### Random intercepts and slopes - [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - see Table 2. <div style='margin-bottom:35px'></div> :::info <div style="text-align:left; margin-top:5px; padding-bottom:11px; padding-left:45px;"> <span style="font-size:70%"> Barr et al. (2013) </span> </div> ::: --- ### Fixed effects - [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - see Table 1. --- ### Convergence challenges - Approach: Prioritising conservativeness - [Brauer and Curtin (2018)](https://psych.wisc.edu/Brauer/BrauerLab/wp-content/uploads/2014/04/Brauer-Curtin-2018-on-LMEMs.pdf) - see Table 17. - [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - see fragment from 'As a first step, it seems advisable to remove the correlations among random slopes'. --- ### Convergence challenges - Approach: Prioritising conservativeness - [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - see fragment from 'When fitting mixed models with complicated random effects structures, convergence warnings appear frequently.' - [Brown (2020)](https://doi.org/10.31234/osf.io/9vghm) - see fragment from 'The one I recommend starting with is changing the optimizer'. --- ### Convergence challenges - Approach: Remaining conservative without losing power - [Matuschek et al. (2017)](https://doi.org/10.1016/j.jml.2017.01.001) --- ### Effect size - Standardised effect sizes are tricky due to random effects structure (Piepho, 2019). - [Lorah (2018)](https://link.springer.com/article/10.1186/s40536-018-0061-2) offers options for fixed and random effects. - see also [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - section 'Effect Sizes For Mixed Models'. --- ### *p* values - [Luke (2017)](https://doi.org/10.3758/s13428-016-0809-y) - Kenward-Roger or Satterthwaite methods the most robust - [Singmann and Kellen (2019)](http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf) - afex::mixed() --- ### Other resources - Tutorials <span style='font-size:60%;'> - https://bbolker.github.io/morelia_2018/notes/mixedlab.html - https://mac-theobio.github.io/QMEE/Mixed_models_examples.html - https://cran.r-project.org/web/packages/afex/vignettes/afex_mixed_example.html </span> --- ### Other resources - Packages <span style='font-size:60%;'> - https://cran.r-project.org/package=lme4 - https://cran.r-project.org/web/packages/afex/afex.pdf - https://cran.r-project.org/web/packages/lmerTest/lmerTest.pdf - http://www.strengejacke.de/sjPlot/index.html </span> --- ### On a different note, a new tool for data simulation - Bernabeu and Lynott (2020) <span style='font-size:60%;'> - https://github.com/pablobernabeu/Experimental-data-simulation/ </span> --- #### References (1/3) <div style="font-size:50%; text-align:left; text-indent:-2em; margin-left:2em;"> Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. *Journal of Memory and Language, 68*, 255–278. http://dx.doi.org/10.1016/j.jml.2012.11.001 Bernabeu, P., & Lynott, D. (2020). *Web application for the simulation of experimental data* (Version 1.2). https://github.com/pablobernabeu/Experimental-data-simulation/ Bolker, B. M. (2015). Linear and generalized linear mixed models. In G. A. Fox, S. Negrete-Yankelevich, & V. J. Sosa (Eds.), *Ecological statistics: Contemporary theory and application*. Oxford, UK: Oxford University Press. Brauer, M., & Curtin, J. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. *Psychological Methods, 23*(3), 389–411. https://doi.org/10.1037/met0000159 Brown, V. A. (2020). *An introduction to linear mixed effects modeling in R.* PsyArXiv. https://doi.org/10.31234/osf.io/9vghm </div> --- #### References (2/3) <div style="font-size:50%; text-align:left; text-indent:-2em; margin-left:2em;"> Clark, H. H. (1973). The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. *Journal of Verbal Learning and Verbal Behavior, 12*(4), 335-359. https://doi.org/10.1016/S0022-5371(73)80014-3 Lorah, J. (2018). Effect size measures for multilevel models: definition, interpretation, and TIMSS example. *Large-scale Assessments in Education, 6*, 8. https://doi.org/10.1186/s40536-018-0061-2 Luke, S. G. (2017). Evaluating significance in linear mixed-effects models in R. *Behavior Research Methods, 49*(4), 1494–1502. https://doi.org/10.3758/s13428-016-0809-y Matuschek, H., Kliegl, R., Vasishth, S., Baayen, H., & Bates, D. (2017). Balancing type 1 error and power in linear mixed models. *Journal of Memory and Language, 94*, 305–315. https://doi.org/10.1016/j.jml.2017.01.001 Meteyard, L., & Davies, R. A. (2020). Best practice guidance for linear mixed-effects models in psychological science. *Journal of Memory and Language, 112*, 104092. https://doi.org/10.1016/j.jml.2020.104092 </div> --- #### References (3/3) <div style="font-size:50%; text-align:left; text-indent:-2em; margin-left:2em;"> Piepho, H.‐P. (2019). A coefficient of determination (*R*²) for generalized linear‐mixed models. *Biometrical Journal, 61*, 860–872. https://doi.org/10.1002/bimj.201800270 Schielzeth, H., Dingemanse, N. J., Nakagawa, S., Westneat, D. F., Allegue, H, Teplitsky, C., Reale, D., Dochtermann, N. A., Garamszegi, L. Z., & Araya-Ajoy, Y. G. (2020). Robustness of linear mixed-effects models to violations of distributional assumptions. *Methods in Ecology and Evolution, 00*, 1– 12. https://doi.org/10.1111/2041-210X.13434 Singmann, H., & Kellen, D. (2019). An Introduction to Mixed Models for Experimental Psychology. In D. H. Spieler & E. Schumacher (Eds.), *New Methods in Cognitive Psychology* (pp. 4–31). Hove, UK: Psychology Press. http://singmann.org/download/publications/singmann_kellen-introduction-mixed-models.pdf Yarkoni, T. (2020). The generalizability crisis. *Behavioral and Brain Sciences*, 1-37. https://doi.org/10.1017/S0140525X20001685 </div>