---
# System prepended metadata

title: Reproducibility studies in the classroom

---

# Reproducibility studies in the classroom
Open Space at the GRN virtual event 02.03.2022

Participants:

- Heidi Seibold (@HeidiBaya)
- Claudia (@FuzzyLeapfrog)
- Matthias König (@konigmatt)
- Stefan Scherbaum (@StefanScherbaum)
- Sabrina Illius
- Florian Pargent
- Lena Hahn 
- Tracey Weissgerber
- Maximilian Frank (maximilian.frank@psy.lmu.de / @epizyklen)
- Leyla Jael Castro

## Key learnings
- Chaos is key
- Something is going to go wrong and that's okay
- Set expectations at start of course that this should be of publishable quality
- In such a course, so many eyes look over everything - from this perspective, the quality of the study is even higher than under normal circumstances
- It's about progress, not perfection
- The big picture is important for motivation, i.e., students must see the final outcome to overcome small hurdles

## Examples

- reproducibility of COVID-19 computational models: https://www.ebi.ac.uk/biomodels/covid-19
- How to run a participant guided learn by doing course: https://journals.plos.org/plosbiology/article/metrics?id=10.1371/journal.pbio.3001073
- Example paper from the course described above: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001161
- EmPra (empirical project in the bachelor curricla) with a focus on two replicationd from the department
- reproducing data from pharmacokinetics publications by digitization (https://pk-db.com). Big part is multiple curators and using data integration via meta-analysis (see for instance https://www.frontiersin.org/articles/10.3389/fphar.2021.752826/full)
- Reproducibility study of MSc course in Stats at LMU Munich https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0251194, Supplementary https://gitlab.com/HeidiSeibold/reproducibility-study-plos-one, Slides: https://docs.google.com/presentation/d/1S6-SJ62ni5vbSjQVmoxBORlrD53vzoAFuRPfm_vrV-M/edit?usp=sharing, Video: https://youtu.be/NswZ6PqsHbU
- ReproHack https://www.reprohack.org/


## Tips
- Only chose study with open data/materials, ideally also with open code)
- Have a data manager involved (e.g., a student)
- Build a community, get people to know each other (e.g use joint communication tool)
- Have specific and clear roles in the group
- Have original researchers available for questions 
- Clarify authorship early on and collaboratively (ranking mechanism and usage of credit categories / alphabetical order)
    - And if there are contributions that will be recognized but will not result in co-authorship
- Answers to (reviewer) doubts
    - Reproducing one paper is already an addition to the scientific knowledge 
    - Quality might be even higher with student groups than when a single PhD student conducts the research ("more eyes on the research")
    - Motivation is high because there is a clear goal
- In addition to authors, you might have as well contributors. At least for software, we recommend to make clear whether or not you as the owner of a code repo, accepts contributions and if any contribution will result in the person been named as co-author for a future release

## Challenges

Group dynamics
- Engaging people
- Deal with diversity
- Voluntary vs. involutary participants
- (Potentially) more workload for teacher and also for the students
- Interpersonal issues are occuring often

Administrative questions
- Difficulties in publishing ("is this real research?"; data/ research process quality)
- How deal with the authorship/ contributorship in the context of a huge student group
- Contacting original authors (not all are happy to get replicated)