## Welcome to the TIER2 ReproHack! <br> ### Event Page: https://osf.io/u5gjh/ This Open Science Framework folder contains links to materials and notebooks for this ReproHack. For more info and today's schedule, check: https://tier2-project.eu/events/tier2-reproducibility-hackathon-fens-forum-2024 --- <!-- Add any housekeeping details here. If running an online event, you might want to include an introduction to the platform here --> ### House Keeping: - No Fire Alarm test today - Toilets & Fire Exit - Refreshnment - Code of Conduct --- # Introductions <!-- Use this section as an ice-breaker. Introduce yourself, then allow others to go around the room and introduce themselves too. If online, consider using break-out rooms of 5-6 people --> --- ## Who am I? ### Dr Alex Bannach-Brown - Research Methodologist _Berlin Institute of Health at Charité_ - Build tools and software for evaluating and synthesising evidence in biomedicin - Product Owner _www.SyRF.org.uk_ - Core Team member: _SOLES: Systematic Online Living Evidence Summaries in R_ --- ## Why am I here? Why should we care about open science and Reproducibility? :::info Quest for truth & trust in science. ::: <img src="https://hackmd.io/_uploads/Bkg2XLnUA.png" alt="quest for truth" width="300"/> --- ## Who is my favorite animated character? Minions! (& fries)  --- --- ## Who am I? ### Dr Friederike Kohrs - Community Engagement Manager _Berlin Institute of Health at Charité_ - supporting projects by helping to build and grow communities - Science communication enthusiast - former member of Pint of Science Berlin and Girls Day organizer --- # Why am I here? - To facilitate efforts in Open Science and Reproducibility and support those who are invested in fostering good science --- --- ## Who is my favorite animated character?  --- <!-- Open it up to participants --> - ### Who are you? - ### Why are you here? - ### Who is your favorite animated character? --- # Welcome Back --- ## ReproHack Objectives 1. **Practical experience in Reproducibility** 3. **Improve the way research is conducted** 5. **Opportunities and challenges for reproducible research** --- ## Lightning Talks by Speakers - Emma Wilson - Dr Anna Krystalli --- # Plan of Action --- ### Reproduce paper 1. **Paper selection and team formation in Zulip** 2. **Select and register your paper on hackpad** 3. **Work on reproducing the paper!** 4. **Re-group part-way through.** 5. **Feedback at the end (group & authors)** --- # Tips for reviewing --- <!-- Remind participant of code of conduct and basic expectations. Bring attention to the additional considerations involved in giving feedback to authors --> ## Code of Conduct Event governed by [**ReproHack Code of Conduct**](https://reprohack.org/code-of-conduct) <br> ### Additional Considerations - #### Reproducibility is hard! - #### Submitting authors are incredibly brave! --- ## Thank you Authors! <img src="https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExMG9uNnI0enEyazlyOW8ybmU0YzB6YWZwbHowamRjYmxvM2F5enl2MiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/FPDZV2JGkNGeUZdi7G/giphy.webp" alt="quest for truth" width="200"/> - #### Without them there would be no ReproHack. - #### Show gratitude and appreciation for their efforts. - #### Constructive criticism only please! --- # Reproduce and Review <img src="https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExOWF6azN4NzQzYzZ4Ynp2YW54ZnE2aXd2dWtsOXd0eXF5ODQ5MTV1dyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/9E40ePI5P22Wngf4mA/giphy.webp" alt="quest for truth" width="400"/> --- ## Selecting Papers - **Comments on Papers:** paper description and why you choose to reproduce. - **Tags:** Tools, languages & research domains --- ## Review as an auditor ### Looking for FAIR principles - Findable - Accessible - Interoperable - Reusable --- ## Access - How easy was it to gain access to the materials? ## Installation - How easy / automated was installation? - Did you have any problems? --- ## Data - Were data clearly separated from code and other items? - Were large data files deposited in a trustworthy data repository and referred to using a persistent identifier? - Were data documented ...somehow... --- ## Documentation Was there adequate documentation describing: - how to install necessary data/analysis/software? - how to use materials to reproduce the paper? - how to cite the materials, ideally in a form that can be copy and pasted? --- ## Analysis - Were you able to fully reproduce the paper? - How automated was the process of reproducing the paper? - How easy was it to link analysis code to: - the plots it generates - sections in the manuscript in which it is described --- ## Analysis ### If the analysis was not fully reproducible - Did results (e.g. model outputs, tables, figures) differ to those published? By how much? - Were missing dependencies? - Was the computational environment not adequately described / captured? --- # Feed back <img src="https://media4.giphy.com/media/v1.Y2lkPTc5MGI3NjExd2h4MzJ5bjF0bWxzeWtkdmdnbHQ1eGltenBobWw2OTBpNHB1ODNkMiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/Hkd41kiT9Zd0AwqVNW/giphy.webp" alt="quest for truth" width="400"/> --- ## Feedback as a community member <br> - Acknowledge author effort - Give feedback in good faith - Focus on community benefits and system level solutions --- # Additional activities --- ## Finished early? ### Explore the work more deeply: - Try and run additional analyses. - Create new plots. - Combine materials with your own or other open materials available on the web! --- # Let's go! <img src="https://media0.giphy.com/media/v1.Y2lkPTc5MGI3NjExNWp0Ymw1b2tnbTJqODRtazhnY2VjeXJsdWR5bzc2N2FiYXkwa3lkbiZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/xrP3yQ0W19SGuguEd8/giphy.webp" alt="quest for truth" width="600"/> --- ## 1. Paper review / Team formation + Have a look at the papers available for reproduction + Add your details to the **hackpad**. + Fine to work individually or tackle a papers as group --- ## 2. Project registration + Register your team and paper on the **hackpad** + Connect with people who want to work on the same paper. --- ## 3. Reproduce Paper - Try to reproduce the paper using the materials provided. - Record your responses to review questions according to your experiences. --- ## 4. Mid-point regroup - Which paper have you selected? Briefly describe what it's about. - Briefly describe the approach to reproducibility the paper has taken. - Anything in particular you like about the paper's approach so far? - Anything you're having difficulty with? --- ## 5. Feedback to authors - **Please complete the feedback form for authors** - Feel free to record general findings the hackpad --- ## 6. Final regroup - So, how did you get on? - Final comments. - On hackMD: One thing you liked, one thing that can be improved. --- # Get involved! - Visit ReproHack Hub for reproducibility resources and future events <https://reprohack.org> - Check out the TIER2 website for opcoming Reproducibility Events <https://tier2-project.eu> --- # THANK YOU ALL! <img src="https://media2.giphy.com/media/v1.Y2lkPTc5MGI3NjExMXk1c25tbGo5dzRmYWx3MTNlbjNtZm82ampqOTQycnJ2aDU4aWZociZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/3o7WICht89LQdQZa2A/giphy.webp" alt="quest for truth" width="700"/>