## Welcome to the <your-event-name> ReproHack! <br> <!-- Put the link to the event page here so people can access all materials --> ### Event Page: ### http://reprohack.org/event/<your-event-id> Contains all event information and links to materials --- <!-- Add any housekeeping details here. If running an online event, you might want to include an introduction to the platform here --> ### House Keeping: - No Fire Alarm test today - Toilets & Fire Exit - Water point - downstairs --- # Introductions <!-- Use this section as an ice-breaker. Introduce yourself, then allow others to go around the room and introduce themselves too. If online, consider using break-out rooms of 5-6 people --> --- <!-- Add details about yourself the organiser in the next few slides: --> ## Who am I? ### Dr Anna Krystalli (@annakrystalli) - Research Software Engineer _University of Sheffield_ - 2019 Fellow _Software Sustainability Institute_ - Software Peer Review Editor _rOpenSci_ - Core Team member: _ReproHack_ --- ## Why am I here? <!-- Edit with your own answer --> > I believe there's lots to learn about Reproducibility from working with real published projects. --- ## Who is my favorite animated character? <!-- Edit with your own answer --> Stitch!  <!-- Let the rest of the organisers do the same --> --- <!-- Open it up to participants --> - ### Who are you? - ### Why are you here? - ### Who is your favorite animated character? <!-- Delete below if participants NOT renaming breakout rooms --> - #### As a group: Name your room! :mountain: :camping: :desert::roller_coaster::national_park::stadium::desert_island::sunrise::Tokyo_tower: --- # Welcome Back --- ## ReproHack Objectives 1. **Practical Experience in Reproducibility** 3. **Feedback to Authors** 5. **Think more broadly about opportunities and challenges** --- # Plan of Action --- ### Reproduce paper 1. **Paper review and team formation** 2. **Select and register your paper on hackpad** 3. **Work on your paper!** 4. **Re-group part-way through.** 5. **Feedback at the end (group & authors)** --- # Tips for reviewing --- <!-- Remind participant of code of conduct and basic expectations. Bring attention to the additional considerations involved in giving feedback to authors --> ## Code of Conduct Event governed by [**ReproHack Code of Conduct**](https://reprohack.org/code-of-conduct) <br> ### Additional Considerations - #### Reproducibility is hard! - #### Submitting authors are incredibly brave! --- ## Thank you Authors! :raised_hands: - #### Without them there would be no ReproHack. - #### Show gratitude and appreciation for their effort and bravery. :pray: - #### Constructive criticism only please! --- # Reproduce and Review # :mag: --- ## Selecting Papers - **Author comments:** paper descroption and why you should choose to reproduce. - **Tags:** Tools, languages & domains - **No. attempts:** No. times reproduction has been attempted - **Mean Repro Score:** Mean reproducibility score (out of 10) - lower == harder! --- ## Review as an auditor :bookmark_tabs: ### Looking for FAIR principles - Findable - Accessible - Interoperable - Reusable --- ## Access - How easy was it to gain access to the materials? ## Installation - How easy / automated was installation? - Did you have any problems? --- ## Data - Were data clearly separated from code and other items? - Were large data files deposited in a trustworthy data repository and referred to using a persistent identifier? - Were data documented ...somehow... --- ## Documentation Was there adequate documentation describing: - how to install necessary software including non-standard dependencies? - how to use materials to reproduce the paper? - how to cite the materials, ideally in a form that can be copy and pasted? --- ## Analysis - Were you able to fully reproduce the paper? :white_check_mark: - How automated was the process of reproducing the paper? - How easy was it to link analysis code to: - the plots it generates - sections in the manuscript in which it is described --- ## Analysis ### If the analysis was not fully reproducible :no_entry_sign: - Did results (e.g. model outputs, tables, figures) differ to those published? By how much? - Were missing dependencies? - Was the computational environment not adequately described / captured? --- ## Review as a user :video_game: <br> #### What did you find easy / intuitive? #### What did you find confusing / difficult #### What did you enjoy? --- # Feed back # :speech_balloon: --- ## Feedback as a community member <br> #### Acknowledge author effort #### Give feedback in good faith #### Focus on community benefits and system level solutions --- # Additional activities --- ## Finished early? ### Explore the work more deeply: - Try and run additional analyses. - Create new plots. - Combine materials with your own or other open materials available on the web! --- # Let's go! :checkered_flag: --- ## 1. Paper review / Team formation + Have a look at the papers available for reproduction + Add your details to the **hackpad**. + Fine to work individually or tackle a papers as group --- ## 2. Project registration + Register your team and paper on the **hackpad** + Register for an account on the Hub. + Feel free to work here or in break-out rooms. --- ## 3. Reproduce Paper - Inspect the review form. - Try to reproduce the paper using the materials provided. - Record your responses to review questions according to your experiences. --- ## 4. Mid-point regroup - Which paper have you selected? Briefly describe what it's about. - Briefly describe the approach to reproducibility the paper has taken. - Anything in particular you like about the paper's approach so far? - Anything you're having difficulty with? --- ## 5. Feedback to authors - **Please complete the feedback form for authors** - Feel free to record general findings the hackpad --- ## 6. Final regroup - So, how did you get on? - Final comments. - If there's time, tackle some discussion topics (see hackpad). - On post-its: One thing you liked, one thing that can be improved. --- # Get involved! ### Visit ReproHack Hub <https://reprohack.org> - [**Submit a paper for review**](https://reprohack.org/paper/new/) - [**Organise your own event**](https://reprohack.org/event/new/) _Check out our [Resources](https://reprohack.org/resources) for more details_ ### Chat to us: [](https://reprohack-autoinvite.herokuapp.com/) --- # THANK YOU ALL! :pray: - #### Thank you PARTICIPANTS for coming! - #### Thank you AUTHORS for submitting! - #### Thanks to <any-sponsors> for sponsoring! # :wave: