## Welcome to the N8 CIR Remote ReproHack!
<!-- Put the link to this slide here so people can follow -->
<br>
### Event Repository:
### [bit.ly/n8-reprohacks](http://bit.ly/n8-reprohacks)
Contains all event information and links to materials
---
## Hello from the ReproHack Team
Sheffield UK
- Anna Krystalli
Leiden NL
- Daniela Gawens
- Linda Nab
- Ricarda Proppert
Buenor Aires, Argentina
- Florencia D'Andrea
---
### House Keeping:
Tour of Zoom
---
# Introductions
<!-- Use this section as an ice-breaker. Introduce yourself, then allow others to
go around the room and introduce themselves too -->
---
<!-- Add details about yourself the organiser here: -->
## Who am I?
### Dr Anna Krystalli (@annakrystalli)
- Research Software Engineer _University of Sheffield_
- 2019 Fellow _Software Sustainability Institute_
- Software Peer Review Editor _rOpenSci_
- Co-organiser _Sheffield R Users Group_
---
## Why am I here?
I believe there's lots to learn about Reproducibility from working with real published projects.
---
## Who is my favorite animated character?
Stitch!
---
<!-- Open it up to participants -->
### Break out rooms (7 mins)
## Who are you?
## Why are you here?
## Who is your favorite animated character?
---
# Agenda
## Welcome back!
---
## ReproHack Objectives
1. **Practical Experience in Reproducibility**
3. **Feedback to Authors**
5. **Think more broadly about opportunities and challenges**
---
### Morning
**10:00 Welcome**
- **Intro to Zoom (3 mins) (any volunteers?)**
- Ice breaker (split into rooms) (10 mins?)
**10:15 Introduction & tips for reviewing (20 mins)**
**10:40 TALK: Daniel Nüst - Title TBC**
***11:00 Reprohack***
- **Review list, form groups, select papers**
- **COFFEE BREAK** (break into Zoom rooms)
- Work on papers
---
### Lunchtime
**12:10 Regroup: get someone from each group to tell us a bit about their experiences.**
**12:30 LUNCH BREAK**
**13:30 :loudspeaker: Daniel Pique - How I discovered a missing data point in a paper with 8000+ citations**
***13:50 Reprohack***
- Work on papers (break into Zoom rooms)
---
### Afternoon
***Reprohack***
- Work on papers (break into Zoom rooms)
- 14:45 COFFEE BREAK
- Complete feedback form
**16:00 Regroup - get someone from each group to tell us a bit about their experiences.**
**16:30 :loudspeaker: Sarah Gibson - Sharing Reproducible Computational Environments with Binder**
**16:50 Final Words (any volunteers?)**
---
## 10:40 - 13:50 Lunch talk: Daniel NUst :loudspeaker:
> ### *"How I discovered a missing data point in a paper with 8000+ citations"*
---
## Materials
- [**Repository**](https://github.com/annakrystalli/n8cir-reprohacks) Instructions and links
- [**Paper list**](https://sheffield-university.shinyapps.io/n8cir-reprohacks/) List of submitted papers
- [**Hackpad**](https://hackmd.io/6nhNpD0VTHaCLuXPBy-dFw?both#Leeds) Event collaborative notebook
- [**Feedback form**](https://forms.gle/wvXfjNT7HbkqH2Rj7) Record feedback for authors
---
### Participation guidelines
#### During talks and information sessions, regroups
- Stay on mute until you are invited to speak.
- Add questions for speakers on the hackpad
#### During break out work
- Feel free to mute, trun of camera and disconnect if you need to
- Encourage you to at least work while connected to th ezoom room.
- We also encourage discussions.
### At any time
- Feel free to add to the collaborative notes on the hackpad
- Contact facilitators through the chat
---
---
## Code of Conduct
Event governed by [**ReproHack Code of Conduct**](https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html)
<br>
### Additional Considerations
- #### Reproducibility is hard!
- #### Submitting authors are incredibly brave!
---
## Thank you Authors! :raised_hands:
- #### Without them there would be no ReproHack.
- #### Show gratitude and appreciation for their effort and bravery. :pray:
- #### Constructive criticism only please!
---
# Reproduce and Review
# :mag:
---
## Selecting Papers
- **No. attempts:** No. times reproduction has been attempted
- **Mean Repro Score:** Mean reproducibility score (out of 10)
- lower == harder!
---
## Review as an auditor :bookmark_tabs:
---
## Access
- How easy was it to gain access to the materials?
## Installation
- How easy / automated was installation?
- Did you have any problems?
---
## Data
- Were data clearly separated from code and other items?
- Were large data files deposited in a trustworthy data repository and referred to using a persistent identifier?
- Were data documented ...somehow...
---
## Documentation
Was there adequate documentation describing:
- how to install necessary software including non-standard dependencies?
- how to use materials to reproduce the paper?
- how to cite the materials, ideally in a form that can be copy and pasted?
---
## Analysis
- Were you able to fully reproduce the paper? :white_check_mark:
- How automated was the process of reproducing the paper?
- How easy was it to link analysis code to:
- the plots it generates
- sections in the manuscript in which it is described and results reported
---
## Analysis
### If the analysis was not fully reproducible :no_entry_sign:
- Were there missing dependencies?
- Was the computational environment not adequately described / captured?
- Was there bugs in the code?
- Did results (e.g. model outputs, tables, figures) differ to those published? By how much?
---
## Review as a user :video_game:
<br>
#### What did you find easy / intuitive?
> Was the file structure and file naming informative / intuitive?
> Was the analysis workflow easy to follow?
#### What did you find confusing / difficult
> Identify pressure points. Constructive suggestions?
#### What did you enjoy?
> Identify aspects that worked well.
---
# Feed back
# :speech_balloon:
---
## Feedback as a community member
<br>
#### Acknowledge author effort
#### Give feedback in good faith
#### Focus on community benefits and system level solutions
---
# Additional activities
---
## Finished early?
### Explore the work more deeply:
- Try and run additional analyses.
- Create new plots.
- Combine materials with your own or other open materials available on the web!
---
# Resources
- [**Example Compendium**](https://github.com/annakrystalli/rrcompendiumDTB): Demo Research compendium
- [**The Turing Way**](https://the-turing-way.netlify.com/introduction/introduction): a lightly opinionated guide to reproducible data science.
- [**Statistical Analyses and Reproducible Research**](): Gentleman and Temple Lang's introduction of the concept of Research Compendia
- [**Packaging data analytical work reproducibly using R (and friends)**](https://peerj.com/preprints/3192/): how researchers can improve the reproducibility of their work using research compendia based on R packages and related tools
---
# Let's go! :checkered_flag:
---
## 11:00 - 11:15
### :mag_right: Paper List review
+ Have a look at the papers available for reproduction
### :busts_in_silhouette: :spiral_note_pad: Team formation / project registration
+ Fine to work individually
+ Add your details to the [**hackpad**](https://hackmd.io/6nhNpD0VTHaCLuXPBy-dFw?both#Leeds).
+ Register your team and paper on the [**hackpad**](https://hackmd.io/6nhNpD0VTHaCLuXPBy-dFw?both#Leeds)
### :house: :coffee: Grab a coffee!
---
## 11:15 - 12:10 :computer: ReproHack in Break out groups
Work on your papers. Feel free to discuss papers and collaboratively troubleshoot problems
> ### Before Lunch-time Regroup :thought_balloon:
> - Briefly describe the approach to reproducibility the paper has taken.
> - Anything in particular you like about the paper's approach so far?
> - Anything you're having difficulty with?
---
## 12:10 - 12:30 :speech_balloon: Lunch regroup
### Welcome back
- Summarise the approach to reproducibility papers in your group have taken.
- Anything in particular that worked well in the paper's approach so far?
- Anything you're having difficulty with?
---
## 12:30 - 13:30 :house::green_salad: :burrito: LUNCH
Will free to disconnect or hang out in the main Zoom room.
Be back for talk no 2 at 13:30
---
## 13:30 - 13:50 Lunch talk: Daniel Pique :loudspeaker:
> ### *"How I discovered a missing data point in a paper with 8000+ citations"*
---
## 13:50 - 16:00 ReproHack in Break out groups :computer:
### 14:45 COFFEE BREAK :house: :coffee:
Work on your papers. Feel free to discuss papers and collaboratively troubleshoot problems
> ### Before Final-time Regroup :thought_balloon:
> - #### **Complete author feedback form** :writing_hand:
> - Discuss how you got on with your papers?
> - Summarise final experiences of the group in hackpad
---
## 16:00 - 16:30 Final regroup :speech_balloon:
- So, how did the groups get on?
- Final comments.
- If there's time, tackle some discussion topics (see hackpad).
- On post-its: One thing you liked, one thing that can be improved.
---
## 16:30 - 16:50 Lunch talk: Sarah Gibson :loudspeaker:
> ### *"Sharing Reproducible Computational Environments with Binder"*
---
# Get involved!
### [reprohack-hq](https://github.com/reprohack/reprohack-hq) repository
#### Check out our [issues](https://github.com/reprohack/reprohack-hq/issues)
#### Chat to us:
[](https://reprohack-autoinvite.herokuapp.com/)
---
# THANK YOU ALL! :pray:
- ### Thank you PARTICIPANTS for coming!
- ### Thank you AUTHORS for submitting!
- ### Thank to the N8 CIR for sponsoring!
# :wave:
---
### Replicate your paper
- Consider attempting **replication**!
- Replications could be considered for publication in **ReScience C** Journal
---
## [ReScience C](http://rescience.github.io/)
> **ReScience C** is an open-access peer-reviewed journal that targets computational research and encourages the **explicit replication** of already published research, promoting new and open-source implementations in order to ensure that the original research is reproducible.
---
{"metaMigratedAt":"2023-06-15T06:22:28.976Z","metaMigratedFrom":"YAML","title":"N8 CIR Northern tour ReproHack slides","breaks":true,"description":"View the slide with \"Slide Mode\".","slideOptions":"{\"theme\":\"white\"}","contributors":"[{\"id\":\"536292a4-16ac-44ba-8cd1-f37e6002d5af\",\"add\":12961,\"del\":2499}]"}