# CODECHECK Video Presentation Nov 2022

**https://youtu.be/_nMzFhYro_U**

## About

- German Reproducibility Network requires all members to have a short video presentation, cf. initatives videos on https://reproducibilitynetwork.de/members/
- Scott Edmunds asked us if we could provide a short video to be shared at a Chinese conference _by the end of October 2022_
- Raniere generously offered to help with video editing
- It would be nice to have something more than just a voice and a slidedeck, BUT that's also fine for the first iteration

## Channel

Publish on the brand new YouTube channel: https://www.youtube.com/channel/UCwgrYIcGvx4MOsodoiC00rQ

## Script

### Idea

* To show that CODECHECK is a community, let's have more than one person narrate the video
* We all speak the text below (with a good camera/mic/lighting if possible) and then we cut that nicely together, and go through this slidedeck in between the video images: TBD
* Make sure to have good breaks between the sentences, small variations in the text are not an issue.
* We might play some sentences as repeats one after the other, or at the same time, for some cool effects (I hope)
* Alternatively, if we find that to much work in the given time, I just use my voice and show mostly slides.
* In the Text below, the bold strings are camera instructions and slide instructions; I set up OBS Studio for the recording
* 1 = only speaker
* 2 = large slide, mini speaker in the corner
* 3 = slide and speaker side by same, roughly same size
* Have some intro and outro music by an AI
* **https://soundraw.io/**

### Slides

https://codecheck.org.uk/slides/codecheck-video.html

### Video description

The CODECHECK initiative supports academic journals and conferences with a workflow and guidance to evaluate computational workflows underlying scientific papers as part of a peer review process. Over 50 successful CODECHECKs to date demonstrate convincingly how a seemingly low bar can have a huge positive impact on the transparency and usefulness of material that is too often neglected in academia: research data and research software.

A CODECHECK increases availability, discoverability, reproducibility, and reusability of data and code. Every CODECHECK is a rewarding learning experience for both author and codechecker that eventually will impact the practices on how we share our work and how we collaborate to solve big societal challenges.

Acknowledgements

CODECHECK received a Mozilla mini science grant; SJE received support from UK SSI; we thank editors @ GigaScience, eLife, Scientific Data, and more for valuable conversations.

DN was supported by the project Opening Reproducible Research (o2r) with funding by the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) under project numbers PE 1632/10-1 and PE 1632/17-1.

### Text

**1**

Hallo! My name is Daniel Nüst and I am a codechecker.

**3**

The CODECHECK initiative supports academic journals and conferences with a workflow and guidance to evaluate computational workflows underlying scientific papers as part of a peer review process.

Unlike other projects and products to improve reproducibility, CODECHECK does _not_ attempt to make code reproducible forever for everyone.
We also do not focus on technology, but on the structural and social procedures.

**SLIDE**

CODECHECK is lead by Stephen Eglen and myself.

So, what is a CODECHECK?

**SLIDE**

In short, an independent person, we call them codechecker, provides a "certificate of executable computation".

**2**

The CODECHECK certificate includes metadata and a short summary.
The existence of a certificate shows that the codechecker was able to execute a provided set of data, code, and configuration files at least _once_ without any critical errors based on the provided documentation.
The codechecker must successfully recreate the expected output files.

The codechecker may include notes on their experiences and created figures in the certificate.
Often the codechecker's feedback will be adopted by authors even before the publication of the research article.

Ideally, the codechecker reports that all outputs match the original figures, maps, or statistical measures from the manuscript under review.

**1**

However, the codechecker does not evaluate scientific merit, validity of code, or correctness of a used method.

That is what peer review is for.

CODECHECK recognises the undesirable development in academic publishing that the main output of research is still just a paper, and that reviewers are overloaded.
Therefore, CODECHECK complements existing procedures to look at all parts of a research project.
And we tap into a new resource, because people currently not active as peer reviewers are great candidates to become codecheckers, for example graduate students or research software engineers.

**SLIDE**

**2**

The implementation of a CODECHECK procedure at a journal or conference is based on **five principles** to balance the needs of auditing complex research pipelines, the reality of the scholarly publication pressure, and the current academic evaluation practices.

1. Codecheckers record but don’t investigate or fix.
2. Communication between humans is key.
3. Credit is given to codecheckers.
4. Workflows must be auditable.
5. Open by default and transitional by disposition.

**3**

Guided by these principles, the codechecker is encouraged to communicate with the author to resolve any issues, but the responsibility to package a research analysis in a reproducible way lies with the author.
In most cases, the codechecker's questions and comments lead to a considerable improvement of the published workflow artefacts.

The principles require the publisher to give appropriate credit for the codechecking, for example by properly publishing certificates and referencing them in the article, or by putting reviewer activity in public databases.

Finally, we understand not all research can be shared openly or use only free and open tools, but we strive for all checked workflows to be as open as possible.
Our goal is that CODECHECKing of workflows becomes the norm so that we eventually just talk about peer review again.

**SLIDE**

The implementation of these principles can vary a lot:
from a publisher's staff codechecker that screens submissions before peer review
to a volunteering community codechecker providing a badge of special merit post publication.

**1**

CODECHECK shows how a seemingly low bar can have a huge positive impact on the transparency and usefulness of the published material.
Over 50 successful CODECHECKs to date demonstrate this convincingly.

A CODECHECK increases availability, discoverability, reproducibility, and reusability of scientific data and code.
It exposes important building blocks for computational research and gives credit for open scholarship.

The approach is flexible enough to handle any level of reproducibility, data size, or data sensitivity, if we can find a codechecker with matching skills and access to suitable resources.

Most importantly, every CODECHECK is a rewarding learning experience for both author and codechecker.
These experiences will eventually impact the practices on how we share our work and how we collaborate to solve big societal challenges.

**SLIDE**

Do you want to learn more about CODECHECK?

**2**

Head over to our public register of checks to see what articles we successfully reproduced.

**SLIDE**

Check out our Open Access paper to learn more about the background and all the details.

Who are the relevant stakeholders?

What roles are involved in a CODECHECK?

What variations of a CODECHECK are possible?

What are the challenges, benefits, and limitations of codechecking?

How will CODECHECKs become standard procedure in the future?

**SLIDE**

Take a look at our website to see what conferences and journals we currently collaborate with and, most importantly, how *you* can get involved in the project.

**3**

If you are an editor, conference organisers, or publisher - get in touch, we would like to talk to you about enhancing peer review with code execution.

Are you an author and want to learn how you can make your workflow more reproducible?

You can suggest to conduct a CODECHECK in your next cover letter.

Or even better:

Become a codechecker yourself!

**1**

Your next CODECHECK is a unique learning opportunity and a great Open Science experience.
Don't miss it!

I see you at your next CODECHECK!

<!-- space for scrolling the text up in the browser window while reading it into the camera -->

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.