How do you review code that accompanies a research project or paper?
===
Weecology Lab Meeting
2018 October 10

# Agenda
* intro (5 min)
* whole lab discussion on the goals for accompanying code (15 min)
* breakout discussions on code criteria (15 min)
* breakout discussions on practices/guidelines (15 min)

# Introduction
(Hao is going to be talking about this topic next week at the [rOpenSci Community Call](https://ropensci.org/blog/2018/10/05/commcall-oct2018/))

*(to simplify text, using this definition)*
**research code** := code that accompanies a research project or manuscript

**Meeting Objectives**
* identify the different kinds of code that accompany research projects
* describe the criteria for code
* discuss practices/guidelines that you use or would be willing to use

## What are the goals for research code (whole lab)

* appropriate and consistent variable names (guides for coding style)
code is understandable to an proficient non-expert?
* code produces all the analyses in the paper
* code produces all the figures in the paper
* code is commented to describe what is what
* documentation of environment, package dependencies, version #s
* how-to guide for running the code
- documentation of manual steps, where they occur in the pipeline (e.g. data cleaning)
- describing what different components of the code do (could be literate programming, or comments)
* the code reliably runs into the future (computational reproducibility)
* flexibility to modify inputs/parameters (ideally)
* what is appropriate to share? w.r.t. old code that is commented out

## Breakout Group 1 (remote + Ethan + Ellen)

### How do you tell if research code meets acceptable standards or is "good enough"?

* Can you read the code itself?
* Small amount of work required to get from reading it to running it?
* Does it run? Does it produce output from the manuscript?
* Runnable examples/something to demonstrate that the code does what you think it does.
* Independent of platform
* Documentation - can you understand how the code relates to the goal of the analysis?

### What practices or guidelines do you currently use to check research code? What practices or guidelines would you be willing to adopt?

* adding tests to code (make sure functions do what they do) and for the whole analysis (does the data as input produce the same results and figures)
* asking other people in the lab to read your code and vice-versa (pair programming review?)
* going over style guide as a group? (some way to devote time to it, look at examples)
* use tools like roxygen or formatting to produce consistent documentation (with internal lab community practices and external scientific community)
* guidelines for how to structure workflows (file/folder organization, saving intermediate objects, naming, etc.) - provide examples for other members of the lab
* follow structure of an R package (if working in R)
* more lab demos and training on practices

## Breakout Group 2 (unconference room)

### How do you tell if research code meets acceptable standards or is "good enough"?

* must run on anyone's computer (provide everything in code, or document external packages and data)
* documentation (examples of how to run the code, and different pieces)
- lots of possibilities here
- describe main flow of what portions of code to run in what order (and what they do)
* tests (and code written in functions to be tested) - would be nice
* how many functions per file? (could be 1 function per file or multiple, whatever is sensible for users trying to find the code)
* listing dependencies among functions in the research code (what order are things run in?)
* style guide (use consistent names, don't reuse names if representing different things)
* self-check "if I come back in a year, will I still be able to understand and use the code?"
* clean up previous drafts that may be commented out (for the publication stage)

### What practices or guidelines do you currently use to check research code? What practices or guidelines would you be willing to adopt?

* Current:
* version control
* testing
* documentation ('self-check')
* core code as packages
* documenting dependencies
*
*
* Adopt:
* testing
* peer review (of code)
* runnable examples (formal)
* containers
* documenting dependencies
* dependency management tool - specify package versions
*
*