Try   HackMD

N8 CIR Northern Tour ReproHack Series Notes

tags: Templates Reprohack hackpad

Plan of Action

Welcome and Introduction

  • Into Slides

:computer: Form teams

Feel free to tackle papers individually or as teams.

:dart: Select papers

  • Choose paper from list of proposed papers
  • Register the paper selected and the participants reproducing below. You can copy, paste and edit the following template: 
    ​​​​### **Paper:** <Title of the paper reproduced>
​​​​**Reviewers:** Reviewer 1, Reviewer 2 etc.

:books: Reproduce

  • Attempt to reproduce papers from available materials and documentation
  • Make notes about your experiences, in particular with respect to how easy it is to:
    • :earth_africa: navigate the materials
    • :repeat: reproduce the analysis
    • :recycle: reuse the materials

:memo: Feedback to authors

  • Fill in the author feedback form, documenting your experiences reproducing your chosen paper

Event Hackpad

Newcastle

  • Location: Newcastle University
  • Date: 2020-01-21
  • Participants:
    • Alison Clarke (Durham University / @alisonrclarke )
    • Jonathan Frawley (Durham University)
    • Marion Weinzierl (Durham University)
    • Juan Ojeda (Newcastle University)
    • Victoria Kurushina (Newcastle University)
    • Helen Clare (Jisc / @helenclare)

Reproduction Log

Log details of the papers reproduced and reviewers (see template in Agenda). You can add links to any derived materials produced here.

Paper: <Spectral measure of color variation of black-orange-black (BOB) pattern in small parasitoid wasps (Hymenoptera: Scelionidae), a statistical approach>

Reviewers: Juan Ojeda

Discussion notes

Easy to reproduce. GitHub repository downloaded and code run on RStudio without major problems. The only issue was to manually create a folder called "Figures", without it figures were not saved. Figures were identical to the ones on the paper.

Paper: <Social-evaluative threat: Stress response stages and influences of biological sex and neuroticism>

Reviewers: Marion Weinzierl

Discussion notes

  • I got two plots that matched those in the paper. There was an additional plot and lots of .xlsx files produced, but I could not easily match them up with tables or plots in the paper.
  • The code gave lots of 'object not found' errors which I could not resolve.
  • The code had very good in-code documentation (comments), and a spreadsheet that explained the types of data. However, it did not have a README or other documentation, and the 'Steps to reproduce' description was extremely sparse.
  • The code was one very long script that was structured in sections. A better structuring, using functions and splitting up the code into more files might have helped to read it. Also, more descriptive variable names would be good (and not assuming that abbreviations are understood by someone else).

Paper: < PREPRINT: Using digital epidemiology methods to monitor influenza-like illness in the Netherlands in real-time: the 2017-2018 season>

Reviewers: Alison Clarke, Helen Clare

Discussion notes

  • It would be useful to give a link in the README to download R Studio, for new R users.
  • It would also be good to say which files to run etc, e.g. I tried running manuscript first but that didn't work without running the source code file.
  • "CAVE: Please Check - gtrendsR developer version required!" was output partway through running the code.
    • It might be more obvious to mention this in the README as I missed it the first time running.
    • It would be better to give a date or commit hash for the development version used, as the current development version will change over time. Describing the problem with the current version would also be useful, as then we could check the release notes to see if it's fixed in a later version.

Paper: <A multiscale Bayesian inference approach to analyzing subdiffusion in particle trajectories

K. Hinsen and G.R. Kneller, J. Chem. Phys. 145, 151101 (2016)>
Reviewers: Victoria Kurushina, Jonathan Frawley, Juan Ojeda

Discussion notes

Lots of issues setting this up:

  • Instructions in general very vague, they just point to the "ActivePapers" tool and do not give instructions on how to use it to run code, etc.
  • Requires "ActivePapers" tool which seems like it is not well supported: Documentation has dead links
  • Does not specify which version of ActivePapers to install (There are at least 3: Python 2, Python 3, JVM). I figured out that they wanted Python 2 by process of elimination
  • Some dependencies missing for ActivePapers: h5py, matplotlib, numpy
  • Requires Python 2 which is now at end of life
  • No Windows install instructions, and we could not get it to work
  • And even once we ran this and got access to the data and code, the code does not work with the latest version of ActivePapers.

Paper: <Evaluation of the ‘Irish Rules’: The Potato Late Blight Forecasting Model and Its Operational Use in the Republic of Ireland by Mladen Cucak, Adam Sparks, Rafael de Andrade Moral, Stephen Kildea, Keith Lambkin and Rowan Fealy>

Reviewers: Jonathan Frawley

Discussion notes

  • No docs on how to run it
  • Takes a long time to install dependencies (Over half an hour!)
  • Some errors when running from a clean install that needed to be fixed: bind_rows -> dplyr::bind_rows, rename -> dplyr::rename, add_column -> tibble::add_column
  • Some system level packages required (on Fedora 31): sudo dnf install libxml2 openssl-devel R-curl
  • Confused by the suffix of (1) on some file names, does this indicate a newer version of the code?
  • But! In the end, I was able to reproduce a lot of the plots in the paper!

Feedback on the Feedback form

  • The introductory slide content + the feedback form are currently under active development. They will form the basis of the Reproducibility Review form in the new Hub and the Reproducibility report to be published in ReScience C. The plans and current thinking is outlined in this issue

How could they be improved? Feel free to add suggestions :point_down:

  • Having more time. Finishing at 17:00
  • Making an exhaustive list of software (with versions!) to reproduce ALL the papers and sending this list in advance to the participants for they could install a bunch of things prior to the session and start directly with the reproduction at the workshop.
  • Starting the workshop session with an easy going paper and reproduce the results together with the instructor with the step-by-step guidance.

HPC Reprohack Special Edition

  • Try and discuss at CW2020

Ten Year Reproducibility challenge

Example discussion Topic 1: Remote ReproHacks

  • Can we make this happen?
  • What technologies would we need?
  • What would loose compared to the localised experience? Could we make up for it?

Have python / django skills or just want to get involved? Check out our open issues in the our new under development hub: https://github.com/reprohack/reprohack_site

Group notes

Leeds

  • Location: University of Leeds
  • Date: 2020-02-14
  • Participants:
    • Anna Krystalli (University of Sheffield, @annakrystalli)
    • Isabel Birds (University of Leeds, @IzBirds)
    • Haroldas Bagdonas (University of York, @GABRAHREX)
    • Nick Sheppard (University of Leeds, @OpenResLeeds)
    • Joanna Leng (University of Leeds)
    • Alistair Curd (University of Leeds)
    • Nick Rhodes (University of Leeds)
    • Nujcharee Haswell ()

Reproduction Log

Log details of the papers reproduced and reviewers (see template in Agenda). You can add links to any derived materials produced here.

Paper: </h3> <p class="part" data-startline="177" data-endline="177" data-position="8220" data-size="0"><strong data-position="8220" data-size="0"><span data-position="8222" data-size="10">Reviewers:</span></strong><span data-position="8234" data-size="28"> Reviewer 1, Reviewer 2 etc.</span></p> <h3 class="raw part" data-startline="179" data-endline="179"><strong data-position="8268" data-size="0"><span data-position="8270" data-size="6">Paper:</span></strong><span data-position="8278" data-size="1"> </span><Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity></h3> <p class="part" data-startline="180" data-endline="180" data-position="8376" data-size="0"><strong data-position="8376" data-size="0"><span data-position="8378" data-size="10">Reviewers:</span></strong><span data-position="8390" data-size="28"> Nick Sheppard, Graham Blyth</span></p> <p class="part" data-startline="182" data-endline="182" data-position="8420" data-size="0"><span data-position="8420" data-size="86">Data and code availability clearly available as Supplementary Info (github and zenodo)</span></p> <p class="part" data-startline="184" data-endline="184" data-position="8508" data-size="0"><span data-position="8508" data-size="50">No description or link back to the paper in Zenodo</span></p> <p class="part" data-startline="186" data-endline="186" data-position="8560" data-size="0"><span data-position="8560" data-size="61">Succinct readme outlining archive contents (data and scripts)</span></p> <p class="part" data-startline="188" data-endline="188" data-position="8623" data-size="0"><span data-position="8623" data-size="186">First barrier (as novice and new to R) that I can't install R on UoL machine without admin rights. Colleague suggested Anaconda will exe without admin and includes Rstudio so trying that</span><span class="smartypants" data-position="8809" data-size="3">…</span></p> <p class="part" data-startline="190" data-endline="195" data-position="8814" data-size="0"><span data-position="8814" data-size="52">R project creatated by pointing at top level folder.</span><br> <span data-position="8867" data-size="44">Tried to run Species-table - packages needed</span><br> <span data-position="8912" data-size="7">ggplot2</span><br> <span data-position="8920" data-size="9">tidyverse</span><br> <span data-position="8930" data-size="5">tidyr</span><br> <span data-position="8936" data-size="18">tinyTex two stages</span></p> <p class="part" data-startline="197" data-endline="197" data-position="8956" data-size="0"><span data-position="8956" data-size="41">Able to reproduce one of graphs in paper.</span></p> <h3 class="raw part" data-startline="200" data-endline="200"><strong data-position="9004" data-size="0"><span data-position="9006" data-size="6">Paper:</span></strong><span data-position="9014" data-size="144"> <Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis></span></a></h3> <p class="part" data-startline="201" data-endline="201" data-position="9163" data-size="0"><strong data-position="9163" data-size="0"><span data-position="9165" data-size="10">Reviewers:</span></strong><span data-position="9177" data-size="19"> Haroldas Bagdonas(</span><strong data-position="9196" data-size="0"><span data-position="9198" data-size="5">MacOS</span></strong><span data-position="9205" data-size="15">), Joanna Leng(</span><strong data-position="9220" data-size="0"><span data-position="9222" data-size="10">Windows 10</span></strong><span data-position="9234" data-size="1">)</span></p> <h3 class="raw part" data-startline="202" data-endline="202"><span data-position="9240" data-size="16">Discussion notes</span></h3> <p class="part" data-startline="203" data-endline="203" data-position="9258" data-size="0"><span data-position="9259" data-size="8">General:</span></p> <ul class="part" data-startline="204" data-endline="206" data-position="9270" data-size="0"> <li class="raw" data-startline="204" data-endline="206" data-position="9272" data-size="0"><span data-position="9272" data-size="10">In github </span><a href="http://readme.md"><span data-position="9282" data-size="9">readme.md</span></a><span data-position="9291" data-size="191">, we think that the authors should consider providing hardware information and give as much detail as possible about hardware configurations that were successful at running jupyter notebooks.</span><br> <span class="smartypants" data-position="9484" data-size="2">–</span><span data-position="9486" data-size="144"> Also consider setting recommended and minimum hardware requirements - running these computations on consumer-grade hardware may be impractical.</span></li> </ul> <p class="part" data-startline="207" data-endline="207" data-position="9633" data-size="0"><span data-position="9633" data-size="6">MacOS:</span></p> <ul class="part" data-startline="208" data-endline="218" data-position="9641" data-size="0"> <li class="raw" data-startline="208" data-endline="208" data-position="9643" data-size="0"><span data-position="9643" data-size="115">All the code is well formulated and written in Jupyter notebook, which is a significant benefit to reproducibility.</span></li> <li class="raw" data-startline="209" data-endline="209" data-position="9762" data-size="0"><span data-position="9762" data-size="53">The package requirement list is present. Huge benefit</span></li> <li class="raw" data-startline="210" data-endline="215" data-position="9819" data-size="0"><span data-position="9819" data-size="109">In general the use of jupyter notebook and having everything there is excellent. Perfect for reproducibility.</span><br> <span class="smartypants" data-position="9930" data-size="2">–</span><span data-position="9932" data-size="72"> Notebook 0: NetCDF4 fails at runtime. "RuntimeError: NetCDF: HDF error"</span><br> <span class="smartypants" data-position="10005" data-size="2">–</span><span data-position="10007" data-size="31"> Notebook 1: Works as intended.</span><br> <span class="smartypants" data-position="10039" data-size="2">–</span><span data-position="10041" data-size="98"> Notebook 2: Works as intended. Long time to wait for computations to finish on consumer Notebook.</span><br> <span class="smartypants" data-position="10140" data-size="2">–</span><span data-position="10142" data-size="31"> Notebook 3: Works as intended.</span><br> <span class="smartypants" data-position="10175" data-size="2">–</span><span data-position="10177" data-size="60"> The rest of the notebooks assumed to be working as inteded.</span></li> <li class="raw" data-startline="216" data-endline="216" data-position="10246" data-size="0"><span data-position="10246" data-size="209">Should consider organising output files better in the code. Currently it saves all files in current directory with all the source .ipynb files. Can get difficult to navigate, when all notebooks finish running.</span></li> <li class="raw" data-startline="217" data-endline="218" data-position="10460" data-size="0"><span data-position="10460" data-size="239">In conclusion and compared within the context, the authors of this paper did a good job at making their analysis relatively reproducible. Jupyter notebook can be attributed as the main factor of this analysis being relatively reproducible.</span></li> </ul> <p class="part" data-startline="219" data-endline="219" data-position="10703" data-size="0"><span data-position="10704" data-size="8">Windows:</span></p> <ul class="part" data-startline="221" data-endline="226" data-position="10716" data-size="0"> <li class="raw" data-startline="221" data-endline="226" data-position="10719" data-size="0"><strong data-position="10719" data-size="0"><span data-position="10721" data-size="8">jupytext</span></strong><span data-position="10731" data-size="103"> dependancy not available on user's Windows 10, therefore can't get the Jupyer environment initialized.</span></li> </ul> <h3 class="raw part" data-startline="227" data-endline="227"><strong data-position="10845" data-size="0"><span data-position="10847" data-size="6">Paper:</span></strong><span data-position="10855" data-size="1"> </span><Quantitive analysis of spec>

Paper: <Cell Contractility Facilitates Alignment of Cells and Tissues to Static Uniaxial Stretch>

(C and MATLAB)

Reviewers: Alistair Curd

  • Documentation consists of instructions to install C compiling software, and an initial command to type.
  • I installed the recommended C compilation software through Miniconda, avoiding problems with administrator privileges.
  • This software failed to compile with errors on my Windows 10 system.

Paper: <Sea level regulated tetrapod diversity dynamics through the Jurassic/Cretaceous interval>

Reviewers: Isabel Birds. (MacOS)

  • Methods dense and not clear.
  • "Raw" data in supplementary has been manipulated.
  • Could not replicate actual raw data from PaleoDB - some detail missing?
  • Gave up.

Paper: <Where should new parkrun events be located? Modelling the potential impact of 200 new events on socio-economic inequalities in access and participation>

Reviewers: Stefano Maffei, Nujcharee Haswell (Ped).

Discussion notes

  • Shinny app is very good and easy to navigate. Successfully ran this locally.
  • Suggestion on alternative of color legend Red/Yellow/Green. Not sure if this is accessible?
  • Some comments on how to run the script in individual R files however it would beneficial to have all instructions in Readme.md
  • Can successfully re-produce p_imd_dist_runs plot as per the paper.
  • No documentation on package dependency
  • It would be useful in vignette to have details of packages and versions last succcessfully ran.
  • src2_data_proc.R - england_sp object is commented out but referred to elsewhere
  • line 144 - error srvd_lsoa merging data frames as number of rows didnt match

Paper: <Multiscale Bayesian inference approach to analyzing subdiffusion in particle trajectories>

Reviewers: Alistair Curd, Reviewer 2 etc.

(Python)

  • In Active Papers format. The Active Papers Python tool did not install to use as documentation says it should. Attempting to access the README through h5py was also unsuccessful.

Paper: <Bayesian determination of the effect of a deep eutectic solvent on the structure of lipid monolayers>

Reviewers: Haroldas Bagdonas(MacOS)

Discussion notes

  • Can't get conda environment setup on MacOS Mojave(v10.14). Some dependancies are impossible to compile.
    – I assume the author ran this analysis on Linux, as one of the packages require gcc to compile. Apple MacOS uses clang to compile C++/C code.
  • Otherwise, I am pretty certain that everything would be working if I was on my Linux machine.
    – The readme.md and jupyter notebooks look relatively complete and offer enough information for the user to find solutions for themselves in case specific dependancies would fail.
  • Didn't manage to go through the reproducibility in detail, due to failing to get conda environment properly setup.

Paper: <Evaluation of the ‘Irish Rules’: The Potato Late Blight Forecasting Model and Its Operational Use in the Republic of Ireland>

Reviewers: Isabel Birds (MacOS)

Discussion notes

  • Straightforward to run in RMarkdown, but not obvious to a new user. Eg how to access data.
  • Comprehensive list of R packages used given, but not version numbers - could cause issues in future.
  • Need to have more clarity for data. E.g. the source of historical weather data (Met Éireann synoptic weather station) is given, but no further information, eg a key. Perhaps a clear download link including metadata etc?
  • Clear description of quality control justifications.
  • Multiple versions of Rmd files on Github - why?
  • Overall good, but improvement possible.

Paper: A multiscale Bayesian inference approach to analyzing subdiffusion in particle trajectories

Reviewers: Nick Rhodes (Linux/Debian)

Discussion notes

  • Packaged code,data and Results together using ActivePapers in a single file made easily accesible on Zenodo.
  • Provided no instructions of how to install software dependancies required to run the provided code.
  • No reference to versions of software dependancies required.
  • By referencing the ActivePapers website I was able to install activepapers via conda and libraries required. Note ActivePapers provides a non-conda install method (for users of Linux).
  • No readme provided within the ActivePapers file, but some clues given in a report.md file.

Paper: <Ecological Network assembly: how the regional meta web influence local food webs>

Reviewers: Euan McDonnell (Linux/Debian)

Discussion notes

Paper: <Algorithm configuration data mining for CMA evolution strategies>

Reviewers: Stefano Maffei, Nujcharee Haswell (Ped)

Discussion notes (Python)

  • Repo is binder compatible
  • Repo has requirements.txt with list of packages dependencies and versions required to test / deploy
  • Successfully run the first diagram however the rest of notebook takes too long to run (10+ mins) for block 4 so we stopped.

Discussion notes

  • Target platforms - Joanna Leng mentioned that it appeared code was designed to run on HPC - this information is very helpful to include.

Group notes

Liverpool

Reproduction Log

Log details of the papers reproduced and reviewers (see template in Agenda). You can add links to any derived materials produced here.

Paper: </h3> <p class="part" data-startline="357" data-endline="357" data-position="17604" data-size="0"><strong data-position="17604" data-size="0"><span data-position="17606" data-size="10">Reviewers:</span></strong><span data-position="17618" data-size="28"> Reviewer 1, Reviewer 2 etc.</span></p> <h3 class="raw part" data-startline="360" data-endline="360"><span data-position="17653" data-size="16">Discussion notes</span></h3> <!-- Feel free to use this space to add topics for discussion and make notes on those discussions. You can remove if you don't plan to have any guided discussion sessions --> <h3 class="raw part" data-startline="365" data-endline="365"><span data-position="17853" data-size="11">Group notes</span></h3> <!-- Any other notes you'd like to add. --> <h3 class="raw part" data-startline="368" data-endline="368"><strong data-position="17915" data-size="0"><span data-position="17917" data-size="6">Paper:</span></strong><span data-position="17925" data-size="88"> Growth Dynamics of Independent Gametophytes of Pleurosoriopsis makinoi ( Polypodiaceae)</span></h3> <p class="part" data-startline="369" data-endline="369" data-position="18014" data-size="0"><strong data-position="18014" data-size="0"><span data-position="18016" data-size="10">Reviewers:</span></strong><span data-position="18028" data-size="13"> Marie Phelan</span></p> <h3 class="raw part" data-startline="371" data-endline="371"><strong data-position="18047" data-size="0"><span data-position="18049" data-size="6">Paper:</span></strong><span data-position="18057" data-size="1"> </span><a href="https://hannes.muehleisen.org/p852-muehleisen.pdf"><span data-position="18059" data-size="64">Don’t Hold My Data Hostage – A Case For Client Protocol Redesign</span></a></h3> <p class="part" data-startline="372" data-endline="372" data-position="18176" data-size="0"><strong data-position="18176" data-size="0"><span data-position="18178" data-size="10">Reviewers:</span></strong><span data-position="18190" data-size="29"> Cai Wingfield (MacOS Mojave)</span></p> <ul class="part" data-startline="374" data-endline="380" data-position="18221" data-size="0"> <li class="raw" data-startline="374" data-endline="374" data-position="18223" data-size="0"><a href="https://github.com/Mytherin/Protocol-Benchmarks"><span data-position="18224" data-size="15">Code repository</span></a><span data-position="18289" data-size="116"> linked to from the paper's PDF, which is very useful. The data deposition is then linked from the code repository.</span></li> <li class="raw" data-startline="375" data-endline="380" data-position="18408" data-size="0"><a href="https://zenodo.org/record/1305845"><span data-position="18409" data-size="4">Data</span></a><span data-position="18449" data-size="182"> is deposited inside an Ubuntu virtual machine image, which is available from a location with a DOI, which is nice. Also this means dependency management is presumably not an issue.</span> <ul data-position="18636" data-size="0"> <li class="raw" data-startline="376" data-endline="376" data-position="18638" data-size="0"><span data-position="18638" data-size="114">Instrutions for running relevant queries and the resultant output locations is given alongside the image download.</span></li> <li class="raw" data-startline="377" data-endline="377" data-position="18759" data-size="0"><span data-position="18759" data-size="9">Requires </span><a href="https://www.virtualbox.org/"><span data-position="18769" data-size="10">VirtualBox</span></a><span data-position="18809" data-size="51"> to be installed, but this is free and open source.</span></li> <li class="raw" data-startline="378" data-endline="380" data-position="18867" data-size="0"><span data-position="18867" data-size="83">The encapsulated machine and data image is over 50GB. Waiting for that to complete</span><span class="smartypants" data-position="18950" data-size="3">…</span> <ul data-position="18962" data-size="0"> <li class="raw" data-startline="379" data-endline="380" data-position="18964" data-size="0"><span data-position="18964" data-size="94">Unfortunately the download keeps failing after 30–40GB so I haven't been able to access it yet</span><span class="smartypants" data-position="19058" data-size="3">…</span></li> </ul> </li> </ul> </li> </ul> <h3 class="raw part" data-startline="381" data-endline="381"><strong data-position="19067" data-size="0"><span data-position="19069" data-size="6">Paper:</span></strong><span data-position="19077" data-size="1"> </span><a href="https://scipost.org/10.21468/SciPostPhys.7.3.039"><span data-position="19079" data-size="63">Supercurrent-induced Majorana bound states in a planar geometry</span></a></h3> <p class="part" data-startline="382" data-endline="382" data-position="19194" data-size="0"><strong data-position="19194" data-size="0"><span data-position="19196" data-size="9">Reviewer:</span></strong><span data-position="19207" data-size="29"> Cai Wingfield (MacOS Mojave)</span></p> <ul class="part" data-startline="384" data-endline="401" data-position="19238" data-size="0"> <li class="raw" data-startline="384" data-endline="384" data-position="19240" data-size="0"><span data-position="19240" data-size="51">All relevant files for reproduction contained in a </span><a href="https://zenodo.org/record/2653483"><span data-position="19292" data-size="14">single archive</span></a><span data-position="19342" data-size="71"> with a DOI, which was nice, but it wasn't referenced or linked in the </span><a href="https://scipost.org/SciPostPhys.7.3.039/pdf"><span data-position="19414" data-size="11">paper's PDF</span></a><span data-position="19471" data-size="18"> that I could see.</span></li> <li class="raw" data-startline="385" data-endline="387" data-position="19492" data-size="0"><code data-position="19493" data-size="9">README.md</code><span data-position="19503" data-size="107"> included instructions on how to install the dependencies and which files were releant to the reproduction.</span> <ul data-position="19615" data-size="0"> <li class="raw" data-startline="386" data-endline="387" data-position="19617" data-size="0"><code data-position="19618" data-size="5">conda</code><span data-position="19624" data-size="56"> installation following README instructions worked fine.</span> <ul data-position="19689" data-size="0"> <li class="raw" data-startline="387" data-endline="387" data-position="19691" data-size="0"><span data-position="19691" data-size="11">Inluding a </span><code data-position="19703" data-size="5">conda</code><span data-position="19709" data-size="43">-readable versioned dependencies file is a </span><em data-position="19752" data-size="0"><span data-position="19753" data-size="4">huge</span></em><span data-position="19758" data-size="6"> help!</span></li> </ul> </li> </ul> </li> <li class="raw" data-startline="388" data-endline="394" data-position="19767" data-size="0"><span data-position="19767" data-size="88">Using a Jupyter notebook is a really handy way to make reproduction easy and convenient.</span> <ul data-position="19860" data-size="0"> <li class="raw" data-startline="389" data-endline="389" data-position="19862" data-size="0"><span data-position="19862" data-size="62">No specific instructions on how to run Jupyter notebook files.</span></li> <li class="raw" data-startline="390" data-endline="390" data-position="19931" data-size="0"><code data-position="19932" data-size="7">jupyter</code><span data-position="19940" data-size="129"> wasn't specificaly included in list of dependencies… but I'm using PyCharm which helps out with this kind of thing so it was ok.</span></li> <li class="raw" data-startline="391" data-endline="393" data-position="20076" data-size="0"><span data-position="20076" data-size="109">Some errors when running the notebook, but didn't seem to actually cause any problems to the generated files:</span> <ul data-position="20194" data-size="0"> <li class="raw" data-startline="392" data-endline="392" data-position="20196" data-size="0"><span data-position="20196" data-size="76">"Notebook kernel does not match project kernel" (can be "fixed" in PyCharm).</span></li> <li class="raw" data-startline="393" data-endline="393" data-position="20283" data-size="0"><span data-position="20283" data-size="59">"Notebook is not trusted, JavaScript hasn't been executed".</span></li> </ul> </li> <li class="raw" data-startline="394" data-endline="394" data-position="20349" data-size="0"><span data-position="20349" data-size="299">Running the notebook produces and saves PDFs of the figures in the paper, but does not reproduce the entire paper itself. Some people like the whole paper to produced by the code, so that a corrected error is automatically encorporated into a generated PDF. (Personally I have no opinion on this.)</span></li> </ul> </li> <li class="raw" data-startline="395" data-endline="395" data-position="20651" data-size="0"><span data-position="20651" data-size="38">Data is stored in undocumented binary </span><code data-position="20690" data-size="2">.p</code><span data-position="20693" data-size="127"> files, but as this is seemingly a simulation paper, the data files are actually generated by the notebook, which is very neat.</span></li> <li class="raw" data-startline="396" data-endline="397" data-position="20823" data-size="0"><span data-position="20823" data-size="150">First run (loading precomputed data) produced most figures from the paper identically (figures not produced were just schematic figures and diagrams).</span> <ul data-position="20978" data-size="0"> <li class="raw" data-startline="397" data-endline="397" data-position="20980" data-size="0"><strong data-position="20980" data-size="0"><span data-position="20982" data-size="4">Yay!</span></strong></li> </ul> </li> <li class="raw" data-startline="398" data-endline="401" data-position="20991" data-size="0"><span data-position="20991" data-size="51">Second run (recomputing data) is taking a long time</span><span class="smartypants" data-position="21042" data-size="3">…</span><span data-position="21045" data-size="104"> but the file does say it will require a cluster to run in a reasonable time, and I'm using a laptop. ;)</span> <ul data-position="21154" data-size="0"> <li class="raw" data-startline="399" data-endline="401" data-position="21156" data-size="0"><span data-position="21156" data-size="130">It didn't finish running in the time we had available, so I couldn't verify if the included precomputed data files were identical.</span> <ul data-position="21295" data-size="0"> <li class="raw" data-startline="400" data-endline="401" data-position="21297" data-size="0"><span data-position="21297" data-size="67">This also makes including the precomputed files very convenient! :)</span></li> </ul> </li> </ul> </li> </ul> <h3 class="raw part" data-startline="402" data-endline="402"><strong data-position="21378" data-size="0"><span data-position="21380" data-size="6">Paper:</span></strong><span data-position="21388" data-size="1"> </span><a href="https://www.joelnitta.com/publication/2019-03-27_pleurosoriopsis/"><span data-position="21390" data-size="86">Growth Dynamics of Independent Gametophytes of Pleurosoriopsis makinoi (Polypodiaceae)</span></a></h3> <p class="part" data-startline="403" data-endline="403" data-position="21545" data-size="0"><strong data-position="21545" data-size="0"><span data-position="21547" data-size="10">Reviewers:</span></strong><span data-position="21559" data-size="29"> Cai Wingfield (MacOS Mojave)</span></p> <ul class="part" data-startline="404" data-endline="410" data-position="21589" data-size="0"> <li class="raw" data-startline="404" data-endline="407" data-position="21591" data-size="0"><span data-position="21591" data-size="91">I didn't have time to delve into this one too much, but I followed the instructions on the </span><a href="https://github.com/joelnitta/pleurosoriopsis"><span data-position="21683" data-size="15">code repository</span></a><span data-position="21745" data-size="38"> and I got a PDF which reproduced the </span><a href="https://drive.google.com/uc?export=download&id=1N_fgFtD2l279R-ZhCUrT1HTvEt5xZIdM"><span data-position="21784" data-size="13">published one</span></a><span data-position="21880" data-size="1">.</span> <ul data-position="21886" data-size="0"> <li class="raw" data-startline="405" data-endline="405" data-position="21888" data-size="0"><span data-position="21888" data-size="76">The code repository was linked to from the paper's PDF, which was very nice.</span></li> <li class="raw" data-startline="406" data-endline="406" data-position="21971" data-size="0"><span data-position="21971" data-size="125">The formatting was a little different (linked pdf was 2 columns, generated one was 1), but figures and tables seem identical.</span></li> <li class="raw" data-startline="407" data-endline="407" data-position="22103" data-size="0"><strong data-position="22103" data-size="0"><span data-position="22105" data-size="5">Nice!</span></strong></li> </ul> </li> <li class="raw" data-startline="408" data-endline="408" data-position="22115" data-size="0"><span data-position="22115" data-size="179">Really nice for all code, data and text to be included in the same repository, this was was very straightforward to run, once I'd got Docker to work, which took a bit of fiddling.</span></li> <li class="raw" data-startline="409" data-endline="410" data-position="22297" data-size="0"><span data-position="22297" data-size="51">To follow the instructions on the code repository, </span><a href="https://docs.docker.com/install/"><span data-position="22349" data-size="6">Docker</span></a><span data-position="22390" data-size="119"> was required, which seems like a lot of machinery. However it worked, even if I didn't understand exactly what I did.</span></li> </ul> <h3 class="raw part" data-startline="411" data-endline="411"><strong data-position="22515" data-size="0"><span data-position="22517" data-size="6">Paper:</span></strong><span data-position="22525" data-size="1"> </span><a href="https://dl.acm.org/doi/10.1145/3071178.3071205"><span data-position="22527" data-size="64">Algorithm configuration data mining for CMA evolution strategies</span></a></h3> <p class="part" data-startline="412" data-endline="412" data-position="22641" data-size="0"><strong data-position="22641" data-size="0"><span data-position="22643" data-size="10">Reviewers:</span></strong><span data-position="22655" data-size="29"> Cai Wingfield (MacOS Mojave)</span></p> <ul class="part" data-startline="414" data-endline="431" data-position="22686" data-size="0"> <li class="raw" data-startline="414" data-endline="414" data-position="22688" data-size="0"><span data-position="22688" data-size="81">The paper was not open-access, so could not be downloaded from the specified link</span><span class="smartypants" data-position="22769" data-size="3">…</span><span data-position="22772" data-size="45"> luckily I have institutional access via VPN.</span></li> <li class="raw" data-startline="415" data-endline="415" data-position="22820" data-size="0"><span data-position="22820" data-size="23">The paper links to the </span><a href="https://github.com/sjvrijn/cma-es-configuration-data-mining"><span data-position="22844" data-size="15">code repository</span></a><span data-position="22921" data-size="22">, which is very handy!</span></li> <li class="raw" data-startline="416" data-endline="416" data-position="22946" data-size="0"><span data-position="22946" data-size="60">I worked out I needed to use Python 2.7 instead of Python 3.</span></li> <li class="raw" data-startline="417" data-endline="424" data-position="23009" data-size="0"><span data-position="23009" data-size="13">The specific </span><code data-position="23023" data-size="11">pip install</code><span data-position="23035" data-size="53"> command given in the instructions produced an error:</span> <div class="part code-block-wrapper" data-startline="418" data-endline="420" data-infoprefix-length="0" data-position="23093"> <pre><code>​​​​ERROR: Can not perform a '--user' install. User site-packages are not visible in this virtualenv. </code></pre> </div> <span data-position="23211" data-size="98">(this may be me not knowing how to use Python 2 virtual environments properly). Rerunning without </span><code data-position="23310" data-size="6">--user</code><span data-position="23317" data-size="56"> seemed to work ok, with the obvious Python 2.7 warning:</span> <div class="part code-block-wrapper" data-startline="422" data-endline="424" data-infoprefix-length="0" data-position="23378"> <pre><code>​​​​DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support </code></pre> </div> </li> <li class="raw" data-startline="425" data-endline="431" data-position="23730" data-size="0"><span data-position="23730" data-size="16">There is also a </span><a href="http://mybinder.org/repo/energya/cma-es-configuration-data-mining"><span data-position="23747" data-size="11">Binder link</span></a><span data-position="23826" data-size="27">, which is also very handy.</span> <ul data-position="23858" data-size="0"> <li class="raw" data-startline="426" data-endline="431" data-position="23860" data-size="0"><span data-position="23860" data-size="39">There were some trivial warnings, e.g.:</span> <div class="part code-block-wrapper" data-startline="427" data-endline="430" data-infoprefix-length="0" data-position="23908"> <pre><code>​​​​​​​​/home/main/.local/lib/python2.7/site-p ackages/matplotlib/font_manager.py:1331: UserWarning: findfont: Font family [u'sans-serif'] not found. Falling back to DejaVu Sans ​​​​​​​​ (prop.get_family(), self.defaultFamily[fontext])) </code></pre> </div> </li> </ul> </li> </ul> <h3 class="raw part" data-startline="432" data-endline="432"><strong data-position="24172" data-size="0"><span data-position="24174" data-size="6">Paper:</span></strong><span data-position="24182" data-size="1"> </span><a href="https://pubs.rsc.org/en/content/articlelanding/2019/CP/C9CP00203K#!divAbstract"><span data-position="24184" data-size="100">Bayesian determination of the effect of a deep eutectic solvent on the structure of lipid monolayers</span></a></h3> <p class="part" data-startline="433" data-endline="433" data-position="24366" data-size="0"><strong data-position="24366" data-size="0"><span data-position="24368" data-size="10">Reviewers:</span></strong><span data-position="24380" data-size="23"> Matthew Carter (MacOS)</span></p> <ul class="part" data-startline="435" data-endline="445" data-position="24405" data-size="0"> <li class="raw" data-startline="435" data-endline="435" data-position="24407" data-size="0"><span data-position="24407" data-size="70">Data, code and paper stored in separate repositories but easy to find.</span></li> <li class="raw" data-startline="436" data-endline="436" data-position="24480" data-size="0"><span data-position="24480" data-size="56">Nice to include all the figures from the report as pdfs.</span></li> <li class="raw" data-startline="437" data-endline="437" data-position="24539" data-size="0"><span data-position="24539" data-size="77">Instructions to setup the Conda environment and compile notebooks were clear.</span></li> <li class="raw" data-startline="438" data-endline="438" data-position="24619" data-size="0"><span data-position="24619" data-size="50">Packages installed after fixing an issue with pip.</span></li> <li class="raw" data-startline="439" data-endline="439" data-position="24672" data-size="0"><span data-position="24672" data-size="35">Simple to compile and run the code.</span></li> <li class="raw" data-startline="440" data-endline="440" data-position="24710" data-size="0"><span data-position="24710" data-size="110">Could more clearly label variables (as someone not familiar with the field, I don't know what 'sp1,2,3' mean).</span></li> <li class="raw" data-startline="441" data-endline="441" data-position="24823" data-size="0"><span data-position="24823" data-size="106">Previous point meant it wasn't intuitive how to run the Python files manually (i.e. without the makefile).</span></li> <li class="raw" data-startline="442" data-endline="442" data-position="24932" data-size="0"><span data-position="24932" data-size="190">Could include one makefile to build the project from start to finish, and another which only compiles LaTeX code. This would make it easy to update the report when running the code manually.</span></li> <li class="raw" data-startline="443" data-endline="443" data-position="25125" data-size="0"><span data-position="25125" data-size="139">Good to include a small dataset as well as the full dataset (note: still takes ages to run the 'small' dataset, need an even smaller one?).</span></li> <li class="raw" data-startline="444" data-endline="445" data-position="25267" data-size="0"><span data-position="25267" data-size="81">Could not verify if the results were reproducible due to the runtime of the code.</span></li> </ul> <h3 class="raw part" data-startline="446" data-endline="446"><strong data-position="25354" data-size="0"><span data-position="25356" data-size="6">Paper:</span></strong><span data-position="25364" data-size="1"> </span><a href="https://aip.scitation.org/doi/10.1063/1.4965881"><span data-position="25366" data-size="106">Communication: A multiscale Bayesian inference approach to analyzing subdiffusion in particle trajectories</span></a></h3> <p class="part" data-startline="447" data-endline="447" data-position="25523" data-size="0"><strong data-position="25523" data-size="0"><span data-position="25525" data-size="10">Reviewers:</span></strong><span data-position="25537" data-size="23"> Matthew Carter (MacOS)</span></p> <ul class="part" data-startline="449" data-endline="454" data-position="25562" data-size="0"> <li class="raw" data-startline="449" data-endline="449" data-position="25564" data-size="0"><span data-position="25564" data-size="85">No instructions on how to install the required packages etc. and get the code to run.</span></li> <li class="raw" data-startline="450" data-endline="450" data-position="25652" data-size="0"><span data-position="25652" data-size="129">Found 'ActivePapers' extremely unintuitive to work with, would've preferred to have the code, data etc. on something like Github.</span></li> <li class="raw" data-startline="451" data-endline="451" data-position="25784" data-size="0"><span data-position="25784" data-size="88">Attempted to extract data and code from AP however, the extracted data files were blank.</span></li> <li class="raw" data-startline="452" data-endline="452" data-position="25875" data-size="0"><span data-position="25875" data-size="122">State which version of Python is used. AP isn't compatible between Python 2 and 3 making it hard to reproduce the results.</span></li> <li class="raw" data-startline="453" data-endline="454" data-position="26000" data-size="0"><span data-position="26000" data-size="93">Unfortunately, couldn't get AP to work, so I couldn't verify the reproducibility of code etc.</span></li> </ul> <h3 class="raw part" data-startline="455" data-endline="455"><strong data-position="26099" data-size="0"><span data-position="26101" data-size="6">Paper:</span></strong><span data-position="26109" data-size="1"> </span><a href="https://pubs.rsc.org/en/content/articlelanding/2019/CP/C9CP00203K#!divAbstract"><span data-position="26111" data-size="100">Bayesian determination of the effect of a deep eutectic solvent on the structure of lipid monolayers</span></a></h3> <p class="part" data-startline="456" data-endline="456" data-position="26293" data-size="0"><strong data-position="26293" data-size="0"><span data-position="26295" data-size="10">Reviewers:</span></strong><span data-position="26307" data-size="20"> Manhui Wang (Linux)</span></p> <ul class="part" data-startline="458" data-endline="466" data-position="26329" data-size="0"> <li class="raw" data-startline="458" data-endline="458" data-position="26331" data-size="0"><span data-position="26331" data-size="31">Generally good notes to follow.</span></li> <li class="raw" data-startline="459" data-endline="459" data-position="26365" data-size="0"><span data-position="26365" data-size="112">Dependency packages are clearly mentioned. It will be good to note the main Python version and Anaconda version.</span></li> <li class="raw" data-startline="460" data-endline="460" data-position="26480" data-size="0"><span data-position="26480" data-size="78">pip installation could be very tricky if you don't have sudo permission ("pip </span><span class="smartypants" data-position="26558" data-size="2">–</span><span data-position="26560" data-size="66">user" could install dependency packages in user's home directory).</span></li> <li class="raw" data-startline="461" data-endline="461" data-position="26629" data-size="0"><span data-position="26629" data-size="105">It may take many hours to run all the analysis. It will be good to provide smaller analysis to reproduce.</span></li> <li class="raw" data-startline="462" data-endline="466" data-position="26737" data-size="0"><span data-position="26737" data-size="75">It will require some dependency packages to generate the pdf file on Linux.</span></li> </ul> <h3 class="raw part" data-startline="467" data-endline="467"><strong data-position="26821" data-size="0"><span data-position="26823" data-size="6">Paper:</span></strong><span data-position="26831" data-size="169"> Where should new parkrun events be located? Modelling the potential impact of 200 new events on geographical and socioeconomic inequalities in access and participation.</span></h3> <p class="part" data-startline="468" data-endline="468" data-position="27001" data-size="0"><strong data-position="27001" data-size="0"><span data-position="27003" data-size="10">Reviewers:</span></strong><span data-position="27015" data-size="20"> Fran Biggin (MacOS)</span></p> <ul class="part" data-startline="470" data-endline="479" data-position="27037" data-size="0"> <li class="raw" data-startline="470" data-endline="470" data-position="27039" data-size="0"><span data-position="27039" data-size="136">General comment: Really nice Shiny App. If viewing before reading the paper it would be nice to have some explanantion of the variables.</span></li> <li class="raw" data-startline="471" data-endline="471" data-position="27178" data-size="0"><span data-position="27178" data-size="175">Everything needed to obtain the data and run the code is available on GitHub, but it would be nice to have some brief detail in the Readme on how to download and run the code.</span></li> <li class="raw" data-startline="472" data-endline="472" data-position="27356" data-size="0"><span data-position="27356" data-size="59">File structure within the src folder is nice and intuitive.</span></li> <li class="raw" data-startline="473" data-endline="473" data-position="27418" data-size="0"><span data-position="27418" data-size="126">R scripts are well commented and I particularly like the the fact that links to open source data are included in the comments.</span></li> <li class="raw" data-startline="474" data-endline="474" data-position="27547" data-size="0"><span data-position="27547" data-size="183">Some problems in src2_data_proc.R: line 102 commented out so subsequent lines don't run; line 149 and 150 create error messages; line 167 produces warninngs regarding missing variables.</span></li> <li class="raw" data-startline="475" data-endline="475" data-position="27735" data-size="0"><span data-position="27735" data-size="90">minor point: rgdal is a required package, but isn't included in the install_n_load function.</span></li> <li class="raw" data-startline="476" data-endline="476" data-position="27830" data-size="0"><span data-position="27830" data-size="236">Some of the summary stats are out when compared to the ones reported in the paper. E.g. Mean distance from LSOA to nearest parkrun 4.74 compared to 4.66. I think this relates back to the lines that were unable to be run in src2_data_proc.</span></li> <li class="raw" data-startline="477" data-endline="477" data-position="28071" data-size="0"><span data-position="28071" data-size="383">As you progress with the analysis script it gets harder to relate the code and output to the relevant parts of the paper, so it's difficult to judge whether the reproduction of the stats matches the paper. Maybe a list of variable names and their descriptions and locations in the paper (e.g. section/table number) would be useful as a comment at the start of each chunk of analysis.</span></li> <li class="raw" data-startline="478" data-endline="479" data-position="28457" data-size="0"><span data-position="28457" data-size="59">In order to create the map figure you need a GoogleMaps API</span></li> </ul> <h3 class="raw part" data-startline="480" data-endline="480"><strong data-position="28522" data-size="0"><span data-position="28524" data-size="6">Paper:</span></strong><span data-position="28532" data-size="142"> Quantitative analysis of spectroscopic Low Energy Electron Microscopy data: High-dynamic range imaging, drift correction and cluster analysis</span></h3> <p class="part" data-startline="481" data-endline="488" data-position="28675" data-size="0"><strong data-position="28675" data-size="0"><span data-position="28677" data-size="10">Reviewers:</span></strong><span data-position="28689" data-size="28"> Kostas Alexandridis (Linux)</span><br> <span data-position="28675" data-size="93">*Some minor issues with installing the packages, (e.g. I had to include a channel in anaconda)</span><br> <span data-position="28675" data-size="132">*Data link was not working I had to manually download all the file from the website. It was not very straightforward but it was okay.</span><br> <span data-position="28675" data-size="158">*I liked the fact that each section of the paper correspond to a specific jupyter notebook. So, the user could understand what the code was doing in each step.</span><br> <span data-position="28675" data-size="112">*A potential improvement could be a general makefile that can compile the whole project and generate all results.</span><br> <span data-position="28675" data-size="46">*I had to install some extra packages like h5py</span><br> <span data-position="28675" data-size="165">*In the accuracy_testing notebook, I could run the notebook as it gave me a memory error, for opera browser, when I switch to chromium it worked, but it was very slow.</span><br> <span data-position="28675" data-size="126">*The PCA components graphs were almost identicall, I am not the expert to juddge but all graphs seemed the same with the paper.</span></p> <h3 class="raw part" data-startline="490" data-endline="490"><span data-position="29571" data-size="16">Discussion Notes</span></h3> <pre class="part" data-startline="491" data-endline="491" data-position="29588"><code class="code_block">​​​​* </code></pre> <h3 class="raw part" data-startline="494" data-endline="494"><span data-position="29601" data-size="10">Resources:</span></h3> <h4 class="raw part" data-startline="496" data-endline="496"><span data-position="29618" data-size="6">Binder</span></h4> <ul class="part" data-startline="498" data-endline="500" data-position="29626" data-size="0"> <li class="raw" data-startline="498" data-endline="498" data-position="29628" data-size="0"><a href="https://mybinder.readthedocs.io/en/latest/sample_repos.html"><span data-position="29629" data-size="26">Sample Binder Repositories</span></a></li> <li class="raw" data-startline="499" data-endline="500" data-position="29720" data-size="0"><a href="https://github.com/alan-turing-institute/the-turing-way/tree/master/workshops/boost-research-reproducibility-binder"><span data-position="29721" data-size="47">Boost Your Research Reproducibility with Binder</span></a><span data-position="29886" data-size="31">: Turing Way workshop materials</span></li> </ul> <h4 class="raw part" data-startline="501" data-endline="501"><code data-position="29925" data-size="7">rrtools</code></h4> <ul class="part" data-startline="503" data-endline="505" data-position="29935" data-size="0"> <li class="raw" data-startline="503" data-endline="503" data-position="29937" data-size="0"><a href="https://github.com/benmarwick/rrtools"><code data-position="29939" data-size="7">rrtools</code><span data-position="29947" data-size="8"> package</span></a></li> <li class="raw" data-startline="504" data-endline="505" data-position="29998" data-size="0"><a href="https://annakrystalli.me/rrtools-repro-research/"><span data-position="29999" data-size="39">Reproducible Research in R with rrtools</span></a><span data-position="30089" data-size="20">: workshop materials</span></li> </ul> <hr> <h4 class="raw part" data-startline="509" data-endline="509"><span data-position="30122" data-size="16">Cautionary tale:</span></h4> <ul class="part" data-startline="510" data-endline="511" data-position="30139" data-size="0"> <li class="raw" data-startline="510" data-endline="511" data-position="30141" data-size="0"><a href="https://www.ncbi.nlm.nih.gov/pubmed/31591889"><span data-position="30142" data-size="185">Characterization of Leptazolines A-D, Polar Oxazolines from the Cyanobacterium Leptolyngbya sp., Reveals a Glitch with the "Willoughby-Hoye" Scripts for Calculating NMR Chemical Shifts.</span></a><span data-position="30374" data-size="6">: see </span><a href="https://pubs.acs.org/doi/10.1021/acs.orglett.9b03216#tbl2"><span data-position="30381" data-size="7">table 2</span></a><span data-position="30448" data-size="63"> for differences in calculated values with Mac, Windows & Linux</span></li> </ul> <h1 class="raw part" data-startline="512" data-endline="512"><span data-position="30515" data-size="9">Sheffield</span></h1> <div class="alert alert-info part" data-startline="514" data-endline="524" data-position="30526" data-size="450"> <ul data-position="30534" data-size="0"> <li class="raw" data-startline="515" data-endline="515" data-position="30536" data-size="0"><strong data-position="30536" data-size="0"><span data-position="30538" data-size="9">Location:</span></strong><span data-position="30549" data-size="1"> </span><em data-position="30550" data-size="0"><span data-position="30551" data-size="23">University of Sheffield</span></em></li> <li class="raw" data-startline="516" data-endline="516" data-position="30578" data-size="0"><strong data-position="30578" data-size="0"><span data-position="30580" data-size="5">Date:</span></strong><span data-position="30587" data-size="1"> </span><em data-position="30588" data-size="0"><span data-position="30589" data-size="10">2020-03-10</span></em></li> <li class="raw" data-startline="517" data-endline="524" data-position="30603" data-size="0"><strong data-position="30603" data-size="0"><span data-position="30605" data-size="13">Participants:</span></strong> <ul data-position="30625" data-size="0"> <li class="raw" data-startline="518" data-endline="518" data-position="30627" data-size="0"><span data-position="30627" data-size="38">Jane Doe (Affiliation / twitter / etc)</span></li> <li class="raw" data-startline="519" data-endline="519" data-position="30672" data-size="0"><span data-position="30672" data-size="51">Florencia D'Andrea (INTA. Argentina / @cantoflor_87)</span></li> <li class="raw" data-startline="520" data-endline="520" data-position="30731" data-size="0"><span data-position="30731" data-size="53">David Wilby (University of Sheffield / @DrDavidWilby)</span></li> <li class="raw" data-startline="521" data-endline="521" data-position="30791" data-size="0"><span data-position="30791" data-size="57">Robert (Bob) Turner (University of Sheffield / @bobatron)</span></li> <li class="raw" data-startline="522" data-endline="522" data-position="30855" data-size="0"><span data-position="30855" data-size="57">Anna Krystalli (University of Sheffield / @annakrystalli)</span></li> <li class="raw" data-startline="523" data-endline="524" data-position="30919" data-size="0"><span data-position="30919" data-size="51">Simon Rolph (University of Sheffield / @simon_rolph)</span></li> </ul> </li> </ul> </div> <h2 class="raw part" data-startline="527" data-endline="527"><span data-position="30981" data-size="16">Reproduction Log</span></h2> <p class="part" data-startline="529" data-endline="529" data-position="30999" data-size="0"><span data-position="30999" data-size="134">Log details of the papers reproduced and reviewers (see template in Agenda). You can add links to any derived materials produced here.</span></p> <h3 class="raw part" data-startline="531" data-endline="531"><strong data-position="31140" data-size="0"><span data-position="31142" data-size="6">Paper:</span></strong><span data-position="31150" data-size="1"> </span><Title of the paper reproduced></h3> <p class="part" data-startline="532" data-endline="532" data-position="31183" data-size="0"><strong data-position="31183" data-size="0"><span data-position="31185" data-size="10">Reviewers:</span></strong><span data-position="31197" data-size="28"> Reviewer 1, Reviewer 2 etc.</span></p> <h3 class="raw part" data-startline="535" data-endline="535"><strong data-position="31232" data-size="0"><span data-position="31234" data-size="6">Paper:</span></strong><span data-position="31242" data-size="132"> [Population structure and phenotypic variation of Sclerotinia sclerotiorum from dry bean (Phaseolus vulgaris) in the United States]</span></h3> <p class="part" data-startline="537" data-endline="537" data-position="31376" data-size="0"><strong data-position="31376" data-size="0"><span data-position="31378" data-size="10">Reviewers:</span></strong><span data-position="31390" data-size="31"> Florencia D'Andrea, Juan Ojeda</span></p> <p class="part" data-startline="539" data-endline="540" data-position="31423" data-size="0"><span data-position="31423" data-size="59">Some problems installing Docker and running RStudio server.</span><br> <span data-position="31483" data-size="191">The Rmd was not straightforward to use. There were too many scripts to reproduce the figures from the paper, most of the scripts produced multiple tables and figures not present in the paper.</span></p> <h3 class="raw part" data-startline="543" data-endline="543"><strong data-position="31681" data-size="0"><span data-position="31683" data-size="6">Paper:</span></strong><span data-position="31691" data-size="1"> </span><a href="https://www.sciencedirect.com/science/article/pii/S0272494418300549?via%3Dihub"><span data-position="31693" data-size="112">Explicit (but not implicit) environmentalist identity predicts pro-environmental behavior and policy preferences</span></a></h3> <p class="part" data-startline="544" data-endline="544" data-position="31887" data-size="0"><strong data-position="31887" data-size="0"><span data-position="31889" data-size="10">Reviewers:</span></strong><span data-position="31901" data-size="11"> Jim Uttley</span></p> <ul class="part" data-startline="545" data-endline="555" data-position="31913" data-size="0"> <li class="raw" data-startline="545" data-endline="545" data-position="31915" data-size="0"><span data-position="31915" data-size="225">Not too clear where to start with the reproduction - presumably some parts rely on completion of other parts - e.g. cleaning of data. Perhaps a readme file giving overview instructions on reproducing the work would be useful.</span></li> <li class="raw" data-startline="546" data-endline="546" data-position="32143" data-size="0"><span data-position="32143" data-size="61">Unable to reproduce SPSS stuff as my SPSS licence had run out</span></li> <li class="raw" data-startline="547" data-endline="547" data-position="32207" data-size="0"><span data-position="32207" data-size="89">Only looking at reproducing meta-analyses and SEMs (parts done in R), as do not use STATA</span></li> <li class="raw" data-startline="548" data-endline="548" data-position="32299" data-size="0"><span data-position="32299" data-size="82">Was it not possible to do all data cleaning and analysis in one software (e.g. R)?</span></li> <li class="raw" data-startline="549" data-endline="549" data-position="32384" data-size="0"><span data-position="32384" data-size="156">Error messages when trying to access ‘mvmeta’, ‘igraph’ and ‘psych’ packages (not included in the ‘wants’ variable. Manually installed these three packages.</span></li> <li class="raw" data-startline="550" data-endline="550" data-position="32543" data-size="0"><span data-position="32543" data-size="110">When reading in ‘merged’ dataset, had warning error of 37633 parsing failures - not sure if this is a problem?</span></li> <li class="raw" data-startline="551" data-endline="551" data-position="32656" data-size="0"><span data-position="32656" data-size="20">Plots reproduce well</span></li> <li class="raw" data-startline="552" data-endline="552" data-position="32679" data-size="0"><span data-position="32679" data-size="185">Variable names are re-used in the script, e.g. ‘model’ - fine if running the script all in one go, but if you run different chunks at different times, this caused a few confusing issues</span></li> <li class="raw" data-startline="553" data-endline="553" data-position="32867" data-size="0"><span data-position="32867" data-size="234">Not very explicit which bits of code and analysis relate to which statistical results. I am not particularly familiar with the statistical methods used though, so perhaps it would be more obvious to someone who was familiar with them.</span></li> <li class="raw" data-startline="554" data-endline="555" data-position="33104" data-size="0"><span data-position="33104" data-size="291">Results appear to be reproducible, but had difficulty identifying which statistics reported in the paper relate to which bits of the analysis and code. For ease and transparency of reproduction, the link between the bits of code and what is reported in the paper could be made more explicit.</span></li> </ul> <h3 class="raw part" data-startline="556" data-endline="556"><strong data-position="33401" data-size="0"><span data-position="33403" data-size="6">Paper:</span></strong><span data-position="33411" data-size="1"> </span><a href="https://link.springer.com/content/pdf/10.1007/s10584-015-1580-2.pdf"><span data-position="33413" data-size="116">Spatial modelling of rice yield losses in Tanzania due to bacterial leaf blight and leaf blast in a changing climate</span></a></h3> <p class="part" data-startline="557" data-endline="557" data-position="33600" data-size="0"><strong data-position="33600" data-size="0"><span data-position="33602" data-size="10">Reviewers:</span></strong><span data-position="33614" data-size="12"> David Wilby</span></p> <h5 class="raw part" data-startline="559" data-endline="559"><span data-position="33634" data-size="6">Access</span></h5> <p class="part" data-startline="560" data-endline="562" data-position="33641" data-size="0"><span data-position="33641" data-size="42">GitHub repo was straightforward to access.</span><br> <span data-position="33684" data-size="95">Updating the repo's README to explain which code to run and how would speed up reproducibility.</span><br> <span data-position="33780" data-size="97">README also refers to a data directory, which seems to be created by running one of the R scripts</span></p> <h5 class="raw part" data-startline="564" data-endline="564"><span data-position="33885" data-size="12">Installation</span></h5> <p class="part" data-startline="565" data-endline="567" data-position="33898" data-size="0"><span data-position="33898" data-size="66">Dependent on Windows, and on ArcGIS which is proprietary software.</span><br> <span data-position="33965" data-size="5">Uses </span><code data-position="33971" data-size="5">arcpy</code><span data-position="33977" data-size="61">, python API for ArcGIS. Would be interesting to port to use </span><code data-position="34039" data-size="4">QGIS</code><span data-position="34044" data-size="5"> and </span><code data-position="34050" data-size="6">pyqgis</code><span data-position="34057" data-size="31"> as an open-source alternative.</span><br> <span data-position="34089" data-size="42">Difficulty installing cropsim via R-Forge.</span></p> <h5 class="raw part" data-startline="569" data-endline="569"><span data-position="34139" data-size="4">Data</span></h5> <p class="part" data-startline="570" data-endline="571" data-position="34144" data-size="0"><span data-position="34144" data-size="100">Data was easily obtained once the correct R script was run, however this could be better documented.</span><br> <span data-position="34245" data-size="81">The README in the github repo refers to a data directory, but this isn't present.</span></p> <h5 class="raw part" data-startline="573" data-endline="573"><span data-position="34334" data-size="13">Documentation</span></h5> <p class="part" data-startline="574" data-endline="576" data-position="34348" data-size="0"><span data-position="34348" data-size="83">It was undocumented how to run the code and in what order to reproduce the results.</span><br> <span data-position="34432" data-size="44">For instance to run the first R script, the </span><code data-position="34477" data-size="7">cropsim</code><span data-position="34485" data-size="164"> package is required. A link is provided to the package on R forge, but it is protracted to work out how to install the package. Subversion was required to install.</span><br> <span data-position="34650" data-size="154">The reproducibility of the analysis would be significantly improved by a walkthrough-style guide of how to run the code in order to reproduce the results.</span></p> <h5 class="raw part" data-startline="578" data-endline="578"><span data-position="34812" data-size="8">Analysis</span></h5> <p class="part" data-startline="579" data-endline="581" data-position="34821" data-size="0"><span data-position="34821" data-size="31">Table 1 was reproduced exactly.</span><br> <span data-position="34853" data-size="47">Running 03_Figures_2_and_3.R resulted in the error </span><code data-position="34905" data-size="52">Error in value[j, ] : incorrect number of dimensions</code><span data-position="34958" data-size="11"> at line 36</span><br> <span data-position="34970" data-size="50">Running 04_Figures_4_5_6_and_7.R resulted in the error: </span><code data-position="35027" data-size="67">Error in loadNamespace(name) : there is no package called ‘mapproj’</code></p> <h6 class="raw part" data-startline="585" data-endline="585"><span data-position="35106" data-size="13">Versions etc.</span></h6> <p class="part" data-startline="586" data-endline="588" data-position="35120" data-size="0"><span data-position="35120" data-size="38">Windows 10 running in a virtal machine</span><br> <span data-position="35159" data-size="43">R version 3.6.3 running in RStudio 1.2.5033</span><br> <span data-position="35203" data-size="21">ArcGIS Desktop 10.7.1</span></p> <h5 class="raw part" data-startline="590" data-endline="590"></h5> <h3 class="raw part" data-startline="592" data-endline="592"><strong data-position="35239" data-size="0"><span data-position="35241" data-size="5">Paper</span></strong><span data-position="35248" data-size="1"> </span><a href="https://doi.org/10.3390/insects9040186"><span data-position="35250" data-size="94">Comparisons of Citizen Science Data-Gathering Approaches to Evaluate Urban Butterfly Diversity</span></a></h3> <p class="part" data-startline="593" data-endline="593" data-position="35386" data-size="0"><strong data-position="35386" data-size="0"><span data-position="35388" data-size="9">Reviewer:</span></strong><span data-position="35399" data-size="12"> Simon Rolph</span></p> <p class="part" data-startline="595" data-endline="595" data-position="35414" data-size="0"><span data-position="35414" data-size="174">The doi is attached to v0.9.1 of the code but there is a 0.9.2 available. I meant to work with v0.9.1 but accidently used the master branch, unsure if there are many changes.</span></p> <p class="part" data-startline="597" data-endline="601" data-position="35591" data-size="0"><strong data-position="35591" data-size="0"><span data-position="35593" data-size="6">Access</span></strong><br> <span data-position="35602" data-size="21">Paper is open access.</span><br> <span data-position="35624" data-size="105">Code on zenodo with link to github repo. Can download straight from zenodo but I have cloned from github.</span><br> <span data-position="35730" data-size="62">Readme includes a few sentences on how the repo is structured.</span><br> <span data-position="35793" data-size="97">Pdf documents describing analyses with extra figures that are not included in the original paper.</span></p> <p class="part" data-startline="603" data-endline="604" data-position="35892" data-size="0"><strong data-position="35892" data-size="0"><span data-position="35894" data-size="12">Installation</span></strong><br> <span data-position="35909" data-size="117">R packages needed are in scripts as library(package) with no indication of the version number used for this analysis.</span></p> <p class="part" data-startline="606" data-endline="607" data-position="36028" data-size="0"><strong data-position="36028" data-size="0"><span data-position="36030" data-size="4">Data</span></strong><br> <span data-position="36037" data-size="65">Data is clearly separated from code but not very well documented.</span></p> <p class="part" data-startline="609" data-endline="609" data-position="36105" data-size="0"><span data-position="36105" data-size="220">Data is stored as csv files. But there is also a bash script for downloading inaturalist data via a GBIF doi. This is good because it means they're not storing a big file on GitHub but this isn't indicated in the readme.</span></p> <p class="part" data-startline="611" data-endline="611" data-position="36327" data-size="0"><span data-position="36327" data-size="97">In order to download this file you must go to a doi link and download the file into a new folder </span><code data-position="36425" data-size="9">data/gbif</code><span data-position="36435" data-size="129"> and run the bash script. I worked out where I had to put the file from the error so clearer instructions would have been useful.</span></p> <p class="part" data-startline="613" data-endline="613" data-position="36566" data-size="0"><span data-position="36566" data-size="183">Reading the paper is needed to understand the data. There are three sources of data which this analysis compares. For each of the sources there are different versions of the data eg. </span><code data-position="36750" data-size="15">BioScanData.csv</code><span data-position="36766" data-size="5"> and </span><code data-position="36772" data-size="23">BioScanDataComplete.csv</code></p> <p class="part" data-startline="615" data-endline="615" data-position="36798" data-size="0"><span data-position="36798" data-size="75">Some column names are not documented eg. opler-wright-min, opler-wright-max</span></p> <p class="part" data-startline="617" data-endline="618" data-position="36875" data-size="0"><strong data-position="36875" data-size="0"><span data-position="36877" data-size="13">Documentation</span></strong><br> <span data-position="36893" data-size="400">Not clear script to run first. Includes instructions to create an output folder. How do the Rmd files relate to the scripts? It is fairly clear when you go into the Rmd scripts what everything does, which is ok for a project of this size and complexity but if the project had been more complex it would be very useful to have a overview readme of an order to run the scripts and what each of them do.</span></p> <p class="part" data-startline="620" data-endline="620" data-position="37295" data-size="0"><strong data-position="37295" data-size="0"><span data-position="37297" data-size="8">Analysis</span></strong></p> <p class="part" data-startline="622" data-endline="622" data-position="37309" data-size="0"><code data-position="37310" data-size="28">gbif-additional-processing.R</code><span data-position="37339" data-size="167"> tries to load a file that doesn't exist. This file has to be downloaded by from GBIF then proccessed using included bash script but it's unclear that it was required.</span></p> <p class="part" data-startline="624" data-endline="624" data-position="37508" data-size="0"><span data-position="37508" data-size="17">Attempted to use </span><code data-position="37526" data-size="12">sites-plot.R</code><span data-position="37539" data-size="51"> to draw the map in the paper but it gave an error:</span></p> <p class="part" data-startline="626" data-endline="627" data-position="37593" data-size="0"><code data-position="37594" data-size="110">Error in scalebar(data = latlongs.scale, dist = 5, location = "bottomleft", : transform should be logical.</code></p> <p class="part" data-startline="629" data-endline="629" data-position="37707" data-size="0"><span data-position="37707" data-size="41">Assumed to be to due to an update in the </span><code data-position="37749" data-size="4">ggsn</code><span data-position="37754" data-size="133"> package. Listing the version numbers of the packages would make this more reproducible and prevent any future code-breaking changes.</span></p> <p class="part" data-startline="631" data-endline="631" data-position="37889" data-size="0"><span data-position="37889" data-size="34">When plotting the map without the </span><code data-position="37924" data-size="8">scalebar</code><span data-position="37933" data-size="28"> function it gave the error </span><code data-position="37962" data-size="36">there is no package called ‘mapproj’</code><span data-position="37999" data-size="74">. After installing the package it worked fine. There's no indication that </span><code data-position="38074" data-size="7">mapproj</code><span data-position="38082" data-size="147"> was needed to be installed. This might have been the cause for the other packages that were required but I happened to have them installed anyway.</span></p> <p class="part" data-startline="633" data-endline="633" data-position="38231" data-size="0"><span data-position="38231" data-size="101">Two figures in the paper could be reproduced. R markdown documents are nicely put toegther and clear.</span></p> <p class="part" data-startline="635" data-endline="635" data-position="38334" data-size="0"><strong data-position="38334" data-size="0"><span data-position="38336" data-size="7">Summary</span></strong></p> <p class="part" data-startline="637" data-endline="637" data-position="38347" data-size="0"><span data-position="38347" data-size="285">Analysis was reproducible, all stats and figures (and more) could be reproduced but the overall workflow is not initially clear to the user. A more extended readme would have been useful. Including package version numbers and dependencies should prevent some bugs. Generally good! 8/10</span></p> <h3 class="raw part" data-startline="640" data-endline="640"><span data-position="38640" data-size="16">Discussion notes</span></h3> <!-- Feel free to use this space to add topics for discussion and make notes on those discussions. You can remove if you don't plan to have any guided discussion sessions --> <p class="part" data-startline="644" data-endline="646" data-position="38835" data-size="0"><span data-position="38835" data-size="5">Rob: </span><a href="https://github.com/lsaravia/MetawebsAssembly"><span data-position="38840" data-size="44">https://github.com/lsaravia/MetawebsAssembly</span></a><br> <span data-position="38885" data-size="67">Progress: From a clean R install, had to install various packages. </span><code data-position="38953" data-size="10">doParallel</code><span data-position="38964" data-size="149"> had to be installed manually and wasn't listed anywhere (that i've found). Chunks have to be ran individually, can't use knit (noted at top of rmd).</span><br> <span data-position="39114" data-size="64">I think it was failing to open PNG device because the directory </span><code data-position="39179" data-size="7">Figures</code><span data-position="39187" data-size="125"> did not exist, but it takes ~40 minutes per chunk so will be a while before i can see if creating that directory has helped.</span></p> <blockquote class="raw part" data-startline="647" data-endline="653"> <p data-position="39315" data-size="0"><span data-position="39315" data-size="83">Error in png("Figures/RegionalFoodWeb.png", width = 6, height = 6, units = "in", :</span><br> <span data-position="39404" data-size="28">unable to start png() device</span><br> <span data-position="39435" data-size="30">In addition: Warning messages:</span><br> <span data-position="39468" data-size="80">1: In png("Figures/RegionalFoodWeb.png", width = 6, height = 6, units = "in", :</span><br> <span data-position="39553" data-size="61">unable to open file 'Figures/RegionalFoodWeb.png' for writing</span><br> <span data-position="39617" data-size="80">2: In png("Figures/RegionalFoodWeb.png", width = 6, height = 6, units = "in", :</span><br> <span data-position="39702" data-size="21">opening device failed</span></p> </blockquote> <h3 class="raw part" data-startline="656" data-endline="656"><span data-position="39730" data-size="11">Group notes</span></h3> <!-- Any other notes you'd like to add. --> <p class="part" data-startline="659" data-endline="660" data-position="39788" data-size="0"><a href="https://github.com/annakrystalli/literate-programming"><span data-position="39789" data-size="31">Rmarkdown: literate-programming</span></a><span data-position="39876" data-size="19"> workshop materials</span><br> <a href="https://github.com/annakrystalli/lit-prog"><span data-position="39897" data-size="35">Literate programming with rmarkdown</span></a><span data-position="39976" data-size="40">: Sheffield R Users Group talk materials</span></p> <hr> <h1 class="raw part" data-startline="666" data-endline="666"><span data-position="40027" data-size="10">Manchester</span></h1> <div class="alert alert-info part" data-startline="668" data-endline="677" data-position="40039" data-size="352"> <ul data-position="40047" data-size="0"> <li class="raw" data-startline="669" data-endline="669" data-position="40049" data-size="0"><strong data-position="40049" data-size="0"><span data-position="40051" data-size="9">Location:</span></strong><span data-position="40062" data-size="1"> </span><em data-position="40063" data-size="0"><span data-position="40064" data-size="24">University of Manchester</span></em></li> <li class="raw" data-startline="670" data-endline="670" data-position="40092" data-size="0"><strong data-position="40092" data-size="0"><span data-position="40094" data-size="5">Date:</span></strong><span data-position="40101" data-size="1"> </span><em data-position="40102" data-size="0"><span data-position="40103" data-size="10">2020-03-12</span></em></li> <li class="raw" data-startline="671" data-endline="671" data-position="40117" data-size="0"><strong data-position="40117" data-size="0"><span data-position="40119" data-size="18">Link to Materials:</span></strong><span data-position="40139" data-size="1"> </span><a href="https://github.com/annakrystalli/n8cir-reprohacks"><span data-position="40140" data-size="49">https://github.com/annakrystalli/n8cir-reprohacks</span></a></li> <li class="raw" data-startline="672" data-endline="677" data-position="40192" data-size="0"><strong data-position="40192" data-size="0"><span data-position="40194" data-size="13">Participants:</span></strong> <ul data-position="40214" data-size="0"> <li class="raw" data-startline="673" data-endline="673" data-position="40216" data-size="0"><span data-position="40216" data-size="56">Anna Krystalli (University of Sheffield / annakrystalli)</span></li> <li class="raw" data-startline="674" data-endline="674" data-position="40279" data-size="0"><span data-position="40279" data-size="39">Daniela Gawehns (Leiden Uni/ @dgawehns)</span></li> <li class="raw" data-startline="675" data-endline="677" data-position="40325" data-size="0"><span data-position="40325" data-size="56">Linda Nab (Leiden University Medical Center/ @lindanab1)</span></li> </ul> </li> </ul> </div> <h2 class="raw part" data-startline="680" data-endline="680"><span data-position="40396" data-size="25">Breakout Rooms and Papers</span></h2> <ul class="part" data-startline="681" data-endline="687" data-position="40422" data-size="0"> <li class="raw" data-startline="681" data-endline="683" data-position="40424" data-size="0"><strong data-position="40424" data-size="0"><span data-position="40426" data-size="13">Paper Awesome</span></strong><span data-position="40441" data-size="7"> Room 1</span> <ul data-position="40453" data-size="0"> <li class="raw" data-startline="682" data-endline="682" data-position="40455" data-size="0"><span data-position="40455" data-size="12">Reviewer one</span></li> <li class="raw" data-startline="683" data-endline="683" data-position="40474" data-size="0"><span data-position="40474" data-size="12">Reviewer two</span></li> </ul> </li> <li class="raw" data-startline="684" data-endline="687" data-position="40489" data-size="0"><strong data-position="40489" data-size="0"><span data-position="40491" data-size="14">Paper Awesome2</span></strong><span data-position="40507" data-size="7"> Room 2</span></li> </ul> <h2 class="raw part" data-startline="688" data-endline="688"><span data-position="40521" data-size="16">Reproduction Log</span></h2> <p class="part" data-startline="690" data-endline="690" data-position="40539" data-size="0"><span data-position="40539" data-size="134">Log details of the papers reproduced and reviewers (see template in Agenda). You can add links to any derived materials produced here.</span></p> <h3 class="raw part" data-startline="692" data-endline="692"><strong data-position="40680" data-size="0"><span data-position="40682" data-size="6">Paper:</span></strong><span data-position="40690" data-size="1"> </span><Title of the paper reproduced></h3> <p class="part" data-startline="693" data-endline="693" data-position="40723" data-size="0"><strong data-position="40723" data-size="0"><span data-position="40725" data-size="10">Reviewers:</span></strong><span data-position="40737" data-size="28"> Reviewer 1, Reviewer 2 etc.</span></p> <h3 class="raw part" data-startline="695" data-endline="695"><strong data-position="40771" data-size="0"><span data-position="40773" data-size="6">Paper:</span></strong><span data-position="40781" data-size="74"> 6. Resolving the Measurement Uncertainty Paradox in Ecological Management</span></h3> <p class="part" data-startline="696" data-endline="696" data-position="40856" data-size="0"><strong data-position="40856" data-size="0"><span data-position="40858" data-size="10">Reviewers:</span></strong><span data-position="40870" data-size="15"> Anna Krystalli</span></p> <h2 class="raw part" data-startline="700" data-endline="700"><span data-position="40892" data-size="16">Discussion notes</span></h2> <h3 class="raw part" data-startline="702" data-endline="702"><span data-position="40915" data-size="10">First Talk</span></h3> <h3 class="raw part" data-startline="705" data-endline="705"><span data-position="40932" data-size="11">Second Talk</span></h3> <h3 class="raw part" data-startline="708" data-endline="708"><span data-position="40950" data-size="10">Third Talk</span></h3> <!-- Feel free to use this space to add topics for discussion and make notes on those discussions. You can remove if you don't plan to have any guided discussion sessions --> <h3 class="raw part" data-startline="713" data-endline="713"><span data-position="41143" data-size="11">Group notes</span></h3> <!-- Any other notes you'd like to add. --> <hr>

Last changed by 
 
Anna Krystalli
0
940

Read more

Mannheim Open Science Day Reprohack - Collaborative Hackpad

:::info :earth_africa: Remote :calendar: 2022-10-11 :watch: 10:00 - 17:00 UTC :arrow_forward: Slides: https://bit.ly/mheim-reprohack-slides :arrow_forward: Joel&apos;s slide https://joelnitta.github.io/reprohack_2022-10-11 :arrow_forward: Camille&apos;s slide https://clandesv.github.io/reproducible-manuscripts-with-RMD/slides.html#1 :page_with_curl: Paper List:

Oct 11, 2022
HPC ReproHack - Collaborative Hackpad

:::info :earth_africa: Remote :calendar: 2022-03-21 to 2022-03-31 :watch: Start at 10:00 UTC :arrow_forward: Slides: https://bit.ly/hpc-reprohack-slides :arrow_forward: Closing Event Slides: https://bit.ly/hpc-reprohack-closing-slides :page_with_curl: Paper List: Event List: https://reprohack.org/event/14/

Apr 1, 2022
ReproHack Hub Launch - Collaborative Hackpad

:::info :earth_africa: Remote :calendar: 2021-11-18 :watch: 10:00 - 17:00 UTC :arrow_forward: Slides: https://bit.ly/reprohack-hub-launch-slides :page_with_curl: Paper List: Central List: https://reprohack.org/paper/ Event List: https://reprohack.org/event/11/

Mar 9, 2022
ReproHack Meeting Minutes

:::info Location: https://meet.google.com/ssx-dfcj-tvc Date: February 16 2022 15:00 PM (UTC+1), 11:00 AM (UTC-3) Agenda Updates on HPC ReproHack [name=Anna]

Feb 16, 2022
Read more from Anna Krystalli
Published on HackMD