---
# System prepended metadata

title: Gource story line

---

# Gource story line

format is:

TS|Caption

where the timestamp TS can be seen on the preview video frames

### Juseless
- Conducting computationally reproducible processing is beautiful. 
- Let's take a look.
- In 2020, we asked: How can we process the UK Biobank dataset in a reproducible, transparent, and efficient way?
- We set up an analysis for voxel-based morphometry.
- Each dot is a file
- It parallelizes over subjects, using their T1-weighted images
- In total, there were 41180 images to process
- each job run for ~1 hour and creates 4 output tarballs, needs 1 CPU core, 4GB RAM and 5GB diskspace
- The initial analysis was finalized Christmas 2020
- "FAIR share" system, the analysis was implemented as some kind of backfill workload
- initially, we scheduled throughput too cautiously. The cluster was in use, with jobs from dozens of other researchers
- processing was interupted to improve throughput
- With less-cautious scheduling, the overall computing time improved considerably
- throughput was great in the beginning, but slowed substantially beyond 20k results
- improvements removed bottleneck and throughput no longer decreased with growing output dataset

```
0000|Unix was born

```

### Jureca
- Large-scale reproducible processing can also scale up hardware-wise
- Here's the same computation on a supercomputer
- dedicated allocation of 25 nodes with XX TB of RAM, running 125 jobs in parallel
- There was no need for cautious scheduling. As many jobs as possible were submitted at once
- KABOOM
- ... and done

```
0000|Unix was born

```