# Snakemake Hackathon
## Instructions for participants using UPPMAX resources

Course project: `NAISS 2024/22-49` (contains resources on both Rackham and Dardel)
Magnetic reservation: `naiss2024-22-49_1` from 09:00-14:00 CET today (reserved nodes will be used by default, no need to specify it in the Slurm job script)
Course project folder: `/proj/introtouppmax/snakemake-hackathon`
Dardel is down this week due to a major update. Feel free to use the resource next week and onwards.

You may use your prefered editor for the files.
Or:
VSCode with Rackham: http://docs.uppmax.uu.se/cluster_guides/vscode_on_rackham/
VDCodium on Bianca: http://docs.uppmax.uu.se/cluster_guides/vscodium_on_bianca/
Bianca is not included as a resource to the course project, you may use your own SUPR project if you wish to run on Bianca instead.

**Environment we will be using during the hackathon**:

module load hyperqueue/0.18.0
source /proj/introtouppmax/snakemake-hackathon/snakemake-8.4.6/venv/bin/activate

If you wish to **rebuild the snakemake environment** created for the course, you may follow the steps below. Please change the `/proj/introtouppmax/snakemake-hackathon` path to a different one.

export PATH=/sw/comp/python/3.11.8/rackham/bin:$PATH
cd /proj/introtouppmax/snakemake-hackathon/snakemake-8.4.6
mkdir /proj/introtouppmax/snakemake-hackathon/snakemake-8.4.6/venv
python -m venv /proj/introtouppmax/snakemake-hackathon/snakemake-8.4.6/venv
source venv/bin/activate
#check `wich pip3` ; it should be in this venv

#Edit `requirements.txt` with the contents:
snakemake==8.4.6
snakemake-executor-plugin-slurm
snakemake-executor-plugin-cluster-generic
pandas
numpy
matplotlib
pygments


pip install -r requirements.txt
cp venv/bin/snakemake .

You may of course tweak if you need additional/less packages for your workflows.

### Scaling exercise
The files for the Snakemake scaling gexercise may be found in the folder `/proj/introtouppmax/snakemake-hackathon/snakemake_scaling`.

Contents of `snakemake_hq_uppmax.sh` (Slurm job example for Snakemake using HyperQueue):

#!/bin/bash
#SBATCH --job-name=snakemake
#SBATCH --account=naiss2024-22-49
#SBATCH --partition=core
#SBATCH --time=00:10:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=20

module load hyperqueue/0.18.0
source /proj/staff/diana/snakemake-hackathon/snakemake-8.4.6/venv/bin/activate

# Specify a location for the HyperQueue server
export HQ_SERVER_DIR=${PWD}/hq-server-${SLURM_JOB_ID}
mkdir -p "${HQ_SERVER_DIR}"
 
# Start the server in the background (&) and wait until it has started
hq server start &
 
# Start the workers in the background and wait for them to start
srun --exact --cpu-bind=none --mpi=none hq worker start --cpus=${SLURM_CPUS_PER_TASK} &

hq worker wait "${SLURM_NTASKS}"

snakemake -s Snakefile --jobs 1 --use-singularity --executor cluster-generic --cluster-generic-submit-cmd "hq submit --cpus 5"

# Wait for all jobs to finish, then shut down the workers and server
hq job wait all
hq worker stop all
hq server stop