# Tower.nf 🗼

## What is it? 🧐

- Web based monitoring for nextflow runs

export TOWER_ACCESS_TOKEN=eyxxxxxxxxxxxxxxxQ1ZTE=
export NXF_VER=22.10.6
nextflow run my-pipelines -with-tower

- Launch nextflow pipelines:
- Cloud based compute resources (GC,AWS,Azure)
- On-premise HPC (SLURM, SGE, etc)
- K8S

Price: https://cloud.tower.nf/pricing/

➡️ https://tower.nf/login

## Use tower.nf with QIB Cloud VM(s)

### Setup a VM for running nextflow & tower
**Requirements**

    - conda/mamba/micromamba
    - docker
    - singularity
    - nextflow
    - tw-agent
    - slurm

* Opt1. ansible https://github.com/thanhleviet/ansible-slurm-tower (10 - 15 mins)

* Opt2. pre-built image `qib-ubuntu-22.04-tower-20230623` (5 mins)

For opt2, once the VM is spawned, it may require to update `slurm.conf`

sudo vim /etc/slurm/slurm.conf
# Line 45: Update RealMemory, CPUs, Sockets
NodeName=localhost RealMemory=60362 CPUs=16 Sockets=16 CoresPerSocket=1 ThreadsPerCore=1

sudo systemctl restart slurmd
sudo systemctl restart slurmctld

### Setup tower
There are two workspace categories in Tower: **User** and **Organisation**
**User**
1. First-thing-first, create a tower token `https://tower.nf/tokens`
2. Create a tower credentials `https://tower.nf/credentials` (can be created on-the-fly via Compute Env)
3. Create a compute environment `https://tower.nf/user/your-account/compute-envs/new`

Back in the VM, open a new `tmux` or `screen` session, run

mkdir work
export TOWER_ACCESS_TOKEN=CHANGE_ME
tw-agent CHANGE_ME  --work-dir=./work

**Organisation**
🤔

### Setup CLIMB S3
https://bryn.climb.ac.uk/teams/V83D0V0XNKZ/buckets

S3 configuration for CLIMB S3 in Tower/Nextflow

aws.client.accessKey            = 'your-access-key' //required
aws.client.secretKey            = 'your-secret-key' //required
aws.client.endpoint             = 's3.climb.ac.uk' //required
aws.client.protocol             = 'https'
aws.client.profile              = 'climb'
aws.client.s3PathStyleAccess    = true //required

### Launch pipelines via Tower web from **user workspace**

- 👋 hello-world https://github.com/nextflow-io/hello

- 🦖 Epi2me `wf-metagenomics` https://github.com/epi2me-labs/wf-metagenomics
**Params:**
- fastq: `${projectDir}/test_data/case02`
- out_dir: `s3://your-bucket/wf-metagenomics-output`

- 🦍 nf-core `fetchngs` https://github.com/nf-core/fetchngs

    SRR23244585
    SRR23244586
    SRR23244587
    SRR23244589
    SRR23244590

https://quadram-bioinfo-demo.s3.climb.ac.uk/PRJNA928401.csv

- 🦕 Telatin's `nextflow-example` pipeline https://github.com/thanhleviet/nextflow-example/tree/2023

### Tower CLI

https://help.tower.nf/23.1/cli/#nextflow-tower-cli

Usage: tw [OPTIONS] [COMMAND]

Nextflow Tower CLI.

Options:
  -t, --access-token=<token>   Tower personal access token (TOWER_ACCESS_TOKEN).
  -u, --url=<url>              Tower server API endpoint URL (TOWER_API_ENDPOINT) [default: 'tower.nf'].
  -o, --output=<output>        Show output in defined format (only the 'json' option is available at the moment).
  -v, --verbose                Show HTTP request/response logs at stderr.
      --insecure               Explicitly allow to connect to a non-SSL secured Tower server (this is not recommended).
  -h, --help                   Show this help message and exit.
  -V, --version                Print version information and exit.

Commands:
  actions              Manage actions.
  collaborators        Manage organization collaborators.
  compute-envs         Manage workspace compute environments.
  credentials          Manage workspace credentials.
  datasets             Manage datasets.
  generate-completion  Generate bash/zsh completion script for tw.
  info                 System info and health status.
  labels               Manage labels.
  launch               Launch a Nextflow pipeline execution.
  members              Manage organization members.
  organizations        Manage organizations.
  participants         Manage workspace participants.
  pipelines            Manage workspace pipeline launchpad.
  runs                 Manage workspace pipeline runs.
  teams                Manage organization teams.
  workspaces           Manage workspaces.
  secrets              Manage workspace secrets.

export TOWER_ACCESS_TOKEN=eyxxxxxxxxxxxxxxxQ1ZTE=

tw pipelines list

tw runs list

### Launch pipelines as other members/participants (external collaborators)
...

### BUGS 🐛🪲

- S3 objects might be mistakenly deleted, e.g. you have a "folder" sra and an object `sra.txt` at the same level, if you delete the "folder", `sra.txt` will be deleted too.😱
- Message on workflow completing fails to push to tower.nf if emojis are used in tag, like `tag "💩"`