---
title: Running nf-core pipelines on de.NBI OpenStack and BibiGrid clusters
tags: denbi,nextflow,tutorial,clusters,nf-core,bibigrid,openstack
author: James A. Fellows Yates
date: 2023-08-30
---

# Running nf-core pipelines on de.NBI Openstack clusters

## Introduction

This tutorial describes how you can set up nf-core pipelines (and likely any Nextflow pipeline) on the de.NBI cloud's OpenStack cluster system using the SLURM executor and docker.

## Infrastructure

The de.NBI cloud provides either single virtual machines (VMs) or direct access to their OpenStack cluster. Every project approved by de.NBI comes with different quotas. Those are limits on the number of VMs, virtual cores (VCPUs), total memory (RAM), and so on. It is up to the project managers to create instances (nodes) with the desired operating systems (OS image) and network them. This can all be done via the interface, but may present some challenges as it requires background knowledge that is potentially unfamiliar to researchers.

For that reason, there are several facilities that can help to automate the process of provisioning compute infrastructure. There is the [openstack Python client](https://pypi.org/project/python-openstackclient/), [scripts provided directly by the BiBiGrid](https://github.com/BiBiServ/bibigrid), and among many options we can use [terraform](https://www.terraform.io/) with the [OpenStack provider](https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs). A good general introduction for getting set up can be found in the [BiBiGrid tutorial](https://github.com/deNBI/bibigrid_clum2022). In order to install software on the provisioned infrastructure, we can use a tool like [Ansible](https://www.ansible.com/).

### terraform

Terraform can provision infrastructure such as on OpenStack by taking as input a description of the desired end state and then performing a series of operations to get to that state. It will only modify infrastructure that it considers under its own management. It also maintains a snapshot of the state that was present after the last operation. As a group of researchers, using terraform with state stored in a shared place, such as S3, can be a good fit, as it allows everyone to interact with the common infrastructure of a project. Of course, communication is still key, as we don't want to destroy nodes that are still in use by someone else. However, terraform strongly supports the infrastructure-as-code (IaC) pattern. Which means, recreating a specific infrastructure configuration should always be just one command away.

In order to get started with terraform on BiBiGrid:

1. Follow the [tutorial](https://github.com/deNBI/bibigrid_clum2022) to get oriented and create your credentials
2. Create an object store container (bucket) through the OpenStack dashboard to store the terraform state (done already with bucket name 'tfstate')
3. Follow [this guide](https://docs.pco.get-cloud.io/docs/tutorials/tf-backend-s3/) to configure your access to the bucket

You should then end up with a terraform section that looks similar to the below.

terraform {
  required_version = ">= 0.14.0"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.51"
    }
  }

  backend "s3" {
    bucket                      = "tfstate"
    key                         = "terraform.tfstate"
    region                      = "us-east-1"
    endpoint                    = "https://openstack.cebitec.uni-bielefeld.de:8080"
    skip_credentials_validation = true
    skip_region_validation      = true
    force_path_style            = true
  }
}

provider "openstack" {
  cloud = "openstack"
}

(Please note that the region `"us-east-1"` is fake and only used to satisfy the requirements of the `s3` backend.)

We can then create one or more compute nodes by using the openstack provider, for example, with a resource section as below.

resource "openstack_compute_instance_v2" "manager" {
  name        = "manager"
  flavor_name = "de.NBI mini"
  image_name  = "Ubuntu 22.04 LTS (2023-08-14)"
}

And further S3 buckets to store data

resource "openstack_objectstorage_container_v1" "results" {
  name       = "results"
  versioning = true
}

We can manage networks, security policies, etc. In the same way. We can also extract information about the environment without allowing terraform to make changes.

data "openstack_networking_network_v2" "network" {
  name = "external"
}

In the end, we can provision our desired infrastructure using

terraform apply

and when we are done, we can remove everything using

terraform destroy