Deploying a Kubernetes cluster on Google Cloud for NextFlow pipeline execution.

# Deploying a Kubernetes cluster on Google Cloud for NextFlow pipeline execution. [![N|Solid](https://www.nextflow.io/img/nextflow2014_no-bg.png)](https://nodesource.com/products/nsolid) [![N|Solid](https://cloud.google.com/_static/7cc3d17baa/images/cloud/gcp-logo.svg)](https://nodesource.com/products/nsolid) This tutorial demonstrates how to start a free Kubernetes cluster on the Google Cloud Platform, for free (with $300 to spend) and run NextFlow pipelines on it. # Before you begin! - Create a Google Cloud account https://cloud.google.com/free/ - It will come with $300 / 365 days to spend credit - In order to fully take advantage of scalabiity and be able to run the tutorial you need to upgrade to a paid account (otherwise you are limited to 8 CPUs which might not be enough) https://cloud.google.com/free/docs/frequently-asked-questions#upgrading-billing - You car will not be charged before thr $300 credit is exchausted - but monitor your account for any rogue running instances that charge you after the $300 are consumed - we bear no responsibility ! ### Tutorial We will be using [Ubuntu Juju](https://www.ubuntu.com/cloud/juju). ```sh sudo add-apt-repository --update ppa:juju/stable sudo apt install juju ``` and Kubernetes client by... installing gcloud first https://cloud.google.com/sdk/docs/quickstart-linux and then kubernetes client immediately after https://kubernetes.io/docs/getting-started-guides/ubuntu/installation/ ### Kubernetes deployment on Google Cloud via Juju The hierarchy of JuJu is as following, with nested layers (top to bottom), each having a 1:N relationship with the next layer in the hierarchy. - Cloud - Controller - Model - Charms ```sh juju bootstrap ``` Select "google" and zone "europe-west" or adjust accordingly. It should take you to your browser to athenticate on Google cloud. We will use a Juju "charm" (think of Chef recipe) called Kubernetes-core (https://jujucharms.com/kubernetes-core), that deploys a small Kubernetes cluster (more charms can be found at https://jujucharms.com/) ```sh juju deploy kubernetes-core ``` Then add shared storage with Ceph (commands below picked from the production kubernetes charm https://jujucharms.com/canonical-kubernetes ) ```sh juju deploy cs:ceph-mon -n 3 juju deploy cs:ceph-osd -n 3 juju add-relation ceph-mon ceph-osd ``` ###Login on Kubernetes master ```sh mkdir -p ~/.kube juju scp kubernetes-master/0:config ~/.kube/config juju ssh kubernetes-master/0 ``` ###Run NextFlow (To enable this executor set the property process.executor = 'k8s' in the nextflow.config file.) ```sh https://github.com/nextflow-io/rnatoy ``` ### Note: bugs might be present, will be fixed eventually.