# Installing Python modules on [Bianca](https://uppmax.uu.se/resources/systems/the-bianca-cluster/) - UPPMAX

Bianca is a research system dedicated for analyzing sensitive personal data from large-scale molecular experiments. Bianca is part of the SNIC-SENS project.

Since Bianca is designed to handle sensitive personal data security is a key aspect of the configuration and features restricted access and lack of direct Internet access to and from the cluster. This make it particularly difficult to use common tools to maintain python modules or software installations.

Here are some tips and advises on different way to approach the problem.

[TOC]

## User installs via pip
pip [documentation](https://pip.pypa.io/en/stable/user_guide/?highlight=user#user-installs)

### Download the necessary modules - [docs.](https://pip.pypa.io/en/stable/reference/pip_download/)
You want to collect the required python modules on a computer with Internet access then transfer the files to Bianca for installation.

>[color=#907bf7] On Rackham (*source machine*)

# Load python from modules
$ module load python/3.9.5
# Update pip, etuptools and wheel
$ python3 -m pip install --user -U  pip setuptools wheel

# Create folder where to download the dependencies for the module
$ mkdir mdownloads
$ cd mdownloads
$ python3 -m pip download module_name

### Transfer the collected files on Bianca
- [Transit user guide](https://uppmax.uu.se/support-sv/user-guides/transit-user-guide/)
- [Bianca user guide](https://www.uppmax.uu.se/support/user-guides/bianca-user-guide/)

### Install modules
> [color=#907bf7] On Bianca (*target machine*)

$ module load python/3.9.5
$ export PYTHONUSERBASE=/proj/snicxxxxx/pythontools

$ python3 -m pip install -U -f ./ --no-index  module_name.whl

### Setup

# make sure you have `PYTHONPATH` and `PATH` setup accordingly.
$ export PYTHONUSERBASE=/proj/snicxxxxx/pythontools
$ export PATH=/proj/snicxxxxx/pythontools/bin:$PATH

Look at this [example](https://hackmd.io/@pmitev/SigProfilerExtractor-on-Bianca) in details.

## User installs via [python-venv](https://docs.python.org/3/tutorial/venv.html)
:::info
Note that this might be done better by using python's venv
https://docs.python.org/3/tutorial/venv.html

**Also**: [User practices: Installation and use of python packages on UPPMAX](/@pmitev/UPPMAX-python-venv)
*Just tar the installation folder and untar it on Bianca and use the same python version.*
:::

**TL;DR** - installing https://github.com/nanoporetech/bonito

>[color=#907bf7] On Rackham (*source machine*)

Collect the module package following the instructions from the previous step.

Copy `mdownload.tar` to Bianca

> [color=#907bf7] On Bianca (*target machine*)

$ module load python/3.9.5

$ cd /proj/PATH/
$ tar -xvf mdownload.tar
$ python3 -m venv bonito
$ source bonito/bin/activate

$(bonito) cd mdownload
$(bonito) python3 -m pip install -f ./ --no-index bonito-xxx.xx.whl
$(bonito) $ bonito -v
bonito 0.3.6

## Using conda

:::success
**Note:** On Bianca there is nearly complete mirror of most common channels, which means that you might be able to manage your **conda** installations the usual way.
:::
The methods above will not allow you to use python version that is not provided by the system setup i.e.

:::spoiler Available Python versions on UPPMAX (2020.12.21)

$ module spider python

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  python:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
     Versions:
        python/2.7.6
        python/2.7.9
        python/2.7.11
        python/2.7.15
        python/3.3
        python/3.3.1
        python/3.4.3
        python/3.5.0
        python/3.6.0
        python/3.6.8
        python/3.7.2

To use another specific version you need to use conda to create new environment with selected Python version.
:::

</br>

[Conda User Guide@UPPMAX](https://uppmax.uu.se/support/user-guides/conda-user-guide/)
[Conda-pack](https://conda.github.io/conda-pack/) a command line tool for creating relocatable conda environments.

:::success
**Hint:** You can install packages with pip that will be packed with the conda environment...
:::

### Create conda environment with selected Python version

> [color=#907bf7] On Rackham /or other linux computer/ (*source machine*)

# Load conda module
$ module load conda
# Setup conda environment 
$ source conda_init.sh
$ export CONDA_ENVS_PATH=/proj/snicxxxxx/conda_enviroments

# create environment with selected Python version
$ conda create -n my_env python=x.x module_name
$ conda activate my_env
$(my_env) 
# make your installation then pack your environment

# Follow the guide at https://conda.github.io/conda-pack/
$(my_env) conda install -c conda-forge conda-pack
# Deactivate the environment
$(my_env) conda deactivate
# Pack environment my_env into out_name.tar.gz
$ conda pack -n my_env -o out_name.tar.gz
...

> [color=#907bf7] On Bianca (*target machine*)

# Load conda module
$ module load conda
# Setup conda environmen 
$ source conda_init.sh
$ export CONDA_ENVS_PATH=/proj/snicxxxxx/conda_enviroments

# Unpack environment into directory `my_env`
#$ cd $CONDA_ENVS_PATH
$ mkdir -p $CONDA_ENVS_PATH/my_env
$ tar -xzf my_env.tar.gz -C $CONDA_ENVS_PATH/my_env

# Activate the environment. This adds `my_env/bin` to your path
$ source $CONDA_ENVS_PATH/my_env/bin/activate

# Cleanup prefixes from in the active environment.
(my_env) $ conda-unpack

# Deactivate the environment to remove it from your path
(my_env) $ source my_env/bin/deactivate

## Contacts:
- [Pavlin Mitev](https://katalog.uu.se/profile/?id=N3-1425)
- [UPPMAX](https://www.uppmax.uu.se/)
- [SNIC AE@UPPMAX - related documentation](/8sqXISVRRquPDSw9o1DizQ)


###### tags: `UPPMAX`, `SNIC`, `RT223073`, `conda`, `pip`