Conda Installation

First create the following

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "$EBROOTANACONDA3/etc/profile.d/conda.sh" ]; then
        . "$EBROOTANACONDA3/etc/profile.d/conda.sh"
    else
        export PATH="$EBROOTANACONDA3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

and save this scriptconda.sh in $HOME directory.

To install conda python package (Tensorflow for example)
login to the compute node. First logon to Lochness and then
srun --partition=datasci --ntasks=1 --mem=10G --pty bash

node430-41 ~ >: module load Anaconda3
node430-41 ~ >: conda create --name tf python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/g/guest24/miniconda3/envs/tf

  added / updated specs:
    - python=3.9


The following packages will be downloaded:

 <output snipped>

Proceed ([y]/n)?y

 <output snipped>
#
# To activate this environment, use
#
#     $ conda activate tf
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Activate the new 'tf' environment

node430-41 ~ >: source $HOME/conda.sh
node430-41 ~ >: conda activate tf
(tf) node430-41 ~ >:

Install tensorflow-gpu

(tf) node430-41 ~ >: conda install -c anaconda tensorflow-gpu
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/g/guest24/miniconda3/envs/tf

  added / updated specs:
    - tensorflow-gpu

<output snipped>

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates                                 pkgs/main --> anaconda
  certifi                                         pkgs/main --> anaconda
  openssl                                         pkgs/main --> anaconda


Proceed ([y]/n)?y

<output snipped>

mkl_fft-1.1.0        | 143 KB    | ####################################################################################### | 100%
urllib3-1.25.9       | 98 KB     | ####################################################################################### | 100%
cudatoolkit-10.1.243 | 513.2 MB  | ####################################################################################### | 100%
protobuf-3.12.3      | 711 KB    | ####################################################################################### | 100%
blinker-1.4          | 21 KB     | ####################################################################################### | 100%
requests-2.24.0      | 54 KB     | ####################################################################################### | 100%
werkzeug-1.0.1       | 243 KB    | ####################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Check to see if tensorflow can be loaded

(tf) node430-41 ~ >: python
Python 3.9.13 (main, Oct 13 2022, 21:15:33)
[GCC 11.2.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Simple tensorflow test program to make sure the virtual env can access a gpu. Program is called tf.gpu.test.py

import tensorflow as tf

if tf.test.gpu_device_name():

    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))

else:

   print("Please install GPU version of TF")

Slurm script to submit the job

#!/bin/bash -l
#SBATCH --job-name=tf_test
#SBATCH --output=%x.%j.out # %x.%j expands to JobName.JobID
#SBATCH --nodes=1
#SBATCH --tasks-per-node=1
#SBATCH --partition=datasci
#SBATCH --gres=gpu:1
#SBATCH --mem=4G

# Purge any module loaded by default
module purge > /dev/null 2>&1
module load Anaconda3
source $HOME/conda.sh
conda activate tf
srun python tf.gpu.test.py

Install PyTorch with GPU

Login to node650 because it's the AVX512 node and faster. From your login node

srun --gres=gpu:1 --partition=datasci --nodelist=node650 --ntasks=16 --mem=10G --pty bash

Create a new environment (DO NOT LOAD ANY EXISTING ENVIRONMENT)

conda create --name torch-cuda python=3.7
conda activate torch-cuda
conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia

Check Torch version

python -c "import torch; print( torch.__version__)"

Check CUDA version

python -c "import torch; print(torch. version .cuda)"

Check whether Torch is compiled with CUDA

python -c "import torch; print(torch.cuda. is_available())"

PyTorch3d

conda create -n pytorch3d python=3.9
conda activate pytorch3d
conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit
conda install pytorch=1.13.0 torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch3d -c pytorch3d

Install picrust2

module load Mamba Miniconda3

mamba create -n picrust2 -c bioconda -c conda-forge picrust2=2.5.1

Once PICRUSt2 is installed, create a file .miniconda3.sh in your $HOME directory and add the following

>>> conda initialize >>>
!! Contents within this block are managed by 'conda init' !!
__conda_setup="$('conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "$EBROOTMINICONDA3/etc/profile.d/conda.sh" ]; then
        . "$EBROOTMINICONDA3/etc/profile.d/conda.sh"
    else
        export PATH="$EBROOTMINICONDA3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

save the file and use

source ~/.miniconda3.sh

conda activate picrust2

PICRUSt2 is now installed, and you can check the installation from the tutorial https://github.com/picrust/picrust2/wiki/PICRUSt2-Tutorial-(v2.5.0)

Next time when you use PICRUSt2 always load the Miniconda3 module, source the .miniconda3.sh, and use conda activate to use picrust2.

So the steps will be

module load Miniconda3

source ~/.miniconda3.sh

conda activate picrust2

Ovito install

conda create -n ovito --strict-channel-priority -c https://conda.ovito.org -c conda-forge ovito