Conda Tutorial ================= What exactly is Conda? ---------------------- - A package and environment manager - Like apt/yum, but much more flexible - Environments are isolated from each other - User-contributed package recipes - Different “channels”, can create your own - Updated constantly - Prebuilt binaries - Linked to libraries in the same environment Conda packages -------------- - Specific versions - Various sources ("channels") - Defined requirements - Usually from the same or predefined other channels Conda channels -------------- - conda-forge: Most dependencies (numpy, scipy, zlib, CRAN packages, etc.) - bioconda: Most bioinf packages (salmon, STAR, samtools, DESeq2, etc.) - defaults: Packages built by Anaconda Inc. - Order matters! Use this one: ```bash= $ conda config --show channels channels: - conda-forge - bioconda - defaults $ conda config --prepend channels bioconda $ conda config --prepend channels conda-forge ``` Finding Packages ---------------- - Search <https://anaconda.org> - Use ``conda search`` ```bash= $ conda search pysam Loading channels: done # Name Version Build Channel [...] pysam 0.16.0 py36h71d3148_1 bioconda pysam 0.16.0 py36h873a209_0 bioconda pysam 0.16.0 py37ha9a96c6_0 bioconda pysam 0.16.0 py37hc501bad_1 bioconda pysam 0.16.0.1 py27ha863e18_1 bioconda pysam 0.16.0.1 py36h4c34d4e_1 bioconda pysam 0.16.0.1 py36h71d3148_0 bioconda pysam 0.16.0.1 py37hc334e0b_1 bioconda pysam 0.16.0.1 py37hc501bad_0 bioconda pysam 0.16.0.1 py38hbdc2ae9_1 bioconda ``` - Packages have versions, build numbers, build hashes/strings - Build hashes include dependency information - Bioconda packages are also listed at <http://bioconda.github.io/conda-package_index.html> # Conda environments - A (mostly) self-contained directory with a set of compatible packages - Uses links to reduce disk space when possible - No more conflicting dependencies between versions! ### common commands - ``conda info --envs`` (or ``conda env list``) - Lists available environments ```bash= $ conda info --envs ``` - You start in ``base`` - The ``*`` indicates the active environment - ``conda create``/``conda env remove`` - Create/remove environments ```bash= conda create --name=myenv python=3.8 numpy 'pysam>=0.16' conda env remove --name=myenv ``` - Packages can have versions specified - Min/max versions can be specified - ``conda activate``/``conda deactivate`` - Activates/deactivates an environment ```bash= command -v nanoplot conda activate nanoplot command -v nanoplot conda deactivate command -v nanoplot ``` - ``conda install``/``conda remove`` - ``conda list`` ```bash= $ conda activate myenv $ conda install snakemake ... a lot of status output ... $ conda list ... many packages ... $ conda remove snakemake ``` - Keep your ``base`` env clean! (only the package manager + its deps) - Generously create/remove environments for different tools/workflows! - ``conda env export``/``conda env create`` - Exports or creates an environment from a YAML file ```bash= $ conda env export --no-builds > env.yaml $ conda env create --name=more-map-and-call --file=env.yaml $ head env.yaml name: map-and-call channels: - conda-forge - bioconda - defaults dependencies: - _libgcc_mutex=0.1 - _openmp_mutex=4.5 - bcftools=1.10.2 - blis=0.7.0 ``` ## Introducing ``mamba`` - Newer package manager called ``mamba`` - A reimplementation of ``conda`` - Compatible with ``conda`` - Installed beside ``conda`` in the ``base`` environment - Much faster than ``conda`` ``` conda activate base conda install mamba ``` from now: every time you see `conda` somewhere, replace it by `mamba`! Sometimes it will complain, if so, you can always resort back to `conda` :) ## Common pitfalls - Wrong channel order - Installing packages in your ``base`` env - Manually manipulating ``$PYTHONPATH`` - Avoid manually (i.e., not via ``conda``/``mamba``) installed packages # Practicals ## Practical 1 What are the most recent versions of samtools and Snakemake? ## Practical 2 - Let's create some new environments! - But first make sure that - ``conda`` is up to date ```bash= $ conda activate base $ conda update --all $ conda --version ``` - The correct channel order is in place ```bash= $ conda config --show channels channels: - conda-forge - bioconda - defaults $ conda info ``` ### Practical 2.1 - Create a new environment named "mapping" with ``bwa-mem2`` and ``pysamstats`` - What versions of ``numpy`` and ``python`` got installed in it? ### Practical 2.2 - Create a new environment for Snakemake with ``mamba`` # Q&A - Questions?