Tsung-Jung Tsai (TJ_Tsai)
    • Create new note
    • Create a note from template
      • Sharing URL Link copied
      • /edit
      • View mode
        • Edit mode
        • View mode
        • Book mode
        • Slide mode
        Edit mode View mode Book mode Slide mode
      • Customize slides
      • Note Permission
      • Read
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Write
        • Only me
        • Signed-in users
        • Everyone
        Only me Signed-in users Everyone
      • Engagement control Commenting, Suggest edit, Emoji Reply
    • Invite by email
      Invitee

      This note has no invitees

    • Publish Note

      Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note No publishing access yet

      Your note will be visible on your profile and discoverable by anyone.
      Your note is now live.
      This note is visible on your profile and discoverable online.
      Everyone on the web can find and read all notes of this public team.

      Your account was recently created. Publishing will be available soon, allowing you to share notes on your public page and in search results.

      Your team account was recently created. Publishing will be available soon, allowing you to share notes on your public page and in search results.

      Explore these features while you wait
      Complete general settings
      Bookmark and like published notes
      Write a few more notes
      Complete general settings
      Write a few more notes
      See published notes
      Unpublish note
      Please check the box to agree to the Community Guidelines.
      View profile
    • Commenting
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
      • Everyone
    • Suggest edit
      Permission
      Disabled Forbidden Owners Signed-in users Everyone
    • Enable
    • Permission
      • Forbidden
      • Owners
      • Signed-in users
    • Emoji Reply
    • Enable
    • Versions and GitHub Sync
    • Note settings
    • Note Insights New
    • Engagement control
    • Make a copy
    • Transfer ownership
    • Delete this note
    • Save as template
    • Insert from template
    • Import from
      • Dropbox
      • Google Drive
      • Gist
      • Clipboard
    • Export to
      • Dropbox
      • Google Drive
      • Gist
    • Download
      • Markdown
      • HTML
      • Raw HTML
Menu Note settings Note Insights Versions and GitHub Sync Sharing URL Create Help
Create Create new note Create a note from template
Menu
Options
Engagement control Make a copy Transfer ownership Delete this note
Import from
Dropbox Google Drive Gist Clipboard
Export to
Dropbox Google Drive Gist
Download
Markdown HTML Raw HTML
Back
Sharing URL Link copied
/edit
View mode
  • Edit mode
  • View mode
  • Book mode
  • Slide mode
Edit mode View mode Book mode Slide mode
Customize slides
Note Permission
Read
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Write
Only me
  • Only me
  • Signed-in users
  • Everyone
Only me Signed-in users Everyone
Engagement control Commenting, Suggest edit, Emoji Reply
  • Invite by email
    Invitee

    This note has no invitees

  • Publish Note

    Share your work with the world Congratulations! 🎉 Your note is out in the world Publish Note No publishing access yet

    Your note will be visible on your profile and discoverable by anyone.
    Your note is now live.
    This note is visible on your profile and discoverable online.
    Everyone on the web can find and read all notes of this public team.

    Your account was recently created. Publishing will be available soon, allowing you to share notes on your public page and in search results.

    Your team account was recently created. Publishing will be available soon, allowing you to share notes on your public page and in search results.

    Explore these features while you wait
    Complete general settings
    Bookmark and like published notes
    Write a few more notes
    Complete general settings
    Write a few more notes
    See published notes
    Unpublish note
    Please check the box to agree to the Community Guidelines.
    View profile
    Engagement control
    Commenting
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    • Everyone
    Suggest edit
    Permission
    Disabled Forbidden Owners Signed-in users Everyone
    Enable
    Permission
    • Forbidden
    • Owners
    • Signed-in users
    Emoji Reply
    Enable
    Import from Dropbox Google Drive Gist Clipboard
       Owned this note    Owned this note      
    Published Linked with GitHub
    • Any changes
      Be notified of any changes
    • Mention me
      Be notified of mention me
    • Unsubscribe
    conda (Anaconda, Miniconda) === ###### tags: `Python`, `conda`, `pip`, `HPC`, `Singularity` ![](https://i.imgur.com/A4bTVx7.png =40%x) :::info :bulb: **關於 conda** - **可以把 conda 想成就是 virtualenv + pip** - `pip list` 相當於 `conda list` - **版本差異** - Anaconda:非常肥大的 conda,什麼套件都幫你裝了,像是 numpy, matplotlib 等 - Miniconda:輕巧版的 conda,一切都要自行安裝 ::: <br> [TOC] <br> <hr> <br> ## conda 安裝指南 > doc: https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html <br> ### Regular installation (一般安裝) - [Windows](https://docs.conda.io/projects/conda/en/latest/user-guide/install/windows.html) - [macOS](https://docs.conda.io/projects/conda/en/latest/user-guide/install/macos.html) - [Linux](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html) (見下面說明) <br> ### Installing on Linux (在 Linux 上安裝) 1. **下載 conda 套件,有兩種安裝套件:** - [Miniconda](https://docs.conda.io/en/latest/miniconda.html#linux-installers) - [Anaconda](https://www.anaconda.com/download/) :::info :bulb: **選擇想要的 python 版本** - 內含 python 套件 - 換言之,即使 local 端沒有安裝對應的 Python 版本 ::: 2. **驗證下載的檔案有沒有完整** ``` $ sha256sum your_filename ``` ``` $ sha256sum Miniconda3-py39_4.10.3-Linux-x86_64.sh 1ea2f885b4dbc3098662845560bc64271eb17085387a70c2ba3f29fff6f8d52f Miniconda3-py39_4.10.3-Linux-x86_64.sh ``` 對照下載點的 SHA256 hash 資訊: [![](https://i.imgur.com/3BFJ23Z.png)](https://i.imgur.com/3BFJ23Z.png) 3. **安裝 conda 指令** - **Miniconda** ```bash= $ bash Miniconda3-latest-Linux-x86_64.sh ``` - 顯示說明 ``` $ bash Miniconda3-latest-Linux-x86_64.sh -h` usage: Miniconda3-latest-Linux-x86_64.sh [options] Installs Miniconda3 py312_24.5.0-0 -b run install in batch mode (without manual intervention), it is expected the license terms (if any) are agreed upon -f no error if install prefix already exists -h print this help message and exit -p PREFIX install prefix, defaults to /home/tjtsai29863/miniconda3, must not contain spaces. -s skip running pre/post-link/install scripts -m disable the creation of menu items / shortcuts -u update an existing installation -t run package tests after installation (may install conda-build) ``` - 安裝到指定目錄: -p <path_to_target> - **Anaconda** ```bash= $ bash Anaconda-latest-Linux-x86_64.sh ``` - 按照提示進行預設安裝、或變更選項 4. **檢查安裝結果** ```bash= $ conda list conda: command not found ``` 蛤? conda 執行檔沒有註冊到全域 `$PATH` ```bash= $ nano ~/.bashrc # 2021.12.02, miniconda PATH="$PATH:/home/tj/miniconda3/bin" $ source ~/.bashrc # 或是開新的 terminal 就生效 ``` - 參考資料 - [【Linux】conda: command not found解决办法](https://blog.csdn.net/weixin_38705903/article/details/86533863) - 測試指令 ```bash= $ conda list # packages in environment at /home/tj/miniconda3: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 4.5 1_gnu brotlipy 0.7.0 py39h27cfd23_1003 ca-certificates 2021.7.5 h06a4308_1 certifi 2021.5.30 py39h06a4308_0 cffi 1.14.6 py39h400218f_0 chardet 4.0.0 py39h06a4308_1003 conda 4.10.3 py39h06a4308_0 conda-package-handling 1.7.3 py39h27cfd23_1 cryptography 3.4.7 py39hd23ed53_0 idna 2.10 pyhd3eb1b0_0 ld_impl_linux-64 2.35.1 h7274673_9 libffi 3.3 he6710b0_2 libgcc-ng 9.3.0 h5101ec6_17 libgomp 9.3.0 h5101ec6_17 libstdcxx-ng 9.3.0 hd4cf53a_17 ncurses 6.2 he6710b0_1 openssl 1.1.1k h27cfd23_0 pip 21.1.3 py39h06a4308_0 pycosat 0.6.3 py39h27cfd23_0 pycparser 2.20 py_2 pyopenssl 20.0.1 pyhd3eb1b0_1 pysocks 1.7.1 py39h06a4308_0 python 3.9.5 h12debd9_4 readline 8.1 h27cfd23_0 requests 2.25.1 pyhd3eb1b0_0 ruamel_yaml 0.15.100 py39h27cfd23_0 setuptools 52.0.0 py39h06a4308_0 six 1.16.0 pyhd3eb1b0_0 sqlite 3.36.0 hc218d9a_0 tk 8.6.10 hbc83047_0 tqdm 4.61.2 pyhd3eb1b0_1 tzdata 2021a h52ac0ba_0 urllib3 1.26.6 pyhd3eb1b0_1 wheel 0.36.2 pyhd3eb1b0_0 xz 5.2.5 h7b6447c_0 yaml 0.2.5 h7b6447c_0 zlib 1.2.11 h7b6447c_3 ``` <br> ### 更新 conda ```bash= $ conda update conda. ``` <br> ### 反安裝 conda - [Uninstalling Anaconda or Miniconda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html#uninstalling-anaconda-or-miniconda) <br> <hr> <br> ## 檢視 conda 系統環境 ### env ```bash= echo $CONDA_DEFAULT_ENV echo $CONDA_PREFIX ``` - [How do I find the name of the conda environment in which my code is running?](https://stackoverflow.com/questions/36539623) ### `conda info` ```bash= $ conda info active environment : None user config file : /home/tj/.condarc populated config files : conda version : 4.10.3 conda-build version : not installed python version : 3.9.5.final.0 virtual packages : __linux=5.4.0=0 __glibc=2.31=0 __unix=0=0 __archspec=1=x86_64 base environment : /home/tj/miniconda3 (writable) conda av data dir : /home/tj/miniconda3/etc/conda conda av metadata url : None channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/tj/miniconda3/pkgs /home/tj/.conda/pkgs envs directories : /home/tj/miniconda3/envs /home/tj/.conda/envs platform : linux-64 user-agent : conda/4.10.3 requests/2.25.1 CPython/3.9.5 Linux/5.4.0-88-generic ubuntu/20.04.3 glibc/2.31 UID:GID : 1000:1000 netrc file : None offline mode : False ``` <br> ### `conda info --env` > 指令同 `$ conda env list` ```bash= $ conda info --env # conda environments: # base * /home/tj/miniconda3 ``` <br> ### `conda list` ```bash= $ conda list # packages in environment at /home/tj/miniconda3: # # Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 4.5 1_gnu brotlipy 0.7.0 py39h27cfd23_1003 ca-certificates 2021.7.5 h06a4308_1 certifi 2021.5.30 py39h06a4308_0 cffi 1.14.6 py39h400218f_0 chardet 4.0.0 py39h06a4308_1003 conda 4.10.3 py39h06a4308_0 conda-package-handling 1.7.3 py39h27cfd23_1 cryptography 3.4.7 py39hd23ed53_0 idna 2.10 pyhd3eb1b0_0 ld_impl_linux-64 2.35.1 h7274673_9 libffi 3.3 he6710b0_2 libgcc-ng 9.3.0 h5101ec6_17 libgomp 9.3.0 h5101ec6_17 libstdcxx-ng 9.3.0 hd4cf53a_17 ncurses 6.2 he6710b0_1 openssl 1.1.1k h27cfd23_0 pip 21.1.3 py39h06a4308_0 pycosat 0.6.3 py39h27cfd23_0 pycparser 2.20 py_2 pyopenssl 20.0.1 pyhd3eb1b0_1 pysocks 1.7.1 py39h06a4308_0 python 3.9.5 h12debd9_4 readline 8.1 h27cfd23_0 requests 2.25.1 pyhd3eb1b0_0 ruamel_yaml 0.15.100 py39h27cfd23_0 setuptools 52.0.0 py39h06a4308_0 six 1.16.0 pyhd3eb1b0_0 sqlite 3.36.0 hc218d9a_0 tk 8.6.10 hbc83047_0 tqdm 4.61.2 pyhd3eb1b0_1 tzdata 2021a h52ac0ba_0 urllib3 1.26.6 pyhd3eb1b0_1 wheel 0.36.2 pyhd3eb1b0_0 xz 5.2.5 h7b6447c_0 yaml 0.2.5 h7b6447c_0 zlib 1.2.11 h7b6447c_3 ``` ### `conda config` > 避開 Anaconda 商業授權問題 - ### 設 strict 模式: `conda config --set channel_priority strict` - 在初始狀態下強調使用高優先權頻道(例如 conda-forge)中的套件,這樣在依賴解析時,若高優先權頻道中有套件可用,就會使用它,而不考慮其他頻道。 - ### 添加 conda-forge 頻道: `conda config --add channels conda-forge` - 將 conda-forge 加入頻道列表。這樣做的目的是把 conda-forge 作為主要來源之一。 <br> <hr> <br> ## 建立 conda 環境 ### 1. `conda create` 建立一個虛擬環境,名稱為 `jupyterlab-ext`,並安裝相關套件 ```bash= $ conda create \ -n jupyterlab-ext \ --override-channels \ --strict-channel-priority \ -c conda-forge \ -c nodefaults \ jupyterlab=3 cookiecutter nodejs jupyter-packaging git ... # # To activate this environment, use # # $ conda activate jupyterlab-ext # # To deactivate an active environment, use # # $ conda deactivate ``` - `-n ENVIRONMENT, --name ENVIRONMENT` > Name of environment. - `--override-channels` > Do not search default or .condarc channels. Requires --channel. - `--strict-channel-priority` > Packages in lower priority channels are not considered if a package with the same name appears in a higher priority channel. > > 如果有相同名稱的套件出現在優先權高的通道中,則不考慮優先權低的通道中的套件。 > 換言之,名稱衝突時,就選優先權高的通道 - `-c CHANNEL, --channel CHANNEL` > Additional channel to search for packages. These are URLs searched in the order they are given (including local directories using the 'file://' syntax or simply a path like '/home/conda/mychan' or '../mychan'). Then, the defaults or channels from .condarc are searched (unless --override-channels is given). You can use 'defaults' to get the default packages for conda. You can also use any name and the .condarc channel_alias value will be prepended. The default channel_alias is http://conda.anaconda.org/. - 套件來源 - `-c conda-forge` 從 `conda-forge` 套件庫,下載要安裝的套件 - 沒有 `-c` 從預設的 `anaconda` 套件庫,下載要安裝的套件 - 也就是說 `jupyterlab` 可以選擇不同的來源(owner/package)下載: - `anaconda/jupyterlab` - `conda-forge/jupyterlab` - 參考資料 - [conda-forge,conda,-c的理解](https://blog.csdn.net/qq_43391414/article/details/115069247) ![](https://i.imgur.com/1AGioPO.png) <br> ### 2. `conda env list` > 查看目前有哪些已建立的 conda 環境 > 指令同 `conda info --env` ```bash= $ conda env # 顯示可用指令 ``` ```bash= $ conda env list # conda environments: # base * /home/tj/miniconda3 jupyterlab-ext /home/tj/miniconda3/envs/jupyterlab-ext ``` 目前虛擬環境位於 `base` <br> ### 3. `conda activate jupyterlab-ext` > 啟動虛擬環境 ```bash= $ conda activate jupyterlab-ext CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'. To initialize your shell, run $ conda init <SHELL_NAME> Currently supported shells are: - bash - fish - tcsh - xonsh - zsh - powershell See 'conda init --help' for more information and options. IMPORTANT: You may need to close and restart your shell after running 'conda init'. ``` 需要對系統環境配置執行 script 類型 <br> ==**解法1**==:這裡有提到:[Using with fish shell](https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html#using-with-fish-shell) ```bash= $ conda init fish no change /home/tj/miniconda3/condabin/conda no change /home/tj/miniconda3/bin/conda no change /home/tj/miniconda3/bin/conda-env no change /home/tj/miniconda3/bin/activate no change /home/tj/miniconda3/bin/deactivate no change /home/tj/miniconda3/etc/profile.d/conda.sh no change /home/tj/miniconda3/etc/fish/conf.d/conda.fish no change /home/tj/miniconda3/shell/condabin/Conda.psm1 no change /home/tj/miniconda3/shell/condabin/conda-hook.ps1 no change /home/tj/miniconda3/lib/python3.9/site-packages/xontrib/conda.xsh no change /home/tj/miniconda3/etc/profile.d/conda.csh modified /home/tj/.config/fish/config.fish ==> For changes to take effect, close and re-open your current shell. <== ``` 再次執行 `conda activate jupyterlab-ext` 還是不行... :::info :bulb: **忘了試 `conda init bash` 可不可行** **2022/03/07** `conda init fish` 是修改 `~/.config/fish/config.fish` `conda init bash` 是修改 `~/.bashrc`,如下所示: ``` $conda init bash no change /root/miniconda3/condabin/conda no change /root/miniconda3/bin/conda no change /root/miniconda3/bin/conda-env no change /root/miniconda3/bin/activate no change /root/miniconda3/bin/deactivate no change /root/miniconda3/etc/profile.d/conda.sh no change /root/miniconda3/etc/fish/conf.d/conda.fish no change /root/miniconda3/shell/condabin/Conda.psm1 no change /root/miniconda3/shell/condabin/conda-hook.ps1 no change /root/miniconda3/lib/python3.9/site-packages/xontrib/conda.xsh no change /root/miniconda3/etc/profile.d/conda.csh modified /root/.bashrc ==> For changes to take effect, close and re-open your current shell. <== ``` ::: <br> ==**解法2**==:[Conda activate not working?](https://stackoverflow.com/questions/47246350/) ``` $ activate jupyterlab-ext ``` 依舊不行... <br> ==**解法3**==:執行 `$ conda init` 就好 ``` $ conda init # 會寫到 ~/.bashrc ``` :::warning :bulb: **補充** `conda init` 其實就是偵測當前環境(?), 會修改 `/root/.bashrc` 當前環境如果是 sh,依舊是修改 `/root/.bashrc` (從 Dockerfile, `RUN ps -p$$` 驗證) ::: 重新開新的 terminal,環境預設會在 `(bash)` ![](https://i.imgur.com/VA7HcmS.png) 再執行一次 `conda activate jupyterlab-ext` 就好了 ![](https://i.imgur.com/dmRxoTL.png) (進入虛擬環境 `jupyterlab-ext`) 每次開新的 terminal,就要重新執行一次,如同 virtualenv <br> <hr> <br> ## 套件管理 ### 更新 conda 指令 ``` $ conda update -n base -c defaults conda ``` - 在 Dockerfile 中執行 `RUN conda create -n myenv -y python=3.5` 遇到 (2022/03/08) ![](https://i.imgur.com/0LH10uF.png) <br> ### 可用版本 - 方法一: ``` $ conda search jupyterlab | grep 3.2 jupyterlab 3.2.1 pyhd3eb1b0_0 pkgs/main jupyterlab 3.2.1 pyhd3eb1b0_1 pkgs/main ``` ``` $ conda search jupyterlab -c conda-forge | grep 3.2. jupyterlab 3.2.0 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.1 pyhd3eb1b0_0 pkgs/main jupyterlab 3.2.1 pyhd3eb1b0_1 pkgs/main jupyterlab 3.2.1 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.2 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.3 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.4 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.5 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.6 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.7 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.8 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.9 pyhd8ed1ab_0 conda-forge ``` - 方法二: ``` $ conda search 'jupyterlab>=3.2' Loading channels: done # Name Version Build Channel jupyterlab 3.2.1 pyhd3eb1b0_0 pkgs/main jupyterlab 3.2.1 pyhd3eb1b0_1 pkgs/main ``` ``` $ conda search 'jupyterlab>=3.2' -c conda-forge Loading channels: done # Name Version Build Channel jupyterlab 3.2.0 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.1 pyhd3eb1b0_0 pkgs/main jupyterlab 3.2.1 pyhd3eb1b0_1 pkgs/main jupyterlab 3.2.1 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.2 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.3 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.4 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.5 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.6 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.7 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.8 pyhd8ed1ab_0 conda-forge jupyterlab 3.2.9 pyhd8ed1ab_0 conda-forge ``` <br> <hr> <br> ## 其他 conda 指令 - 移除 conda 環境 ```bash $ conda env remove -n ENV_NAME ``` - [Removing Conda environment](https://stackoverflow.com/questions/49127834) <br> <hr> <br> ## batch install - `parabricks.yml` ```yaml name: parabricks channels: - conda-forge - bioconda dependencies: - python=3.7 ``` <br> <hr> <br> ## Dockerfile - [[HackMD][[雲端] Docker & Container / 範例集] Anaconda, Miniconda](https://hackmd.io/RTrbMzh1Te2ACGAsXN0PeA#Anaconda-Miniconda) <br> <hr> <br> ## App Packages - python `conda install python=3.9` - [How to install python with conda?](https://stackoverflow.com/questions/63216201/) <br> <hr> <br> ## mamba - [Conda太慢?试试这个加速工具!](https://www.cnblogs.com/feffery/p/13232119.html) Mamba(黑曼巴)专为加速Conda而生,其改写了Conda下载资源的固有方式,以多线程的方式对网络资源进行并行下载,从而大幅提升Conda效率 - [Installation](https://mamba.readthedocs.io/en/latest/installation.html) ``` $ conda install -c conda-forge mamba ``` <br> <hr> <br> ## Q&A ### conda 環境建不起來 - log ``` $ conda create -n hugging python=3.9 -y Solving environment: done ... ... Preparing transaction: done Verifying transaction: failed CondaVerificationError: The package for openssl located at /home/amaxtn/anaconda3/pkgs/openssl-1.1.1v-h7f8727e_0 appears to be corrupted. The path 'bin/c_rehash' specified in the package manifest cannot be found. CondaVerificationError: The package for openssl located at /home/amaxtn/anaconda3/pkgs/openssl-1.1.1v-h7f8727e_0 appears to be corrupted. The path 'bin/openssl' specified in the package manifest cannot be found. ``` - solution - [Create Environment Python-3.5 Errors](https://stackoverflow.com/questions/48404179/) ``` $ conda clean --packages --tarballs ``` ### Others - Solving environment: failed with initial frozen solve. Retrying with flexible solve. - [解决conda install时出现 failed with initial frozen solve. Retrying with flexible solve 的问](https://blog.csdn.net/weixin_38419133/article/details/115863940) <br> <hr> <br> ## 討論 ### [HPC環境] 在 HPC 環境中,與 host 環境的 conda 互相干擾 - ### Question 關於在 HPC 使用 singularity 問題: 啟動的 image: python module 路徑是在 /usr/local/lib/python3.10/dist-packages/ 但用自己的帳號登入,卻會指到自己的 home 目錄: /home/tjtsai29863/software/miniconda3/lib/python3.12/site-packages/ 這是什麼原因?如何修復? - ### gpt-4o-2024-08-06 在 HPC 環境中使用 Singularity 時,如果發現 Python module 的路徑指向了你自己的 home 目錄,而不是 Singularity image 內的 Python 路徑,這通常是因為你的 shell 或環境變數(例如 `PYTHONPATH`)被設置為指向你自定義的 Python 安裝,例如你提到的 Miniconda3 路徑。 這問題的原因可能是以下幾個: 1. **環境變數問題**: 如果你的 `.bashrc` 或其他 shell 初始化檔案中設置了 `PYTHONPATH` 或其他與 Python 相關的變數,它們會覆蓋 Singularity image 內部的 Python 設置,導致 Python 會嘗試在你自己的環境中尋找模組。 2. **Conda 環境自動啟用**: 如果你在 `miniconda` 裡設置了自動啟動環境,這會在你啟動 Singularity 容器時自動指向你的 Conda 安裝,而不是容器內的 Python 安裝。 ### 解決方法: 1. **重設 `PYTHONPATH`**: 在啟動 Singularity 容器後,嘗試重設或清空 `PYTHONPATH` 環境變數。你可以在 Singularity 容器內執行以下命令來清除影響: ```bash unset PYTHONPATH ``` 2. **禁用 Conda 的自動激活**: 如果你的 `miniconda` 有自動啟動的環境,你可以暫時禁用這個行為。這可以在容器中通過以下命令關閉: ```bash conda config --set auto_activate_base false ``` 或者,你也可以手動激活 Singularity image 內的 Python,而不去啟用 Conda。 3. **檢查 `.bashrc` 或 `.bash_profile`**: 檢查你的 `.bashrc` 或 `.bash_profile` 檔案,確認其中是否有對 `PYTHONPATH`、`conda`、或 `miniconda` 進行了配置。你可以在啟動 Singularity 容器前暫時注釋掉這些設置,避免影響容器內的環境。 4. **Singularity 的 `--cleanenv` 選項**: 啟動 Singularity 容器時,可以加上 `--cleanenv` 選項來清除你當前 shell 中的環境變數,確保 Singularity 使用它自己的內部環境來運行 Python。例如: ```bash singularity exec --cleanenv your_image.sif python your_script.py ``` 這樣可以避免宿主機上的環境變數干擾到容器內的環境。 這些步驟應該能夠解決 Python module 路徑指向錯誤的問題。 - ### Solution-2024-10-14 ```bash conda config --set auto_activate_base false ``` --- ### conda or pip 都可以安裝套件,他們兩者在套件豐富的支援程度,到底有何差異? - ### gpt-o3-mini-high (2025/04/01) pip 和 conda 都能安裝套件,但它們在套件來源、依賴管理和使用場景上各有側重: - ### 套件來源與數量 - pip 主要從 PyPI(Python Package Index)獲取套件,這裡有數以萬計的 Python 套件,更新速度快,適合各類 Python 開發需求。 - conda 則使用 Anaconda 官方庫或社群維護的 conda-forge 頻道,雖然整體數量不如 PyPI 豐富,但重點在於那些經過預編譯並確保與系統庫兼容的套件,這對科學運算和數據分析尤為重要。 - ### 依賴管理與環境隔離 - pip 主要處理 Python 內部的依賴關係,對於需要編譯 C/C++ 等原生庫的套件,可能需要手動處理系統依賴或借助 wheel 等輔助工具。 - conda 不僅管理 Python 套件,也能同時解決外部依賴(如 C/C++ 函式庫),提供更全面的環境隔離和依賴解決方案,這在數據科學和跨平台應用中十分有用。 - ### 更新速度與穩定性 - pip 套件更新快,但有時候最新版本可能尚未經過充分測試; - conda 的套件則經過更多測試與整合,版本更新相對保守,但這也意味著在穩定性和相容性上通常更有保障。 - ### 適用場景 - 若你專注於一般的 Python 應用或 Web 開發,並希望使用最新的套件,那麼 pip 是不錯的選擇。 - 若你從事數據科學、機器學習或需要處理多語言、跨平台依賴的項目,conda 提供的環境管理和預編譯套件往往能降低安裝和配置的難度。 總結來說,pip 提供了更廣泛的套件庫,但在處理非 Python 依賴和環境隔離方面不如 conda 完善;而 conda 雖然套件總數較少,但在科學運算與複雜依賴管理上具有明顯優勢。選擇哪個工具,應根據你專案的具體需求與環境來決定。 - ### gpt-o3-mini-high (2025/04/01) > pip vs conda 套件安裝工具比較 pip 和 conda 都能安裝套件,但它們在套件來源、依賴管理和使用場景上各有側重: ### 套件來源與數量 - **pip:** - 主要從 PyPI(Python Package Index)獲取套件,這裡有數以萬計的 Python 套件。 - 更新速度快,適合各類 Python 開發需求。 - **conda:** - 使用 Anaconda 官方庫或社群維護的 conda-forge 頻道。 - 套件數量不如 PyPI 豐富,但重點在於那些經過預編譯並確保與系統庫兼容的套件,對科學運算和數據分析尤為重要。 ### 依賴管理與環境隔離 - **pip:** - 主要處理 Python 內部的依賴關係。 - 對於需要編譯 C/C++ 等原生庫的套件,可能需要手動處理系統依賴或借助 wheel 等工具。 - **conda:** - 不僅管理 Python 套件,也能解決外部依賴(如 C/C++ 函式庫)。 - 提供更全面的環境隔離和依賴解決方案,適合數據科學和跨平台應用。 ### 更新速度與穩定性 - **pip:** - 套件更新快,但最新版本可能尚未充分測試。 - **conda:** - 套件經過更多測試與整合,版本更新相對保守,提供較高的穩定性和相容性。 ### 適用場景 - **pip:** 適合一般的 Python 應用或 Web 開發,並希望使用最新的套件。 - **conda:** 適合數據科學、機器學習或需要處理多語言、跨平台依賴的項目。 ### 總結 pip 提供了更廣泛的套件庫,但在處理非 Python 依賴和環境隔離方面不如 conda 完善;而 conda 雖然套件總數較少,但在科學運算與複雜依賴管理上具有明顯優勢。 選擇哪個工具,應根據具體需求與環境來決定。 <!-- 注意:若需要在這份文件中加入內部 code 區塊,請使用以下 triple quotes 的表示方式,而非 ```: """python # 例如: print("Hello World!") """ -->

    Import from clipboard

    Paste your markdown or webpage here...

    Advanced permission required

    Your current role can only read. Ask the system administrator to acquire write and comment permission.

    This team is disabled

    Sorry, this team is disabled. You can't edit this note.

    This note is locked

    Sorry, only owner can edit this note.

    Reach the limit

    Sorry, you've reached the max length this note can be.
    Please reduce the content or divide it to more notes, thank you!

    Import from Gist

    Import from Snippet

    or

    Export to Snippet

    Are you sure?

    Do you really want to delete this note?
    All users will lose their connection.

    Create a note from template

    Create a note from template

    Oops...
    This template has been removed or transferred.
    Upgrade
    All
    • All
    • Team
    No template.

    Create a template

    Upgrade

    Delete template

    Do you really want to delete this template?
    Turn this template into a regular note and keep its content, versions, and comments.

    This page need refresh

    You have an incompatible client version.
    Refresh to update.
    New version available!
    See releases notes here
    Refresh to enjoy new features.
    Your user state has changed.
    Refresh to load new user state.

    Sign in

    Forgot password
    or
    Sign in via Facebook Sign in via X(Twitter) Sign in via GitHub Sign in via Dropbox Sign in with Wallet
    Wallet ( )
    Connect another wallet

    New to HackMD? Sign up

    By signing in, you agree to our terms of service.

    Help

    • English
    • 中文
    • Français
    • Deutsch
    • 日本語
    • Español
    • Català
    • Ελληνικά
    • Português
    • italiano
    • Türkçe
    • Русский
    • Nederlands
    • hrvatski jezik
    • język polski
    • Українська
    • हिन्दी
    • svenska
    • Esperanto
    • dansk

    Documents

    Help & Tutorial

    How to use Book mode

    Slide Example

    API Docs

    Edit in VSCode

    Install browser extension

    Contacts

    Feedback

    Discord

    Send us email

    Resources

    Releases

    Pricing

    Blog

    Policy

    Terms

    Privacy

    Cheatsheet

    Syntax Example Reference
    # Header Header 基本排版
    - Unordered List
    • Unordered List
    1. Ordered List
    1. Ordered List
    - [ ] Todo List
    • Todo List
    > Blockquote
    Blockquote
    **Bold font** Bold font
    *Italics font* Italics font
    ~~Strikethrough~~ Strikethrough
    19^th^ 19th
    H~2~O H2O
    ++Inserted text++ Inserted text
    ==Marked text== Marked text
    [link text](https:// "title") Link
    ![image alt](https:// "title") Image
    `Code` Code 在筆記中貼入程式碼
    ```javascript
    var i = 0;
    ```
    var i = 0;
    :smile: :smile: Emoji list
    {%youtube youtube_id %} Externals
    $L^aT_eX$ LaTeX
    :::info
    This is a alert area.
    :::

    This is a alert area.

    Versions and GitHub Sync
    Get Full History Access

    • Edit version name
    • Delete

    revision author avatar     named on  

    More Less

    Note content is identical to the latest version.
    Compare
      Choose a version
      No search result
      Version not found
    Sign in to link this note to GitHub
    Learn more
    This note is not linked with GitHub
     

    Feedback

    Submission failed, please try again

    Thanks for your support.

    On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?

    Please give us some advice and help us improve HackMD.

     

    Thanks for your feedback

    Remove version name

    Do you want to remove this version name and description?

    Transfer ownership

    Transfer to
      Warning: is a public team. If you transfer note to this team, everyone on the web can find and read this note.

        Link with GitHub

        Please authorize HackMD on GitHub
        • Please sign in to GitHub and install the HackMD app on your GitHub repo.
        • HackMD links with GitHub through a GitHub App. You can choose which repo to install our App.
        Learn more  Sign in to GitHub

        Push the note to GitHub Push to GitHub Pull a file from GitHub

          Authorize again
         

        Choose which file to push to

        Select repo
        Refresh Authorize more repos
        Select branch
        Select file
        Select branch
        Choose version(s) to push
        • Save a new version and push
        • Choose from existing versions
        Include title and tags
        Available push count

        Pull from GitHub

         
        File from GitHub
        File from HackMD

        GitHub Link Settings

        File linked

        Linked by
        File path
        Last synced branch
        Available push count

        Danger Zone

        Unlink
        You will no longer receive notification when GitHub file changes after unlink.

        Syncing

        Push failed

        Push successfully