# Docker 設置 contributer:TiTi, yssu ### 安裝 nvidia driver ```=bash sudo apt-get install gcc -y sudo apt-get install make sudo service gdm3 stop sudo reboot sudo add-apt-repository ppa:graphics-drivers/ppa sudo apt-get update ubuntu-drivers devices ## <VERSION_STRING>版本請依照所需 sudo apt-get install nvidia-driver-<VERSION_STRING> sudo service gdm3 start sudo reboot ``` ### 驗證 nvidia driver ```=bash nvidia-smi ``` ### 安裝 Docker ```=bash sudo apt-get update sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - sudo apt-key fingerprint 0EBFCD88 sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io -y apt-cache madison docker-ce ## <VERSION_STRING>版本請依照所需 sudo apt-get install docker-ce=<VERSION_STRING> docker-ce-cli=<VERSION_STRING> containerd.io ## e.g. sudo apt-get install docker-ce=5:19.03.5~3-0~ubuntu-bionic docker-ce-cli=5:19.03.5~3-0~ubuntu-bionic containerd.io distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit sudo systemctl restart ``` ### 更改 Docker 檔案位置(Optional) 將14行改成如下(目標資料夾為/docker_disk) ```=bash sudo vim /lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd --data-root /docker_disk -s overlay2 -H fd:// --containerd=/run/containerd/containerd.sock ExecReload=/bin/kill -s HUP $MAINPID ``` 儲存後輸入 ```=bash sudo systemctl daemon-reload sudo service docker restart ``` ### 創建container ```=bash sudo docker run --gpus all -it --name (NAME) --shm-size 1G --ipc=host -p (B):22 -p (A):8888 -p (C):6006 (image) ## NAME、A、B、C為自取(將NAME取代成想要的名子,A取代為想要的port(不是改8888)) ## A為開啟 jupyter notebook 的port ## B為遠端 ssh 的 port,C為開啟 tensorboard 的 port ## EX:sudo docker run --gpus all -it --name test --shm-size 1G --ipc=host -p 50000:22 -p 50050:8888 -p 50051:6006 titi861203/cuda11.0-cudnn8-tf2.4-torch1.7.1 ``` ### 啟動Container ```=bash sudo docker start (name) ``` ### 確認GPU是否支援Tensorflow與PyTorch ```=bash ## tensorflow python3 -c "import tensorflow as tf;print('Can your tensorflow use GPU?');print(tf.test.is_gpu_available());" ## pytorch python3 -c "import torch as t;print('Can your pytorch use GPU?');print(t.cuda.is_available());" ``` ## 建立 Dockerfile 需先申請docker hub帳號 ### 建立dockerfile ```=bash mkdir docker_file && cd docker_file ``` ### 編輯dockerfile ```=bash vim Dockerfile ``` 指令介紹: FROM:使用到的 Docker Image 名稱 MAINTAINER: 用來說明,撰寫和維護這個 Dockerfile 的人是誰,也可以給 E-mail的資訊 RUN: RUN 指令後面放 Linux 指令,用來執行安裝和設定這個 Image 需要的東西 ADD: 把 Local 的檔案複製到 Image 裡,如果是 tar.gz 檔複製進去 Image 時會順便自動解壓縮。 ENV: 用來設定環境變數 CMD: 在指行 docker run 的指令時會直接呼叫開啟 Tomcat Service ```=bash FROM nvidia/cuda:11.0-cudnn8-devel-ubuntu18.04 MAINTAINER Titi Wei <a2699560@gmail.com> RUN apt-get update RUN apt-get upgrade -y RUN apt update RUN apt upgrade RUN echo 'root:12345678' | chpasswd RUN apt-get install -y openssh-server python3 python3-pip vim wget git libcupti-dev iputils-ping curl screen RUN echo "PermitRootLogin yes">>/etc/ssh/sshd_config RUN /etc/init.d/ssh restart RUN pip3 install --upgrade pip RUN pip install tensorflow-gpu==2.4.0 RUN pip install torch==1.7.1 torchvision==0.8.2 RUN pip install jupyter jupyterlab RUN jupyter notebook --generate-config RUN pip install matplotlib pillow lxml pandas Cython opencv-python openpyxl RUN pip install scikit-image --upgrade RUN apt-get install dbus-x11 RUN echo "X11Forwarding yes">> /etc/ssh/sshd_config RUN echo "X11UseLocalhost no">> /etc/ssh/sshd_config RUN grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config RUN echo "service ssh start">>~/.bashrc RUN echo "alias python='/usr/bin/python3'">>~/.bashrc RUN echo "c.NotebookApp.terminado_settings = { 'shell_command': ['bash'] }">>/root/.jupyter/jupyter_notebook_config.py RUN service ssh start RUN wget https://developer.download.nvidia.com/compute/cuda/11.1.0/local_installers/cuda_11.1.0_455.23.05_linux.run RUN chmod +x cuda_11.1.0_455.23.05_linux.run RUN sh cuda_11.1.0_455.23.05_linux.run --tar mxvf ``` Example: Pytorch1.4 version by yssu ```=bash FROM nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04 MAINTAINER YuSheng Su <yushengsu.ai09@nycu.edu.tw> RUN apt-get update RUN apt-get upgrade -y RUN apt update RUN apt upgrade RUN echo 'root:12345678' | chpasswd RUN apt-get install -y openssh-server python3 python3-pip vim wget git libcupti-dev iputils-ping curl screen RUN echo "PermitRootLogin yes">>/etc/ssh/sshd_config RUN /etc/init.d/ssh restart RUN pip3 install --upgrade pip RUN pip install tensorflow-gpu==2.3.0 RUN pip install torch==1.4.0 torchvision==0.5.0 RUN pip install jupyter jupyterlab RUN jupyter notebook --generate-config RUN pip install matplotlib pillow lxml pandas Cython opencv-python openpyxl RUN pip install scikit-image --upgrade RUN apt-get install dbus-x11 RUN echo "X11Forwarding yes">> /etc/ssh/sshd_config RUN echo "X11UseLocalhost no">> /etc/ssh/sshd_config RUN grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config RUN echo "service ssh start">>~/.bashrc RUN echo "alias python='/usr/bin/python3'">>~/.bashrc RUN echo "c.NotebookApp.terminado_settings = { 'shell_command': ['bash'] }">>/root/.jupyter/jupyter_notebook_config.py RUN service ssh start RUN wget https://developer.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.105_418.39_linux.run RUN chmod +x cuda_10.1.105_418.39_linux.run RUN sh cuda_10.1.105_418.39_linux.run --tar mxvf ``` ### Build Docker Image ```=bash sudo docker build -t="(your-name)/(image-name)" . --no-cache ``` ### 將 Docker image 上傳至 Docker Hub ```=bash sudo docker login sudo docker push (your-name)/(image-name) ``` 要push之前記得logout ### docker logout ```=bash sudo docker logout ``` 執行container裡面的東東 ```=bash sudo docker exec -d [container name] /bin/bash [file] ``` 起jupyer的時候可以把指令寫成一個shell script