Install Docker

tags: tutorials Ubuntu 18.04 Docker PairLab

安裝 Docker

驗證 nvidia driver

nvidia-smi

安裝 nvidia driver (以上若是有驗證成功，則不需要執行這段。)

sudo apt-get install gcc -y
sudo apt-get install make
sudo service gdm3 stop
sudo reboot

sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update
ubuntu-drivers devices
## <VERSION_STRING>版本請依照所需
sudo apt-get install nvidia-driver-<VERSION_STRING>
sudo service gdm3 start
sudo reboot

安裝 Docker

sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io -y
apt-cache madison docker-ce

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker

更改 Docker 檔案位置（Optional）

將14行改成如下(目標資料夾為/docker_disk)

sudo vim /lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd --data-root /docker_disk -s overlay2 -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP $MAINPID

儲存後輸入

sudo systemctl daemon-reload
sudo service docker restart

創建container

sudo docker run --gpus all -it --name (NAME) --shm-size 1G --ipc=host -p (B):22 -p (A):8888 -p (C):6006 (image)

## NAME、A、B、C為自取(將NAME取代成想要的名子，A取代為想要的port(不是改8888))
## A為開啟 jupyter notebook 的port
## B為遠端 ssh 的 port，C為開啟 tensorboard 的 port
## EX:sudo docker run --gpus all -it --name test --shm-size 1G --ipc=host -p 50000:22 -p 50050:8888 -p 50051:6006 titi861203/cuda11.0-cudnn8-tf2.4-torch1.7.1

啟動Container

sudo docker start (name)

確認GPU是否支援Tensorflow與PyTorch

## tensorflow
python3 -c "import tensorflow as tf;print('Can your tensorflow use GPU?');print(tf.test.is_gpu_available());"
## pytorch
python3 -c "import torch as t;print('Can your pytorch use GPU?');print(t.cuda.is_available());"

若是要設定與開啟 Jupyter Lab 請點這兒

建立 Dockerfile

需先申請docker hub帳號

建立dockerfile

mkdir docker_file && cd docker_file

編輯dockerfile

vim Dockerfile

指令介紹:

FROM：使用到的 Docker Image 名稱
MAINTAINER： 用來說明，撰寫和維護這個 Dockerfile 的人是誰，也可以給 E-mail的資訊
RUN： RUN 指令後面放 Linux 指令，用來執行安裝和設定這個 Image 需要的東西
ADD： 把 Local 的檔案複製到 Image 裡，如果是 tar.gz 檔複製進去 Image 時會順便自動解壓縮。
ENV： 用來設定環境變數
CMD： 在指行 docker run 的指令時會直接呼叫開啟 Tomcat Service

PyTorch2.0.0 CUDA11.7.1 Ubuntu22.04

FROM nvidia/cuda:11.7.1-cudnn8-devel-ubuntu22.04
MAINTAINER Titi Wei <[e-mail]>
RUN apt-get update
RUN apt-get upgrade -y
RUN echo 'root:12345678' | chpasswd
RUN apt-get install -y openssh-server python3 python3-pip vim wget git libcupti-dev iputils-ping curl screen
RUN echo "PermitRootLogin yes">>/etc/ssh/sshd_config
RUN /etc/init.d/ssh restart
RUN service ssh start
RUN echo "service ssh start">>~/.bashrc
RUN pip3 install --upgrade pip
RUN pip3 install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1
RUN pip install jupyter jupyterlab
RUN jupyter notebook --generate-config
RUN echo "c.NotebookApp.password = u'sha1:1170ee6efd7d:7868110829c753e16815394c7507dc7450dd1655'" >> /root/.jupyter/jupyter_notebook_config.py
RUN echo "c.NotebookApp.terminado_settings = { 'shell_command': ['bash'] }">>/root/.jupyter/jupyter_notebook_config.py
RUN echo "jupyter lab --allow-root --ip=0.0.0.0" >> /root/.jupyter.sh
RUN chmod +x /root/.jupyter.sh

FROM nvidia/cuda:11.0-cudnn8-devel-ubuntu18.04
MAINTAINER Titi Wei <[e-mail]>
RUN apt-get update
RUN apt-get upgrade -y
RUN apt update
RUN apt upgrade
RUN echo 'root:12345678' | chpasswd
RUN apt-get install -y openssh-server python3 python3-pip vim wget git libcupti-dev iputils-ping curl screen
RUN echo "PermitRootLogin yes">>/etc/ssh/sshd_config
RUN /etc/init.d/ssh restart
RUN pip3 install --upgrade pip
RUN pip install tensorflow-gpu==2.4.0
RUN pip install torch==1.7.1 torchvision==0.8.2
RUN pip install jupyter jupyterlab
RUN jupyter notebook --generate-config
RUN pip install matplotlib pillow lxml pandas Cython opencv-python openpyxl
RUN pip install scikit-image --upgrade
RUN apt-get install dbus-x11
RUN echo "X11Forwarding yes">> /etc/ssh/sshd_config
RUN echo "X11UseLocalhost no">> /etc/ssh/sshd_config
RUN grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config
RUN echo "service ssh start">>~/.bashrc
RUN echo "alias python='/usr/bin/python3'">>~/.bashrc
RUN echo "c.NotebookApp.terminado_settings = { 'shell_command': ['bash'] }">>/root/.jupyter/jupyter_notebook_config.py
RUN service ssh start
RUN wget https://developer.download.nvidia.com/compute/cuda/11.1.0/local_installers/cuda_11.1.0_455.23.05_linux.run
RUN chmod +x cuda_11.1.0_455.23.05_linux.run
RUN sh cuda_11.1.0_455.23.05_linux.run --tar mxvf

Pytorch1.4 version by yssu

FROM nvidia/cuda:10.1-cudnn7-devel-ubuntu18.04
MAINTAINER YuSheng Su <[e-mail]>
RUN apt-get update
RUN apt-get upgrade -y
RUN apt update
RUN apt upgrade
RUN echo 'root:12345678' | chpasswd
RUN apt-get install -y openssh-server python3 python3-pip vim wget git libcupti-dev iputils-ping curl screen
RUN echo "PermitRootLogin yes">>/etc/ssh/sshd_config
RUN /etc/init.d/ssh restart
RUN pip3 install --upgrade pip
RUN pip install tensorflow-gpu==2.3.0
RUN pip install torch==1.4.0 torchvision==0.5.0
RUN pip install jupyter jupyterlab
RUN jupyter notebook --generate-config
RUN pip install matplotlib pillow lxml pandas Cython opencv-python openpyxl
RUN pip install scikit-image --upgrade
RUN apt-get install dbus-x11
RUN echo "X11Forwarding yes">> /etc/ssh/sshd_config
RUN echo "X11UseLocalhost no">> /etc/ssh/sshd_config
RUN grep "^X11UseLocalhost" /etc/ssh/sshd_config || echo "X11UseLocalhost no" >> /etc/ssh/sshd_config
RUN echo "service ssh start">>~/.bashrc
RUN echo "alias python='/usr/bin/python3'">>~/.bashrc
RUN echo "c.NotebookApp.terminado_settings = { 'shell_command': ['bash'] }">>/root/.jupyter/jupyter_notebook_config.py
RUN service ssh start
RUN wget https://developer.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.105_418.39_linux.run
RUN chmod +x cuda_10.1.105_418.39_linux.run
RUN sh cuda_10.1.105_418.39_linux.run --tar mxvf

Build Docker Image

sudo docker build -t="(your-name)/(image-name)" . --no-cache

將 Docker image 上傳至 Docker Hub

sudo docker login
sudo docker push (your-name)/(image-name)

如果要push到自己帳號記得logout

sudo docker logout

執行container裡面的東東

sudo docker exec -d [container name] /bin/bash [file]