# NVIDIA DRIVER AND INSTALL DOCKER
## 1. 安裝 NIVIDA 驅動程式 (GPU)
```bash
# 更新系統
sudo apt update && sudo apt upgrade -y
# 移除所有nvidia套件
sudo apt autoremove nvidia* --purge -y
# 安裝ubuntu nvudia 套件
sudo apt install ubuntu-drivers-common -y
# 列出nvidia可安裝DRIVER版本
ubuntu-drivers devices
ubuntu-drivers list
# 並動更新, 若移除以下第二行之# 指定版本
sudo ubuntu-drivers autoinstall
#sudo apt install nvidia-driver-525
# 列出顯卡硬體內容
sudo lshw -C display
# 重開機
sudo reboot now
```
## 2. 安裝 Docker (GPU)
- 移除
```bash
sudo apt-get remove docker docker-engine docker.io containerd runc -y
sudo apt-get update -y
docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1
docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker
```
- 安裝 docker-ce (GPU/CPU)
```bash
sudo apt-get update -y
sudo apt-get install php-cli zip jq apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable" -y
sudo apt-get update -y
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
```
- 安裝NVIDIA Container Toolkit (nvidia-docker2) (GPU)
```bash
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update -y
sudo apt-get install nvidia-docker2 -y
sudo systemctl restart docker.service
sudo systemctl restart docker
```
- 測試是否成功 (GPU)
```
sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
```
- 加入 Docker 群組, 記得你的身分為ubuntu (GPU/CPU)
```
sudo groupadd docker
sudo usermod -aG docker $USER
sudo systemctl enable docker # 啟動 Docker 服務
docker --version
```
- 測試安裝結果, 記得你的身分為ubuntu (GPU/CPU)
```
sudo su
su ubuntu
docker run hello-world
```
## :a: 到這裡就可以了, 以下參考用
## 3. 安裝 Docker Composer
```
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose --version
```
## 4. GPU Support (Optional)
If you have a GPU and want to leverage its power within a Docker container, follow these steps to install the NVIDIA Container Toolkit:
```
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
# Configure NVIDIA Container Toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
# Test GPU integration
docker run --gpus all nvidia/cuda:11.5.2-base-ubuntu20.04 nvidia-smi
```
# old Docker
https://unimimic.github.io/posts/docker-gpu/
## 移除舊版Docker (可選)
```
sudo apt-get remove docker docker-engine docker.io containerd runc
```
## 更新apt安裝包以及所需的相關套件
```
sudo apt-get update
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
```
## 加入docker的官方GPG Key
```
sudo mkdir -m 0755 -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
```
## 設置repository
```
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
```
## 更新apt安裝包
```
sudo apt-get update
```
## 安裝docker
```
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
## 加入 Docker 群組, 記得你的身分為ubuntu
```
sudo groupadd docker
sudo usermod -aG docker $USER
sudo systemctl enable docker # 啟動 Docker 服務
docker --version
```
## 確認是否安裝成功
```
docker run hello-world
```
# :a: 新版
## 1. 安裝 NIVIDA 驅動程式 (GPU)
```bash
# 更新系統
sudo apt update && sudo apt upgrade -y
# 移除所有nvidia套件
sudo apt autoremove nvidia* --purge -y
# 安裝ubuntu nvudia 套件
sudo apt install ubuntu-drivers-common -y
# 列出nvidia可安裝DRIVER版本
ubuntu-drivers devices
ubuntu-drivers list
# 並動更新, 若移除以下第二行之# 指定版本
sudo ubuntu-drivers autoinstall
#sudo apt install nvidia-driver-525
# 列出顯卡硬體內容
sudo lshw -C display
# 重開機
sudo reboot now
```
## 2. 移除 Docker (GPU/CPU)
```bash
sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
docker volume ls -q -f driver=nvidia-docker | xargs -r -I{} -n1
docker ps -q -a -f volume={} | xargs -r docker rm -f
sudo apt-get purge -y nvidia-docker
```
## 3. 安裝 Docker (GPU/CPU)
```bash
sudo apt-get update -y
sudo apt-get install php-cli zip jq apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable" -y
sudo apt-get update -y
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y
```
### 4. Install the NVIDIA Container Toolkit.
- Configure the repository
```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update -y
```
- Install the NVIDIA Container Toolkit packages
```bash
sudo apt-get install -y nvidia-container-toolkit
```
### 5. Configure Docker to use Nvidia driver
```bash
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```
### 6. 測試是否成功 (GPU)
```bash
sudo docker run --rm --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
```
### 7. 加入 Docker 群組, 記得你的身分為ubuntu (GPU/CPU)
```
sudo groupadd docker
sudo usermod -aG docker $USER
sudo systemctl enable docker # 啟動 Docker 服務
docker --version
```
### 8. 測試安裝結果, 記得你的身分為ubuntu (GPU/CPU)
```
sudo su
su ubuntu
docker run hello-world
```
### 9. Start the container
```bash
docker run -d --gpus=all -v ollama:/root/.ollama -p 443:11434 --name ollama ollama/ollama
```
```bash
docker run -d --gpus=all -v ollama:/root/.ollama -p 443:11434 -e OLLAMA_MAX_LOADED_MODELS=4 -e OLLAMA_NUM_PARALLEL=16 --name ollama ollama/ollama
```
啟動 ollama 前面加上這個
OLLAMA_NUM_PARALLEL=4 OLLAMA_MAX_LOADED_MODELS=4 ollama xxxxxxxxxxxxx
### 10. 模型下載
```bash
docker exec ollama ollama pull jcai/llama3-taide-lx-8b-chat-alpha1:Q4_K_M
docker exec ollama ollama pull jcai/llama3-taide-lx-8b-chat-alpha1:f16
docker exec ollama ollama pull jcai/breeze-7b-32k-instruct-v1_0:f16
```
### 11. 驗測
```bash
curl http://localhost:443/api/generate -d '{"model": "jcai/breeze-7b-32k-instruct-v1_0:f16","prompt":"Why is the sky blue?"}'
curl http://localhost:443/api/generate -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:f16","prompt":"Why is the sky blue?"}'
curl http://localhost:443/api/generate -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:Q4_K_M","prompt":"Why is the sky blue?"}'
curl http://localhost:443/api/chat -d '{"model": "jcai/breeze-7b-32k-instruct-v1_0:f16","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
curl http://localhost:443/api/chat -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:f16","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
curl http://localhost:443/api/chat -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:Q4_K_M","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
```
```bash
curl http://203.145.222.186:443/api/generate -d '{"model": "jcai/breeze-7b-32k-instruct-v1_0:f16","prompt":"Why is the sky blue?"}'
curl http://203.145.222.186:443/api/generate -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:f16","prompt":"Why is the sky blue?"}'
curl http://203.145.222.186:443/api/generate -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:Q4_K_M","prompt":"Why is the sky blue?"}'
curl http://203.145.222.186:443/api/chat -d '{"model": "jcai/breeze-7b-32k-instruct-v1_0:f16","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
curl http://203.145.222.186:443/api/chat -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:f16","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
curl http://203.145.222.186:443/api/chat -d '{"model": "jcai/llama3-taide-lx-8b-chat-alpha1:Q4_K_M","messages": [{"role": "user", "content": "why is the sky blue?" }]}'
```
Sign in with Wallet
Connect another wallet