# 使用 Docker 安裝 ollama on ubuntu 24.04

## 檢查 GPU

$ nvidia-smi
Fri Jul 11 15:02:12 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.142                Driver Version: 550.142        CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3070        Off |   00000000:00:10.0 Off |                  N/A |
|  0%   49C    P0             48W /  270W |       1MiB /   8192MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

## CPU 安裝 ollama

$ sudo docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

## GPU 安裝 ollama
* 新增 `nvidia-container-toolkit` 套件來源

$ curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

* 開始安裝 `nvidia-container-toolkit`

$ sudo apt-get update

$ export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
  
$ sudo apt-get install -y \
      nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

* 設定 Docker 使用 NVIDIA Runtime

$ sudo nvidia-ctk runtime configure --runtime=docker

* 重新啟動 Docker

$ sudo systemctl restart docker

* 部署 ollama container

$ sudo docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

## 下載 `deepseek-r1:1.5b` model

$ sudo docker exec -it ollama ollama pull deepseek-r1:1.5b

## 檢查已下載模型

$ sudo docker exec -it ollama ollama list
NAME                ID              SIZE      MODIFIED
deepseek-r1:1.5b    e0979632db5a    1.1 GB    48 minutes ago

## 使用指令呼叫 LLM

$ curl -X POST http://192.168.11.119:11434/api/generate -d '{
  "model": "deepseek-r1:1.5b",
  "prompt": "Hello"
}'

## ollama 所有 modle

https://ollama.com/library

## 參考

https://ollama.com/blog/ollama-is-now-available-as-an-official-docker-image

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html