---
tags: NVIDIA, NGC , Docker, Kubernetes, Kubeflow
---
# NVIDIA T4 benchmark
###### tags: NVIDIA T4

## HW equipment

OS: Ubuntu 18.04 LTS Desktop, kernel 5.4.0 (UEFI)
Cuda: 11.1
NVIDIA TensorRT docker image version: 20.11-py3
NVIDIA TensorFlow docker image version: 20.11-tf2-py3

## Environment using SOP

### Since we are using NVIDIA GPU so we have to follow the NVIDIA SOP

## Setting up Docker
Docker-CE on Ubuntu can be setup using Docker’s official convenience script:

curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker

Setting up NVIDIA Container Toolkit
Setup the stable repository and the GPG key:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
   && curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list

Install the nvidia-docker2 package (and dependencies) after updating the package listing:

sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

At this point, a working setup can be tested by running a base CUDA container:

sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

This should result in a console output shown below:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
| N/A   34C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

## Step 2: Install Kubernetes Components
First, install some dependencies:

sudo apt-get update \
   && sudo apt-get install -y apt-transport-https curl

Add the package repository keys:

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

And the repository:

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

Update the package listing and install kubelet:

sudo apt-get update \
   && sudo apt-get install -y -q kubelet kubectl kubeadm

Configure the cgroup driver for kubelet:

sudo cat << EOF | sudo tee  /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock --cgroup-driver='systemd'"
EOF

Restart kubelet:

sudo systemctl daemon-reload \
   && sudo systemctl restart kubelet

And init using kubeadm:

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

Finish the configuration setup with Kubeadm:

mkdir -p $HOME/.kube \
   && sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config \
   && sudo chown $(id -u):$(id -g) $HOME/.kube/config

## Step 3: Configure Networking
Now, setup networking with Calico:

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Untaint the control plane, so it can be used to schedule GPU pods in our simplistic single-node cluster:

kubectl taint nodes --all node-role.kubernetes.io/master-