---
# System prepended metadata

title: NVIDIA T4 benchmark
tags: [' Kubeflow', ' NGC ', ' Docker', NVIDIA, ' Kubernetes']

---

---
tags: NVIDIA, NGC , Docker, Kubernetes, Kubeflow
---
# NVIDIA T4 benchmark
###### tags: NVIDIA T4

## HW equipment

OS: Ubuntu 18.04 LTS Desktop, kernel 5.4.0 (UEFI)
Cuda: 11.1
NVIDIA TensorRT docker image version: 20.11-py3
NVIDIA TensorFlow docker image version: 20.11-tf2-py3

## Environment using SOP

### Since we are using NVIDIA GPU so we have to follow the NVIDIA SOP

## Setting up Docker
Docker-CE on Ubuntu can be setup using Docker’s official convenience script:

```javascript=
curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker
```
Setting up NVIDIA Container Toolkit
Setup the stable repository and the GPG key:
```javascript=
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
   && curl -s -L https://nvidia.github.io/nvidia-container-runtime/experimental/$distribution/nvidia-container-runtime.list | sudo tee /etc/apt/sources.list.d/nvidia-container-runtime.list
```
Install the nvidia-docker2 package (and dependencies) after updating the package listing:

```javascript=
sudo apt-get update
sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker
```
At this point, a working setup can be tested by running a base CUDA container:

```javascript=
sudo docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
```
This should result in a console output shown below:

```javascript=
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.51.06    Driver Version: 450.51.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            On   | 00000000:00:1E.0 Off |                    0 |
| N/A   34C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
```
## Step 2: Install Kubernetes Components
First, install some dependencies:

```javascript=
sudo apt-get update \
   && sudo apt-get install -y apt-transport-https curl
```
Add the package repository keys:
```javascript=
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
```
And the repository:

```javascript=
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
```
Update the package listing and install kubelet:

```javascript=
sudo apt-get update \
   && sudo apt-get install -y -q kubelet kubectl kubeadm
```
Configure the cgroup driver for kubelet:
```javascript=
sudo cat << EOF | sudo tee  /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock --cgroup-driver='systemd'"
EOF
```
Restart kubelet:

```javascript=
sudo systemctl daemon-reload \
   && sudo systemctl restart kubelet
```
And init using kubeadm:

```javascript=
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
```
Finish the configuration setup with Kubeadm:

```javascript=
mkdir -p $HOME/.kube \
   && sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config \
   && sudo chown $(id -u):$(id -g) $HOME/.kube/config
```
## Step 3: Configure Networking
Now, setup networking with Calico:

```javascript=
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
```

Untaint the control plane, so it can be used to schedule GPU pods in our simplistic single-node cluster:

```javascript=
kubectl taint nodes --all node-role.kubernetes.io/master-
```