Docker / Docker-compose

Prerequisite

• Operating system

Docker

• Source code: written by Golang
  • Recommand to learn Golang and Rust in your free time!
  • sudo is going to be rewritten in Rust (link)
• Use OS-level virtualization to deliver software in packages called containers
• Docker&VM difference
• 
• Considering GPU and PyTorch:

GPU hardware || Host OS: cuda driver || Docker: cuda toolkits/cudnn/torch ...

Pros/Cons

• Pros
  • Build any enviroment you want easily
  • Safety
  • Reproducible
  • Make you Productive
• Cons
  • Require to remember many commands
    • Don't remember them, but internalize them!

Docker image/container

Image and container

• Docker image -> template
• Docker container -> realized runtime instance

Where to find images

• Docker Hub

Basic useful commands

Image

$ docker pull <IMAGE_NAME>:<IMAGE_TAG>
# e.g., docker pull python:3.9
$ docker run -it -d <IMAGE_NAME> <COMMAND>
# e.g., docker run -it -d python:3.9 bash
$ docker images # | grep ...
$ docker rmi <IMAGE_NAME>

Container

$ docker ps --all # | grep ...
$ docker exec -it <CONTAINER_ID> <COMMAND>
$ docker stop <CONTAINER_ID>
$ docker start <CONTAINER_ID>
$ docker rm <CONTAINER_ID>
$ docker inspect <CONTAINER_ID>
$ docker attach <CONTAINER_ID>
$ docker logs <CONTAINER_ID>

To leave a container

$ exit

Others

$ docker build -t <IMAGE_NAME>:<TAG> .

Docker run go deeper

$ docker run -it -d -w <PATH> --name <CONTAINER_NAME> <IMAGE_NAME> <COMMAND>
$ docker run -it -d -v <host>:<container> -p <host>:<container> -w <PATH> --name <CONTAINER_NAME> <IMAGE_NAME> <COMMAND>

• -i: Keep STDIN open even if not attached
• -t: Allocate a pseudo-TTY
• -d: Run container in background and print container ID
• -v: Bind mount a volume
  • support multiple ones
  • e.g., -v /home/xx:/home/xx -v /home/app:/app
• -p: Publish a container’s port(s) to the host
  • support multiple ones
  • e.g., -p 8000:8000 -p 5000:80
• -w: Working directory inside the container
• --rm: Automatically remove the container when it exits
• --ipc: IPC mode to use
  • --ipc=host
• --name: Assign a name to the container
• --privileged: Give extended privileges to this container
  • Important if you want to access GPU resource with root privileges
• --gpus: Specify which GPUs (or all) to use
  • --gpus all
• --shm-size: Size of /dev/shm

Dockerfile

• The available docker images cannot fulfill your environment
• You might need to install other packages after run the container
  • e.g., install numpy with python:3.9 image

$ docker run -it --rm python:3.9 bash
# in the container
$ pip install numpy

• Use Dockerfile and build your image!

FROM python:3.9
RUN pip install numpy
CMD ["/bin/bash"]

Then

$ docker build -t my-numpy-env:v1 .
$ docker run -it --rm my-numpy-env bash

# in the container
$ echo -e "import numpy\nprint(numpy.__version__)" > test.py
$ python3 test.py

• Dockerfile is complicated and only the basic instruction will be mentioned here. For the details, please see the tutorial and document.
• Basic instructions
  • FROM
    • FROM <image name>
    • FROM <image name>:<tag>
  • WORKDIR
    • WORKDIR <path>
  • RUN
    • RUN <command>
  • COPY
    • COPY [--chown=<user>:<group>] <source path> <dist path>
  • CMD
    • CMD [“command name”, argv1, argv2, …]
    • CMD <command>
  • ENTRYPOINT
    • ENTRYPOINT <command>
    • Please read this article to understand the difference between CMD
  • EXPOSE
    • Expose <PORT1> <PORT2> ...
  • ENV
    • ENV <key> <value>
    • ENV <key1>=<value1> <key2>=<value2>
  • ARG
    • ARG <arg>
    • docker build --build-arg <arg>=...

Labs

Lab 2-1: An interesting experiment of C/C++

Please skip the explanation part if you are a novice！

#include <iostream>
    
int main() {
    while (1)
        ;
}

void unreachable() {
    std::cout << "Hello world!" << std::endl;
}

# Try different versions of clang
$ clang++ loop.cpp -O1 -Wall -o loop
$ ./loop  # clang12 -> non-stop; clang14 -> Hello world!

• Why?
• clang 12.0.0 -O1

__cxx_global_var_init:                  # @__cxx_global_var_init
        push    rax
        mov     edi, offset std::__ioinit
        call    std::ios_base::Init::Init() [complete object constructor]
        mov     edi, offset std::ios_base::Init::~Init() [complete object destructor]
        mov     esi, offset std::__ioinit
        mov     edx, offset __dso_handle
        call    __cxa_atexit
        pop     rax
        ret
main:                                   # @main
.LBB1_1:                                # =>This Inner Loop Header: Depth=1
        jmp     .LBB1_1
unreachable():                       # @unreachable()
        push    rax
        mov     edi, offset std::cout
        mov     esi, offset .L.str
        call    std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)
        mov     esi, offset std::basic_ostream<char, std::char_traits<char> >& std::endl<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&)
        mov     rdi, rax
        call    std::basic_ostream<char, std::char_traits<char> >::operator<<(std::basic_ostream<char, std::char_traits<char> >& (*)(std::basic_ostream<char, std::char_traits<char> >&))
        pop     rax
        ret
_GLOBAL__sub_I_example.cpp:             # @_GLOBAL__sub_I_example.cpp
        push    rax
        call    __cxx_global_var_init
        pop     rax
        ret
.L.str:
        .asciz  "Hello world!"

• clang 14.0.0 -O1

main:                                   # @main
unreachable():                       # @unreachable()
        push    rbx
        mov     edi, offset std::cout
        mov     esi, offset .L.str
        mov     edx, 12
        call    std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long)
        mov     rax, qword ptr [rip + std::cout]
        mov     rax, qword ptr [rax - 24]
        mov     rbx, qword ptr [rax + std::cout+240]
        test    rbx, rbx
        je      .LBB1_5
        cmp     byte ptr [rbx + 56], 0
        je      .LBB1_3
        mov     al, byte ptr [rbx + 67]
        jmp     .LBB1_4
.LBB1_3:
        mov     rdi, rbx
        call    std::ctype<char>::_M_widen_init() const
        mov     rax, qword ptr [rbx]
        mov     rdi, rbx
        mov     esi, 10
        call    qword ptr [rax + 48]
.LBB1_4:
        movsx   esi, al
        mov     edi, offset std::cout
        call    std::basic_ostream<char, std::char_traits<char> >::put(char)
        mov     rdi, rax
        call    std::basic_ostream<char, std::char_traits<char> >::flush()
        pop     rbx
        ret
.LBB1_5:
        call    std::__throw_bad_cast()
_GLOBAL__sub_I_example.cpp:             # @_GLOBAL__sub_I_example.cpp
        push    rax
        mov     edi, offset std::__ioinit
        call    std::ios_base::Init::Init() [complete object constructor]
        mov     edi, offset std::ios_base::Init::~Init() [complete object destructor]
        mov     esi, offset std::__ioinit
        mov     edx, offset __dso_handle
        call    __cxa_atexit
        pop     rax
        ret
.L.str:
        .asciz  "Hello world!"

Lab 2-2: Build a segment-anything frontend service

Recommand to see the source code if you are (going to be) a ReactJS developer
Prepare a docker file

FROM nikolaik/python-nodejs:python3.10-nodejs16
RUN apt-get update
RUN git clone https://github.com/MiscellaneousStuff/meta-sam-demo.git
RUN cd meta-sam-demo && yarn install
WORKDIR /meta-sam-demo/
EXPOSE 3000
CMD ["yarn", "start"]

Then

$ docker build -t sam-demo:v1 .
$ docker run -d -p 9196:3000 --name sam-demo sam-demo:v1

Lab 2-3: Build a segment-anything frontend service with NGINX load balancer

FROM nikolaik/python-nodejs:python3.10-nodejs16 AS builder
WORKDIR /app
RUN git clone https://github.com/MiscellaneousStuff/meta-sam-demo.git
RUN cd meta-sam-demo && yarn install && yarn build

FROM nginx:alpine
WORKDIR /usr/share/nginx/html
RUN rm -rf ./*
COPY --from=builder /app/meta-sam-demo/build .
COPY ./nginx/ .
RUN rm /etc/nginx/conf.d/default.conf
RUN cp nginx.conf /etc/nginx/conf.d/default.conf
ENTRYPOINT ["nginx", "-g", "daemon off;"]

Then

$ docker build -t sam-demo-nginx:v1 .
$ docker run -d -p 9296:80 --name sam-demo-nginx-test sam-demo-nginx:v1

Docker Compose

It is troublesome to configurate something like port, mount, and network every time with docker command!

docker-compose.yml

Only basic configurations are mentioned here; please see the document for the details.

Lab 2-4: A convenient PyTorch enviroment

• docker-compose.yml

version: '3'
services:
  torch-env:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: torch-env
    privileged: true
    shm_size: 512g
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
    volumes:
      - ${HOME}:${HOME}
    working_dir: ${HOME}
    ports:
      - 15000:8888
    stdin_open: true
    tty: true
    networks:
      - backend
    depends_on:
      - time-backend

  time-backend:
    build: ./backend
    container_name: time-backend
    expose:
      - "5000"
    networks:
      - backend

networks:
  backend:
    driver: bridge

• Dockerfile (backend)

FROM python:3.10
RUN apt-get update
COPY . /root/timer/
WORKDIR /root/timer/
RUN pip install -r requirements.txt
ENTRYPOINT ["uvicorn", "main:app", "--reload", "--host", "0.0.0.0", "--port", "5000"]

• Backend API

from datetime import datetime

import pytz
from fastapi import FastAPI

app = FastAPI()


@app.get("/time")
async def get_time():
    current_time = datetime.now(pytz.utc)
    current_time_with_tz = current_time.strftime("%Y-%m-%d %H:%M:%S %Z")
    return {"time": current_time_with_tz}

• To test backend APIs

import asyncio
import logging

import aiohttp


async def get_time():
    async with aiohttp.ClientSession() as session:
        async with session.get('http://time-backend:5000/time') as response:
            json = await response.json()
            return json['time']


if __name__ == '__main__':

    loop = asyncio.get_event_loop()
    current_time = loop.run_until_complete(get_time())
    logging.warning(f'Get current time from the backend: {current_time}')

Problem

...
COPY ...
RUN pip install -r requirements.txt

Or

...
RUN pip install -r requirements.txt
COPY ...

Practice more!

• [Easy] How to run a jupyter notebook every time you run the services
• [Medium] Write a Dockerfile & docker-compose.yml for project/research
• [Medium] NGINX configuration
• [Medium] FastAPI
• [Medium] Python docker package (docker SDK)
• [Medium] Python asyncio and aiohttp
  • requests or aiohttp
  • async/await
• [Hard] docker-compose configuration