---
tags: devtools2022
---

# 06A-Docker
**[Day4 PM]** Brief overview of Docker, a simple idea on how it works, and deploying applications with Docker using `docker-compose`. If time permits, we go through some basics of `git-hooks` to tie everything up before we conclude our 4 days course.

## Schedule and Learning Objectives
* **1330-1400**: Motivation, differences between VM and Containers, installation (Docker Desktop)
* **1400-1430**: `chroot` jail
* **1430-1500**: Docker components, writing Dockerfile and `docker-compose` file
* **1500-1530**: Break
* **1530-1630** **Lab**:
* Deploying Docker in EC2
* Git Hooks
* **1630-1700**: Discussion, Q&A

## Containers vs VM

Docker is a **set** of platform as a service products that use OS-level virtualization to deliver software in packages called **containers**.

Containers means an **entire runtime environment**: an application, plus all its dependencies, libraries and other binaries, and configuration files needed to run it, bundled into **one** package.

> Note: many organizations are adopting containers to develop and manage **stable** applications. Docker is one of the most feature-rich and widely used tools in this space, but there are other alternatives to Docker that offer unique use cases and features such as [Podman](https://podman.io) and [LXD](https://linuxcontainers.org/lxd/introduction/) to name a few.

We will be using **Docker** for the purpose of this course and learning more about *containerisation*.

Here's a brief diagram to show you the differences between a VM and a Container (image taken from [here](https://www.docker.com/resources/what-container/#/package_software)).
### How does virtualization work at a low level?

**Hypervisor** virtualisation: 1 or more independent machine run virtually on physical hardware

A VM **manager** (hypervisor) takes over the CPU ring 0 (or the "root mode" in newer CPUs) and **intercepts** all privileged calls made by the guest OS to create the *illusion* that the guest OS has its own hardware.
> Fun fact: Before 1998 it was thought to be impossible to achieve this on the x86 architecture because there was no way to do this kind of interception. The folks at VMware were the **first** who had an idea to **rewrite** the executable bytes in memory for privileged calls of the guest OS to achieve this.

The **effect**: virtualization allows you to run **two** completely different OSes on the **same** hardware.
* Each guest OS goes through all the processes of **bootstrapping**, **loading** kernel, etc. You can have very tight **security**.
* For example, a guest OS **can't** get full access to the host OS or other guests and mess things up.

### How do containers work at a low level?

In 2006, a new **kernel** level feature called **namespaces** were created (the idea long before existed in FreeBSD).

One function of the OS is to **allow** sharing of global resources like **network** and **disks** among processes. The trick is to wrap these global resources in **namespaces** so that they are **visible** only to those processes that run in the same **namespace** (more explanation will come).

> This is how we limit what a process can see.

Explanation:
* Suppose you want a process in **namespace** X to be able to read/write to a protected disk location.
* You can assign a chunk of disk in **namespace** X.
* Processes in other namespace Y can't read or write to it.
* Similarly, processes in namespace X can't access anything in memory that is allocated to namespace Y.
* Do not forget that processes in X can't see or talk to processes in namespace Y (isolation) by default, unless IPC attempts are made.

This provides a *kind* of **virtualization** and **isolation** for global resources.

> This is how Docker works in a nutshell: **Each container runs in its own namespace but uses exactly the same kernel as all other containers**.

The isolation happens because the kernel knows the namespace that was assigned to the process and during API calls it makes sure that the process can only access resources in its own namespace.

### Limitations
We can't run completely different OSes in containers like in VMs **because they share the same Kernel**

> However you can run different distros of Linux because they do share the same kernel.

The isolation level **is not as strong** as in a VM. In fact, there was a way for a "guest" container to *take over* the host in early implementations.

Also, when you load a new container, an entire new copy of the OS doesn't start like it does in a VM. **Like we said before, all containers share the same kernel**.

> This is why containers are "light weight".

Also unlike a VM, you don't have to **pre-allocate** a significant chunk of **memory** to containers because **we are not running a new copy of the OS**.

This enables running **thousands** of containers on one OS while sandboxing them, which might not be possible if we were running separate copies of the OS in their own VMs.

### Quick Summary
**Containers**: run in user space on top of OS Kernel
* OS level virtualisation
* multiple isolated user space instances run on a single host
* Containers can only run same or similar guest OS as the Host (unlike VMs --- run Windows VM on Ubuntu VM, but can't run Windows on Ubuntu server)
* Seen "less secure" (vs full isolation of VM) -- but less attack surface (doesn't need hypervisor and a full OS needed by the VM)

**Popular for**:
1. Hyperscale deployments
2. Lightweight sandboxing
3. Process isolation environments (despite concern about security)

## chroot jail
Common example of a very simple **container** is a`chroot` jail: isolated directory environment for running processes.
> Attackers, if they breach the running process in the jail, then find themselves **trapped** in this **environment** and unable to further compromise a host.

> Docker container can be thought of an extension of `chroot`. It adds additional isolation not available when just using `chroot`.

**What is chroot jail?**
A chroot (short for change root) is a Unix operation that changes the apparent root directory to the one specified by the user.

Any process you run after a chroot operation only has access to the newly defined root directory and its subdirectories. This operation is colloquially known as a chroot jail since these processes cannot read or write outside the new root directory.

Chroot jail is used to create a **limited** sandbox for a process to run in. This means a process **cannot** maliciously change data **outside** the prescribed directory tree.

Another use for chroot jails is as a **substitute** for virtual machines. This method is called kernel-level virtualization and requires fewer resources than virtual machines. This operation allows users to create multiple isolated instances on the same system.

1. Create a new directory called chroot_jail:

mkdir chroot_jail

2. Create a new subdirectory tree inside chroot_jail:

mkdir -p chroot_jail/bin chroot_jail/lib64/x86_64-linux-gnu chroot_jail/lib/x86_64-linux-gnu

These subdirectories will store all the necessary elements of the `bash` and `ls` commands.

3. Using the `cp` command with the `which` command lets copy `bash` and `ls` commands **without** specifying the path you are copying from.

cp $(which ls) chroot_jail/bin/
cp $(which bash) chroot_jail/bin/

4. For `bash` and `ls` to work in the new root folder, add all associated libraries to `chroot_jail/libraries`. Use the `ldd` command to find out which libraries are associated with which command:

ldd $(which bash)
ldd $(which ls)

5. Copy the appropriate libraries to the chroot_jail subdirectories lib and lib64.

For **bash**

cp /lib/x86_64-linux-gnu/libtinfo.so.6 chroot_jail/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libdl.so.2 chroot_jail/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libc.so.6 chroot_jail/lib/x86_64-linux-gnu/
cp /lib64/ld-linux-x86-64.so.2 chroot_jail/lib64/

For the **ls** command:

cp /lib/x86_64-linux-gnu/libselinux.so.1 chroot_jail/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libc.so.6 chroot_jail/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libpcre2-8.so.0 chroot_jail/lib/x86_64-linux-gnu/
cp /lib/x86_64-linux-gnu/libdl.so.2 chroot_jail/lib/x86_64-linux-gnu/
cp /lib64/ld-linux-x86-64.so.2 chroot_jail/lib64/
cp /lib/x86_64-linux-gnu/libpthread.so.0 chroot_jail/lib/x86_64-linux-gnu/

6. Use the chroot command to change the root to the chroot_jail directory. Run this command to change the root to the chroot_jail directory.

sudo chroot chroot_jail /bin/bash

Now we can use the **ls** command here. If want to exit, use `exit`.

A new shell must pop up when we `chroot`, our **jailed** bash. We currently have only 2 commands installed, `bash` and `ls`. Fortunately **cd** and **pwd** are builtin commands in bash shell, and so you can use them as well.

Common ways of using `chroot`:

   chroot /path/to/new/root command
                OR
   chroot /path/to/new/root /path/to/server
                OR
   chroot [options] /path/to/new/root /path/to/server

### Interesting points

If you run `ps aux` inside the jail (but we need to move necessary stuffs for ps, and mount the proc, **not trivial**), there will only be one process.

Interestingly, processes in the **jailed** shell run as a simple child process of this shell. All the processes inside the **JAILED** environment, are just **simple user level process** in the host OS and are **isolated** by the **namespaces** provided by the kernel, thus there is **minimal** overhead and as an added benefit we get isolation.

Similarly, you can add more commands to you virtual jailed environment. To add more complex programs, you might need to create more directories, like, ‘/proc’ and ‘/dev’.

### Namespace Isolation
In a single-user computer, a single system environment may be fine.

But on a server, where you want to run multiple services, it is essential to security and stability that the services are as isolated from each other as possible.

> Imagine a server running multiple services, one of which gets compromised by an intruder. In such a case, the intruder may be able to exploit that service and work his way to the other services, and may even be able compromise the entire server.

Namespace isolation can provide a secure environment to eliminate this risk.

The `initd/systemd`: everytime time a computer with Linux boots up, it starts with just one process, with process identifier (PID) 1.
* This process is the **root** of the process tree, and it initiates the rest of the system by performing the appropriate maintenance work and **starting** the correct daemons/services. All the other processes start **below** this process in the tree.
* The PID namespace allows a process to **spin** off a new tree, **with its own PID 1 process**.

The process that does this remains in the parent namespace, in the original tree, but makes the child the root of its own process tree.
* With PID namespace isolation, processes in the child namespace have no way of knowing of the parent process’s existence. However, processes in the parent namespace have a complete view of processes in the child namespace, as if they were any other process in the parent namespace.


**Nested namespace**: It is possible to create a nested set of child namespaces: one process starts a child process in a new PID namespace, and that child process spawns yet another process in a new PID namespace, and so on.

Can track multiple `pid` using `upid` (Linux source code)

struct upid {
  int nr;                     // the PID value
  struct pid_namespace *ns;   // namespace where this PID is relevant
  // ...
};

struct pid {
  // ...
  int level;                  // number of upids
  struct upid numbers[0];     // array of upids
};

#### Create a new namespace example

A PID namespace can only be created at the **time** a new process is spawned using `clone()`.

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

static char child_stack[1048576];

static int child_fn() {
  printf("PID: %ld\n", (long)getpid());
  printf("Parent PID: %ld\n", (long)getppid());
  return 0;
}

int main() {
  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWPID | SIGCHLD, NULL);
  printf("clone() = %ld\n", (long)child_pid);

  waitpid(child_pid, NULL, 0);
  return 0;
}

Compile and run this program with root privileges and you will notice an output that resembles this:

clone() = 5486
PID: 1
Parent PID: 0

The PID, as printed from within the `child_fn`, will be 1.

For the PPID, it will return parent PID of `0`: no parent (doesn't know).

Try running the same program again, but this time, remove the `CLONE_NEWPID` flag from within the `clone()` function call:

pid_t child_pid = clone(child_fn, child_stack+1048576, SIGCHLD, NULL);

You will no longer have ppid of 0.

#### `clone` vs `fork`
`fork()`: create a new process, and child process has **identical** data to its parent process. However, both processes have separate address spaces.
> The child process created receives a unique Process Identifier (PID) but retains the parent’s PID as its Parent Process Identifier (PPID).

The `clone()` system call is an **upgraded** version of the fork call. It’s powerful since it creates a child process and provides more **precise** control over the data shared between the parent and child processes. The caller of this system call can control the table of file descriptors, the table of signal handlers, and whether the two processes share the same address space.

`clone()` system call allows the child process to be placed in different namespaces. With the flexibility that comes with using the `clone()` system call, we can **choose**:
1. To share an address space with the parent process, emulating the vfork() system call.
2. To share file system information, open files, and signal handlers using different flags available.

#### Network namespace

#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
#include <unistd.h>

static char child_stack[1048576];


static int child_fn() {
  sleep(3);
  // Process namespace
  printf("PID: %ld\n", (long)getpid());
  printf("Parent PID: %ld\n", (long)getppid());

  // Linux network namespace
  printf("New `net` Namespace:\n");
  system("ip link");
  printf("\n\n");
  return 0;
}

int main() {
  printf("Original `net` Namespace:\n");
  system("ip link");
  printf("\n\n");
  pid_t child_pid = clone(child_fn, child_stack+1048576, CLONE_NEWPID | CLONE_NEWNET | CLONE_NEWNS | SIGCHLD, NULL);
  printf("clone() = %ld\n", (long)child_pid);

  char command[256];
  sprintf(command, "sudo ip link add name veth0 type veth peer name veth1 netns %d\n", child_pid);
  system(command);
  waitpid(child_pid, NULL, 0);
  return 0;
}

A network namespace allows each of these processes to see an **entirely** different set of networking **interfaces**. Even the loopback interface is different for each network namespace.

Without the `command` ran by the parent, we have:

Original `net` Namespace:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 06:30:d2:d8:6a:d8 brd ff:ff:ff:ff:ff:ff


clone() = 7709
PID: 1
Parent PID: 0
New `net` Namespace:
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

The physical ethernet device `eth0` belongs to the **global** network namespace. However, the physical interface is not available in the new network namespace. Moreover, the loopback device is **active** in the original network namespace, but is “**down**” in the child network namespace.

#### Routing
In order to provide a **usable** network interface in the child namespace, it is necessary to set up **additional** “virtual” network interfaces which span multiple namespaces.

Once that is done, it is then possible to create Ethernet **bridges**, and even **route** packets between the namespaces. Finally, to make the whole thing work, a “routing process” must be running in the global network namespace to receive traffic from the physical interface, and route it through the appropriate virtual interfaces to to the correct child network namespaces.

Running this command:

sudo ip link add name veth0 type veth peer name veth1 netns [child_pid]

establishes a **pipe**-like connection between these two namespaces. The parent namespace retains the `veth0` device, and passes the `veth1` device to the child namespace. Anything that enters one of the ends, comes out through the other end, just as you would expect from a real Ethernet connection between two real nodes. Accordingly, both sides of this virtual Ethernet connection **must** be assigned IP addresses: `06:30:d2:d8:6a:d8` and `d6:75:ed:cc:f7:3a`.

Original `net` Namespace:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 06:30:d2:d8:6a:d8 brd ff:ff:ff:ff:ff:ff


clone() = 7383
PID: 1
Parent PID: 0
New `net` Namespace:
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: veth1@if4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether d6:75:ed:cc:f7:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0

Linux also maintains a data structure for all the mountpoints of the system. We can also separate mount namespace (similar to doing `chroot`):
- What disk partitions are mounted,
- where they are mounted,
- whether they are readonly
- file permissions, etc
With Linux namespaces, one can have this **data structure** cloned, so that processes under different namespaces can change the **mountpoints** without affecting each other.

### Summary
A container is a **sandboxed** process on your machine that is **isolated** from all other processes on the host machine:
* It is a **runnable** instance of an image.
* You can create, start, stop, move, or delete a container using the DockerAPI or CLI.
* Can be run anywere: local machines, virtual machines or deployed to the cloud.
* Is **portable** (can be run on **any** OS because well, it comes with Linux Kernel).
* Will be running natively in Linux-based distros
* Is **isolated** from other containers and runs its own software, binaries, and configurations.

That isolation leverages kernel **namespaces** and **cgroups** (features that have been in Linux for a long time).
> A control group (**cgroup**) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, and so on) of a collection of processes. You can read more about them [here](https://docs.kernel.org/admin-guide/cgroup-v1/cgroups.html).

## Docker Containers

### Install Docker
You can follow the official instruction [here](https://docs.docker.com/engine/install/ubuntu/) to install Docker Engine (for Ubuntu or other Linux Distro), or install Docker Desktop for macOS/Windows.
> While Docker Desktop is required to run Docker Engine on macOS and Windows (VM), it is not required for Linux. Docker Engine **sits** directly on its kernel and runs natively.

Linux distro users, you can also see the tl;dr below:

# Install Docker packages
sudo apt-get update
sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
    
# Add docker official gpg key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

# Setup docker repo
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install docker engine
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
 
# Start the service
sudo service docker start

# Run docker without sudo,
sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker 
docker ps 

### Basics

In Docker’s case, having **modern** Linux kernel **features**, such as **control groups** and **namespaces**, means that containers can have strong isolation, their own network and storage stacks, as well as resource management capabilities to allow friendly co-existence of multiple containers on a host.

Docker adds an **application deployment engine** on top of a virtualized **container** execution environment:
1. **Fast**: compared to VM
2. Local **segregation** of duties:
* **Devs**: running what's inside the container
* **Ops**: managing the container

Enhance **consistency**: ensuring the environment in which your devs write code matches the deploy environment (library versions, tools used, etc).

### Docker Client and Docker Server (Daemon)
Docker client (aka `docker`) talks to server (daemon) --- aka the Docker engine. Docker ships with a **command** line client binary, docker, as well as a full RESTful API to interact with the daemon: `dockerd`.

> `dockerd` and docker client can be running on different hosts.


The Docker client is the primary way that many Docker **users** interact with Docker. It talks to the Docker daemon, which does the **heavy** lifting of building, running, and distributing your Docker containers. The Docker client can communicate with more than one daemon. The `docker` command uses the **Docker API**.

The Docker client and daemon communicate using a REST API, over **UNIX** sockets or a network interface.

> Another Docker client is Docker Compose, that lets you work with applications consisting of a set of containers.

#### Privileges
Docker runs as a **root**-privileged daemon process to allow it to handle op- erations that can’t be executed by normal users (e.g., mounting filesystems). The docker binary runs as a client of this daemon and also requires **root** privileges to run.

### Docker Desktop
Docker **server** is not to be confused with Docker Desktop. Docker

Desktop is an **easy-to-install** application for your Mac, Windows or Linux environment that enables you to build and share containerized applications and microservices.

Docker Desktop **includes** the Docker daemon (`dockerd`), the Docker client (`docker`), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper.

### Docker Images
You launch your **containers** from **images**. Images are the “build” part of Docker’s life cycle: the source code of the containers.

We can use existing images or build our own images (mostly we do the former).

### Docker Registries (not so important yet)
Docker stores the images you build in **registries**: public and private.

When you use the `docker pull` or `docker run` commands, the required images are pulled from your configured registry. When you use the docker push command, your image is pushed to your configured registry.

### Containers
Docker helps you build and deploy **containers** inside of which you can package your applications and services:
* Containers are launched from **images** and can contain one or more running processes
* It is an image format, set of standard ops, execution env

Each container contains a software **image** – its ‘cargo’ – and, like its physical counterpart, allows a set of **operations** to be performed.
> For example, it can be created, started, stopped, restarted, and destroyed.

Like a **shipping** container, Docker doesn’t care about the contents of the container when performing these actions; for example, whether a container is a web server, a database, or an application server. **Each container is loaded the same as any other containers.**

### Compose
Docker Compose (a docker client) - which allows you to run **stacks** of containers to represent application stacks, for example web server, application server and database server containers running **together** to serve a specific application.

## Technical Component
Docker can be run **natively** on any x64 host running a modern Linux kernel.

* Docker does not have an OS in its **containers**. In simple terms, a docker container image just has a kind of filesystem snapshot of the linux-image the container image is dependent on.
* However, "Docker **Desktop** for Mac/Windows" (not the container) **does** run some sort of **Linux host VM**, with a replacement for the older days `boot2docker` - LinuxKit developed and maintained by Docker for the **purpose** of making lightweight distributions.

In the early days: Docker Machine + VirtualBox + boot2docker,
The new days: provisioning is done internally by "Docker [Desktop] for Mac" and VirtualBox is replaced by [Apple's Hypervisor](https://developer.apple.com/documentation/hypervisor).

> Docker for Mac is a complete development environment deeply integrated with the MacOS Hypervisor framework, networking and filesystem.

Types of **isolation**:
- **Filesystem** isolation: each container is its own root filesystem.
- **Process** isolation: each container runs in its own process environment.
- **Network** isolation: separate virtual interfaces and IP addressing between containers.
- **Resource** isolation and grouping: resources like CPU and memory are allo- cated individually to each Docker container using the cgroups, or control groups, kernel feature.
- **Copy-on-write**: filesystems are created with copy-on-write, meaning they are layered and fast and require limited disk usage.
- **Logging**: `STDOUT`,`STDERR` and `STDIN` from the container are collected,logged, and available for analysis or trouble-shooting.
• **Interactive shell**: You can create a `pseudo-tty` and attach to `STDIN` to provide an interactive shell **to your container**.

### Copy-on-write
If a resource is **duplicated** but not modified, it is not necessary to create a **new** resource; the resource can be **shared** between the copy and the original.

Modifications must still create a copy, hence the technique: the copy operation is *deferred* until the first write.

By sharing resources in this way, it is possible to significantly reduce the resource consumption of unmodified copies, while adding a small overhead to resource-modifying operations.

Implemented in `fork`: Usually the child process does not modify any memory and immediately **executes** (`exec`) a new process, replacing the address space entirely. Thus, it would be wasteful to copy all of the parent process's memory during a fork, and instead the `copy-on-write` technique is used.

## Part 1: Creating a Dockerfile
**Goal:** learn the very basics about building a container image and created a Dockerfile to do so. Once we built an image, we started the container and saw the running app.

This [docker cheatsheet ](https://gist.github.com/ShawnClake/8650930c9283d1e6473371f5db7d9478)is very useful to have at hand.

Open this [app starter project](https://www.dropbox.com/sh/ccqo3a4hc6p3eoi/AABrR_X4cU7B17mpVnoMxM-xa?dl=0), and create a `Dockerfile` in the same dir as `package.json`:
> You can view the full tutorial [here](https://docs.docker.com/get-started/). We shorten it in the interest of time.

# syntax=docker/dockerfile:1
FROM node:12-alpine
RUN apk add --no-cache python2 g++ make
WORKDIR /app
COPY . .
RUN yarn install --production
CMD ["node", "src/index.js"]
EXPOSE 3000

1. We instructed the builder that we wanted to start from the `node:12-alpine` image.
2. The `RUN` instruction will execute any commands in a new layer on top of the current image and commit the results
3. The `WORKDIR` instruction sets the working directory for any `RUN`, `CMD`, `ENTRYPOINT`, `COPY` and `ADD` instructions that follow it in the Dockerfile.
4. The `COPY` instruction copies **new** files or directories from **host `[src]` and adds them to the filesystem of the container at the path `[dest]`.

### Docker Base Image
Most Docker images aren’t built from scratch. Instead, you take an existing image and use it as the basis for your image using the FROM command in your Dockerfile. There are also many **other** starting images for other projects like [Python](https://hub.docker.com/_/python), [Jekyll](https://hub.docker.com/r/jekyll/jekyll/), etc.

FROM python:3.9

Docker has a series of “official” Docker base images based on various Linux distributions, and also base images that package specific programming languages, in particular Python.

### About RUN vs CMD
`RUN` is an image build step, the **state** of the container **after** a `RUN` command will be **committed** to the container image. A Dockerfile can have many RUN steps that layer on top of one another to build the image.

`CMD` is the command the container **executes** by default when you **launch** the built image. A Dockerfile will only use the **final** CMD defined.

In short, do all the `RUN` needed to setup your environment, and your (only) CMD **launches** the process running in your container,

---

Then, we need to **build** the image:

docker build -t site-todo .

Afterwards, we can **start an app container**

docker run -dp 3000:3000 site-todo

It will return the container **hash**.
- `-d`: detached options (in the background)
- `-p`: mapping between host port 3000 to container port 3000

## Part 2: Persist the db
We need to prevent our to-do list to be wiped out every time we launch the container.

When a container runs, it uses the various **layers** FROM an image for its **filesystem**. Each container also gets its own “scratch space” to **create/update/remove** files. It doesn't run from it's previous run state.

**Any changes won’t be seen in another container, even if they are using the same image, because the image remains the same**.
> Every time you do a docker `run` it will spin up a fresh container based on your image.

In our toy project, the todo app stores its data in a SQLite Database at `/etc/todos/todo.db` in the **container’s filesystem**.

### Using Docker Named Volume
We want to store this data using a volume and **mounting** it into the container.
> Docker maintains the physical location on the disk and you only need to remember the name of the volume.

To do this, create the named volume:

docker volume create todo-db

Then stop it, and re-run it with this volume mapped to `/etc/todos/todo.db`:

docker ps
docker -rm -f [ID]
docker run -dp 3000:3000 -v todo-db:/etc/todos site-todo

Whenever you stop and rerun the container from now on, any changes you made in the to-do list will persist. You can inspect the volume from your Docker Desktop, or using the command:

docker volume inspect [VOLUME]

### Using bind mount

You can also map it directly to a local directory by running it as such:

docker run -dp 3000:3000 -v [PATH_LOCAL_DIR]:/etc/todos site-todo

Depending on your OS, you might need to **enable file sharing** (macOS). Ubuntu and other Linux distro doesn't require any other settings.

You can then get your EC2 public ip:

dig +short myip.opendns.com @resolver1.opendns.com

and open http://[YOUR-PUBLIC-IP]:3000 in your browser.

## Part 3: Using `docker-compose`

Instead of repeatedly keying each command one by one, we can use `docker-compose` to spin up or tear down our containers with a single command. It is a huge topic in itself, and there are already [plenty of examples](https://github.com/docker/awesome-compose) out there for every common use case.

For our toy project, create `docker-compose.yml` file in the root project path.

version: "3.7"

services:
  site-todo:
    build:
      context: "."
      dockerfile: Dockerfile
    ports:
      - 3000:3000
    volumes:
      - ./todo-db-local:/etc/todos
#      - todo-db:/etc/todos   # if using Docker volume

# if using Docker volume
# volumes:
#   todo-db:

Then, type the following to run the container in the background:

docker compose up -d

Once you're done, type the following to delete your container:

docker compose down

> Remember if you change your project and didn't use bind mount, then you need to `build` a new image each time you want to test your app.

## Exercise
You can create a **network** of containers, and each container will have it's own IP addresses.

Create a new network:

docker network create todo-app

Use standard `mysql` container and spawn it in that network:

docker run -d \
     --network todo-app --network-alias mysql \
     -v todo-mysql-data:/var/lib/mysql \
     -e MYSQL_ROOT_PASSWORD=secret \
     -e MYSQL_DATABASE=todos \
     mysql:5.7

You can go into your database by the following command:

docker exec -it [CONTAINER_ID] mysql -u root -p


Now we just need to run our todo-app container **in the same network**:

 docker run -dp 3000:3000 \
   -w /app -v "$(pwd):/app" \
   --network todo-app \
   -e MYSQL_HOST=mysql \
   -e MYSQL_USER=root \
   -e MYSQL_PASSWORD=secret \
   -e MYSQL_DB=todos \
   node:12-alpine \
   sh -c "yarn install && yarn run dev"

As you open your app in the browser and add the todo list:

You can inspect the same in your mysql container:

Notice the environment: `MYSQL_HOST=mysql`. This is the **hostname** of our mysql service, as we set before when running the mysql container: `--network-alias mysql`.

We can use [this](https://github.com/nicolaka/netshoot) tool and run `dig` command inside it to confirm that indeed our mysql container is assigned a local IP address within the docker network `todo-app`.

docker run -it --network todo-app nicolaka/netshoot


Now, notice that we don't utilise any `Dockerfile` or `docker-compose.yaml` file to run any of these containers. Everything is done via the one-liner cli. Can you create:
1. `Dockerfile`
2. `docker-compose.yaml` file
.... which will reach the same effect?