Book mode https://hackmd.io/@ncnu-opensource/book

LSA 番外篇 - 從入門到放棄 - Docker/Container

Slide

從入門到放棄 - Docker/Container

docker

Docker components/docker daemon
- https://docs.docker.com/get-started/overview/
```
$ curl -s --unix-socket /var/run/docker.sock http:/localhost/info | jq .
```
- You can also access other docker APIs
docker lifecycle
- How the docker terminate the running container. Please refer docker stop.
  
  The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL.
- docker stop
- docker ps -a
- Exitcodes: http://tldp.org/LDP/abs/html/exitcodes.html
- nginx - sigterm : https://github.com/nginxinc/docker-nginx/blob/594ce7a8bc26c85af88495ac94d5cd0096b306f7/mainline/buster/Dockerfile#L101

namespace

namespace and cgroup
- Linux capabilities
How many type namespace does docker use? Ans: Docker overview

The pid namespace: Process isolation (PID: Process ID).
The net namespace: Managing network interfaces (NET: Networking).
The ipc namespace: Managing access to IPC resources (IPC: InterProcess Communication).
The mnt namespace: Managing filesystem mount points (MNT: Mount).
The uts namespace: Isolating kernel and version identifiers. (UTS: Unix Timesharing System).
docker inspect <YOUR_DOCKER>
cgroup: https://docs.docker.com/config/containers/runmetrics/
https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
OOM
- Please verify your Ubuntu enable the cgroup swap limit capabilities(cgroup_enable=memory ).
How to find
- pstree
- ps
- /proc/<pid>/cgroup
How to mapping the host pid and the container process. Thanks for yiyu assistance.

logging

log driver

$ docker inspect ff7c3b30abd4 | jq .[].HostConfig.LogConfig
{
  "Type": "json-file",
  "Config": {}
}

How the docker logs

docker logs
https://docs.docker.com/config/containers/logging/

$ docker inspect ff7c3b30abd4 | jq .[].LogPath
"/var/lib/docker/containers/ff7c3b30abd46c2d4c138e9a59bd9f1e0bc367da5db81e384d32d2659e5cfc7b/ff7c3b30abd46c2d4c138e9a59bd9f1e0bc367da5db81e384d32d2659e5cfc7b-json.log"

Nginx Dockerfile

storage

What is docker image layer
https://docs.docker.com/storage/storagedriver/

Network

docker network
iptable: how the docker run -d -p 80:8080 works?

If you want to check the network namespace, we can use nsenter command to run the command.

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
c51caebc6999        test-oom:latest     "bash"              10 minutes ago      Up 10 minutes                           my-test-oom
$ for id in `docker ps -n 1 -q`; do pid=`docker inspect $id -f '{{.State.Pid}}'`; echo container $id; sudo nsenter -t $pid -n ip link show; done
container c51caebc6999
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
16: eth0@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default 
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0

Test environment

$ sudo lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:        18.04
Codename:       bionic

$ docker info
  Running: 0
 Server Version: 19.03.8                                                                                                                                              [0/53]
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.3.0-1017-aws
 Operating System: Ubuntu 18.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 1
 Total Memory: 978.8MiB
 Name: ip-172-31-32-58
 ID: DXL4:POV4:6DLW:IBTW:2ZHX:NX6S:3ULN:4ME5:QQUY:4LYC:ZSYX:FRME
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support

By default, Ubuntu did not enable the cgroup_enable=memory, You may see messages similar to the following when working with an image.
Please follow the
Post-installation steps for Linux - Your kernel does not support cgroup swap limit capabilities section.

WARNING: Your kernel does not support swap limit capabilities. Limitation discarded.

Test OOM via stress-ng

Add a Dcokerfile.


From ubuntu:20.04
RUN apt update && apt install stress-ng -y

Built a image with the stress-ng tool.

$ docker build . -t test-OOM
invalid argument "test-OOM" for "-t, --tag" flag: invalid reference format: repository name must be lowercase
See 'docker build --help'.
125 ec2-user@ip-172-31-35-188:~$ docker build . -t test-oom
Sending build context to Docker daemon  4.755MB
Step 1/2 : From ubuntu:20.04
 ---> 1d622ef86b13
Step 2/2 : RUN apt update && apt install stress-ng -y
 ---> Running in bd3213d5bc74

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [107 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal InRelease [265 kB]
Get:3 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [1077 B]
Get:4 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [13.2 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [134 kB]
Get:6 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [42.3 kB]
Get:7 http://archive.ubuntu.com/ubuntu focal-updates InRelease [107 kB]
Get:8 http://archive.ubuntu.com/ubuntu focal-backports InRelease [98.3 kB]
Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages [11.3 MB]
Get:10 http://archive.ubuntu.com/ubuntu focal/restricted amd64 Packages [33.4 kB]
Get:11 http://archive.ubuntu.com/ubuntu focal/multiverse amd64 Packages [177 kB]
Get:12 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages [1275 kB]
Get:13 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [13.2 kB]
Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [251 kB]
Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [136 kB]
Get:16 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [1077 B]
Get:17 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [2900 B]
Fetched 14.0 MB in 5s (2712 kB/s)
Reading package lists...
Building dependency tree...
Reading state information...
5 packages can be upgraded. Run 'apt list --upgradable' to see them.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libaio1 libapparmor1 libbsd0 libipsec-mb0 libjudydebian1 libsctp1
Suggested packages:
  lksctp-tools
The following NEW packages will be installed:
  libaio1 libapparmor1 libbsd0 libipsec-mb0 libjudydebian1 libsctp1 stress-ng
0 upgraded, 7 newly installed, 0 to remove and 5 not upgraded.
Need to get 2377 kB of archives.
After this operation, 20.2 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapparmor1 amd64 2.13.3-7ubuntu5.1 [34.1 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 libbsd0 amd64 0.10.0-1 [45.4 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio1 amd64 0.3.112-5 [7184 B]
Get:4 http://archive.ubuntu.com/ubuntu focal/universe amd64 libipsec-mb0 amd64 0.53-1 [491 kB]
Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjudydebian1 amd64 1.0.5-5 [94.6 kB]
Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 libsctp1 amd64 1.0.18+dfsg-1 [7876 B]
Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 stress-ng amd64 0.11.07-1 [1697 kB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 2377 kB in 2s (1077 kB/s)
Selecting previously unselected package libapparmor1:amd64.
(Reading database ... 4122 files and directories currently installed.)
Preparing to unpack .../0-libapparmor1_2.13.3-7ubuntu5.1_amd64.deb ...
Unpacking libapparmor1:amd64 (2.13.3-7ubuntu5.1) ...
Selecting previously unselected package libbsd0:amd64.
Preparing to unpack .../1-libbsd0_0.10.0-1_amd64.deb ...
Unpacking libbsd0:amd64 (0.10.0-1) ...
Selecting previously unselected package libaio1:amd64.
Preparing to unpack .../2-libaio1_0.3.112-5_amd64.deb ...
Unpacking libaio1:amd64 (0.3.112-5) ...
Selecting previously unselected package libipsec-mb0.
Preparing to unpack .../3-libipsec-mb0_0.53-1_amd64.deb ...
Unpacking libipsec-mb0 (0.53-1) ...
Selecting previously unselected package libjudydebian1.
Preparing to unpack .../4-libjudydebian1_1.0.5-5_amd64.deb ...
Unpacking libjudydebian1 (1.0.5-5) ...
Selecting previously unselected package libsctp1:amd64.
Preparing to unpack .../5-libsctp1_1.0.18+dfsg-1_amd64.deb ...
Unpacking libsctp1:amd64 (1.0.18+dfsg-1) ...
Selecting previously unselected package stress-ng.
Preparing to unpack .../6-stress-ng_0.11.07-1_amd64.deb ...
Unpacking stress-ng (0.11.07-1) ...
Setting up libapparmor1:amd64 (2.13.3-7ubuntu5.1) ...
Setting up libjudydebian1 (1.0.5-5) ...
Setting up libipsec-mb0 (0.53-1) ...
Setting up libsctp1:amd64 (1.0.18+dfsg-1) ...
Setting up libaio1:amd64 (0.3.112-5) ...
Setting up libbsd0:amd64 (0.10.0-1) ...
Setting up stress-ng (0.11.07-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9) ...
Removing intermediate container bd3213d5bc74
 ---> 4b3493a0fd76
Successfully built 4b3493a0fd76
Successfully tagged test-oom:latest

Verify the the image you built with test-oom tag.

$ docker images
REPOSITORY                                                      TAG                 IMAGE ID            CREATED             SIZE
test-oom                                                        latest              4b3493a0fd76        35 seconds ago      116MB

Launch and attach into the image.

$ docker run --rm --name my-test-oom -m 200m -it test-oom:latest bash

Open another terminal to view docker stats.

$ docker stats
CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT   MEM %               NET I/O             BLOCK I/O           PIDS
c51caebc6999        my-test-oom         0.00%               1.082MiB / 200MiB   0.54%               906B / 0B           0B / 0B             1

Run the following stress-ng command in the container. More information for the stress-ng.

root@c51caebc6999:/# stress-ng --brk 2 --stack 2 --bigheap 2

We can view the OOM message in your syslog(/var/log/syslog).

Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25511 (stress-ng-brk) score 1275 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25511 (stress-ng-brk) total-vm:124528kB, anon-rss:56264kB, file-rss:804kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25511 (stress-ng-brk), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25516 (stress-ng-stack) score 1244 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25516 (stress-ng-stack) total-vm:118184kB, anon-rss:49912kB, file-rss:868kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25516 (stress-ng-stack), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25515 (stress-ng-bighe) score 1294 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25515 (stress-ng-bighe) total-vm:128540kB, anon-rss:60076kB, file-rss:864kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25515 (stress-ng-bighe), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25517 (stress-ng-stack) score 1323 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25517 (stress-ng-stack) total-vm:134212kB, anon-rss:66008kB, file-rss:868kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25517 (stress-ng-stack), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25514 (stress-ng-brk) score 1328 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25514 (stress-ng-brk) total-vm:135320kB, anon-rss:67064kB, file-rss:804kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25514 (stress-ng-brk), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25535 (stress-ng-stack) score 1248 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25535 (stress-ng-stack) total-vm:119088kB, anon-rss:50708kB, file-rss:868kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25535 (stress-ng-stack), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25521 (stress-ng-bighe) score 1295 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25521 (stress-ng-bighe) total-vm:128668kB, anon-rss:60344kB, file-rss:864kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25521 (stress-ng-bighe), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25536 (stress-ng-bighe) score 1279 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25536 (stress-ng-bighe) total-vm:125212kB, anon-rss:56912kB, file-rss:864kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25536 (stress-ng-bighe), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25537 (stress-ng-stack) score 1311 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25537 (stress-ng-stack) total-vm:131972kB, anon-rss:63636kB, file-rss:868kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25537 (stress-ng-stack), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25534 (stress-ng-brk) score 1353 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25534 (stress-ng-brk) total-vm:140428kB, anon-rss:72176kB, file-rss:804kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25534 (stress-ng-brk), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25540 (stress-ng-bighe) score 1250 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25540 (stress-ng-bighe) total-vm:119452kB, anon-rss:51104kB, file-rss:864kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25540 (stress-ng-bighe), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25539 (stress-ng-stack) score 1291 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25539 (stress-ng-stack) total-vm:127784kB, anon-rss:59404kB, file-rss:868kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25539 (stress-ng-stack), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25541 (stress-ng-bighe) score 1304 or sacrifice child
Jun 17 01:05:28 ip-172-31-35-188 kernel: Killed process 25541 (stress-ng-bighe) total-vm:130460kB, anon-rss:62188kB, file-rss:864kB, shmem-rss:4kB
Jun 17 01:05:28 ip-172-31-35-188 kernel: oom_reaper: reaped process 25541 (stress-ng-bighe), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:29 ip-172-31-35-188 kernel: oom_kill_process: 28 callbacks suppressed
Jun 17 01:05:29 ip-172-31-35-188 kernel: stress-ng-bighe invoked oom-killer: gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null),  order=0, oom_score_adj=1000
Jun 17 01:05:29 ip-172-31-35-188 kernel: stress-ng-bighe cpuset=c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573 mems_allowed=0
Jun 17 01:05:29 ip-172-31-35-188 kernel: CPU: 0 PID: 25547 Comm: stress-ng-bighe Not tainted 4.14.177-139.253.amzn2.x86_64 #1
Jun 17 01:05:29 ip-172-31-35-188 kernel: Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
Jun 17 01:05:29 ip-172-31-35-188 kernel: Call Trace:
Jun 17 01:05:29 ip-172-31-35-188 kernel: dump_stack+0x66/0x82
Jun 17 01:05:29 ip-172-31-35-188 kernel: dump_header+0x94/0x229
Jun 17 01:05:29 ip-172-31-35-188 kernel: oom_kill_process+0x223/0x420
Jun 17 01:05:29 ip-172-31-35-188 kernel: out_of_memory+0x112/0x4d0
Jun 17 01:05:29 ip-172-31-35-188 kernel: mem_cgroup_out_of_memory+0x49/0x80
Jun 17 01:05:29 ip-172-31-35-188 kernel: mem_cgroup_oom_synchronize+0x2ed/0x330
Jun 17 01:05:29 ip-172-31-35-188 kernel: ? mem_cgroup_css_reset+0xd0/0xd0
Jun 17 01:05:29 ip-172-31-35-188 kernel: pagefault_out_of_memory+0x32/0x77
Jun 17 01:05:29 ip-172-31-35-188 kernel: __do_page_fault+0x4b4/0x4c0
Jun 17 01:05:29 ip-172-31-35-188 kernel: ? page_fault+0x2f/0x50
Jun 17 01:05:29 ip-172-31-35-188 kernel: page_fault+0x45/0x50
Jun 17 01:05:29 ip-172-31-35-188 kernel: RIP: 6309a0f0:0x10000
Jun 17 01:05:29 ip-172-31-35-188 kernel: RSP: ad423010:00007ffd6309a0e0 EFLAGS: 00001000
Jun 17 01:05:29 ip-172-31-35-188 kernel: Task in /docker/c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573 killed as a result of limit of /docker/c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573
Jun 17 01:05:29 ip-172-31-35-188 kernel: memory: usage 204800kB, limit 204800kB, failcnt 49204
Jun 17 01:05:29 ip-172-31-35-188 kernel: memory+swap: usage 204800kB, limit 409600kB, failcnt 0
Jun 17 01:05:29 ip-172-31-35-188 kernel: kmem: usage 2700kB, limit 9007199254740988kB, failcnt 0
Jun 17 01:05:29 ip-172-31-35-188 kernel: Memory cgroup stats for /docker/c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573: cache:2184KB rss:199916KB rss_huge:0KB shmem:2164KB mapped_file:2184KB dirty:0KB writeback:0KB swap:0KB inactive_anon:2152KB active_anon:199912KB inactive_file:0KB active_file:0KB unevictable:12KB
Jun 17 01:05:29 ip-172-31-35-188 kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes nr_pmds swapents oom_score_adj name
Jun 17 01:05:29 ip-172-31-35-188 kernel: [22120]     0 22120     1028      250       7       3        0             0 bash
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23961]     0 23961    17158     1175      13       3        0          1000 stress-ng
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23965]     0 23965    17158      378      13       3        0          1000 stress-ng-brk
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23966]     0 23966    17158      333      13       3        0          1000 stress-ng-stack
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23967]     0 23967    17158      374      13       3        0          1000 stress-ng-bighe
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23968]     0 23968    17158      345      13       3        0          1000 stress-ng-brk
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23969]     0 23969    17158      322      13       3        0          1000 stress-ng-stack
Jun 17 01:05:29 ip-172-31-35-188 kernel: [23970]     0 23970    17158      291      13       3        0          1000 stress-ng-bighe
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25538]     0 25538    33440    16578      45       3        0          1000 stress-ng-brk
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25542]     0 25542    31002    14146      40       3        0          1000 stress-ng-stack
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25543]     0 25543    21217     4356      21       3        0          1000 stress-ng-brk
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25544]     0 25544    25143     8301      29       3        0          1000 stress-ng-bighe
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25546]     0 25546    23040     6163      25       3        0          1000 stress-ng-stack
Jun 17 01:05:29 ip-172-31-35-188 kernel: [25547]     0 25547    18615     1737      16       3        0          1000 stress-ng-bighe
Jun 17 01:05:29 ip-172-31-35-188 kernel: Memory cgroup out of memory: Kill process 25538 (stress-ng-brk) score 1320 or sacrifice child
Jun 17 01:05:29 ip-172-31-35-188 kernel: Killed process 25538 (stress-ng-brk) total-vm:133760kB, anon-rss:65504kB, file-rss:804kB, shmem-rss:4kB
Jun 17 01:05:29 ip-172-31-35-188 kernel: oom_reaper: reaped process 25538 (stress-ng-brk), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
Jun 17 01:05:29 ip-172-31-35-188 kernel: stress-ng-stack invoked oom-killer: gfp_mask=0x14000c0(GFP_KERNEL), nodemask=(null),  order=0, oom_score_adj=1000
Jun 17 01:05:29 ip-172-31-35-188 kernel: stress-ng-stack cpuset=c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573 mems_allowed=0
Jun 17 01:05:29 ip-172-31-35-188 kernel: CPU: 0 PID: 25542 Comm: stress-ng-stack Not tainted 4.14.177-139.253.amzn2.x86_64 #1
Jun 17 01:05:29 ip-172-31-35-188 kernel: Hardware name: Xen HVM domU, BIOS 4.2.amazon 08/24/2006
Jun 17 01:05:29 ip-172-31-35-188 kernel: Call Trace:
Jun 17 01:05:29 ip-172-31-35-188 kernel: dump_stack+0x66/0x82
Jun 17 01:05:29 ip-172-31-35-188 kernel: dump_header+0x94/0x229
Jun 17 01:05:29 ip-172-31-35-188 kernel: oom_kill_process+0x223/0x420
Jun 17 01:05:29 ip-172-31-35-188 kernel: out_of_memory+0x112/0x4d0
Jun 17 01:05:29 ip-172-31-35-188 kernel: mem_cgroup_out_of_memory+0x49/0x80
Jun 17 01:05:29 ip-172-31-35-188 kernel: mem_cgroup_oom_synchronize+0x2ed/0x330
Jun 17 01:05:29 ip-172-31-35-188 kernel: ? mem_cgroup_css_reset+0xd0/0xd0
Jun 17 01:05:29 ip-172-31-35-188 kernel: pagefault_out_of_memory+0x32/0x77
Jun 17 01:05:29 ip-172-31-35-188 kernel: __do_page_fault+0x4b4/0x4c0
Jun 17 01:05:29 ip-172-31-35-188 kernel: ? page_fault+0x2f/0x50
Jun 17 01:05:29 ip-172-31-35-188 kernel: page_fault+0x45/0x50
Jun 17 01:05:29 ip-172-31-35-188 kernel: RIP: 6309a0e0:0x1000
Jun 17 01:05:29 ip-172-31-35-188 kernel: RSP: 40000:00007ffd5ef54d80 EFLAGS: 7ffd5ef54d90
Jun 17 01:05:29 ip-172-31-35-188 kernel: Task in /docker/c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573 killed as a result of limit of /docker/c51caebc69999ea567d7acb8b1d7a30eb7f7b3d00b5e7cad7c80ad519e066573
Jun 17 01:05:29 ip-172-31-35-188 kernel: memory: usage 204800kB, limit 204800kB, failcnt 49236
Jun 17 01:05:29 ip-172-31-35-188 kernel: memory+swap: usage 204800kB, limit 409600kB, failcnt 0
Jun 17 01:05:29 ip-172-31-35-188 kernel: kmem: usage 2700kB, limit 9007199254740988kB, failcnt 0

LSA 番外篇 - 從入門到放棄 - Docker/Container

Slide

docker

namespace

logging

storage

Network

延伸閱讀

Test environment

Test OOM via stress-ng

Read more

Week 15 & 16 (2025/05/29, 2025/06/05) 期末報告

NCNU OpenSource - LSA 課程共筆

Monitoring

Week 11 (2025/05/01) 負載平衡與高可用性