owned this note
owned this note
Published
Linked with GitHub
# Moby v24.0 + containerd/runwasi
## Moby
- codebase
https://github.com/moby/moby.git
branch v24.0
commit 2bef272269e43eda5e06433456b3ffecc0f5a690 (2023.6.27)
- Pre-pocess
```bash=
cp {vendor,go}.mod
cp {vendor,go}.sum
cp hack/dockerfile/etc/docker/daemon.json ./daemon.json
```
- Edit this daemon.json to make `containerd-snapshotter` enable
```json=
{
"runtimes": {
"crun": {
"path": "/usr/local/bin/crun"
}
},
"features": {
"containerd-snapshotter": true
}
}
```
- Run dockerd/moby (create test context first)
```bash=
docker context create local --docker "host=unix:///tmp/docker.sock"
docker context use local
hack/make.sh binary && sudo ./bundles/binary-daemon/dockerd -D -H unix:///tmp/docker.sock --data-root /tmp/root --pidfile /tmp/docker.pid --config-file daemon.json
```
## containerd/runwasi
Test below commits
| commit | status |
| - | -|
| 55f9b32e27d8ee3fad0cd01d542b86a1c8061082 | ok |
| 68b20ad3f2f85b465ef0274f04dccf977ed7cbe9 | ok |
| 298e3c0f8bd47bd9f6be01cd4b6d6c76bf3d4d82 | ok |
| 84f5c50e537515268f984ab8cc5e0f0bd8ba9fda | ok |
| ca5260fd34635ca56d81db3f59ee8672fc7fde68 | ng |
moby log error in line 47-48, 65
```log=
DEBU[2023-06-30T02:09:13.153791520+08:00] Calling HEAD /_ping
DEBU[2023-06-30T02:09:13.156148614+08:00] Calling GET /v1.42/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.project%3Dwasmedge_hyper_demo%22%3Atrue%7D%7D&limit=0
DEBU[2023-06-30T02:09:13.158362803+08:00] Calling GET /v1.42/images/demo-client/json
DEBU[2023-06-30T02:09:13.159841529+08:00] Calling GET /v1.42/info
DEBU[2023-06-30T02:09:13.163141033+08:00] Calling GET /v1.42/networks?filters=%7B%22name%22%3A%7B%22wasmedge_hyper_demo_default%22%3Atrue%7D%7D
DEBU[2023-06-30T02:09:13.163911286+08:00] Calling GET /v1.42/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.oneoff%3DFalse%22%3Atrue%2C%22com.docker.compose.project%3Dwasmedge_hyper_demo%22%3Atrue%2C%22com.docker.compose.service%3Dclient%22%3Atrue%7D%7D&limit=0
DEBU[2023-06-30T02:09:13.165028342+08:00] Calling GET /v1.42/containers/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27/json
DEBU[2023-06-30T02:09:13.165991384+08:00] Calling POST /v1.42/containers/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27/attach?stderr=1&stdin=1&stdout=1&stream=1
DEBU[2023-06-30T02:09:13.166052748+08:00] attach: stdout: begin
DEBU[2023-06-30T02:09:13.166060489+08:00] attach: stderr: begin
DEBU[2023-06-30T02:09:13.166124761+08:00] Calling GET /v1.42/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.oneoff%3DFalse%22%3Atrue%2C%22com.docker.compose.project%3Dwasmedge_hyper_demo%22%3Atrue%2C%22com.docker.compose.service%3Dclient%22%3Atrue%7D%7D&limit=0
DEBU[2023-06-30T02:09:13.166217944+08:00] Calling GET /v1.42/events?filters=%7B%22label%22%3A%7B%22com.docker.compose.project%3Dwasmedge_hyper_demo%22%3Atrue%7D%7D
DEBU[2023-06-30T02:09:13.167697797+08:00] Calling POST /v1.42/containers/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27/start
DEBU[2023-06-30T02:09:13.168271404+08:00] container mounted via snapshotter: /tmp/root/rootfs/overlayfs/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 container=ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27
DEBU[2023-06-30T02:09:13.168421406+08:00] Assigning addresses for endpoint wasmedge_hyper_demo-client-1's interface on network wasmedge_hyper_demo_default
DEBU[2023-06-30T02:09:13.168431677+08:00] RequestAddress(LocalDefault/172.28.0.0/16, <nil>, map[])
DEBU[2023-06-30T02:09:13.168447170+08:00] Request address PoolID:172.28.0.0/16 Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3 Serial:false PrefAddress:invalid IP
DEBU[2023-06-30T02:09:13.190091257+08:00] Assigning addresses for endpoint wasmedge_hyper_demo-client-1's interface on network wasmedge_hyper_demo_default
DEBU[2023-06-30T02:09:13.190231506+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, true) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.190248827+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.190254612+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(client, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.190258899+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(ed1f07cc6532, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
INFO[2023-06-30T02:09:13.196437549+08:00] No non-localhost DNS nameservers are left in resolv.conf. Using default external servers: [nameserver 8.8.8.8 nameserver 8.8.4.4]
INFO[2023-06-30T02:09:13.196446501+08:00] IPv6 enabled; Adding default IPv6 external servers: [nameserver 2001:4860:4860::8888 nameserver 2001:4860:4860::8844]
DEBU[2023-06-30T02:09:13.202586104+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, true) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.202599794+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.202605472+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(client, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.202609377+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).addSvcRecords(ed1f07cc6532, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:13.208806011+08:00] Programming external connectivity on endpoint wasmedge_hyper_demo-client-1 (baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526)
DEBU[2023-06-30T02:09:13.222419711+08:00] DeleteConntrackEntries purged ipv4:0, ipv6:0
DEBU[2023-06-30T02:09:13.222456735+08:00] DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0
DEBU[2023-06-30T02:09:13.228887876+08:00] EnableService ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 START
DEBU[2023-06-30T02:09:13.228926616+08:00] EnableService ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 DONE
DEBU[2023-06-30T02:09:13.241186758+08:00] bundle dir created bundle=/var/run/docker/containerd/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 module=libcontainerd namespace=moby root=/tmp/root/rootfs/overlayfs/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27
DEBU[2023-06-30T02:09:14.385931551+08:00] event module=libcontainerd namespace=moby topic=/tasks/create
DEBU[2023-06-30T02:09:14.408787692+08:00] /usr/sbin/iptables, [--wait -t nat -C OUTPUT -d 127.0.0.11 -j DOCKER_OUTPUT]
DEBU[2023-06-30T02:09:14.409473793+08:00] /usr/sbin/iptables, [--wait -t nat -N DOCKER_OUTPUT]
DEBU[2023-06-30T02:09:14.410087604+08:00] /usr/sbin/iptables, [--wait -t nat -I OUTPUT -d 127.0.0.11 -j DOCKER_OUTPUT]
DEBU[2023-06-30T02:09:14.410758999+08:00] /usr/sbin/iptables, [--wait -t nat -C POSTROUTING -d 127.0.0.11 -j DOCKER_POSTROUTING]
DEBU[2023-06-30T02:09:14.411264678+08:00] /usr/sbin/iptables, [--wait -t nat -N DOCKER_POSTROUTING]
DEBU[2023-06-30T02:09:14.411801319+08:00] /usr/sbin/iptables, [--wait -t nat -I POSTROUTING -d 127.0.0.11 -j DOCKER_POSTROUTING]
DEBU[2023-06-30T02:09:14.412360571+08:00] /usr/sbin/iptables, [--wait -t nat -I DOCKER_OUTPUT -d 127.0.0.11 -p udp --dport 53 -j DNAT --to-destination 127.0.0.11:32907]
DEBU[2023-06-30T02:09:14.413207815+08:00] /usr/sbin/iptables, [--wait -t nat -I DOCKER_POSTROUTING -s 127.0.0.11 -p udp --sport 32907 -j SNAT --to-source :53]
DEBU[2023-06-30T02:09:14.414016107+08:00] /usr/sbin/iptables, [--wait -t nat -I DOCKER_OUTPUT -d 127.0.0.11 -p tcp --dport 53 -j DNAT --to-destination 127.0.0.11:33415]
DEBU[2023-06-30T02:09:14.414803745+08:00] /usr/sbin/iptables, [--wait -t nat -I DOCKER_POSTROUTING -s 127.0.0.11 -p tcp --sport 33415 -j SNAT --to-source :53]
DEBU[2023-06-30T02:09:14.493198790+08:00] sandbox set key processing took 84.827686ms for container ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27
ERRO[2023-06-30T02:09:14.649479888+08:00] stream copy error: read /proc/self/fd/24: file already closed
ERRO[2023-06-30T02:09:14.649536417+08:00] stream copy error: read /proc/self/fd/25: file already closed
DEBU[2023-06-30T02:09:14.649574594+08:00] attach: stdout: end
DEBU[2023-06-30T02:09:14.649607341+08:00] attach: stderr: end
DEBU[2023-06-30T02:09:14.649619478+08:00] attach done
DEBU[2023-06-30T02:09:14.651069707+08:00] event module=libcontainerd namespace=moby topic=/tasks/delete
INFO[2023-06-30T02:09:14.651085123+08:00] ignoring event container=ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
ERRO[2023-06-30T02:09:14.693690065+08:00] ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27 cleanup: failed to delete container from containerd: container "ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27" in namespace "moby": not found
DEBU[2023-06-30T02:09:14.693931434+08:00] Revoking external connectivity on endpoint wasmedge_hyper_demo-client-1 (baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526)
DEBU[2023-06-30T02:09:14.696621525+08:00] DeleteConntrackEntries purged ipv4:0, ipv6:0
DEBU[2023-06-30T02:09:14.696630381+08:00] DeleteConntrackEntriesByPort for udp ports purged ipv4:0, ipv6:0
DEBU[2023-06-30T02:09:14.782622966+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).deleteSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, true) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:14.782647840+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).deleteSvcRecords(wasmedge_hyper_demo-client-1, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:14.782652614+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).deleteSvcRecords(client, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:14.782656390+08:00] baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526 (c62524e).deleteSvcRecords(ed1f07cc6532, 172.28.0.2, <nil>, false) updateSvcRecord sid:baf81166b86404586202574883855b69f5ea9cd42e3f69145a8eeb3bc2e76526
DEBU[2023-06-30T02:09:14.916624003+08:00] Releasing addresses for endpoint wasmedge_hyper_demo-client-1's interface on network wasmedge_hyper_demo_default
DEBU[2023-06-30T02:09:14.916693306+08:00] ReleaseAddress(LocalDefault/172.28.0.0/16, 172.28.0.2)
DEBU[2023-06-30T02:09:14.916758372+08:00] Released address Address:172.28.0.2 Sequence:Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:3
ERRO[2023-06-30T02:09:14.948552364+08:00] Handler for POST /v1.42/containers/ed1f07cc6532a2ea247a25eff392c29637953f820f3f12eecdbc2100fd060c27/start returned error: failed to create container: unknown
DEBU[2023-06-30T02:09:14.950289771+08:00] Client context cancelled, stop sending events
DEBU[2023-06-30T02:09:14.950351753+08:00] Closing buffered stdin pipe
```
## Test Steps
Make build and install runwasi and dependency first. [Ref](https://github.com/containerd/runwasi#building)
```bash=
git clone https://github.com/WasmEdge/wasmedge_hyper_demo.git
cd wasmedge_hyper_demo
docker compose up client
```
Normal output
```
[+] Running 1/0
⠿ Container wasmedge_hyper_demo-client-1 Created 0.0s
Attaching to wasmedge_hyper_demo-client-1
wasmedge_hyper_demo-client-1 |
wasmedge_hyper_demo-client-1 | GET as byte stream: http://eu.httpbin.org/get?msg=Hello
wasmedge_hyper_demo-client-1 | Response: 502 Bad Gateway
wasmedge_hyper_demo-client-1 | Headers: {
wasmedge_hyper_demo-client-1 | "server": "awselb/2.0",
wasmedge_hyper_demo-client-1 | "date": "Thu, 29 Jun 2023 18:38:27 GMT",
wasmedge_hyper_demo-client-1 | "content-type": "text/html",
wasmedge_hyper_demo-client-1 | "content-length": "122",
wasmedge_hyper_demo-client-1 | "connection": "keep-alive",
wasmedge_hyper_demo-client-1 | }
wasmedge_hyper_demo-client-1 |
wasmedge_hyper_demo-client-1 | b"<html>\r\n<head><title>502 Bad Gateway</title></head>\r\n<body>\r\n<center><h1>502 Bad Gateway</h1></center>\r\n</body>\r\n</html>\r\n"
wasmedge_hyper_demo-client-1 |
wasmedge_hyper_demo-client-1 | GET and get result as string: http://eu.httpbin.org/get?msg=WasmEdge
:
(omitted)
```