PyTorch Image and Module Loading

--- tags: David, OpenSSD, ftrace, UNet --- # PyTorch Image and Module Loading ## Environment ```shell! $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.1 LTS Release: 22.04 Codename: jammy $ uname -r 5.15.0-47-generic $ python3 -V Python 3.10.4 $ python3 -c "import torch; print(torch.__version__)" 1.12.1+cpu ``` 最新版 [PyTorch-UNet](https://github.com/milesial/Pytorch-UNet) 以及 [Pretrained Model](https://github.com/milesial/Pytorch-UNet/releases/tag/v3.0)。 ## Module ```clike! openat(AT_FDCWD, "data/unet_carvana_scale0.5_epoch2.pth", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=124239509, ...}, AT_EMPTY_PATH) = 0 ioctl(3, TCGETS, 0x7fff96473dc0) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, "PK\3\4\0\0\10\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\22\0ar"..., 4096) = 4096 lseek(3, 0, SEEK_CUR) = 4096 lseek(3, 0, SEEK_CUR) = 4096 lseek(3, 0, SEEK_END) = 124239509 lseek(3, 0, SEEK_CUR) = 124239509 lseek(3, 0, SEEK_SET) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "PK\3\4\0\0\10\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\22\0ar"..., 4096) = 4096 lseek(3, 124235413, SEEK_SET) = 124235413 read(3, "\0\0\10\0\0\17\0\0\0\0\0\0\0\0\0\0\0\0\0\220\32\225\0archive/d"..., 4096) = 4096 lseek(3, 124239487, SEEK_SET) = 124239487 read(3, "PK\5\6\0\0\0\0x\0x\0\241\34\0\0\222\241g\7\0\0", 4096) = 22 lseek(3, 124239467, SEEK_SET) = 124239467 read(3, "PK\6\7\0\0\0\0003\276g\7\0\0\0\0\1\0\0\0PK\5\6\0\0\0\0x\0x\0"..., 4096) = 42 lseek(3, 124239411, SEEK_SET) = 124239411 read(3, "PK\6\6,\0\0\0\0\0\0\0\36\3-\0\0\0\0\0\0\0\0\0x\0\0\0\0\0\0\0"..., 4096) = 98 lseek(3, 124232082, SEEK_SET) = 124232082 read(3, "PK\1\2\0\0\0\0\10\10\0\0\0\0\0\0\246\231\31-\227K\0\0\227K\0\0\20\0\0\0"..., 7329) = 7329 lseek(3, 124231952, SEEK_SET) = 124231952 read(3, "PK\3\4\0\0\10\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\17\0C\0ar"..., 4096) = 4096 lseek(3, 0, SEEK_SET) = 0 read(3, "PK\3\4\0\0\10\10\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20\0\22\0ar"..., 4096) = 4096 read(3, "\0\0M\0\1tr1\1\0\0QK\0M\0\1\205r2\1\0\0K\1\205r3\1\0\0\211"..., 15319) = 15319 ``` ## Image ![UNet](https://i.imgur.com/l11l7zi.png) ### Case 1 - Small Image ```clike! $ strace python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input1.jpg -o data/imgs/output1.jpg [...] openat(AT_FDCWD, "data/imgs/input1.jpg", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=35885, ...}, AT_EMPTY_PATH) = 0 ioctl(3, TCGETS, 0x7fff96474290) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376\0\23Created "..., 4096) = 4096 [...] read(3, "\253\271\316\317\237L\233E\t\342\2353\340;\230\266u7\207\177\233\275\316\260\3155{\27>\335\2466"..., 61440) = 31789 read(3, "", 28672) = 0 brk(0x563e7556a000) = 0x563e7556a000 brk(0x563e7560d000) = 0x563e7560d000 close(3) [...] --- $ strace python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input2.jpg -o data/imgs/output2.jpg [...] openat(AT_FDCWD, "data/imgs/input2.jpg", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=36022, ...}, AT_EMPTY_PATH) = 0 ioctl(3, TCGETS, 0x7ffdcc9ae370) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376\0\23Created "..., 4096) = 4096 [...] brk(0x560dc36b5000) = 0x560dc36b5000 read(3, "\246$.\0\2\0022\"\3\30\3024\347\230\236@r\6\250\327DcL\332T\251X\213Kfh\1"..., 61440) = 31926 read(3, "", 28672) = 0 brk(0x560dc3757000) = 0x560dc3757000 close(3) [...] ``` 第一次 read 只讀 4 KB（header），第二次 read 再讀 60 KB（data），總共是 64 KB。但第二次 read 時由於圖片大小小於 64 KB，因此實際只讀了不到 60KB。 :::danger 已經讀到 EOF 了，但還是會再 read 一次？讀的大小 28 KB？ ::: ### Case 2 - Large Image (data) ```clike! $ strace python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input3.jpg -o data/imgs/output3.jpg [...] openat(AT_FDCWD, "data/imgs/input3.jpg", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=108736, ...}, AT_EMPTY_PATH) = 0 ioctl(3, TCGETS, 0x7ffcfaac7d00) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376\0&Created "..., 4096) = 4096 [...] brk(0x5591e3e87000) = 0x5591e3e87000 read(3, "Z+LP\254\214\255\273j\264\275j\213\254\300\231\t\362\325\366\312[\310\252\311\31\255\354\364\267hU"..., 61440) = 61440 brk(0x5591e3f29000) = 0x5591e3f29000 read(3, "(\261n2\367f\36\375!\200\322\f\303s0C\253\32k/w\4\323\302]\225\241\363\32\250\304\303"..., 65536) = 43200 read(3, "", 20480) = 0 close(3) [...] ``` 這次選擇大小大於 64 KB 的圖片，第一次 read 一樣先讀 4 KB（header），接著再讀 60 KB（data），但還沒讀到檔案結尾，所以需要再讀一次，但這次直接讀 64 KB（data）。同樣在 EOF 後再讀了一次，但這次一次讀 20 KB。 ### Case 3 - Large Image (header) ```clike! $ strace python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input4.jpg -o data/imgs/output4.jpg [...] openat(AT_FDCWD, "data/imgs/input4.jpg", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0664, st_size=119798, ...}, AT_EMPTY_PATH) = 0 ioctl(3, TCGETS, 0x7ffc7d3398d0) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376+\\Created "..., 4096) = 4096 [...] read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376+\\Created "..., 65536) = 65536 read(3, "\364\20z\267Fo\364\342z\205\300 x\203\304\3htz\4\36\260 \222+\321R\245\177\370HQ"..., 65536) = 54262 read(3, "", 8192) = 0 close(3) [...] --- $ strace python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input6.jpg -o data/imgs/output6.jpg [...] openat(AT_FDCWD, "data/imgs/input6.jpg", O_RDONLY|O_CLOEXEC) = 3 lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\3760\4Created "..., 4096) = 4096 [...] read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 8192) = 8192 read(3, "AAAAAAAAAAAAAAAAAAAAAAAAAA\377\342\2\260IC"..., 4096) = 4096 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\3760\4Created "..., 65536) = 65536 read(3, "\351?|\241GB\4]\301J<\345\30'\200\23S\243E_\2264[\351\\{\313\0034A\366\10"..., 65536) = 55454 read(3, "", 8192) = 0 close(3) = 0 [...] ``` 嘗試在 JPEG 的 header 中加入大量註解，可以發現第一次讀 4 KB 後因為 header 還沒讀完，所以再讀了兩次 4 KB（==也有一次讀 8 KB 的情況？==）。讀完 header 後再**透過 `lseek` 把位置設定到檔案開頭**，然後一次讀 64 KB 直到把 EOF。而 EOF 後，同樣再呼叫了一次 read，但大小為 8 KB。 ```clike! $ strace -e trace=openat,read,lseek,close python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input7.jpg -o data/imgs/output7.jpg [...] openat(AT_FDCWD, "data/imgs/input7.jpg", O_RDONLY|O_CLOEXEC) = 3 lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376,RCreated "..., 4096) = 4096 [...] read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 read(3, "\1\372\244B\20\204!\10B\20\204!\10B\20\204!\10B\20\204!\10\10b\214\241\231\3\225\207\""..., 4096) = 4096 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376,RCreated "..., 65536) = 65536 read(3, "\243+\332tgG\3760$\202H \212\225*W\377\0\4\211\37\371\37\361\241\333\36\226;T^W"..., 65536) = 54508 read(3, "", 8192) = 0 close(3) = 0 [...] --- $ strace -e trace=openat,read,lseek,close python3 predict.py -m data/unet_carvana_scale0.5_epoch2.pth -i data/imgs/input8.jpg -o data/imgs/output8.jpg [...] openat(AT_FDCWD, "data/imgs/input8.jpg", O_RDONLY|O_CLOEXEC) = 3 lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376,QCreated "..., 4096) = 4096 [...] read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 read(3, "OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO"..., 4096) = 4096 lseek(3, 0, SEEK_SET) = 0 read(3, "\377\330\377\340\0\20JFIF\0\1\1\1\1,\1,\0\0\377\376,QCreated "..., 65536) = 65536 read(3, "+\332tgG\3760$\202H \212\225*W\377\0\4\211\37\371\37\361\241\333\36\226;T^W\374"..., 65536) = 54507 read(3, "", 8192) = 0 close(3) = 0 [...] ``` 透過調整 header 中註解的量，可以發現在 `FFDA` 的相關資料讀完後，就不繼續讀 header： ```! $ xxd input7.jpg 00002fe0: 0101 0000 0000 0000 0000 0000 0102 0304 ................ 00002ff0: 0506 07ff da00 0c03 0100 0210 0310 0000 ................ 00003000: 01fa a442 1084 2108 4210 8421 0842 1084 ...B..!.B..!.B.. 00003010: 2108 4210 8421 0808 628c a199 0395 8722 !.B..!..b......" $ xxd input8.jpg [...] 00002fe0: 0100 0000 0000 0000 0000 0001 0203 0405 ................ 00002ff0: 0607 ffda 000c 0301 0002 1003 1000 0001 ................ 00003000: faa4 4210 8421 0842 1084 2108 4210 8421 ..B..!.B..!.B..! 00003010: 0842 1084 2108 0862 8ca1 9903 9587 229a .B..!..b......". [...] ``` 而 `FFDA` 對應到的是 JPEG Header 中的 Start Of Scan 的部份： ![](https://i.imgur.com/x5VQejS.png) > - [0xFF, 0xDA, Start Of Scan](https://en.wikipedia.org/wiki/JPEG#Syntax_and_structure): > Begins a top-to-bottom scan of the image. In baseline DCT JPEG images, there is **generally a single scan**. Progressive DCT JPEG images usually contain multiple scans. This marker specifies which slice of data it will contain, and is immediately followed by entropy-coded data.