---
# System prepended metadata

title: Yocto Project (6)：Testing and Debugging Openbmc service
tags: [Yocto Project]

---

##    Yocto Project (6)：Testing and Debugging Openbmc service 

前篇：[Yocto Project (5)：在樹梅派 4 上跑 Openbmc](https://hackmd.io/@zhonwei/BJqZEhSTbe)
目標：測試與確認 Openbmc service 是否可以正常運作

---

確認 OpenBMC 版本
```bash
$ cat /etc/os-release
```

確認基本服務是否正常運作
```bash
$ systemctl list-units --state=failed
```

出現三個 fail
```
bmcweb.service failed
obmc-phosphor-sysd.service failed
start-ipkvm.service failed
````

查看那三個服務的狀態
```bash
$ systemctl status bmcweb.service
$ systemctl status obmc-phosphor-sysd.service
$ systemctl status start-ipkvm.service
```


---

## bmcweb.service
::: info
bmcweb 是 OpenBMC 的前端網頁伺服器與 API 通道。它負責處理所有進入 BMC 的 HTTP/HTTPS 請求
:::
研讀：[[OpenBMC] 快速上手OpenBMC的Redfish](https://iris123321.blogspot.com/2022/02/openbmc-openbmcredfish.html)


查看日記，裡面會記載錯誤原因
```b
$ journalctl -u bmcweb.service -n 50
Process: 215 ExecStart=/usr/bin/bmcweb (code=exited, status=255/EXCEPTION)
```

參考 https://github.com/openbmc/bmcweb/issues/126

::: danger
錯誤的原因是缺少了 `/var/log/redfish`，因為 bmcweb 啟動時會嘗試初始化 Redfish 的事件日誌系統，它的程式碼裡有一段邏輯會去開啟或寫入 `/var/log/redfish` 這個資料夾。所以我們就手動去建立資料夾，就可以解決這個問題了。
:::

手動建立資料夾，重新啟動服務
```bash
$ mkdir -p /var/log/redfish
$ systemctl daemon-reload
$ systemctl restart bmcweb.service
$ systemctl status bmcweb.service
```

在重新查看一下 bmcweb.service 就會是 active（running）了

但之後重開機的話又會顯示 bmcweb.service failed，這是因為

```b
$ df -h /var/log
tmpfs   3.9G   8.0K   3.9G   0%   /var/volatile
```
```b
$ ls -la /var/log
/var/log -> volatile/log
```
`/var/log` 是指向 `/var/volatile/log` 的符號連結

::: danger
又發生錯誤的原因是 OpenBMC 把 /var/volatile 掛載為 tmpfs（記憶體檔案系統），/var/log 只是指向 /var/volatile/log 的符號連結。
tmpfs 的特性是：
* 資料存在記憶體裡，不寫入儲存裝置
* 每次重開機記憶體清空，所有資料消失
* 速度快，適合暫存資料

所以手動建立的 `/var/log/redfish` 實際上是建立在記憶體裡，重開機後會自然消失。
:::

解決方法是寫一個開機自動建立 /var/log/redfish 目錄的 service
```b
cat > /etc/systemd/system/create-redfish-log-dir.service << EOF
[Unit]
Description=Create /var/log/redfish directory
Before=bmcweb.service

[Service]
Type=oneshot
ExecStart=/bin/mkdir -p /var/volatile/log/redfish
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target
EOF
```

啟用服務
```
$ systemctl enable create-redfish-log-dir.service
$ systemctl start create-redfish-log-dir.service
```

重新開機確認
```
$ reboot
$ systemctl status bmcweb.service
```

---

## obmc-phosphor-sysd.service

::: info
obmc-phosphor-sysd 是 OpenBMC 架構中負責「系統狀態監控與管理」的背景服務，監控 BMC（基板管理控制器）內部的各種系統事件，並確保系統依照預定的狀態執行。
:::
> 參考：https://github.com/openbmc/phosphor-state-manager

查看 obmc-phosphor-sysd 的狀態
```b
$ systemctl status obmc-phosphor-sysd.service

Active: failed (Result: exit-code) since Thu 1970-01-01 00:00:06 UTC
Process: 328 ExecStart=/usr/bin/env obmc-phosphor-sysd (code=exited, status=1/FAILURE)
Main PID: 328 (code=exited, status=1/FAILURE)
Scheduled restart job, restart counter is at 2.
Failed to start Phosphor System Manager.
```
查看日記
```bash
$ journalctl -u obmc-phosphor-sysd.service -n 50

Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
LookupError: no codec search functions registered: can't find encoding
```

確認 Python encodings 資料夾內容
```b
$ ls /usr/lib/python3.8/encodings/
__pycache__
```
確認只有 __pycache__，沒有任何 .py 檔案。

測試 Python 是否能正常執行
```b
$ python3 -c "print('hello')"

Could not find platform independent libraries <prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python3'
  sys.path = [
    '/usr/lib/python38.zip',
    '/usr/lib/python3.8',
    '/usr/lib/python3.8/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
LookupError: no codec search functions registered: can't find encoding
```

::: danger
錯誤原因是 OpenBMC 為了縮小 image 大小，在編譯時透過 python3_%.bbappend 這個檔案，把所有 Python .py 檔案刪除，只保留 __pycache__。導致 obmc-phosphor-sysd 啟動時呼叫 Python，Python 找不到 encodings 模組，直接崩潰。
:::
> 參考：https://github.com/openbmc/openbmc/issues/3661

查看 python3_%.bbappend 內容（在 Ubuntu 虛擬機上）

```b
$ cat ~/openbmc/meta-phosphor/recipes-devtools/python/python3_%.bbappend

inherit update-alternatives

ALTERNATIVE_${PN}-core += "python"
ALTERNATIVE_LINK_NAME[python] = "${bindir}/python"
ALTERNATIVE_TARGET[python] = "${bindir}/python3"

# python3 takes up a lot of space that most embedded systems
# do not have, so remove some un-needed files from the rootfs
do_install_append_class-target() {
    # Even though python3 is built with --without-ensurepip, it still installs
    # a large, compressed version of pip.  Remove it to free up the space.
    rm -rf ${D}${libdir}/python${PYTHON_MAJMIN}/ensurepip

    # Remove all python .py files from python recipe. Only the .pyc
    # files are required.
    # The _sysconfigdata*.py files are system configuration files generated
    # during build time. It's used in the yocto packaging process so
    # it is required to remain in the image.
    find ${D}${libdir}/python${PYTHON_MAJMIN} -name \*.py ! -name _sysconfigdata*.py -exec rm {} \;
}
```

把這行整段刪除
```bash
$ find ${D}${libdir}/python${PYTHON_MAJMIN} -name \*.py ! -name _sysconfigdata*.py -exec rm {} \;
```

之後再重新編譯
```b
$ cd ~/openbmc
$ source openbmc-env
$ bitbake obmc-phosphor-image
```

編譯好之後，image 檔會在 `/home/zhonwei/openbmc/build/tmp/deploy/images/`

複製 image 檔到共享資料夾
```b
$ sudo cp /home/zhonwei/openbmc/build/tmp/deploy/images/raspberrypi4/obmc-phosphor-image-raspberrypi4-20260427072528.rootfs.rpi-sdimg /media/sf_shared/
```

重新燒錄到 SD 卡中開機，會出現前面有出現的錯誤，再按照前面的方式解決一次就可以了

確認 Python 正常
```b    
$ python3 -c "print('hello')"
hello
```

確認 obmc-phosphor-sysd 服務正不正常
```b   
$ systemctl status obmc-phosphor-sysd.
Active: failed
```

還是失敗，再次查看詳細錯誤原因
```b    
$ journalctl -u obmc-phosphor-sysd.service -n 30

SyntaxError: Missing parentheses in call to 'print'. Did you mean print("obmc-phosphor-watchdog starting...")?
File "/usr/sbin/obmc-phosphor-sysd", line 26
```
::: danger
錯誤原因：obmc-phosphor-sysd 是用 Python 2 語法寫的，但系統上的 python 已經指向 Python 3.8.2，Python 3 不支援舊的 print 語法。
:::

確認 python 版本
```b    
$ python --version
$ python3 --version
$ ls -la /usr/bin/python

Python 3.8.2
Python 3.8.2
/usr/bin/python -> /usr/bin/python3
```

修正 /usr/sbin/obmc-phosphor-sysd 裡的 print 語法
```bash
$ sed -i 's/print "obmc-phosphor-watchdogd starting..."/print("obmc-phosphor-watchdogd starting...")/' /usr/sbin/obmc-phosphor-sysd
```

重新啟動服務
```bash
$ systemctl restart obmc-phosphor-sysd.service
$ systemctl status obmc-phosphor-sysd.service
```
obmc-phosphor-sysd.service 成功啟動！

::: info
補充：可以查看 service 的 Unit file 來了解這個 service 的內容
:::
```b
$ cat /lib/systemd/system/obmc-phosphor-sysd.service

[Unit]
Description=Phosphor System Manager

[Service]
Restart=always
ExecStart=/usr/bin/env obmc-phosphor-sysd
SyslogIdentifier=obmc-phosphor-sysd

[Install]
WantedBy=multi-user.target
```


---

## start-ipkvm.service
```
$ journalctl -u start-ipkvm.service -n 50
Process: 343 ExecStart=/usr/bin/obmc-ikvm -v /dev/video0 -k /dev/hidg0 -p /dev/hidg1 (code=dumped, signal=ABRT)
```

KVM 服務直接崩潰，這個在樹莓派上是正常的，因為樹莓派沒有 KVM 硬體，可以忽略。