ubuntu 22.04 & containerd 踩坑測試

緣由：

主機安裝好 k8s 叢集 (使用 rke2) 後，部署 .net 6 web api 服務時發生以下問題

Unhandled exception. System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.
   at System.IO.FileSystemWatcher.StartRaisingEvents()
   at System.IO.FileSystemWatcher.StartRaisingEventsIfNotDisposed()
   at System.IO.FileSystemWatcher.set_EnableRaisingEvents(Boolean value)
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
   at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
   at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider.<.ctor>b__1_0()
   at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
   at Microsoft.Extensions.Configuration.FileConfigurationProvider..ctor(FileConfigurationSource source)
   at Microsoft.Extensions.Configuration.Json.JsonConfigurationSource.Build(IConfigurationBuilder builder)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
   at Microsoft.Extensions.Hosting.HostBuilder.Build()
   at WebService.Program.Main(String[] args) in /src/WebService/Program.cs:line 00

測試方式

# 確認主機上有沒有裝 lsof 這個套件
sudo dpkg -l | grep lsof

# 如果沒有裝的話需要安裝一下
# sudo apt-get install lsof

# 查詢 inotify + container
lsof -l | grep inotify | grep container

# 可以用這個命令對照 PID
ps aux

查詢後，在 ubuntu 22.04 上可以發現 inotify + container 的執行緒資料特別多，但是回頭使用 20.04 則沒有這個問題。

因此，如果是使用 .net 建立服務的話，建議暫時不使用 ubuntu 22.04 來建立 k8s 叢集，或是需要針對 appsettings.json 的 hot reload 行為進行調整。

(因為 .net 的 appsettings.json 的 hot reload 行為需要使用 inotify 資源)

各項參考資料

1. http://blog.travisgosselin.com/configured-user-limit-inotify-instances/
2. https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?source=recommendations&view=aspnetcore-7.0
3. https://stackoverflow.com/a/67111195
4. https://github.com/dotnet/aspnetcore/issues/3475#issuecomment-491702858
5. https://blackie1019.gitlab.io/2022/02/22/qemu-exception-throwing-on-x64-emulator-for-docker-with-NET6-NET-Core-on-the-Apple-M1-chip/
6. https://github.com/dotnet/AspNetCore.Docs/issues/19814#issue-697147144
7. https://blog.gtwang.org/linux/linux-lsof-command-list-open-files-tutorial-examples/