# Cilium Notes

## 目的

討論 Cilium eBPF 共享同步知識

## 議題

1. eBPF
2. Cilium
3. Service Mesh
4. eBPF 有人質疑圖靈不完備

## 入門相關資料

- [XDP ACM Paper 中譯](https://arthurchiao.art/blog/xdp-paper-acm-2018-zh/)
- XDP技術分析 bottom up
- [eBPF + Kubernetes](http://arthurchiao.art/blog/ebpf-and-k8s-zh/)
- Cilium eBPF 特性及概覽
- [Cilium Datapath 封包旅程](https://arthurchiao.art/blog/understanding-ebpf-datapath-in-cilium-zh/)

## eBPF
### 討論重點
- Cilium eBPF 實作細節
- cgroup eBPF
- tc eBPF
- sock_ops eBPF
- gXDP eBPF

### cgroup eBPF
- [cgroup 實作(2016, 非常早期)](https://github.com/torvalds/linux/commit/ca89fa77b4488ecf2e3f72096386e8f3a58fe2fc#)
- [用 cgroup ebpf 來實作防火牆](https://nfil.dev/coding/security/ebpf-firewall-with-cgroups/)
- 本篇作者想要用ebpf實現防火牆, 內文開頭有提到做這件 packets dropping 有2個hook points 可以做包含
- XDP(但目前僅能支援 ingress traffic)
- 透過Linux traffic control subsystem's hook points(但不能支援作者的CentOS環境)
- cgroup hooks
- 分別寫下 ingress/egress eBPF 程式透過LLVM編譯成.o檔案。透過 `__section` 可以看到這兩段控制 ingress/egress 的code被注入到 cgroup skb

    #include <stdbool.h>
    #include <linux/bpf.h>
    #include <netinet/ip.h>
    #include "bpf_helpers.h"

    #define __section(NAME)                  \
        __attribute__((section(NAME), used))

    /* Ingress hook - handle incoming packets */
    __section("cgroup_skb/ingress")
    int ingress(struct __sk_buff *skb) {
        return false;
    }

    /* Egress hook - handle outgoing packets */
    __section("cgroup_skb/egress")
    int egress(struct __sk_buff *skb) {
        return false;
    }

    char __license[] __section("license") = "GPL";

- 最後必須要透過cilium的函式庫 將這兩段注入至 kernel 流程大概是
- 到ebpf map 註冊(pin), 因為go程序離開之後必須還要能操作這2段ebpf程式
- 獲得root cgroup 的file handle
- 將兩段ebpf code attach 到root group

### tc eBPF
- Linux Kernel 4.1開始支援 eBPF 程式在 tc 的 classifier 及 action 附著(attach)這兩個 hooks
> Extends the "classic" BPF programmable tc classifier by extending its scope also to native eBPF code, thus allowing userspace to implement own custom, 'safe' C like classifiers that can then be compiled with the LLVM eBPF backend to an eBPF elf file and loaded into the kernel via iproute2's tc, and be JITed in the kernel [來源](https://kernelnewbies.org/Linux_4.1)
- tc ebpf direct-action
- tc ebpf classifier with direct-action
- 本來行為是返回0/1/classID 因為透過ebpf改成返回action
- classID 可以從其他地方看到 `tc_classid`
- tc ebpf action direct-classifier? 不行
- actions 必須接在classifer之後
- 但可以做邏輯處理透過hook點滿足條件drop

### 觀點或討論內容

### 相關參考資料
- https://blog.yadutaf.fr/2021/08/22/ebpf-cgroup-per-cpu-storage-low-overhead-accounting/
- [理解tc ebpf的 direct-action](https://arthurchiao.art/blog/understanding-tc-da-mode-zh/)

## Cilium
### 討論重點
- 透過 eBPF 達到 Linux networking stack shortcut
### 觀點或討論內容
xxx

## Service Mesh
### 討論重點
- Sidecar based service mesh
- 因為 sidecar based 將架構及路由變得更複雜，在這個框架之下是否有價值
- eBPF based service mesh

### 觀點或討論內容
xxx

### 相關參考資料

- [社群討論](https://twitter.com/louiscryan/status/1522661442138238976?s=21&t=SAPgpU07Uo2TiZzv76Ok9A)

- isovalent 提出 [eBPF based service mesh] (https://isovalent.com/blog/post/2021-12-08-ebpf-servicemesh) proxyless
- 反對 eBPF based service mesh 並支持 [sidecar based service mesh](https://www.infoq.com/news/2022/01/ebpf-wasm-service-mesh/)