# 第二题: Falco ###### tags: `真题讲解` 切换集群kubectl config use-context k8s You may use your browser to open one additional tab to access sysdig's documentation or Falco's documentation. **Task:** Use runtime detection tools to detect anomalous processes spawning and executing frequently in the single container belorging to Pod redis. Two tools are available to use: * sysdig * falco The tools are pre-installed on the cluster's worker node only; they are not available on the base system or the master node. Using the tool of your choice (including any non pre-installed tool), analyse the container's behaviour for at least 30 seconds, using filters that detect newly spawning and executing processes. Store an incident file at /opt/2/report , containing the detected incidents, one per line, in the following format: [timestamp],[uid], [processName] Keep the tool's original timestamp-format as-is. Make sure to store the incident file on the cluster's **worker** node. ## 解法 SSH到安装了Falco的Worker Node, 命令行中直接启动Falco: ``` falco Sun May 2 03:03:47 2021: Falco version 0.27.0 (driver version 5c0b863ddade7a45568c0ac97d037422c9efb750) Sun May 2 03:03:47 2021: Falco initialized with configuration file /etc/falco/falco.yaml Sun May 2 03:03:47 2021: Loading rules from file /etc/falco/falco_rules.yaml: Sun May 2 03:03:48 2021: Loading rules from file /etc/falco/falco_rules.local.yaml: Sun May 2 03:03:48 2021: Loading rules from file /etc/falco/k8s_audit_rules.yaml: Sun May 2 03:03:49 2021: Starting internal webserver, listening on port 8765 Sun May 2 03:03:49 2021: Runtime error: Could not create embedded webserver: null context when constructing CivetServer. Possible problem binding to port.. Exiting. ``` 如果遇到如上启动失败提示, 则先关闭falco服务, 再启动: ``` service falco stop falco ``` 可以看到Falco会输出较多的检测日志, 找到题目中所说的, 一个叫Redis的Pod频繁尝试 `anomalous processes spawning and executing`的日志. 确认该Pod的唯一标识, 比如Container ID. 正式考试时认真读题, 确认Falco应该捕捉的日志内容. 我模拟一个Pod叫Redis, 每5秒使用cat /etc/shadow > /dev/null触发Falco产生日志: ``` 03:11:04.575542811: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:04.575542811],[0],[cat]) 03:11:09.579324389: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:09.579324389],[0],[cat]) 03:11:14.583081954: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:14.583081954],[0],[cat]) 03:11:19.586782553: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:19.586782553],[0],[cat]) 03:11:24.590820985: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:24.590820985],[0],[cat]) 03:11:29.594303387: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:29.594303387],[0],[cat]) 03:11:34.598374821: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:34.598374821],[0],[cat]) 03:11:39.602268038: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:39.602268038],[0],[cat]) 03:11:44.606287103: Warning Sensitive file opened for reading by non-trusted program (user=root user_loginuid=-1 program=cat command=cat /etc/shadow file=/etc/shadow parent=bash gparent=containerd-shim ggparent=containerd gggparent=systemd container_name=k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 container_id=b6cecc037c8c image=rock981119/net-tools [03:11:44.606287103],[0],[cat]) ``` <br> <br> **Tips: 修改Falco规则前请确保Falco服务已关闭.** 此时你已确认日志信息, 和通过该Container ID锁定了Pod就是Redis, 接下来按照要求修改Falco输入日志的样式. [考试时可参考该文档: falco.org/Outputs Introduction](https://falco.org/docs/rules/supported-fields/) Falco进入规则默认位置, 在编辑falco_rules.yaml之前将其备份, 相同目录即可. ``` root@ubuk8s-vm02:/etc/falco# ls falco_rules.local.yaml falco_rules.yaml falco_rules.yaml.bak falco_rules.yaml.dpkg-dist falco.yaml k8s_audit_rules.yaml rules.available rules.d ``` ``` vim falco_rules.yaml // 搜索Sensitive file opened for reading by non-trusted program // 原Output格式 output: > Sensitive file opened for reading by non-trusted program (user=%user.name user_loginuid=%user.loginuid program=%proc.name command=%proc.cmdline file=%fd.name parent=%proc.pname gparent=%proc.aname[2] ggparent=%proc.aname[3] gggparent=%proc.aname[4] container_name=%container.name container_id=%container.id image=%container.image.repository [%evt.time],[%user.uid],[%proc.name]) ``` 修改成如下格式, 除了删除多余的字段以外, 留下了%container.id 目的是方便待会做日志过滤 ``` output: > Sensitive file opened for reading by non-trusted program container_id=%container.id [%evt.time],[%user.uid],[%proc.name] ``` 再次启动Falco ``` falco Sun May 2 03:28:15 2021: Falco version 0.27.0 (driver version 5c0b863ddade7a45568c0ac97d037422c9efb750) Sun May 2 03:28:15 2021: Falco initialized with configuration file /etc/falco/falco.yaml Sun May 2 03:28:15 2021: Loading rules from file /etc/falco/falco_rules.yaml: Sun May 2 03:28:15 2021: Loading rules from file /etc/falco/falco_rules.local.yaml: Sun May 2 03:28:15 2021: Loading rules from file /etc/falco/k8s_audit_rules.yaml: Sun May 2 03:28:16 2021: Starting internal webserver, listening on port 8765 03:28:16.123689000: Notice Privileged container started (user=root user_loginuid=0 command=container:b6cecc037c8c k8s_sec-ctx-demo_redis_default_d530fb6b-6fb1-45f5-be1a-eea22b9ef7e5_0 (id=b6cecc037c8c) image=rock981119/net-tools:v2) 03:28:16.149939000: Notice Privileged container started (user=root user_loginuid=0 command=container:788ffcbfd6f5 k8s_calico-node_calico-node-cxbmc_calico-system_328695e7-0098-4a3f-b776-b2ab2c43354c_10 (id=788ffcbfd6f5) image=calico/node:v3.17.1) 03:28:20.324144620: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:20.324144620],[0],[cat] 03:28:25.327681647: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:25.327681647],[0],[cat] 03:28:30.331311604: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:30.331311604],[0],[cat] 03:28:35.335309567: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:35.335309567],[0],[cat] 03:28:40.339056296: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:40.339056296],[0],[cat] 03:28:45.342569083: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:45.342569083],[0],[cat] 03:28:50.346103461: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:28:50.346103461],[0],[cat] ``` 输出的日志格式已经接近我们要的形式了, 实际考试直接让falco输出日志会有其他日志对你进行干扰, 这时使用Container ID进行输出过滤: ``` falco |grep b6cecc037c8c 03:30:50.428849365: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:30:50.428849365],[0],[cat] 03:30:55.434386125: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:30:55.434386125],[0],[cat] 03:31:00.437839569: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:00.437839569],[0],[cat] 03:31:05.441275106: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:05.441275106],[0],[cat] 03:31:10.444851223: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:10.444851223],[0],[cat] 03:31:15.448447760: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:15.448447760],[0],[cat] 03:31:20.451896325: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:20.451896325],[0],[cat] 03:31:25.455105207: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:25.455105207],[0],[cat] 03:31:30.458795790: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:30.458795790],[0],[cat] 03:31:35.463714724: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:35.463714724],[0],[cat] ``` 目测30秒的日志量, 将其粘贴至临时文件中保存. 通过Linux命令组合修改成题目所需格式保存为指定的文件名. 尝试使用cut来裁剪原日志, 发现-f为12-刚好满足题意. ``` root@ubuk8s-vm02:/etc/falco# cat temp.log 03:30:50.428849365: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:30:50.428849365],[0],[cat] 03:30:55.434386125: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:30:55.434386125],[0],[cat] 03:31:00.437839569: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:00.437839569],[0],[cat] 03:31:05.441275106: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:05.441275106],[0],[cat] 03:31:10.444851223: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:10.444851223],[0],[cat] 03:31:15.448447760: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:15.448447760],[0],[cat] 03:31:20.451896325: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:20.451896325],[0],[cat] 03:31:25.455105207: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:25.455105207],[0],[cat] 03:31:30.458795790: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:30.458795790],[0],[cat] 03:31:35.463714724: Warning Sensitive file opened for reading by non-trusted program container_id=b6cecc037c8c [03:31:35.463714724],[0],[cat] root@ubuk8s-vm02:/etc/falco# root@ubuk8s-vm02:/etc/falco# cat temp.log |cut -d ' ' -f 11- container_id=b6cecc037c8c [03:30:50.428849365],[0],[cat] container_id=b6cecc037c8c [03:30:55.434386125],[0],[cat] container_id=b6cecc037c8c [03:31:00.437839569],[0],[cat] container_id=b6cecc037c8c [03:31:05.441275106],[0],[cat] container_id=b6cecc037c8c [03:31:10.444851223],[0],[cat] container_id=b6cecc037c8c [03:31:15.448447760],[0],[cat] container_id=b6cecc037c8c [03:31:20.451896325],[0],[cat] container_id=b6cecc037c8c [03:31:25.455105207],[0],[cat] container_id=b6cecc037c8c [03:31:30.458795790],[0],[cat] container_id=b6cecc037c8c [03:31:35.463714724],[0],[cat] root@ubuk8s-vm02:/etc/falco# cat temp.log |cut -d ' ' -f 12- [03:30:50.428849365],[0],[cat] [03:30:55.434386125],[0],[cat] [03:31:00.437839569],[0],[cat] [03:31:05.441275106],[0],[cat] [03:31:10.444851223],[0],[cat] [03:31:15.448447760],[0],[cat] [03:31:20.451896325],[0],[cat] [03:31:25.455105207],[0],[cat] [03:31:30.458795790],[0],[cat] [03:31:35.463714724],[0],[cat] ``` 输出文件, 注意将文件放置在制定位置上(举例, 具体路径审题): ``` cat temp.log |cut -d ' ' -f 12- > /opt/falco.log ```