# This week in Cloud Native 009
Date: September 13th, 2021
Host: @mauilion

### COC
This is an official livestream of the CNCF, and as such is subject to the CNCF Code of Conduct. Please do not add anything to the chat or questions that would be in violation of that code of conduct; basically, please be respectful of all of your fellow participants and presenters.

### This week on cloudnative.tv!
#### [Playlists for your favorite shows.](https://www.youtube.com/c/cloudnativefdn/playlists)

#### New content every day of the week

### Kubernetes!
[good first issues for docs!](https://twitter.com/celeste_horgan/status/1435700144792645634?s=20)
[other first issues here: go.k8s.io/good-first-issue](https://go.k8s.io/good-first-issue)

Also follow [@k8sContributors](https://twitter.com/K8sContributors) on twitter!

### Kubernetes CVEs
* [The security announce group](https://groups.google.com/g/kubernetes-security-announce)
* [hackerone kubernetes list](https://hackerone.com/kubernetes/hacktivity?type=team)

### CNCF Things!
* [Kubernetes Weekly](https://www.cncf.io/kubeweekly/)
* [envoy with Scott Lowe, Kong](https://konghq.com/blog/envoy-service-mesh-configuration/?utm_source=hs_email&utm_medium=email)
* []

## Playtime!

https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/2579-psp-replacement/README.md

https://kubernetes.io/docs/concepts/security/pod-security-standards/

related: https://github.com/kubernetes/kubernetes/pull/104715

Migration cases:
https://kubernetes.io/docs/tasks/configure-pod-container/migrate-from-psp/

Lachies blog:
https://medium.com/@LachlanEvenson/hands-on-with-kubernetes-pod-security-admission-b6cac495cd11

Jims stuff!
https://twitter.com/liggitt/status/1428354415237341186

Auditing things:
https://gist.github.com/mauilion/1a4f4ab0de860b86057c971e352e8ea8

Here is the link to the spec on [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) that was shown on episode 008 (or 8 if you are go 1.17 and can't deal with leading zeros (call out to an issue mentioned in the episode))

## Questions

It'll be interesting to see applying a policy on top of existing pods and see what it does. Once an old non-compliant pod restarts, it might not come back up - thus app down (RW)

Although deprecated, Pod Security Policies are still support through to 1.25. How badly can you break a cluster with both enabled? Is it even posible to enable the feature gate if Pod Security Policies are in place on the cluster already? (RW)

Do duplicates of mode cause an error or is it a last one declared wins (or maybe first)? (RW)

I assume the Pod Security is cluster wide, so you couldn't have different rules per node? The Runtime Class exception makes me think there could be a way to make it per node as a Node's CRI could be set to support an extra RuntimeClass if I remember right... I might be wrong about that though. (RW)

If I understand it right (big if), there are only 3 built-in policies and they can't be changed: Privileged, Baseline, Restricted. Is there a way to add your own, or is it too soon and that's why there are links to other projects like OPA Gatekeeper, Kubewarden and Kyverno? (RW)

Other than getting the feature flag enabled, should this Pod Security Admission mechanism work on cloud providers like GKE, EKS, AKS once they upgrade their services to offer a Kubernetes version that is compatible? (RW)

Issue:

confusing output when setting audit to an invalid value. 

$▶ kubectl label ns nginx-test pod-security.kubernetes.io/audit=pickle --overwrite 
The namespaces "nginx-test" is invalid

### References:

https://github.com/JimBugwadia/pod-security-tests

## config files:

create adv-audit.yaml and kind.yaml in the same dir. Paths are relative so kind create cluster from the dir where these are located.

kind create cluster --config=kind.yaml --image=kindest/node:v1.22.0@sha256:b8bda84bb3a190e6e028b1760d277454a72267a5454b57db34437c34a588d047

then you can verify audit log with
docker exec -ti kind-control-plane cat /var/log/kubernetes/kube-apiserver-audit.log

adv-audit.yaml

# https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v4/audit/
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
# Do not log from kube-system accounts
  - level: None
    userGroups:
    - system:serviceaccounts:kube-system
  - level: None
    users:
    - system:apiserver
    - system:kube-scheduler
    - system:volume-scheduler
    - system:kube-controller-manager
    - system:node

# Do not log from collector
  - level: None
    users:
    - system:serviceaccount:collectorforkubernetes:collectorforkubernetes

# Don't log nodes communications
  - level: None
    userGroups:
    - system:nodes

  # Don't log these read-only URLs.
  - level: None
    nonResourceURLs:
    - /healthz*
    - /version
    - /swagger*

  # Log configmap and secret changes in all namespaces at the metadata level.
  - level: Metadata
    resources:
    - resources: ["secrets", "configmaps"]

  # We want to catch a little more then outcoldsolutions specified ;)
  - level: RequestResponse
    omitStages:
    - RequestReceived

kind.yaml

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
featureGates:
 PodSecurity: true
nodes:
- role: control-plane  
  extraMounts:
  - containerPath: /etc/kubernetes/policies/adv-audit.yaml
    hostPath: ./adv-audit.yaml
    readOnly: true
- role: worker
kubeadmConfigPatches:
- |
  apiVersion: kubeadm.k8s.io/v1beta2
  kind: ClusterConfiguration
  metadata:
    name: config
  apiServer:
    extraArgs:
      audit-policy-file: "/etc/kubernetes/policies/adv-audit.yaml"
      audit-log-path: "/var/log/kubernetes/kube-apiserver-audit.log"
      audit-log-format: "json"
    extraVolumes:
    - name: "audit-policies"
      hostPath: "/etc/kubernetes/policies"
      mountPath: "/etc/kubernetes/policies"
      readOnly: true
      pathType: DirectoryOrCreate
    - name: "audit-logs"
      hostPath: "/var/log/kubernetes"
      mountPath: "/var/log/kubernetes"
      readOnly: false
      pathType: DirectoryOrCreate