First Steps to Full Lifecycle Security with Open Source Tools
Introduction
-
Anaïs Urlichs
- Open Source Developer Advocate @ Aqua
- CNCF Ambassador
- YouTube AnaisUrlichs
-
Rory McCune
- Cloud Native Security Advocate @ Aqua
- Twitter/Github - @raesene
High Level Workshop Objectives
- Scanning Containers and IaC in Development
- Testing in the CI/CD pipeline
- Security in Production
Course Pre-requisites
- Ability to run a local Kubernetes cluster
- KinD
- minikube
- microk8s
- …
- Ability to download and run binaries in :-
- Linux
- MacOS
- FreeBSD?!
Course Logistics
- Ground rules
- Materials
- Slides - http://slides.pwndland.uk
- commands - http://commands.pwndland.uk
- setup notes - http://setup.pwndland.uk
- Questions? Just Ask!
Security in Development
- Vulnerability Scanning
- IaC Mis-configuration Scanning
Security Scanning Process
- Before using any third-party resources
- During Develpment
- Before Deployment
Vulnerability Scanning
- Using a container image vulnerability scanning tool is a useful way of assessing base images and also built container images.
- We'll demonstrate this with Trivy
How do Container Vulnerability Scanners work?
- Generally look at two types of information
- OS packages (e.g.debian, alpine, RHEL)
- Programming language packages (e.g. npm, rubygems)
- Assess whether there are known vulnerabilities in the installed versions
Installing Trivy
- Several options on the install page
- Homebrew for MacOS
- APT repository for Debian/Ubuntu
- YUM repo for RHEL/CentOS
- Let's install Trivy!
Using Trivy to scan Images
trivy i ubuntu:20.04
trivy i public.ecr.aws/docker/library/ubuntu:20.04
Ignore unfixed
trivy i --ignore-unfixed ubuntu:20.04
trivy i --ignore-unfixed public.ecr.aws/docker/library/ubuntu:20.04
Looking for High/Critical OS Issues
trivy image --severity HIGH,CRITICAL --vuln-type os postgres:10.6
trivy image --severity HIGH,CRITICAL --vuln-type os public.ecr.aws/docker/library/postgres:10.15
Looking for High/Critical Library Issues
trivy image --severity HIGH,CRITICAL --vuln-type library node:10.6
trivy image --severity HIGH,CRITICAL --vuln-type library public.ecr.aws/docker/library/node:10.23-slim
Scanning GitHub repositories
trivy repo --vuln-type library https://github.com/raesene/sycamore
Scanning the filesystem
- Pick a project on your machine or
git clone https://github.com/raesene/sycamore
- Scan a whole directory
trivy fs ./sycamore/
- Scan a file
trivy fs ./sycamore/yarn.lock
JSON output
trivy i --format json raesene/spring4shelldemo:latest
Using jq to find a specific issue
trivy -q i --format json raesene/spring4shelldemo:latest | jq '.Results[].Vulnerabilities[] | select(.VulnerabilityID == "CVE-2022-22965")'
Configuration Scanning
- A good practice during development or when using 3rd party projects
- Can flag up where good security practices aren't being followed
- Rulesets vary by tool, although some can be based on standards (e.g. Kubernetes PSS, CIS Benchmarks)
Configuration Scanning - Docker
git clone https://github.com/AnaisUrlichs/trivy-demo.git
cd trivy-demo
trivy config bad_iac/docker/
Fixing a Docker issue
- Uncomment the USER line in the Dockerfile
trivy config bad_iac/docker/
Configuration Scanning - Kubernetes
trivy config bad_iac/kubernetes/
Fixing a Kubernetes Issue
- Pick an issue from the ones flagged up and see if you can fix it, then re-scan
trivy config bad_iac/kubernetes/
Configuration Scanning - Terraform
trivy config bad_iac/terraform/
Trivy SBOM
trivy sbom ubuntu:20.04
*also available as Docker Desktop Extension
Security Scanning in CI/CD
- Applying the same checks as are available in development, in CI/CD pipelines provides an additional layer of security.
- Using GitHub Actions and SARIF, we can automate security checks either periodically or as code is checked in
Using a Github Action to build and scan a Docker image
- Here's an example https://github.com/raesene/sycamore/blob/main/.github/workflows/docker-publish.yml
- It's a modified version of GitHub's basic Docker+cosign action
Permissions to build+scan an image
permissions:
contents: read
packages: write
# This is used to complete the identity challenge
# with sigstore/fulcio when running outside of PRs.
id-token: write
security-events: write # To upload sarif files
Running Trivy
- name: Run trivy
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}
format: sarif
output: 'trivy-results.sarif'
Uploading results to GitHub Security
- name: Upload Trivy scan results to GitHub Security tab
uses: github/codeql-action/upload-sarif@v1
with:
sarif_file: 'trivy-results.sarif'
Config scanning in CI/CD with Trivy
- name: Run Trivy in Config mode to generate SARIF
uses: aquasecurity/trivy-action@master
with:
scan-type: 'config'
hide-progress: false
format: 'sarif'
output: 'trivy-results.sarif'
Open Source Security in Production
- Once our workloads are deployed to clusters we need on-going security
- Regular scans for compliance and assurance
- Runtime security to detect attacks
Kubernetes Operator
Automating human behaviour through controllers
https://www.cncf.io/wp-content/uploads/2021/07/CNCF_Operator_WhitePaper.pdf
Installing Starboard Operator
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/starboard/v0.15.4/deploy/static/starboard.yaml
CRDs
- Starboard creates a number of CRDs to hold scan data.
ciskubebenchreports.aquasecurity.github.io
clustercompliancedetailreports.aquasecurity.github.io
clustercompliancereports.aquasecurity.github.io
clusterconfigauditreports.aquasecurity.github.io
configauditreports.aquasecurity.github.io
vulnerabilityreports.aquasecurity.github.io
Confirming all is well
- Checking the deployment of the operator tells us if the install was successful
kubectl get deployment -n starboard-system
Creating a Deployment
kubectl create ns app
kubectl apply -f https://raw.githubusercontent.com/AnaisUrlichs/trivy-demo/main/manifests/kubernetes.yaml -n app
VulnerabilityReport
Automatically scans the containers that are used inside of your cluster.
Deployment-scoped
kubectl get vulnerabilityreports -o wide -n app
Cluster-scoped
kubectl get clustervulnerabilityreports -o wide -n app
Configuration Auditing
Kubernetes configurations are checked against built-in policies
Deployment-scoped
kubectl get configauditreports -o wide -n app
Cluster-scoped
kubectl get clusterconfigauditreports -o wide -n app
Infrastructure Scanning
- CIS benchmark for Kubernetes nodes provided by kube-bench.
- Penetration test results for a Kubernetes cluster provided by kube-hunter.
CISKubeBenchReport
Maps CIS Benchmarks against Kubernetes version
kubectl get nodes
kubectl get ciskubebenchreports -o wide
kubectl describe ciskubebenchreports/<insert report name>
Other tool to produce CIS benchmarks
Cluster Compliance Report
NSA report
ClusterCompliance and ClusterComplianceDetail Report
kubectl apply -f https://raw.githubusercontent.com/aquasecurity/starboard/main/deploy/specs/nsa-1.0.yaml
kubectl get clustercompliancereport -o wide
kubectl describe clustercompliancereport nsa
kubectl get clustercompliancedetailreport -o wide
Custom Policies
- Trivy/Starboard: Write custom policies using Rego
- tfsec: Custom policies in JSON/YAML
trivy config --policy ./custom-policies --namespaces user ./manifests
What is next?
- Integrate Starboard metrics into your observability stack
- Try out Trivy's functionality in your own projects
- Let us know what use cases you would like to see
Links
Repository https://github.com/AnaisUrlichs/trivy-demo
Aqua GitHub https://github.com/aquasecurity
Rory's Twitter https://twitter.com/raesene
Anais' Twitter https://twitter.com/urlichsanais
Thank you!
- Rory's Twitter/GitHub: @raesene
- Anais' Twitter: @urlichsanais
First Steps to Full Lifecycle Security with Open Source Tools
{"metaMigratedAt":"2023-06-16T22:41:05.615Z","metaMigratedFrom":"YAML","title":"First Steps to Full Lifecycle Security with Open Source Tools","breaks":true,"description":"A key element of successfully integrating security into the DevOps lifecycle is embedding it right from the start. Helping developers and operators build security controls in from day-one with easy to use open source tooling can make that a reality. This workshop will take a hands-on approach to demonstrate how to install, configure and customize open source security tools to be used throughout the DevOps process. The workshop will focus on a couple of core tools. Firstly understanding how Trivy can be used to help secure container images, Dockerfiles, Kubernetes manifests and IaC code such as Terraform. Then the workshop will move on to operationalizing security controls using Starboard to automate the operation of Trivy and other security tools, providing continuous security assurance of workloads and Kubernetes clusters.","slideOptions":"{\"theme\":\"solarized\",\"allottedMinutes\":90}","contributors":"[{\"id\":\"6119c9a4-8578-427f-9c70-02b185f05c99\",\"add\":3740,\"del\":500},{\"id\":\"d371f3af-4727-4a8c-863f-ebcf30897cef\",\"add\":13800,\"del\":2290}]"}