16:45 to 18:00
Attendee registration link: https://community.cncf.io/e/mryy58/
Please note that the content must be live coding/demo - no presentations with slides and no product pitches are allowed.
# Building, Analyzing, Optimizing, and Securing Containerized Apps
Built from the popular "Containers 101" Twitch stream on [https://twitch.tv/SlimDevOps](https://github.com/slimdevops/slim-containers), this workshop and tutorial leverages open source tools, free SaaS platforms, and the command line to show developers how to inspect their containers from the inside out, looking for optimizations, ways to slim and secure them, and final checks to do before sending a container to production.
* What are the most important things to consider when containerizing an app?
* What's the difference between a production-ready container and one used for development?
* What are some open-source tools I should know about when doing container development?
# Introduction
As highlighted in the recent CNCF [Software Supply Chain Best Practices](https://github.com/cncf/tag-security/blob/main/supply-chain-security/supply-chain-security-paper/sscsp.md) whitepaper tools like DockerSlim can be used to limit the number of files in a container image, thus limiting the attack surface.
While our talk today applies to any OCI-compliant container paradigm, we'll be focusing on Docker images, as they are the most familiar and most prevalent. We're going to cover:
* Dockerfile best practice
* Using `docker-slim` to analyse the container layer construction
* Security scan container images
* Generate an SBOM for container images
* Using `docker-slim` to minify a container and analyse what changed
* Exploring and diffing container images
## Dockerfile best practice
Review the Dockerfile and highlight the good practice.
```Dockerfile
#References:
# - https://dev.to/wimpress/creating-production-ready-containers-the-basics-3k6f
# - https://pythonspeed.com/articles/smaller-docker-images/
FROM python:3.10.0-slim
WORKDIR /app
COPY --chown=www-data:www-data app/app.py app/requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt && \
apt-get -y autoremove && \
apt-get -y clean && \
rm -rf /var/lib/apt/lists/*
USER www-data
EXPOSE 8008
ENTRYPOINT ["python","app.py"]
```
* 🛑 **TALKING POINT**: Not using Alpine, why is that?
### Dockerfile linter
Let's see how our `Dockerfile` checks out with `docker-slim lint`
```bash
docker-slim lint --target Dockerfile
```
Now take a look at `slim.report.json` and we see things are good shape although
we have one finding: *Missing .dockerignore*
### Build the Docker image
* Build: `docker build -t slim-demo:prod-fat .`
* See: `docker images`
* Run: `docker run -it --rm -p 8008:8008 slim-demo:prod-fat`
Add some dev tools (`git` and `nano`) to the `Dockerfile` and rebuild.
* Build: `docker build -f Dockerfile.dev -t slim-demo:dev-fat .`
* See: `docker images`
* Run: `docker run -it --rm -p 8008:8008 slim-demo:dev-fat`
## Security scanning containers
Container images should be securty scanned and analysed to generate a software bill of materials (SBOM) prior to being slimmed, and that metadata stored and published. Scanning tools effectiveness can be reduced as a result of the slimming process.
### docker scan
```bash
docker scan slim-demo:prod-fat
```
```
Package manager: deb
Project name: docker-image|wimpress/slim-demo
Docker image: wimpress/slim-demo:prod-fat
Platform: linux/amd64
Base image: python:3.10.0-slim-bullseye
Licenses: enabled
Tested 106 dependencies for known issues, found 39 issues.
According to our scan, you are currently using the most secure version of the selected base image
```
### grype
**[`gripe`](https://github.com/anchore/grype) is a vulnerability scanner for container images and filesystems**.
```bash
grype slim-demo:prod-fat
grype slim-demo:prod-dev
```
This is just the summary information
```
✔ Vulnerability DB [updated]
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [114 packages]
✔ Scanned image [67 vulnerabilities]
```
* **Go and find**: `libgnutls30 3.7.1-5 CVE-2011-3389 Medium`
What about that dev container?
```bash
grype slim-demo:dev-fat
```
```
✔ Cataloged packages [185 packages]
✔ Scanned image [120 vulnerabilities]
```
### trivy
**[`trivy`](https://github.com/aquasecurity/trivy) is a simple and comprehensive scanner for vulnerabilities in container images**.
```bash
trivy slim-demo:prod-fat
```
Just the headlines:
```
slim-demo:prod-fat (debian 11.1)
================================
Total: 65 (UNKNOWN: 0, LOW: 59, MEDIUM: 2, HIGH: 2, CRITICAL: 2)
```
And the dev container?
```bash
trivy slim-demo:dev-fat
```
```
slim-demo:dev-fat (debian 11.1)
===============================
Total: 118 (UNKNOWN: 0, LOW: 92, MEDIUM: 15, HIGH: 4, CRITICAL: 7)
```
## Software Bill of Materials
### syft
**[`syft`](https://github.com/anchore/syft) CLI tool and library for generating a Software Bill of Materials from container images and filesystems**.
```bash
syft slim-demo:prod-fat
```
An example of the output.
```
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [114 packages]
NAME VERSION TYPE
Flask 2.0.2 python
Jinja2 3.0.3 python
MarkupSafe 2.0.1 python
Werkzeug 2.0.2 python
adduser 3.118 deb
apt 2.2.4 deb
base-files 11.1+deb11u1 deb
.
.
.
zlib1g 1:1.2.11.dfsg-2 deb
```
And the dev container?
```bash
syft slim-demo:dev-fat
```
```
✔ Loaded image
✔ Parsed image
✔ Cataloged packages [185 packages]
```
The attach tooling in the container for a hijacker to use is looing much more tasty too, with the inclusion of openssh-client.
71 new packages and nearly twice as many vulnerabilies.
## Considerations when slimming
Circle back to the CNCF comments about the impact of slimming.
* As the whitepaper notes, *"tools like DockerSlim and MiniCon can be used to limit the number of files the container image that are only used by the container runtime process, thus limiting the attack surface to that process."*
* *"Container scanners often rely on standard OS files and package manifests for programming language ecosystems to determine package names and versions. If you choose to use these slimming techniques, make sure to store and publish all the metadata using the recommendations in this whitepaper. It should be noted that complications can arise from use of container slimming which impact the information necessary to produce content leveraged later in the supply chain1. For organizations considering this, be sure to weigh this impact into your overall supply chain security strategy."*
## Using docker-slim xray
`docker-slim` can create a detailed breakdown of Layers, Instructions, and Files.
While much of this information is more interesting for container authors and
maintainers, it is useful to provide insight to what happen during the layer
construction.
```bash
docker-slim xray slim-demo:prod-fat > slim-demo-prod-fat.txt
```
Useful search strings:
* `cmd=xray info=layer.start` - find the start of each layer analysis
* `cmd=xray info=change.instruction index` - find instructions used in layer construction
* `cmd=xray info=image.exposed_ports` - find the exposed ports
The output from `docker-slim xray` is verbose, but we can visualise this data
in a human-friendly way using the [Slim.AI Developer Platform](https://portal.slim.dev/).
```bash
docker-slim xray --export-all-data-artifacts . slim-demo:prod-fat
```
Upload `data-artifacts.tar` to [https://portal.slim.dev/xrayupload](https://portal.slim.dev/xrayupload)
## Slim the container
🛑 **TALKING POINTS:**
* Other techniques exist for creating "slim" containers
* Container size is not a vanity metric, but an indicator of container quality.
* Smaller container images a faster to deploy.
* Container report - dev tools left in "production" containers.
* Recently highlighted in the [ContainerDays 2021 - Hijack a Kubernetes Cluster - a Walkthrough](https://www.youtube.com/watch?v=kmtvUTYynmM), which highlights how leaving unnecessary files in a container can be used to island hop.
* [Hijack Kubernetes Cluster Hands On](https://github.com/nmeisenzahl/hijack-kubernetes/blob/main/docs/hands-on.md)
Let's slim the container
* Slim: `docker-slim build --tag slim-demo:prod-slim slim-demo:prod-fat`
* See: `docker images`
* Run: `docker run -it --rm -p 8008:8008 slim-demo:prod-slim`
### Stimulating the container
By default `docker-slim` will issue a `GET /` request with https and then https
on every exposed port and then attempt to crawl any URIs is discovers.
Your app may require more comprehensive probing to fully stimulate the app while
it is observed by `docker-slim` to ensure all runtime requirements are included
in the slim container image.
`docker-slim` has [http probe commands](https://github.com/docker-slim/docker-slim#http-probe-commands)
to help facilitate that. Here's just a couple of examples:
My app only has two endpoints `/` and `/hello`. If `/hello` relies on decencies that `/`
do not, we could end up with a slim container that is not fully functional.
#### --http-probe-cmd
The `--http-probe-cmd` option is good when you want to specify a small number of
simple commands. We can add `--http-probe-cmd /hello` to `docker-slim build` to
add the additional `GET /hello` to the probe.
```bash
docker-slim build --http-probe-cmd /hello --tag slim-demo:prod-slim slim-demo:prod-fat
```
#### --http-probe-cmd-file
The `--http-probe-cmd-file` option is good when you have a lot of commands and/or
you want to select additional http command options:
Here a `probe.json` for my simple app:
```json
{
"commands": [
{
"protocol": "http",
"method": "GET",
"resource": "/"
},
{
"protocol": "http",
"method": "GET",
"resource": "/hello"
}
]
}
```
Which would be invoked like this:
```bash
docker-slim build --http-probe-cmd-file probe.json --tag slim-demo:prod-slim slim-demo:prod-fat
```
#### --http-probe-exec
You can use the `--http-probe-exec` option to run user provided commands when
the http probes are executed. This example shows how you can run `curl` against
the temporary docker-slim created container when the http probes are executed.
```bash
docker-slim build --http-probe-exec 'curl http://localhost:8008/hello' --publish-port 8008 --tag slim-demo:prod-slim slim-demo:prod-fat
```
The `--http-probe-exec-file` option can be used to provide a file of commands to
run that stimulate the container when the http probes run.
## Slim the dev container
Now, let's also slim our dev container.
* Slim: `docker-slim build --http-probe-cmd /hello --tag slim-demo:dev-slim slim-demo:dev-fat`
* See `docker images`
Both slimmed containers are the same size.
* Run: `docker run -it --rm -p 8008:8008 slim-demo:dev-slim`
## Using Slim.AI
We can simplify the ability to explore and diff containers by pushing our
container images to Docker Hub (or other registry) and connecting them to the
[Slim Developer Platform](https://portal.slim.dev/)
I pushed the fat and slim images to DockerHub like this:
```bash
docker login -u YOUR_DOCKERHUB_NAME
docker tag slim-demo:prod-fat YOUR_DOCKERHUB_NAME/slim-demo:prod-fat
docker push YOUR_DOCKERHUB_NAME/slim-demo:prod-fat
docker tag slim-demo:prod-slim YOUR_DOCKERHUB_NAME/slim-demo:prod-slim
docker push YOUR_DOCKERHUB_NAME/slim-demo:prod-slim
```
### Connect Docker Hub
* Login to [https://portal.slim.dev/](https://portal.slim.dev/) and
* Add *slim-demo* (prod-fat and prod-slim) to Favorites
### Exploring images
* Explore prod-fat
* Overview: Ports, Shell, Special Permissions
* File Explorer: Each Layer with Instructions open
* Dockerfile: Everything
* Explore prod-slim
* Overview: No shells, only `/tmp` with special permissions
* File Explorer: There is only one layer
* **libgnutls**
### Diffing images
* Diff the fat vs. slim containers
* File system diff: Flask had some removals, in though we use it.
* Image metadata: Shells, they were removed. Attack surface reduced.
# Wrap up
* Twitch Tomorrow: twitch.tv/SlimDevOps
* 3PM ET/US
* 8PM GMT
* Twitter: `@SlimDevOps`
* Discord: Links on the [slim.ai](https://slim.ai) website:
* * <https://discord.gg/uBttmfyYNB>
Sign in with Wallet
Connect another wallet