What can subPath do for you?

I've always been confused as to what purpose subPath serves. I've come across subPath whenever a configmap was mounted: it looked like it was used as a way to control which keys in a configmap or secret are projected… but I was never sure.

In this page, I'll try to uncover what exactly is happening when using subPath and finally get some firm understanding.

We will be using Kind to test things out. Let's create a kind cluster:

kind create cluster

We will need the tree utility too:

docker exec kind-control-plane sh -c "apt update && apt install tree -y"

Let's consider the following configmap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: conf
data:
  conf.yaml: "contents"
  other.yaml: "other"

And the following pod manifest; pay close attention to volumes and volumeMounts:

apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
    - name: example
      image: busybox:1.28
      command: ['sh', '-c', 'echo "Running!" && tail -f /dev/null']
      volumeMounts:
        - name: conf
          mountPath: /etc/conf.yaml
          subPath: conf.yaml
  volumes:
    - name: conf
      configMap:
        name: conf

Now, I'll need to inspect the rootfs of the example container above. I'll use the command podbash defined as follows:

#!/bin/bash

#shellcheck disable=SC2093
exec docker exec -it kind-control-plane bash -c "$(grep -v 'exec docker' "$0")" podbash "$@"

CONTAINER_ID=$(crictl ps --name example -o json | jq -r ".containers[].id")
PID="$(ctr -n k8s.io t ls | grep "$CONTAINER_ID" | awk '{print $2}')"
cd "/proc/$PID" || exit 1
exec bash "$@"
EOF

With podbash, I can inspect the container's rootfs as well as look at the mountinfo. For example:

# To inspect the container's filesystem:
podbash -c 'ls root/etc'
# To inspect the mount points:
podbash -c 'cat mountinfo' | grep configmap

It's interesting to look at the mount information to understand the role of subPath. The output of mountinfo looks like this:

4031 4028 259:5 /lib/kubelet/pods/84f85/volumes/kubernetes.io~configmap/conf/..2024_07_01_14_15_02.3624755718/conf.yaml /etc/conf.yaml ro,relatime - ext4 /dev/nvme0n1p3 rw,errors=remount-ro

To make this a little more readable, I've turned this into a table:

Host volume	Pod mount point
/lib/kubelet/pods/bc2bf/volumes/kubernetes.io~configmap/conf/..2024_07_01_14_15_02.3624755718/conf.yaml	/etc/conf.yaml

Now, let's make the distinction between mountPath and subPath:

• mountPath is about the pod mount point.
• subPath is about the host volume path.

Without subPath, the outcome would have been very different. Let's remove the subPath from the manifest:

apiVersion: v1
kind: Pod
metadata:
  name: example
spec:
  containers:
    - name: example
      image: busybox:1.28
      command: ['sh', '-c', 'echo "Running!" && tail -f /dev/null']
      volumeMounts:
        - name: conf
          mountPath: /etc/conf.yaml
  volumes:
    - name: conf
      configMap:
        name: conf

The host "volume" (a directory, really) looks like this:

# From the host:
$ sudo tree /lib/kubelet/pods/X/volumes/kubernetes.io~configmap/conf
├── conf.yaml -> ..data/conf.yaml
└── other.yaml -> ..data/other.yaml

The pod mount point looks off:

$ podbash -c 'tree root/etc/conf.yaml'
root/etc/conf.yaml
├── conf.yaml -> ..data/conf.yaml
└── other.yaml -> ..data/other.yaml

The problem is that /etc/conf.yaml is now a directory rather than a file…

That's where subPath is useful: when you need to mount a single file rather than an entire folder.

More specifically, subPath allows you to select which path under the host volume path /lib/kubelet/pods/bc2bf/volumes/kubernetes.io~configmap/conf.yaml/ needs to be mounted.

Important learnings:

• If the path given in subPath isn't found, what's mounted at mountPath is a host-mounted volume that is made of an empty directory. The subPath name doesn't matter in this case.
• If the path given in subPath corresponds to a file in the host volume (e.g., it matches a configmap key), that file alone is mounted to mountPath.
• If the path given in subPath corresponds to a directory in the host-mounted volume (only happens when mounting a PV volume), then mountPath is a directory that corresponds to the sub- directory subPath in the host-mounted volume.

If your concern is that you want to mount multiple configmap keys to a directory with existing files (such as /etc), the following will cause /etc to be replaced by the host-mounted volume:

volumeMounts:
  - name: conf
    mountPath: /etc

The subPath is exactly what you want in this case. You will have to have one volume mount per file, though:

volumeMounts:
  - name: conf
    mountPath: /etc/conf.yaml
    subPath: conf.yaml
  - name: conf
    mountPath: /etc/other.yaml
    subPath: other.yaml

What about mistakenly using a subPath that doesn't exist in the host's volume?

On the host, since baz doesn't exist in the configmap myconf, the host volume is an empty directory:

$ sudo ls -al /lib/kubelet/pods/bc2bf/volumes/kubernetes.io~configmap/myconf/foo
drwxrwxrwx 2 root root 4096 Jul  1 15:56 .

On the host, when foo exists in the configmap myconf, it appears as a file in the host volume:

$ sudo ls -al /lib/kubelet/pods/bc2bf/volumes/kubernetes.io~configmap/myconf/..2024_07_01_14_15_02.3624755718/foo
-rw-r--r-- 1 root root 8612 Jul  1 16:15 /lib/kubelet/pods/bc2bf/volumes/kubernetes.io~configmap/myconf/..2024_07_01_14_15_02.3624755718/foo