---
# System prepended metadata

title: Explore cluster API for docker infrastructure
tags: [cluster-api]

---

---
tags: cluster-api
---
# Explore cluster API for docker infrastructure
## Official page
https://cluster-api.sigs.k8s.io/user/quick-start.html

## Steps
### Clone clusterAPI github repo
https://github.com/kubernetes-sigs/cluster-api

### Install kubectl
https://kubernetes.io/docs/tasks/tools/install-kubectl/

### Install kind
https://github.com/kubernetes-sigs/kind

### Install clusterctl
https://cluster-api.sigs.k8s.io/user/quick-start.html#install-clusterctl

### Generate the infrastructure components yaml
```
kustomize build ~/go/src/github.com/cluster-apitest/infrastructure/docker/config > ~/tech-explore/capd/infrastructure-docker/infrastructore-components.yaml
```

### Create images and manifests in order to use the docker provider
https://cluster-api.sigs.k8s.io/clusterctl/developers.html#additional-steps-in-order-to-use-the-docker-provider
```
cd ~/go/src/github.com/cluster-api
```
```
make -C test/infrastructure/docker docker-build REGISTRY=danielguo/k8s-
staging-capi-docker
```
```
make -C test/infrastructure/docker generate-manifests REGISTRY=danielguo/k8s-staging-capi-docker
```

### Create kind cluster
```
cat > kind-cluster-with-extramounts.yaml <<EOF
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
  - role: control-plane
    extraMounts:
      - hostPath: /var/run/docker.sock
        containerPath: /var/run/docker.sock
EOF

kind create cluster --config ./kind-cluster-with-extramounts.yaml
kind load docker-image danielguo/k8s-staging-capi-docker/capd-manager-amd64:dev

```

### Additional steps in order to use the docker provider
```
cd ~/go/src/github.com/cluster-api
```
```
# https://cluster-api.sigs.k8s.io/clusterctl/developers.html#create-a-clusterctl-settingsjson-file
cat > cluster-settings.json << EOF
{
  "providers": ["cluster-api","bootstrap-kubeadm","control-plane-kubeadm", "infrastructure-docker"],
  "provider_repos": ["./test/infrastructure/docker"]
}
EOF
```
```
#https://cluster-api.sigs.k8s.io/clusterctl/developers.html#available-providers
cat ./test/infrastructure/docker/cluster-settings.json << EOF
{
  "name": "infrastructure-docker",
  "config": {
    "componentsFile": "infrastructure-components.yaml",
    "nextVersion": "v0.3.0"
  }
}
EOF
```
```
#https://cluster-api.sigs.k8s.io/clusterctl/developers.html#run-the-local-overrides-hack
cmd/clusterctl/hack/local-overrides.py

Output:
--> clusterctl local overrides generated from local repositories for the cluster-api, bootstrap-kubeadm, control-plane-kubeadm, infrastructure-docker providers.
in order to use them, please run:
--> clusterctl init --core cluster-api:v0.3.0 --bootstrap kubeadm:v0.3.0 --control-plane kubeadm:v0.3.0 --infrastructure docker:v0.3.0
--> please check the documentation for additional steps required for using the docker provider

```

### Initialize the management cluster
```
clusterctl init -i docker -v10
Fetching providers
Installing cert-manager
Waiting for cert-manager to be available...
Installing Provider="cluster-api" Version="v0.3.0" TargetNamespace="capi-system"
Installing Provider="bootstrap-kubeadm" Version="v0.3.0" TargetNamespace="capi-kubeadm-bootstrap-system"
Installing Provider="control-plane-kubeadm" Version="v0.3.0" TargetNamespace="capi-kubeadm-control-plane-system"
Installing Provider="infrastructure-docker" Version="v0.3.0" TargetNamespace="capd-system"

Your management cluster has been initialized successfully!

You can now create your first workload cluster by running the following:

  clusterctl config cluster [name] --kubernetes-version [version] | kubectl apply -f -
```
Now you should have all the cluster API components up and running
```
kubectl get pods -A
NAMESPACE                           NAME                                                             READY   STATUS    RESTARTS   AGE
capd-system                         capd-controller-manager-b765fdf4b-qbmpp                          2/2     Running   0          42s
capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager-7d5bf99fcd-9fvv7       2/2     Running   0          44s
capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager-5b78c89b66-t72bq   2/2     Running   0          43s
capi-system                         capi-controller-manager-548c58995f-gjx85                         2/2     Running   0          45s
capi-webhook-system                 capi-controller-manager-84f48bf9bd-xtrqh                         2/2     Running   0          45s
capi-webhook-system                 capi-kubeadm-bootstrap-controller-manager-545bbb995c-8nvq4       2/2     Running   0          44s
capi-webhook-system                 capi-kubeadm-control-plane-controller-manager-777fc96489-hldct   2/2     Running   0          44s
cert-manager                        cert-manager-69b4f77ffc-6ns6g                                    1/1     Running   0          70s
cert-manager                        cert-manager-cainjector-576978ffc8-klzdk                         1/1     Running   0          70s
cert-manager                        cert-manager-webhook-c67fbc858-th5zw                             1/1     Running   0          70s
kube-system                         coredns-6955765f44-bgskj                                         1/1     Running   0          22h
kube-system                         coredns-6955765f44-q9cg9                                         1/1     Running   0          22h
kube-system                         etcd-kind-control-plane                                          1/1     Running   0          22h
kube-system                         kindnet-wtbvn                                                    1/1     Running   0          22h
kube-system                         kube-apiserver-kind-control-plane                                1/1     Running   0          22h
kube-system                         kube-controller-manager-kind-control-plane                       1/1     Running   0          22h
kube-system                         kube-proxy-fnld8                                                 1/1     Running   0          22h
kube-system                         kube-scheduler-kind-control-plane                                1/1     Running   0          22h
local-path-storage                  local-path-provisioner-7745554f7f-lbzwz                          1/1     Running   0          22h
```
And your single controle plane management cluster is running
```
kubectl get nodes
NAME                 STATUS   ROLES    AGE   VERSION
kind-control-plane   Ready    master   22h   v1.17.0
```

### Create a workload cluster
https://cluster-api.sigs.k8s.io/user/quick-start.html#create-your-first-workload-cluster
> The clusterctl config cluster command by default uses cluster templates which are provided by the infrastructure providers.

However, I could not find any `cluster-template.yaml` available under `.cluster-api/overrides/infrastructure-docker/v0.3.0/cluster-template.yaml`. The `cluterctl config cluster <cluster_name>` command will fail with error complaining about the missing `cluster-template.yaml` file

For now, I just copied the sample from https://github.com/vmware-tanzu/tgik/blob/master/episodes/110/capd-v3/infrastructure-docker/v0.3.2/cluster-template.yaml and put it under `.cluster-api/overrides/infrastructure-docker/v0.3.0/cluster-template.yaml`

Then run:
```
clusterctl config cluster test > capi-test.yaml
# Modify the capi-test.yaml if necessary, e.g. the machineDeployment replica. It is 0 by default

kubectl apply -f capi-test.yaml
```

After apply the yaml, you should be able to see the following output
```
~/tech-explore/capd  kubectl get cluster --all-namespaces

NAMESPACE   NAME   PHASE
default     test   Provisioned
 ~/tech-explore/capd  kubectl get kubeadmcontrolplane --all-namespaces

NAMESPACE   NAME   READY   INITIALIZED   REPLICAS   READY REPLICAS   UPDATED REPLICAS   UNAVAILABLE REPLICAS
default     test           true          1                           1                  1
 ~/tech-explore/capd  kubectl get machine -A
NAMESPACE   NAME         PROVIDERID                   PHASE
default     test-jgrph   docker:////test-test-jgrph   Running
 ~/tech-explore/capd  docker ps
CONTAINER ID        IMAGE                          COMMAND                  CREATED              STATUS              PORTS                                  NAMES
94d14d4b0cc9        kindest/node:v1.17.0           "/usr/local/bin/entr…"   About a minute ago   Up 58 seconds       36323/tcp, 127.0.0.1:36323->6443/tcp   test-test-jgrph
9dbd182555fb        kindest/haproxy:2.1.1-alpine   "/docker-entrypoint.…"   About a minute ago   Up About a minute   38337/tcp, 0.0.0.0:38337->6443/tcp     test-lb
bd6eaef3f13c        kindest/node:v1.17.0           "/usr/local/bin/entr…"   10 minutes ago       Up 10 minutes       127.0.0.1:32771->6443/tcp              kind-control-plane
```

(Optional) Just in case you want to scale your workload cluster machine deployment, you could do the following:
```
~/tech-explore/capd  kubectl get machinedeployments -A
NAMESPACE   NAME        PHASE     REPLICAS   AVAILABLE   READY
default     test-md-0   Running
 ~/tech-explore/capd  kubectl scale machinedeployments test-md-0 --replicas=3
machinedeployment.cluster.x-k8s.io/test-md-0 scaled
 ~/tech-explore/capd  kubectl get machine -A
NAMESPACE   NAME                        PROVIDERID                                  PHASE
default     test-jgrph                  docker:////test-test-jgrph                  Running
default     test-md-0-db7cb7668-8mdbz   docker:////test-test-md-0-db7cb7668-8mdbz   Running
default     test-md-0-db7cb7668-dlrvg   docker:////test-test-md-0-db7cb7668-dlrvg   Running
default     test-md-0-db7cb7668-nlt4f   docker:////test-test-md-0-db7cb7668-nlt4f   Running
```
You might have notices that the kubeadmcontrolplane is not ready. That is because you have to deploy CNI. https://cluster-api.sigs.k8s.io/user/quick-start.html#deploy-a-cni-solution

### Get access to a workload cluster
https://cluster-api.sigs.k8s.io/clusterctl/developers.html#connecting-to-a-workload-cluster-on-docker
```
kubectl --namespace=default get secret/test-kubeconfig -o jsonpath={.data.value} \
  | base64 --decode \
  > ./test.kubeconfig
```
(Optional) If you are using Mac, the following steps are required.
```
# Point the kubeconfig to the exposed port of the load balancer, rather than the inaccessible container IP.
sed -i -e "s/server:.*/server: https:\/\/$(docker port test-lb 6443/tcp | sed "s/0.0.0.0/127.0.0.1/")/g" ./test.kubeconfig

# Ignore the CA, because it is not signed for 127.0.0.1
sed -i -e "s/certificate-authority-data:.*/insecure-skip-tls-verify: true/g" ./test.kubeconfig

```

### Deploy CNI
https://cluster-api.sigs.k8s.io/clusterctl/developers.html#connecting-to-a-workload-cluster-on-docker

### You are all set !!!
```
 ~/tech-explore/capd  kubectl --kubeconfig=./test.kubeconfig get pods -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-77c4b7448-dfpv2   1/1     Running   0          7m26s
kube-system   calico-node-2sljv                         1/1     Running   0          7m27s
kube-system   calico-node-cqrvs                         1/1     Running   0          7m27s
kube-system   calico-node-kbh9r                         1/1     Running   0          7m27s
kube-system   calico-node-nb26v                         1/1     Running   0          7m27s
kube-system   coredns-6955765f44-5p47h                  1/1     Running   0          51m
kube-system   coredns-6955765f44-dr4dk                  1/1     Running   0          51m
kube-system   etcd-test-test-jgrph                      1/1     Running   0          51m
kube-system   kube-apiserver-test-test-jgrph            1/1     Running   0          51m
kube-system   kube-controller-manager-test-test-jgrph   1/1     Running   0          51m
kube-system   kube-proxy-2wfxr                          1/1     Running   0          46m
kube-system   kube-proxy-7dpq5                          1/1     Running   0          51m
kube-system   kube-proxy-x5p6h                          1/1     Running   0          46m
kube-system   kube-proxy-z8nhd                          1/1     Running   0          46m
kube-system   kube-scheduler-test-test-jgrph            1/1     Running   0          51m
```