# Flux Bootstrap Hands-on Tasks - Prepare the dev environment - Install the tools and create a Kubernetes cluster - Configure the GitLab PAT - Bootstrap operations - Setup cluster sync over SSH - Migrate to HTTP/S and PAT - Cluster inspection and identification - Upgrades, maintenance and fine-tuning - Commit status updates - Create a notification provider for GitLab - Add an alert for the GitLab API - Disaster recovery - Run various failure scenarios - Debugging and troubleshooting - Restoration techniques ## Prerequisites ### Tools - [kind](https://kind.sigs.k8s.io/docs/user/quick-start/) - [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) - [flux](https://fluxcd.io/docs/installation/) - [helm](https://helm.sh/docs/intro/install/) - [kustomize](https://kubectl.docs.kubernetes.io/installation/kustomize/) - [yq](https://mikefarah.gitbook.io/yq/) ### Cluster prerequisites Check that your cluster is compatible with Flux: ```shell flux check --pre ``` ## Bootstrap the cluster ### GitLab PAT For accessing the GitLab API, the boostrap command requires a GitLab personal access token (PAT) with complete read/write access to the GitLab API. Export the username and token as: ```shell export GITLAB_TOKEN=<gl-token> export GITLAB_USER=<gl-username> ``` ### Flux Bootstrap Bootstrap the cluster with the following command: ```shell flux bootstrap gitlab \ --owner=$GITLAB_USER \ --repository=flux-workshop \ --branch=main \ --path=clusters/dev-cluster \ --personal ``` ### Clone the repository Clone the repository locally: ```shell git clone https://gitlab.com/$GITLAB_USER/flux-workshop.git cd flux-workshop tree . ``` ## Flux operations ### Cluster inspection Verify the cluster state: ```shell flux check ``` Inspect the Git pull secret: ```shell kubectl -n flux-system get secret flux-system -o yaml | yq '.data | map_values(. | @base64d)' ``` ### Migrate from SSH to HTTPS and PAT Delete the auth secret: ```shell kubectl -n flux-system delete secret flux-system ``` Create a new auth secret: ```shell flux create secret git flux-system \ --url=https://gitlab.com/$GITLAB_USER/flux-workshop \ --username=$GITLAB_USER \ --password=$GITLAB_TOKEN ``` Check Flux status (the Git source should error out): ```shell flux get sources git ``` Rerun the bootstrap command with the `--token-auth` flag: ```shell flux bootstrap gitlab \ --owner=$GITLAB_USER \ --repository=flux-workshop \ --branch=main \ --path=clusters/dev-cluster \ --personal \ --token-auth ``` Inspect the GitRepositoy definition: ```shell flux export source git flux-system ``` Check Flux status (the Git source should resume syncing): ```shell flux get sources git ``` Check the events: ```shell flux events --for GitRepository/flux-system ``` ### Cluster info Create a `cluster-info` ConfigMap with the cluster details: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: cluster-info namespace: flux-system data: CLUSTER_NAME: "dev-stefan" CLUSTER_ENV: "dev" CLUSTER_REGION: "ro-bucharest-1" ``` Save the file as `cluster-info.yaml` in the `clusters/dev-cluster` directory, commit and push the changes to the repository: ```shell git add clusters/dev-cluster/cluster-info.yaml git commit -m "Add cluster info" git push origin main ``` Wait for Flux to pull the latest commit and create the ConfigMap: ```shell kubectl -n flux-system get cm -w ``` Check the `cluster-info` ConfigMap: ```shell kubectl -n flux-system get cm cluster-info -o yaml ``` Delete the `cluster-info` ConfigMap: ```shell kubectl -n flux-system delete cm cluster-info ``` Tell Flux to restore the cluster state (or wait for the next reconciliation): ```shell flux reconcile ks flux-system --with-source ``` Verify the `cluster-info` ConfigMap is restored: ```shell kubectl -n flux-system get cm cluster-info ``` ### Upgrades, maintenance and fine-tuning Deploy additional Flux components: ```shell flux bootstrap gitlab \ --owner=$GITLAB_USER \ --repository=flux-workshop \ --branch=main \ --path=clusters/dev-cluster \ --personal \ --token-auth \ --components-extra=image-automation-controller,image-reflector-controller ``` Check the components: ```shell flux check ``` Pull the changes from the repository: ```shel git pull origin main ``` Fine-tuning the controllers by patching the Flux distribution in `clusters/dev-cluster/flux-system/kustomization.yaml`: ```yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - gotk-components.yaml - gotk-sync.yaml patches: - patch: | - op: add path: /spec/template/spec/containers/0/args/- value: --concurrent=10 - op: add path: /spec/template/spec/containers/0/args/- value: --requeue-dependency=5s target: kind: Deployment name: "(kustomize-controller|helm-controller|source-controller)" ``` Commit and push the changes to the repository: ```shell git add clusters/dev-cluster/flux-system/kustomization.yaml git commit -m "Fine-tune Flux components" git push origin main ``` More examples of fine-tuning the Flux components can be found at https://fluxcd.io/flux/installation/configuration/ ## Commit Status Updates Create a notification provider for GitLab in `clusters/dev-cluster/gitlab-status.yaml`: ```shell flux create alert-provider gitlab-status \ --type gitlab \ --address=https://gitlab.com/$GITLAB_USER/flux-workshop \ --secret-ref flux-system \ --export > clusters/dev-cluster/gitlab-status.yaml ``` Add an alert to `clusters/dev-cluster/gitlab-status.yaml`: ```yaml --- apiVersion: notification.toolkit.fluxcd.io/v1beta3 kind: Alert metadata: name: gitlab-status namespace: flux-system spec: providerRef: name: gitlab-status eventSources: - kind: Kustomization name: flux-system ``` Commit the changes to the `flux-workshop` repository: ```shell git add -A git commit -m "Enable commit status updates" git push origin main ``` Verify that the commit status updates are working by checking the GitLab repository. Navigate to `https://gitlab.com/$GITLAB_USER/flux-workshop/-/pipelines` and inspect the latest pipeline. ## Disaster recovery In case of a disaster, you can restore the cluster state by running the `flux bootstrap` command again. If the `flux-system` namespace is stuck in a `Terminating` state, you cleanup any dangling resources with: ```shell flux uninstall --namespace=flux-system ``` ### Debugging Various commands for status checking and debugging: ```shell flux get all --all-namespaces flux tree kustomization flux-system flux events flux logs flux trace Deployment/source-controller flux export source git flux-system flux export kustomization flux-system ```