# ArgoCD To deploy ArgoCD to kubernetes: ```shell= $ kubectl create namespace argocd $ kubectl apply -n argocd -f https://raw.githubusercontent.com/codefresh-contrib/gitops-certification-examples/main/argocd-noauth/install.yaml ``` ArgoCD along with its services and application resources will be hosted inside the `argocd` namespace. Verify it's running: ```shell= $ kubectl get pods -n argocd # Output NAME READY STATUS RESTARTS AGE argocd-redis-5b6967fdfc-grh4p 1/1 Running 0 109s argocd-dex-server-5fc596bcdd-8n9hz 1/1 Running 0 109s argocd-repo-server-98598b6c7-x9hn9 1/1 Running 0 109s argocd-application-controller-0 1/1 Running 0 108s argocd-server-5b4b7b868b-krt44 1/1 Running 0 109s ``` Note that this is simple installation method without any authentication. In a real setting you would probably setup SSO with your ArgoCD instance. > Alternatively you can use an OpenShift cluster and the namespace would be `openshift-gitops` ## Expose the UI to the user By default, Argo CD is only accessible from within the cluster. To expose the UI to users you can utilize any of the standard Kubernetes networking mechanisms such as Ingress (recommended for production) Load balancer (affects cloud cost) NodePort (simple but not very flexible) For this exercise we will do the simplest way and use a Nodeport service. In a production environment you should use an [Ingress](https://argo-cd.readthedocs.io/en/stable/operator-manual/ingress/). Apply the following `service.yml` file: ```yaml= # cat service.yml apiVersion: v1 kind: Service metadata: labels: app: argocd-server managedFields: name: argocd-server-nodeport namespace: argocd spec: ports: - nodePort: 30443 port: 8080 protocol: TCP selector: app.kubernetes.io/name: argocd-server type: NodePort ``` After the service is created, check the "Argo CD UI" tab to access the Argo CD Web interface. ![](https://i.imgur.com/UX5vdnb.png) ## How to Login Argo CD has a flexible user model that also supports SSO with popular identify providers. For this simple example we will use the default admin account. Argo CD initially has an admin user with an autogenerated password. You can get the admin password from the default Configmap: ```shell= kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d > admin-pass.txt # in my case the password was: pNx6K3Fn6PWXGLe3 ``` Just to make things simple we have disabled authentication in the ArgoCD UI. ![](https://i.imgur.com/bvhN3RN.png) Apart from the Graphical User interface, Argo CD also comes with a Command Line Interface (CLI) that can be used for common operations. The CLI is great for administration work, while the GUI is best for inspecting applications and settings. For OpenShift: ```bash= ARGOCD_SERVER=$(oc -n openshift-gitops get routes openshift-gitops-server -o jsonpath="{.spec.host}") ARGOCD_PASSWORD=$(oc -n openshift-gitops get secret openshift-gitops-cluster -o jsonpath="{.data.admin\.password}" | base64 --decode) ``` ## ArgoCD CLI To install the CLI: ```shell= $ curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64 $ chmod +x /usr/local/bin/argocd ``` Verify it works: ```shell= $ argocd help # Output argocd controls a Argo CD server Usage: argocd [flags] argocd [command] Available Commands: account Manage account settings admin Contains a set of commands useful for Argo CD administrators and requires direct Kubernetes access app Manage applications cert Manage repository certificates and SSH known hosts entries cluster Manage cluster credentials completion output shell completion code for the specified shell (bash or zsh) context Switch between contexts gpg Manage GPG keys used for signature verification help Help about any command login Log in to Argo CD logout Log out from Argo CD proj Manage projects relogin Refresh an expired authenticate token repo Manage repository connection parameters repocreds Manage repository connection parameters version Print version information Flags: --auth-token string Authentication token --client-crt string Client certificate file --client-crt-key string Client certificate key file --config string Path to Argo CD config (default "/root/.argocd/config") --core If set to true then CLI talks directly to Kubernetes instead of talking to Argo CD API server --grpc-web Enables gRPC-web protocol. Useful if Argo CD server is behind proxy which does not support HTTP2. --grpc-web-root-path string Enables gRPC-web protocol. Useful if Argo CD server is behind proxy which does not support HTTP2. Set web root. -H, --header strings Sets additional header to all requests made by Argo CD CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) -h, --help help for argocd --http-retry-max int Maximum number of retries to establish http connection toArgo CD server --insecure Skip server certificate and domain verification --logformat string Set the logging format. One of: text|json (default "text") --loglevel string Set the logging level. One of: debug|info|warn|error (default "info") --plaintext Disable TLS --port-forward Connect to a random argocd-server port using port forwarding --port-forward-namespace string Namespace name which should be used for port forwarding --server string Argo CD server address --server-crt string Server certificate file Use "argocd [command] --help" for more information about a command ``` Before we can use your CLI we need to authentication against our own Argo CD instace. In a production environment you might have different authentication options and also change the default admin password (or remove the admin account completely). ## Login with CLI Authenticate: ```shell= $ argocd login localhost:30443 --insecure # Output: Username: admin Password: 'admin:login' logged in successfully Context 'localhost:30443' updated ``` We use the insecure option because Argo CD has a self-signed certificate. In a production installation you would have a real certificate and this option should not be used. Notice that for security reasons the password you paste is not shown in the terminal (not even with asterisks). Just paste the password value and press Enter. Verify it works: ```shell= $ argocd version # Output: argocd: v2.2.2+03b17e0 BuildDate: 2022-01-01T06:27:52Z GitCommit: 03b17e0233e64787ffb5fcf65c740cc2a20822ba GitTreeState: clean GoVersion: go1.16.11 Compiler: gc Platform: linux/amd64 argocd-server: v2.1.5+a8a6fc8 BuildDate: 2021-10-20T15:16:40Z GitCommit: a8a6fc8dda0e26bb1e0b893e270c1128038f5b0f GitTreeState: clean GoVersion: go1.16.5 Compiler: gc Platform: linux/amd64 Ksonnet Version: v0.13.1 Kustomize Version: v4.2.0 2021-06-30T22:49:26Z Helm Version: v3.6.0+g7f2df64 Kubectl Version: v0.21.0 Jsonnet Version: v0.17.0 ``` ```shell= $ argocd app list # Output: NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET ``` ## Create a new app ### Via the UI 1. Navigate to the __+NEW APP__ on the left-hand side of the UI. 2. Then add the following to create the application: ``` General Section: Application Name: TBD Project: default Sync Policy: Automatic ``` ``` Source Section: Repository URL/Git: this is the GitHub repository URL Branches: main Path: TBD ``` ``` Destination Section: Cluster URL: select the cluster URL you are using Namespace: default ``` 3. Then, click __CREATE__ and you have now created your ArgoCD application. ### Via the CLI ```shell= argocd app create {APP NAME} \ --project {PROJECT} \ --repo {GIT REPO} \ --path {APP FOLDER} \ --dest-namespace {NAMESPACE} \ --dest-server {SERVER URL} ``` * `{APP NAME}` is the name you want to give the application * `{PROJECT}` is the name of the project created or "default" * `{GIT REPO}` is the url of the git repository where the gitops config is located * `{APP FOLDER}` is the path to the configuration for the application in the gitops repo * `{DEST NAMESPACE}` is the target namespace in the cluster where the application will be deployed * `{SERVER URL}` is the url of the cluster where the application will be deployed. Use https://kubernetes.default.svc to reference the same cluster where ArgoCD has been deployed Once this completes, you can see the status and configuration of the application. `$ argocd app list` For a more detailed view of the application configuration, run: `$ argocd app get {APP NAME}` ## Synchronize an ArgoCD application Initially, the application is in **OutOfSync** or **Unknown** state since the application has yet to be deployed, and no Kubernetes resources have been created. ### via the UI To synchronize/deploy the ArgoCD app: 1. Choose the tile and then select SYNC. This will provide you options of what you want to synchronize. Select the default options and synchronize all manifests 2. Once it's deployed, you will see the resources deployed in the UI and a Healthy status. ### via the CLI Run: `argocd app sync {APP NAME}` This synchronizes the application. To confirm it’s running, you can execute a kubectl command. `kubectl -n {NAMESPACE} get all` The application will have a status “Running” if synchronized successfully. For exercise purposes, we will use Github for this exercise. The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/simple-app Take a look: **deployment.yml**: ```yaml= --- apiVersion: apps/v1 kind: Deployment metadata: name: simple-deployment spec: replicas: 1 selector: matchLabels: app: trivial-go-web-app template: metadata: labels: app: trivial-go-web-app spec: containers: - name: webserver-simple image: docker.io/kostiscodefresh/gitops-simple-app:v1.0 ports: - containerPort: 8080 ``` **service.yaml** ```yaml= apiVersion: v1 kind: Service metadata: name: simple-service spec: type: NodePort selector: app: trivial-go-web-app ports: - nodePort: 31000 protocol: TCP port: 8080 ``` The UI starts empty because nothing is deployed on our cluster. Click the "New app" button on the top left ... ![](https://i.imgur.com/HiPEFNJ.png) ... and fill the following details: > * application name : `demo` > * project: `default` > * repository URL: `https://github.com/codefresh-contrib/gitops-certification-examples` > * path: `./simple-app` > * Cluster: `https://kubernetes.default.svc` (this is the same cluster where ArgoCD is installed) > * Namespace: `default` Leave all the other values empty or with default selections. Finally click the Create button. The application entry will appear in the main dashboard. Click on it. You should see the following: ![](https://i.imgur.com/t14jegc.png) In YAML view, this looks like this: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: demo spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./simple-app repoURL: 'https://github.com/codefresh-contrib/gitops-certification-examples' targetRevision: HEAD project: default ``` ![](https://i.imgur.com/FKQbXBW.png) Click on the **demo** tile to get inside the app. You should see: ![](https://i.imgur.com/PzPJAJ5.png) Congratulations! Your have setup your first application with GitOps. However if you check your cluster with ```shell= # kubectl get deployments No resources found in default namespace. ``` You will see that nothing is deployed yet. The cluster is still empty. The "Out of sync" message means 3 things: 1. The cluster is empty 2. The Git repository has an application 3. Therefore the Git state and the cluster state are different. Syncing in ArgoCD means matching the state described in Git and what is in the cluster ## Syncing the app We have created our application in Argo CD, but it is still not deployed as it only exists in Git right now. We need to tell Argo CD to make the Git state equal to the cluster state. Visit the Argo CD UI tab and click on your application to see all the details. ![](https://i.imgur.com/3xq5G7m.png) Click the **Sync button** and leave all the default options in all choices. ![](https://i.imgur.com/vhUeHXy.png) Finally click the **Synchronize** button at the top. ArgoCD sees that the cluster is empty and it will deploy your application in order to make the cluster have the same state as what is in Git. You should see the following: ![](https://i.imgur.com/DCqq2z2.png) Your application is now deployed. You can check it with: ```shell= # kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE simple-deployment 1/1 1 1 86s ``` Also visiting the app via your browser: ![](https://i.imgur.com/uBdZLoq.png) ## Via the CLI You can manage your Argo CD installations either from the Web Interface or using the Command Line. ### Step 1 Apart from the UI, ArgoCD also has a CLI. We have installed already the cli for you and authenticated against the instance. Try the following commands ```shell= argocd app list argocd app get demo argocd app history demo ``` ```shell= root@kubernetes-vm:~/workdir# argocd app list NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET demo https://kubernetes.default.svc default default Synced Healthy <none> <none> https://github.com/codefresh-contrib/gitops-certification-examples ./simple-app HEAD root@kubernetes-vm:~/workdir# argocd app get demo Name: demo Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/demo Repo: https://github.com/codefresh-contrib/gitops-certification-examples Target: HEAD Path: ./simple-app SyncWindow: Sync Allowed Sync Policy: <none> Sync Status: Synced to HEAD (083f8d3) Health Status: Healthy GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE Service default simple-service Synced Healthy service/simple-service created apps Deployment default simple-deployment Synced Healthy deployment.apps/simple-deployment created root@kubernetes-vm:~/workdir# argocd app history demo ID DATE REVISION 0 2022-01-04 20:41:10 +0000 UTC HEAD (083f8d3) ``` Let's delete the application and deploy it again but from the CLI this time. First delete the app ```shell= # argocd app delete demo Are you sure you want to delete 'demo' and all its resources? [y/n] y ``` Confirm the deletion by answering yes in the terminal. The application will disappear from the Argo CD dashboard after some minutes. Now deploy it again. ```shell= # argocd app create demo2 \ > --project default \ > --repo https://github.com/codefresh-contrib/gitops-certification-examples \ > --path "./simple-app" \ > --dest-namespace default \ > --dest-server https://kubernetes.default.svc application 'demo2' created ``` The application will appear in the ArgoCD dashboard. Now sync it with ```shell= # argocd app sync demo2 TIMESTAMP GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE 2022-01-04T20:45:20+00:00 Service default simple-service Synced Healthy 2022-01-04T20:45:20+00:00 apps Deployment default simple-deployment Synced Healthy 2022-01-04T20:45:20+00:00 apps Deployment default simple-deployment Synced Healthy deployment.apps/simple-deployment unchanged 2022-01-04T20:45:20+00:00 Service default simple-service Synced Healthy service/simple-service unchanged Name: demo2 Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/demo2 Repo: https://github.com/codefresh-contrib/gitops-certification-examples Target: Path: ./simple-app SyncWindow: Sync Allowed Sync Policy: <none> Sync Status: Synced to (083f8d3) Health Status: Healthy Operation: Sync Sync Revision: 083f8d306ccb4acc5ee8b9318b1cef091cbe44a6 Phase: Succeeded Start: 2022-01-04 20:45:20 +0000 UTC Finished: 2022-01-04 20:45:20 +0000 UTC Duration: 0s Message: successfully synced (all tasks run) GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE Service default simple-service Synced Healthy service/simple-service unchanged apps Deployment default simple-deployment Synced Healthy deployment.apps/simple-deployment unchanged ``` ## Questions **What is ArgoCD?** A GitOps Agent that implements the GitOps principles. **Which other Argo products does Argo CD need to function correctly?** None. Argo CD is a standalone project. It works great with the other Argo projects, but it does not depend on them. **How does ArgoCD interact with clusters?** You can have any combination of clusters and ArgoCD instances. ArgoCD can deploy applications on the cluster it is installed on, or other external clusters that are authenticated correctly. So basically you can create one ArgoCD instance and then link additional clusters to this instance. **How can you install ArgoCD on your cluster Your organization might have different requirements according to your use case. You can just grab the manifests from Git and run a `kubectl apply`. If you are more serious, you might want to try `Helm chart` or if you want the ArgoCD to manage itself and handle its own upgrades, consider using the Argo Autopilot. **What is the relationship between the ArgoCD Web interface and the Argo CD Command line executable?** The Argo CD UI and the CLI can be used interchangeably according to your needs. Most actions should be available to both interaction methods. ## How to auto-sync applications So far we saw how to sync an application **on-demand** using either the GUI or CLI. But, ArgoCD will also auto-sync an application on its own (if you enable it). By default **every 3 minutes** argo CD will: 1. Take one by one, all the applications you have marked as *auto-sync* and fetch their latest Git state from their respective repositories. 2. It will compare the Git state (desired) with the cluster state (current) and take a decision about it. 3. If both states are the same, nothing has to be done, so it will mark the application as *Synced* 4. If there are differences, it will mark the application as *Out-Of-Sync*. In case of *Out-of-sync* situations, it's up to the administrator to decide what action ArgoCD would like to do. In any case, this loop is happening every 3 minutes or so. If you want to change this timeframe, you can do it my changing the `argcd-cm` **configmap** found on the *same namespace* as ArgoCD: ```yaml= apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd labels: app.kubernetes.io/name: argocd-cm app.kubernetes.io/part-of: argocd data: timeout.reconciliation: 240s ``` This example changes the sync period to 4 minutes (240 sec). **To take effect** you have to redploy the `argocd-repo-server` service. If for some reason you want to completely disable this auto-sync functionallity, you can do it by setting to zero: `timeout.reconciliation: 240s`. Finally, you can also change the way ArgoCD learns about Git changes. So far we said that ArgoCD is polling the Git provider. Alternatively you can also use webhooks ([read here](https://argo-cd.readthedocs.io/en/stable/operator-manual/webhook/) about it). The benefit of using webhooks is that there are no delays. By the time you merge a PR, ArgoCD will wake up -- otherwise, you will have to wait 3 minutes, which is also not to bad. Let's do an exercise: You need to have a GitHub account for this exercise. The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/simple-app. Fork this repository in your own account: My fork: `https://github.com/drpaneas/gitops-certification-examples` Take a look at the Kubernetes manifests to understand what we will deploy. It is a very [simple application](https://github.com/drpaneas/gitops-certification-examples/tree/main/simple-app) with one deployment and one service. Create a new app: ``` application name : demo project: default repository URL: https://github.com/<your user>/gitops-certification-examples path: ./simple-app Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: default ``` Leave all the other values empty or with default selections. It should look like this: ![](https://i.imgur.com/WyfU98U.png) in YAML is looks like this: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: demo spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./simple-app repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD project: default ``` And click **Create** button. ![](https://i.imgur.com/PbUKARJ.png) Now sync the application to make sure that is deployed. You should see: ![](https://i.imgur.com/EZuRlw9.png) and if you click on it: ![](https://i.imgur.com/umW668C.png) You can also verify with CLI: ```shell= $ kubectl get deploy NAME READY UP-TO-DATE AVAILABLE AGE simple-deployment 1/1 1 1 64s ``` Now Let's deploy a second version by changing the Git state. We want to deploy another version of our application. We will change the Git and see how Argo CD detects the change. Perform a commit in your simple-app/deployment.yml file in your own account (You can use GitHub on another tab for this) and change the container tag at line 18 from v1.0 to v2.0 ![](https://i.imgur.com/ZrR5Vhk.png) Normally Argo CD checks the state between Git and the cluster every 3 minutes on its own. Just to speed things up you should click manually on the application in the Argo CD dashboard and press the "Refresh" button ![](https://i.imgur.com/bhUOb8P.png) Here Argo CD tells us that the Git state is no longer the same as the cluster state. ![](https://i.imgur.com/ZsdKghZ.png) From the yellow arrows you can see that the from all the Kubernetes components the deployment is now different. And thus the whole application is different. ![](https://i.imgur.com/28dCxN5.png) You can click on the "App diff" button and see the exact change. Enable the "compact diff" checbox. ![](https://i.imgur.com/IGZFnTg.png) You should see your change: ![](https://i.imgur.com/LqFUFKy.png) The Argo CD controller also detects cluster changes Step 1 Let's bring the cluster back to the same state as Git. Click the Sync button in the UI to make the application synced with the new version. You can also execute the following from the CLI: ```shell= # argocd app sync demo TIMESTAMP GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE 2022-01-05T11:32:01+00:00 Service default simple-service Synced Healthy 2022-01-05T11:32:01+00:00 apps Deployment default simple-deployment OutOfSync Healthy 2022-01-05T11:32:01+00:00 Service default simple-service Synced Healthy service/simple-service unchanged 2022-01-05T11:32:01+00:00 apps Deployment default simple-deployment OutOfSync Healthy deployment.apps/simple-deployment configured 2022-01-05T11:32:01+00:00 apps Deployment default simple-deployment Synced Progressing deployment.apps/simple-deployment configured Name: demo Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/demo Repo: https://github.com/drpaneas/gitops-certification-examples Target: HEAD Path: ./simple-app SyncWindow: Sync Allowed Sync Policy: <none> Sync Status: Synced to HEAD (124c929) Health Status: Progressing Operation: Sync Sync Revision: 124c929538042459fc646aa08e3ed0ebad2f08a0 Phase: Succeeded Start: 2022-01-05 11:32:01 +0000 UTC Finished: 2022-01-05 11:32:01 +0000 UTC Duration: 0s Message: successfully synced (all tasks run) GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE Service default simple-service Synced Healthy service/simple-service unchanged apps Deployment default simple-deployment Synced Progressing deployment.apps/simple-deployment configured root@kubernetes-vm:~/workdir# argocd app wait demo Name: demo Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/demo Repo: https://github.com/drpaneas/gitops-certification-examples Target: HEAD Path: ./simple-app SyncWindow: Sync Allowed Sync Policy: <none> Sync Status: Synced to HEAD (124c929) Health Status: Healthy Operation: Sync Sync Revision: 124c929538042459fc646aa08e3ed0ebad2f08a0 Phase: Succeeded Start: 2022-01-05 11:32:01 +0000 UTC Finished: 2022-01-05 11:32:01 +0000 UTC Duration: 0s Message: successfully synced (all tasks run) GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE Service default simple-service Synced Healthy service/simple-service unchanged apps Deployment default simple-deployment Synced Healthy deployment.apps/simple-deployment configured ``` So it's green again: ![](https://i.imgur.com/h4pl62U.png) Detecting changes in Git and applying is a well known scenario. The big strength of GitOps, is that Argo CD works in the opposite direction as well. If you make any change in the cluster then Argo CD will detect it and again tell you that something is different between Git and your cluster. Let's say that somebody changes manually the replicas of the deployment without creating an official Pull Request (a bad practice in general). Execute the following ```shell= kubectl scale --replicas=3 deployment simple-deployment # Output: deployment.apps/simple-deployment scaled ``` Normally Argo CD checks the state between Git and the cluster every 3 minutes on its own. Just to speed things up you should click manually on the application in the Argo CD dashboard and press the "Refresh" button. ![](https://i.imgur.com/Ndgj1pP.png) The click the "App diff" button and enable the "Compact Diff" checkbox. ![](https://i.imgur.com/WKey0Id.png) ## Application Health Apart from the synced/out-of-sync status, ArgoCD also keeps track of the service health for each deployed application. The health status is shown in the UI and can also be retrieved by the CLI. The possible values are: * **Healthy** -> Resource is 100% healthy * **Progressing** -> Resource is not healthy but still has a chance to reach healthy state * **Suspended** -> Resource is suspended or paused. The typical example is a cron job * **Missing** -> Resource is not present in the cluster * **Degraded** -> Resource status indicates failure or resource could not reach healthy state in time * **Unknown** -> Health assessment failed and actual health status is unknown The decision whether an application is healthy or not depends on the type of underlying Kubernetes resources. For the built-in Kubernetes resources, the rules are the following: #### Deployments, ReplicaSets, StatefulSets, and Daemon sets These are considered “healthy” if: 1. observed current generation is equal to desired generation. 2. Number of updated replicas equals the number of desired replicas. #### For a service of type Loadbalancer or Ingress These are consiered "healthy" if: 1. the `status.loadBalancer.ingress` list is non-empty, with at least one value for hostname or IP. #### For custom Kubernetes resources Health is defined in Lua scripts. Some examples of resources that have custom health definitions are: * Argo Rollout (and associated Analysis and Experiments) * Bitnami Sealed secrets * Cert Manager * Elastic Search * Jaeger * Kafka * CrossPlane provider You can see a list of all custom health checks at https://github.com/argoproj/argo-cd/tree/master/resource_customizations. You can add your own health check for a custom resource by implementing the check in Lua which is a self-contained scripting language. Here is a simple example of the health check from Argo Rollouts Experiments that simply maps the experiment status to Argo CD health status: ```lua= hs = {} if obj.status ~= nil then if obj.status.phase == "Pending" then hs.status = "Progressing" hs.message = "Experiment is pending" end if obj.status.phase == "Running" then hs.status = "Progressing" hs.message = "Experiment is running" end if obj.status.phase == "Successful" then hs.status = "Healthy" hs.message = "Experiment is successful" end if obj.status.phase == "Failed" then hs.status = "Degraded" hs.message = "Experiment has failed" end if obj.status.phase == "Error" then hs.status = "Degraded" hs.message = "Experiment had an error" end return hs end hs.status = "Progressing" hs.message = "Waiting for experiment to finish: status has not been reconciled." return hs ``` You can write your own health check in a similar manner for your custom resource, if it is not already supported by Argo CD. ### No Health check? It's possible. Note that having a custom health check just provides a better experience with ArgoCD. You can still deploy your custom application even without a health check, and it will never show as Healthy in the Argo CD dashboards. ### K8s healthcheck vs ArgoCD healthcheck: 2 different things Argo CD health checks are completely independent from [Kubernetes health probes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/). ## Sync Strategies Previously we created an ArgoCD application and we chose to go with a manuall sync approach/strategy. However, ArgoCD can also automatically sync and it also support many automatic variations of this auto-sync. You can find them [here](https://argo-cd.readthedocs.io/en/stable/user-guide/auto_sync/). There are 2 basic parameters to fine-tune your auto-sync strategy: 1. Auto-Pruning of resources. 2. Self-Healing of the cluster. **NOTE**: This two strategies are configurable only when you enable **auto-sync**. They are **not** available in Manual mode. So, we set auto-sync, ArgoCD will detect the changes and apply them to the cluster. If we set manual sync, ArgoCD will detect the changes and do nothing in the cluster. ### Auto Pruning You go to your Git repo and you `git rm $FILE` remove and delete some files and folders there. If **Auto-Pruning** is **enabled** , ArgoCD will also remove the respective resources in the cluster as well. If **Auto-Pruning** is **disabled**, ArgoCD will never delete anything from the cluster. ### Self Healing You go to your cluster and you make changes using `kubectl` or any other way. Note that you shouldn't do that in the first place if you are using GitOps workflow and you adopt GitOps principles (as all changes should pass through Git). If **Self-Healing** is **enabled**, then ArgoCD will revert everything you did and restore the cluster back to the desired state that is described in Git. Here's a table of options you have: ![](https://i.imgur.com/QwkzPci.png) * **Policy A** (nothing is done by ArgoCD) is probably how you should start adopting ArgoCD , especially if you want to apply GitOps on an existing project. This is your chance to see how ArgoCD works, without actually affecting your deployments. * **Policy B** (auto-sync) is the first step towards following GitOps with its automation capabilities. As soon as any change happens in Git, your cluster will auto-update. Disabling auto-prune means that you must still delete resources manually, and disabled self-heal means that you can still make manual changes to the cluster (if you want to have a migration period). * **Policies C** and **D** are orthogonal and can act as stepping stones to **Policy E** which is the one you should look up to. Under that policy, everything is automated both ways. Changes in Git are also reflected automatically to the cluster (including removals of resources), and manual changes in the cluster are simply discarded. Adopting GitOps in its purest form may require organizational changes or a reexamination of policies and your organization may not be ready at this point. While correcting drift automatically will help organizations get the most out of GitOps, it may take some time to be ready. Let's do an exercise in syncing strategies to learn better about them. You will need to have forked the `gitops-certification-examples/sync-strategies/` to your repo. This is mine: `https://github.com/drpaneas/gitops-certification-examples/tree/main/sync-strategies` ### Autosync Click the "New app" button on the top left and fill the following details: ``` application name : demo project: default SYNC POLICY: automatic repository URL: https://github.com/<your user>/gitops-certification-examples path: ./sync-strategies Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: default ``` > Don't forget: SYNC POLICY: automatic Leave all the other values empty or with default selections. Finally click the Create button. The application entry will appear in the main dashboard. Click on it. ![](https://i.imgur.com/CEyunye.png) The yaml looks like this: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: demo spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./sync-strategies repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD project: default syncPolicy: automated: prune: false selfHeal: false ``` Since we have selected the auto-sync strategy, Argo CD will autodeploy the application right away (because the cluster state is not the same as the commit state) You should see the following: ![](https://i.imgur.com/9eisy0t.png) You can also verify with CLI: ```shell= # kubectl get deploy NAME READY UP-TO-DATE AVAILABLE AGE simple-deployment 1/1 1 1 72s ``` Fully automated deployments with auto sync If you look at the "Deployed app" tab you will see that our application is at verion 1.0 ![](https://i.imgur.com/E5CJUUm.png) We want to deploy another version of our application. We will change the Git and see how Argo CD detects the change and automatically deploys (since we have set sync strategy to auto). ![](https://i.imgur.com/EsHAGpQ.png) Normally Argo CD checks the state between Git and the cluster every 3 minutes on its own. Just to speed things up you should click manually on the application in the Argo CD dashboard and press the "Refresh" button You should see the following: ![](https://i.imgur.com/QcvDSuD.png) ### Avoid manual changes in your cluster #### Enable self-heal Autosync will automatically deploy your application as soon as the Git repository changes. But if somebody makes a manual change in the cluster state, Argo CD will not do anything by default (it will still mark the application as out-of-sync). You can enable the self-heal option to tell Argo CD to discard any changes done manually to the cluster. This is a great advantage as it always makes your environments bulletproof against ad-hoc changes via kubectl. Execute the following: ```shell= # kubectl scale --replicas=3 deployment simple-deployment deployment.apps/simple-deployment scaled ``` You have manually changed the cluster. Now go the Argo CD dashboard and click on the application. ![](https://i.imgur.com/2OqKtGn.png) In the Application screen click the top left "App Details" button. ![](https://i.imgur.com/4QGEhPu.png) Scroll down and find the "Sync policy section". Click on the "Self-heal" button to enable it. Answer ok to the confirmation dialog. ![](https://i.imgur.com/Lw2ub38.png) The manual change will be discarded and replicas are back to one . ![](https://i.imgur.com/lTZREMU.png) You can now try changing anything on the cluster and all changes will always be discarded (as they are not part of Git). Try in the terminal again. ```shell # kubectl scale --replicas=3 deployment simple-deployment deployment.apps/simple-deployment scaled # kubectl get deployment simple-deployment NAME READY UP-TO-DATE AVAILABLE AGE simple-deployment 1/1 1 1 11m ``` Your application will always have 1 pod deployed. This is what the Git state says and therefore Argo CD automatically makes the cluster state the same. ### Delete resources with just a Git commit #### Enable auto-prune Even after having auto-sync on and self-heal on, Argo CD will never delete resources from the cluster if you remove them in Git. To enable this behavior you need to enable the auto-prune option. First make a commit at your Git repo by deleting the `sync-strategies/deployment.yml` file. ![](https://i.imgur.com/p5gksuN.png) Then click the "Refresh" button in the ArgoCD dashboard. ArgoCD will detect the change and mark the application as "Out of sync". But the deployment will still be there. You should see the following. ![](https://i.imgur.com/shVcL9v.png) You can still see the deployment with CLI as well: ```shell= # kubectl get deploy NAME READY UP-TO-DATE AVAILABLE AGE simple-deployment 1/1 1 1 14m ``` Go the Argo CD dashboard and click on the application. In the Application screen click the top left "App Details" button. Scroll down and find the "Sync policy section". Click on the "Prune resources" button to enable it. Answer ok to the confirmation dialog. ![](https://i.imgur.com/25tbvWm.png) Close the settings window and click again the "Refresh" button in the application. Now your deployment will be removed (since it does not exist in Git). ![](https://i.imgur.com/eRZUuym.png) Also via CLI: ```shell= # kubectl get deploy No resources found in default namespace ``` ## Secrets How can you use secrets with GitOps? This has been one of the most popular questions from teams that adopt GitOps. The truth is that there is no single accepted practice for how secrets are managed with GitOps. If you already have a solid solution in place such as HashiCorp vault, then it would make sense to use that even though technically it is against GitOps practices. If you are starting from scratch on a new project, there are ways to store secrets in Git so that you can manage them using GitOps principles as well. It goes without saying that you should never ever commit raw secrets in Git. ArgoCD can be used with multiple [secret solutions](https://argo-cd.readthedocs.io/en/stable/operator-manual/secret-management/) that you might already have deployed. All solutions that handle secrets using Git are storing them in an encrypted form. This means that you can get the best of both worlds. Secrets can be managed with GitOps, and they can also be placed in a secure manner in any Git repository (even public repositories). As an example, we will use the [Bitnamic Sealed secrets controller](https://github.com/bitnami-labs/sealed-secrets) to encrypt secrets before storing them in Git and decrypt them before passing them to the application. ### How Kubernetes secrets work Over the next paragraphs, we will talk about two kinds of secrets, the [built-in Kubernetes secrets](https://kubernetes.io/docs/concepts/configuration/secret/) (present in every Kubernetes cluster) and the sealed secrets as introduced by the Bitnami Sealed secrets controller. Before we talk about sealed secrets, let’s talk about normal/plain secrets first. Kubernetes includes a native secret resource that you can use in your application. By default, these secrets are not encrypted in any way and the base64 encoding used should be never seen as a security feature. While there are ways to encrypt the Kubernetes secrets within the cluster itself, we are more interested in encrypting them outside the cluster, so that we can store them externally in Git as required by one the founding principles of GitOps (everything stored in Git). Using Kubernetes application secrets is straightforward. You can use the same mechanisms as configmaps, namely mounting them as files on your application or passing them as environment variables. ![](https://i.imgur.com/JC0nMjf.png) Sealed secrets are just an extension built on top of Kubernetes native secrets. This means that after the encryption/decryption takes place, all secrets function as plain Kubernetes secrets, and this is how your application should access them. If you don’t like how plain Kubernetes secrets work, then you need to find an alternative security mechanism. The Bitnami Sealed secrets controller is a Kubernetes controller you install on the cluster that performs a single task. It converts sealed secrets (that can be committed to Git) to plain secrets (that can be used in your application). Installing the controller is straightforward: ```shell= helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets helm repo update helm install sealed-secrets-controller sealed-secrets/sealed-secrets ``` Once installed, the controller creates two keys on its own: 1. The private key is used for secret decryption. This key should stay within the cluster, and you should never give it to anyone. 2. The public key is used for secret encryption. This can (and will) be used outside the cluster, so it is ok to give it to somebody else. Once the controller is installed, you install and use your application in the standard way. You don’t need to change your application code or tamper with your Kubernetes manifests. If your application can use vanilla Kubernetes secrets, then it can work with sealed secrets as well. ![](https://i.imgur.com/Pviu3dl.png) It should be clear that the controller does not come in direct contact with your application. It converts sealed secrets to Kubernetes secrets, and afterwards it is up to your application how to use them. The application does not even know that its secrets were originally encrypted in Git. ### Encrypting your secrets We have seen how the controller decrypts secrets. But how do we encrypt secrets in the first place? The controller comes with the associated kubeseal executable that is created for this purpose. It is a single binary, so you can install it by copying it to your favorite directory (probably in your PATH variable). ```shell= $ wget https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.17.0/kubeseal-linux-amd64 -O kubeseal $ sudo install -m 755 kubeseal /usr/local/bin/kubeseal ``` If you have a Mac, you can also get it via `brew install kubeseal`. Kubeseal does the opposite operation from the controller. It takes an existing Kubernetes secret and encrypts it. Kubeseal requests the public key that was created during the installation process from the cluster and encrypts all secrets with that key. This means that: 1. Kubeseal needs access to the cluster in order to encrypt secrets. (It expects a kubeconfig like kubectl.) 2. Encrypted secrets can only be used in the cluster that was used for the encryption process. The last point is very important as it means that **all secrets are cluster specific**. The namespace of the application is also used by default, so **secrets are cluster and namespace specific**. If you want to use the same secret for different clusters, you need to encrypt it for each cluster individually. To use kubeseal, just take any existing secret in yaml or json format and encrypt it: `$ kubeseal -n my-namespace < .db-creds.yml > db-creds.json` This creates a SealedSecret which is a custom Kubernetes resource specific to the controller. This file is safe to commit in Git or store in another external system. You can then apply the secret on the cluster `$ kubectl apply -f db-creds.json -n my-namespace` The secret is now part of the cluster and will be decrypted by the controller when an application needs it. Here is the full diagram of encryption/decryption: ![](https://i.imgur.com/gXFSViy.png) The full process is the following: 1. You create a plain Kubernetes secret locally. You should never commit this anywhere. 2. You use kubeseal to encrypt the secret in a SealedSecret. 3. You delete the original secret from your workstation and apply the sealed secret to the cluster. 4. You can optionally commit the Sealed secret to Git. 5. You deploy your application that expects normal Kubernetes secrets to function. (The application needs no modifications of any kind.) 6. The controller decrypts the Sealed secrets and passes them to your application as plain secrets. 7. The application works as usual. By using the Sealed Secrets controller, we can finally store all our secrets in Git (in an encrypted form) right along the application configuration. In the example repository, you can look at the folder https://github.com/codefresh-contrib/gitops-secrets-sample-app/tree/main/safe-to-commit and it contains all manifests of the application, including secrets. Exercise time! The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/secret-app. Fork this repository in your own account. Mine: `https://github.com/drpaneas/gitops-certification-examples/tree/main/secret-app` It is a very simple application with one deployment and one service that also needs two secrets files (for db connection and paypal certificate) that are passed to the application as files under `/secrets/`. The two secrets are stored in the files `db-creds-encrypted.yaml` and `paypal-cert-encrypted.yaml`. These files are empty in the Git repository because we want to encrypt them first. You can also see the full source code at `source-code-secrets` if you want to understand how the application is loading secrets from files. ### Install the Sealed Secrets controller We have placed the _raw/unencrypted_ secrets in your work directory that are needed by the application: ```shell= # ls admin-pass.txt unsealed_secrets # ls unsealed_secrets/ db-creds.yml paypal-cert.ym ``` ```yaml= # cat unsealed_secrets/db-creds.yml apiVersion: v1 kind: Secret type: Opaque metadata: name: mysql-credentials data: connection: bXktZGItY29ubmVjdGlvbi5leGFtcGxlLmNvbTozMzA2 password: bXlmYW5jeXBhc3N3b3Jk username: bXlmYW5jeXVzZXI= ``` ```yaml= # cat unsealed_secrets/paypal-cert.yml apiVersion: v1 kind: Secret type: Opaque metadata: name: paypal-cert data: paypal.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMxVENDQWIyZ0F3SUJBZ0lKQUxNRkRxakg3czRGTUEwR0NTcUdTSWIzRFFFQkJRVUFNQm94R0RBV0JnTlYKQkFNVEQzZDNkeTVsZUdGdGNHeGxMbU52YlRBZUZ3MHlNVEEzTWpJeE1qUTFOVE5hRncwek1UQTNNakF4TWpRMQpOVE5hTUJveEdEQVdCZ05WQkFNVEQzZDNkeTVsZUdGdGNHeGxMbU52YlRDQ0FTSXdEUVlKS29aSWh2Y05BUUVCCkJRQURnZ0VQQURDQ0FRb0NnZ0VCQVBOcWc5clZFRmN2dnpiYkptV1B4VFQ5djBZL3NBUmtTNTJybzJ3NGFJSU8KUjdCMEtmVTRtWjNUMUNBL0tBcTBoQVNMNk9JNFowVXpMelpybzZLM2tmd2hzWjdBNmx0QzZBVUlZdXc0eC9VWgpGeDBBaDZ3MWFTYjBHTFJjc1ZHTENjYjAwMGJKMW5hUkFjSlY3dm1JRUZ4U2l2cEhXRTFuU28rWkpXQ1lHNG1vCk5FQTNvTjB4Nm55d2t4TXRHN0twd0ZtZlRiekxlTjd5K3R4eFQ2RHE5RXpReVpnVE5UQmJCSDdNQzI4eHdDQmYKcnZJSXVRMnZqWkIvTklKY0V2OUdXelZsSlRnNXBXRHdUMUI0YjVnYlpGWlErUSttQ1UwRGRtOWJXSUcvam5lcQplVjN6MkJkR0p4S3NUbk90b2ZKMFFna1h6UU92OEsvR05rS2xpS1VYcDI4Q0F3RUFBYU1lTUJ3d0dnWURWUjBSCkJCTXdFWUlQZDNkM0xtVjRZVzF3YkdVdVkyOXRNQTBHQ1NxR1NJYjNEUUVCQlFVQUE0SUJBUUE2V3VkdnJtYmkKd1g5Y0s5c0JaUnd1dnNlZWliK2tZb2tDdTNIYlZlK055cFl2VVpMam5BRG05VzZrcnp0cGNLT3dpRnJIbTBZVwpSK01Fb1dMdUxzNTlvaXVPa2VianRHdVN5VkRUTUk5eS9JeE4wcm5HaE1tY3BveG9JQ0s5SU8vVHFJZlFyU3V2CmJKRktVRnNQSHJGdXY2SWJtR0RNN3loek5Td2lseTVlY2tYT3EvYTZBeEoyT3BKR0pEbk5vbDBrS1FxUzRyVHIKblYzOXlWOEdpMVAyakd1U2xoYVRuR2NWejk3eXlvazRhdmZXMU1rNG5NUFpRNURLM2JSQW1aUUtURnE5VDlhZwpXSGJIR0lZcGZyOWtuVHJFSkFBNlhtTmo4WnZsVVU4RDZWY2NPZUZ5ZHBGNWhsNXAwYW9ESmtvTlhnTXk0cG5NClZnRmZOdjJudzNnTAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg== ``` These secrets are plain Kubernetes secrets and are not encrypted in any way. Let's install the Bitnami Sealed secrets controller to encrypt our secrets. We installed Argo CD for you and you can login in the UI tab. The UI starts empty because nothing is deployed on our cluster. Click the "New app" button on the top left and fill the following details: ```application name : controller project: default SYNC POLICY: automatic repository URL: https://github.com/codefresh-contrib/gitops-certification-examples path: ./bitnami-sealed-controller Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: kube-system ``` In YAML it looks like this: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: controller spec: destination: name: '' namespace: kube-system server: 'https://kubernetes.default.svc' source: path: ./bitnami-sealed-controller repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD project: default syncPolicy: automated: prune: false selfHeal: false ``` Notice that are using a specific namespace for the controller and not the default one. It is imperative to enter `kube-system` otherwise the secrets controller will not work. You can see that GitOps is not only useful for your own applications but also for other supporting applications as well. #### Install kubeseal binary Use Kubeseal to encrypt secrets The Sealed Secrets controller is running now on the cluster and it is ready to decrypt secrets. We now need to encrypt our secrets and commit them to git. Encryption happens with the `kubeseal` executable. It needs to be installed in the same way as `kubectl`. It re-uses the cluster authentication already used by `kubectl`. We have already installed `kubeseal` for you in this exercise. You can use it right away to encrypt your plain Kubernetes secrets and convert them to encrypted secrets Run the following: ```shell= kubeseal < unsealed_secrets/db-creds.yml > sealed_secrets/db-creds-encrypted.yaml -o yaml kubeseal < unsealed_secrets/paypal-cert.yml > sealed_secrets/paypal-cert-encrypted.yaml -o yaml ``` This created the encrypted forms of the previous files: ```shell= # ls sealed_secrets/ db-creds-encrypted.yaml paypal-cert-encrypted.yaml ``` ```yaml= # cat sealed_secrets/db-creds-encrypted.yaml apiVersion: bitnami.com/v1alpha1 kind: SealedSecret metadata: creationTimestamp: null name: mysql-credentials namespace: default spec: encryptedData: connection: AgChakYoCfsXmJlqjWYvPfCOI3Kt5tqmR6fjrMMw/9O4OvS25EV0Y5xlaYrkjmK9IUOMisz0pU217zMaeIP6kxkeKfvxrOU5LL8h0YAQfqOBQZ1YX7RB12v/KRlVcC9AbCki8LPgFT7gTlv+hx6AoWidOwTmnEs+xA6OJf77HR7Lli1/ly0E4DyjbGJ8O2j/JxTH4aZwsP8cMvGoJSZPXgxJJMWZULT5cZVLknUGQbzIusEA1MuB/bsH5heHahjVTeK7oyb7IPUYrtMLagQkYuV6zPALwww5c3kCBvTsa37xMD3mVACjPhtVcSET8t792082DQ/Atd4pkObXbM1PkEcUrwdU8uT5xeSuYb0QK6PMQMG+hh9pPXjo96nDKNdPhJ0R1+DbVv5auTT6gU02eB8k6MVcztjZj+q7Bw+PN+R51X/XpBrP0TWXrdS16GThrH91ZXnMMBatgBG5doJvc1WCTFw7+IGdp9rMs4IAmYDPUvIR5K++oXcDCTdbTgEnsnYyxv0S42AoaayWiYD/R2RoqsV97c6ERnfWd76EvsM5GItOA/uiADWDU7TmMcar0bPZh+lIFHKnoa04cwGQ5Pd5l8jMywWfrgsaVNrKO7HiooHdrb6Qi41dyTkAYKRiv9TsW9O2/DYxByJ8T1vQeRFgXx5XgNCBhtPTzqpaGgFrp/eur8h1Gx/4tLw1KIOo9UVQvlKTuL0oj78Bx5cIWcTph6OUISrYTTxXFoe99mtWe+s= password: AgAlTDm/c393DBaWCRE3J5KagWdqCijJaSvx/ThpLa19rGgCu5v9vRc6Y/QIgNSvM47mPIMQZAFz5b9BxpszS5P8ko7tF2n9wpzOpKSeK7e/5lk+hgLL662/PWmpsPrXH8vz2i6vSs16rYXX+5gqvrtgUXpvdOpvRVquwkylRQDSpoL5PkO+SZsfnioOlteLjzu3XZu01gbKssjkKHnBwofFHRuieRAxkDpnQE+jXjqy7t4VWOjN4Cv4SQujuu4MPCgWy9CDYrfpgRwCuHLCBUGZcsgz7mKCPGIRDHJAdFCyCBQS9PQP/AU+C251uUxROMekUUER8zlkP0C3GYrOLvsbWpV/YjyWlUiNzI1UhI8dJuB28VVhLuAHipS7p8ikZ+wKQ7Q+Oaqe2FRcDZSWDdWnBC7XwuSCE6IZK9JX5p+3SwP5rjWAgiitiFcWI7BhfQ0SsGJxkheYr7nkMZkuHMI7iQNz0kRQyp8XjN2i18BzJHqzAokISDwrb2kMT+5RkdX1IUeaybUM6joQkRshlOXNs8oqxEQ3koGVEZxUdG2UGYYvxBndmpbCmhvdX1s/HJvocXg8WFoa+Sd1IchaNVDg5HV+/eO7aHeSnbSRViIYHUktRftQd6fX1H52qiXqiQegsgvNSUPDITfCk+bh9ETvtSJJPUX4d4egA2k6HOr3OmLJ1RbVCoOFKdW9aGztpWiafRGYw4hPPJnrPuswX6w= username: AgCi35nTQHz24dFtzfkNv1+IptOVR9T2vaRTFUd34FFJJgct4U4qtFvRqkM16Y+piJ6ksAo56RMYDCDmmRuUkr+HmFcz6X4TXnVWAr1eI5qkz0+0y7v6BHmUb+Lp7cF3ql6AkKbfxHtf3zItuKm2KkM7vSo+p4BOQ/cKLlXU98+6rl233ZxwG2vXtn18lwI6huTA8rT97zlhdm3PKI1UV3SRjQ4uosDaiaPBMsT3mAl/aEsVrTLzVoHLcxQ50MJIb47dYCk3cEuxe/pzswA/HxvjSSDXGFpqHl2bBvFoedi7F0meaOZG6NHAZrdx3vRW99ShjrOpayGK4HQYuhYe6crAT3smTWQ3Ai8F7NySsQN8NoSc1AP4+ERUwMMVjZ22A2zbQaB4ejwGHLZYLHqF7MKYuFjb0Lqb1rEklW6ghSXKP+6zsycVxVuw6iqvhwDqzeJJLA8SnH/feri5bgXePzWMIrUIIWfxiDoR3ly6016weP90y2XlFX/EYvg5HyxB7lJK0il8RchhJp8rGD2ns6IgURl7WvqyhWkAsbYaL4etum4KLk2da/c+VbqqLFCbR3KxFYSHG7sCt/fBoAAwsr+n+4XaKN1Z9CAH6bT0jVyqDWdu2Fv5ouqAo4XPs408zHT3lHPu0Eq41nuPhTGAFzqvYY2eC91ajXyPt0m3McvRKcBasZxXm85UnhVFNH6mckBCW/7T1gpLfGyLxg== template: data: null metadata: creationTimestamp: null name: mysql-credentials namespace: default type: Opaque ``` and ```yaml= apiVersion: bitnami.com/v1alpha1 kind: SealedSecret metadata: creationTimestamp: null name: paypal-cert namespace: default spec: encryptedData: paypal.crt: AgBXBmnrRExNpDfAtOuu9ESHo5IxgZB3i30Hq9GBGF/fG2b+Wa5OdOSzIwU6gSEsL6F7V7ne0LRtq/2njUaE65TdkVkv8119ATbejmdKRJb9lXNmVLSbdZuVO31Fs7IWgKXK7ZnYhAZQCrfWOrUt6oXQAfbBUJom3wL07h51CCwGOOxEBV6KaCevz4ycwMgRtXqK8quklsaBuDytio31Atno46Tqr8gKlWs7SDriNc7i2q5Z6mZsMR1i/mnufMq9XfEmBUy2UUQcrGvkEk2fcOhruFn4ETNw95sDNTixRrigr1q6dDtOprGPkZ3ZWtY6cnBSkXQx2qtAMRldOpyZCKss3U/SHbDhmmi/omeU9+xjTxdgKg1DnPSLYvgsSkNDSjljl9b79bpsQQ2t0X7d7tjV+k6mEOfi0ckpYTdoe8I/ScRfFsjttPzZxW/9yJLFlKWxKtvatWLErRE2Pv2MBd9qEPoXLpxVgPX2bwRN84XIt759F/ScXXsi1GnloU3Gl3DkQhnwSMQ7ZpATFFSCxo3OsT+U8IWuhHYy2JyfBVkBV9NVpALbJ0owf/29KDffdSyPUzQfiZEjpNkzUqzmbh2hp3YrUuLfyTXbKtXTXGIFgHqrkxOfP6fHfQ+ysqwC/CsOkjwye+LzRltbpVKeuqqd4qfS0oiyg8cvvPaFZ50xBlCu4SFAmparxPzeHuo78KYXxa0aBs9U625TtHBhPOax93hI5kJrK4JlWAwEW2SZkxIHaOByVtz3ytbBp74TL71z/UiBYuGNtjU79fE92YiKyqJC9/bz2j08WS6naQIHFZ/LQtLJ6yfFRPxGh1mSV8xcFp25u9NUnxHZxXfU/oirZLSxiVo9CtJ9tIRcHBc9m9BI5MrBMG4M9OA7scesK8Xgf4oQ3kR2KYA0LOreaXHZkCJ4o2FCYo4j/Sru7xi4EiEc+7zJJGDX8AVdxs1KlDyFxru3emUksnmaafx7xZpLysCxbWhRnyqvcajykWJQzHhhUt2oUE6DabgJvzDVtVT8uQ5SHkeCWrc+SZjZB2QQn11l+t/YNW7f9mD+fJmyPl7eJkZSmRKlyDzg/tfbQSm2HmjL7XOEZEfu3/y2EMocOLa6j1MKu0R0D41SL+hPbhuhPgXK14Oui0ovE8OtXMmV/C6qCuhlzvTCcXMnXbT9IZolk0vOGFghjposWS7THjKTCP+gdRmqgudBGcDtzUgikgHDSBaffk8mPsdpHw04b+p/FKQHVDwLqIFL2b2fSvrEzLwQ0+ZIuUGjstVgmtuqUcOl30+jiJnBbZohPCkF3RRsofXsfxaN3yMC4zoAAgVIWOU33Wc304xLsvUDnutY9idluagUeKtwCA1NRbPxQcfJwO2wukWypqUDiXWU8Ld1pncAyhH2P7enKwBga32iq9y0RY+K/VXhyIScO6RheHyK0/N1QVHKtehobz0laK4KmtpdTRairnBEV4klCKvUnd8CKVEiqQ3Jz2cAZaV18YryZPQUlaHJBGUGGztuUMz4GZik9OL/yaVvi8BQ/4SyigM7wMKhXOy8Kxh/dKuILqbGPgJ1it65rVAz5xjS//LlcwuZQ/X5O/qO36MsIVUomYKglnnytYC6YFi+uCgU56RRa0R7CU6z1Qpf1WmYYY3r3VR892Ty9HGu5PfweSeSeAtjv0tays9ALYZE0xpbuT1Z0AAG+TqYsrC76D7PK7/Z+6csh3mZBSiECH1SLZLIP8PMj35Xu711hWQgKQslzu9bSFpr4NRobz+nkq8FcsadkRgMoz5R6dflJALume8FNr3JUwni/apoiH5AUthh7vnRvQqkXBtvNa1wBOqzARTgS0LzH9uk+5fB4YU0MQdHeMCGBnoIaO06dl16AZ3fy8DNo0SAaUlS/uPd2TFu6ERcwmfTvRGeBmMRflEFKoAfh6kypLeKY4m0FqfLIgWTXJgcWy4NLYydGG+W26AeRdBpujHRRAZpuSyzIMeDrsebB02ziSLVyEKZX36KVFTKusCbayMzonWGBuOM6YO1B0xwBNE/Lxw1H40IWLW8+uJJVX5Cuj9lPFXdUugKwL8uekMA2Y/a0z4vwnQIAgV9cENJ template: data: null metadata: creationTimestamp: null name: paypal-cert namespace: default type: Opaque ``` Now you have encrypted secrets. Open the files in the "Editor" tab and copy the contents in your clipboard. Then go the Github UI in another browser tab and commit/push their contents in your own fork of the application, filling the empty files at `gitops-certification-examples/secret-app/manifests/db-creds-encrypted.yaml` and `gitops-certification-examples/secret-app/manifests/paypal-cert-encrypted.yaml` Once you do this The Sealed secrets controller will automatically decrypt secrets and pass them to your application Now that all our secrets are in Git in an encrypted form we can deploy our application as normal. Login in the Argo CD UI. Click the "New app" button on the top left and fill the following details: application name : demo project: default SYNC POLICY: automatic repository URL: https://github.com//gitops-certification-examples path: ./secret-app/manifests Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: default Leave all the other values empty or with default selections. Finally click the Create button. The application entry will appear in the main dashboard. Click on it. ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: demo spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./secret-app/manifests repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD directory: jsonnet: tlas: - {} project: default syncPolicy: automated: prune: false selfHeal: false ``` The application should now be deployed. Once it is healthy you can see it running in the "Deployed app" tab. Just for illustration purposes, the application is showing the secrets it can access. This way you can verify that the sealed secrets controller is working as expected, decrypting secrets on the fly. ![](https://i.imgur.com/IPzI8h1.png) ## Declarative setup: ArgoCD Application Instead of creating an application via the UI, it's better to do it via files and CLI. For example: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: canary-demo spec: destination: server: 'https://kubernetes.default.svc' namespace: demo project: default source: repoURL: 'https://github.com/kostis-codefresh/summer-of-k8s-app-manifests' path: ./ targetRevision: HEAD ``` This file has the same information as defined from the ArgoCD UI. But since it is a standard Kubernetes resource, you can use all your favorite tools for Kubernetes resources including committing it to Git and managing it with ArgoCD. For the full details of all ArgoCD objects, you should consult https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/ Apart from applications, all other ArgoCD entities have the respective Kubernetes resource such as: Projects Secrets RBAC configuration Git repositories Etc This means that in all scenarios, it is much better to fully adopt GitOps and work exclusively via Kubernetes resources and git commits instead of GUI actions. **The GUI of ArgoCD is best used for verifying and inspecting resources instead of changing them by hand**. Let's do it an exercise where we describe applications as Kubernetes resources. The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/declarative. mine: `https://github.com/drpaneas/gitops-certification-examples/tree/main/declarative` So far in all the previous challenges, you have been creating applications via the ArgoCD UI or CLI. An ArgoCD application is essentially a combination of a git repository, an Argo project, several sync options and other values. This information doesn't need to be confined in Argo CD itself. It can be modeled in a Kubernetes resource so that it can be stored in Git and managed with GitOps as well. Take a look at: https://github.com/drpaneas/gitops-certification-examples/blob/main/declarative/single-app/my-application.yml ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: demo # You'll usually want to add your resources to the argocd namespace. namespace: argocd # Add a this finalizer ONLY if you want these to cascade delete. finalizers: - resources-finalizer.argocd.argoproj.io spec: # The project the application belongs to. project: default # Source of the application manifests source: repoURL: https://github.com/codefresh-contrib/gitops-certification-examples.git targetRevision: HEAD path: ./declarative/manifests # directory directory: recurse: false # Destination cluster and namespace to deploy the application destination: server: https://kubernetes.default.svc namespace: default # Sync policy syncPolicy: automated: # automated sync by default retries failed attempts 5 times with following delays between attempts ( 5s, 10s, 20s, 40s, 80s ); retry controlled using `retry` field. prune: true # Specifies if resources should be pruned during auto-syncing ( false by default ). selfHeal: true # Specifies if partial app sync should be executed when resources are changed only in target Kubernetes cluster and no git change detected ( false by default ). allowEmpty: false # Allows deleting all application resources during automatic syncing ( false by default ). ``` Since our ArgoCD application is now a Kubernetes resource we can handle it like any other kubernetes resource. We have checked out for you locally the file my-application.yml Apply it with: ```shell= $ kubectl apply -f my-application.yml # Output: application.argoproj.io/demo created ``` You should now see the application already in the ArgoCD UI. ![](https://i.imgur.com/i88TQey.png) We can delete the application like any other Kubernetes resource ```shell= kubectl delete -f my-application.yml application.argoproj.io "demo" deleted ``` After a while the application should disappear from the ArgoCD UI. ![](https://i.imgur.com/Dba0Pdp.png) The App-of-Apps pattern allows you to group/deploy multiple applications together! Remember however that the whole point of GitOps is to avoid manual kubectl commands. We want to store this application manifest in Git. And if we store it in Git, we can handle it like another GitOps application! ArgoCD can manage any kind of Kubernetes resources and this includes its own applications (inception). So we can commit multiple applications in Git and pass them to ArgoCD like any other kind of manifest. This means that we can handle multiple applications as a single one. We already have such example at https://github.com/drpaneas/gitops-certification-examples/tree/main/declarative/multiple-apps. ``` In the ArgoCD UI, click the "New app" button on the top left and fill the following details: application name : 3-apps project: default SYNC POLICY: automatic repository URL: https://github.com/codefresh-contrib/gitops-certification-examples path: ./declarative/multiple-apps Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: argocd ``` or in YAML: ```yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: 3-apps spec: destination: name: '' namespace: argocd server: 'https://kubernetes.default.svc' source: path: ./declarative/multiple-apps repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD project: default syncPolicy: automated: prune: false selfHeal: false ``` Notice that the namespace value is the namespace where the parent Application is deployed and not the namespace where the individual applications are deployed. ArgoCD applications must be deployed in the same namespace as ArgoCD itself. Leave all the other values empty or with default selections. Finally click the Create button. ArgoCD will deploy the parent application and its 3 children. Click on the parent application. You should see the following. ![](https://i.imgur.com/4IkrX5U.png) ## Deploy with Helm Helm is a package manager for Kubernetes. It’s a convenient way for packaging collections of YAML files with a Helm chart for the Kubernetes application and allowing distribution with a Helm repository. When using Helm, there are some powerful commands you will find useful. To install a new package: ```shell helm install <release name> <name of chart you want to install> ``` The chart install performs the Kubernetes deployment and service creation of the application. The chart is essentially a collection of files used to describe a set of Kubernetes resources, and Helm manages the creation of these resources. To confirm the deployment and view the currently deployed release: `helm list or helm ls` To view revision numbers for a particular release: `helm history <release name>` To execute a rollback: `helm rollback <release name> <revision id>` To upgrade a release: `helm upgrade <release name> <chart name>` To learn more details about these commands, visit Helm’s [official documentation](https://helm.sh/docs/helm/). Something important to note is that ArgoCD provides **native support for Helm**, meaning you can directly connect a packaged Helm chart and ArgoCD will monitor it for new versions. When this takes place, the Helm chart will no longer function as a Helm chart and instead, is rendered with the Helm template when Argo is installed, using the ArgoCD application manifest. ArgoCD then deploys and monitors the application components until both states are identical. The application is no longer a Helm application and is now recognized as an Argo app that can only operated by ArgoCD. Hence if you execute the helm list command, you should no longer be able to view your helm release because the Helm metadata no longer exists. Here’s an example of the command output. As you can see, the ArgoCD application is NOT detected as a Helm application. ![](https://i.imgur.com/AV3Bx6j.png) Because GitOps is all about Git, we will use Github for this exercise. The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/helm-app Mine: `https://github.com/drpaneas/gitops-certification-examples/tree/main/helm-app` Take a look at the Helm charts to understand what we will deploy. It is a very simple application with one deployment and one service. The Ingress template in the chart is disabled by default. The UI starts empty because nothing is deployed on our cluster. Click the "NEW APP" button on the top left and fill the following details: ``` application name : helm-gitops-example project: default sync policy: automatic repository URL: https://github.com/codefresh-contrib/gitops-certification-examples path: ./helm-app/ Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: default ``` Leave all the other values empty or with default selections. In YAML: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: helm-gitops-example spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./helm-app/ repoURL: 'https://github.com/drpaneas/gitops-certification-examples' targetRevision: HEAD project: default syncPolicy: automated: prune: false selfHeal: false ``` Note that in the case of Helm charts you can also override specific values at the bottom of the dialog. This is completely optional as our Helm chart already has the correct `values.yaml` file. Finally click the Create button. The application entry will appear in the main dashboard. Click on it. Congratulations! Your have setup your first Helm application with GitOps. You should see the following: ![](https://i.imgur.com/RRBDHQW.png) also with CLI: ```shell= # kubectl get deployment NAME READY UP-TO-DATE AVAILABLE AGE helm-gitops-example-helm-example 1/1 1 1 32s ``` Note that if you use any of the helm commands such as `helm list` you will not see anything at all. A Helm chart deployed by ArgoCD no longer registers as a Helm deployment. This is because ArgoCD doesn't include the Helm payload information. When deploying a Helm application, Argo CD runs "helm template" and deploys the resulting manifests. ```shell= # helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION ``` #### You can manage your Argo CD installations either from the Web Interface or using the Command Line. Apart from the UI, ArgoCD also has a CLI. We have installed already the cli for you and authenticated against the instance. Try the following commands: ```shell= # argocd app list NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET helm-gitops-example https://kubernetes.default.svc default default Synced Healthy Auto <none> https://github.com/codefresh-contrib/gitops-certification-examples ./helm-app/ HEAD ``` ```shell root@kubernetes-vm:~/workdir# argocd app get helm-gitops-example Name: helm-gitops-example Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/helm-gitops-example Repo: https://github.com/codefresh-contrib/gitops-certification-examples Target: HEAD Path: ./helm-app/ SyncWindow: Sync Allowed Sync Policy: Automated Sync Status: Synced to HEAD (083f8d3) Health Status: Healthy GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE ServiceAccount default helm-gitops-example-helm-example Synced serviceaccount/helm-gitops-example-helm-example created Service default helm-gitops-example-helm-example Synced Healthy service/helm-gitops-example-helm-example created apps Deployment default helm-gitops-example-helm-example Synced Healthy deployment.apps/helm-gitops-example-helm-example created ``` ```shell root@kubernetes-vm:~/workdir# argocd app history helm-gitops-example ID DATE REVISION 0 2022-01-05 15:32:54 +0000 UTC HEAD (083f8d3) ``` Let's delete the application and deploy it again but from the CLI this time. First delete the app: ```shell= $ argocd app delete helm-gitops-example # Output Are you sure you want to delete 'helm-gitops-example' and all its resources? [y/n] y ``` Confirm the deletion by answering yes in the terminal. The application will disappear from the Argo CD dashboard after some minutes. ![](https://i.imgur.com/hNva1ho.png) Now deploy it again: ```shell= # argocd app create demo \ > --project default \ > --repo https://github.com/codefresh-contrib/gitops-certification-examples \ > --path "./helm-app/" \ > --sync-policy auto \ > --dest-namespace default \ > --dest-server https://kubernetes.default.svc application 'demo' created ``` The application will appear again in the ArgoCD dashboard. Confirm the deployment: ```shell= # kubectl get all NAME READY STATUS RESTARTS AGE pod/demo-helm-example-5546f55d74-hpw6x 1/1 Running 0 23s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 11m service/demo-helm-example ClusterIP 10.43.23.195 <none> 80/TCP 23s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/demo-helm-example 1/1 1 1 23s NAME DESIRED CURRENT READY AGE replicaset.apps/demo-helm-example-5546f55d74 1 1 1 23s ``` ![](https://i.imgur.com/3u2MT6r.png) ## Deploy with Kustomize [Kustomize](https://kustomize.io/) is a CLI configuration manager for Kubernetes objects with a [kustomization.yaml](https://kubectl.docs.kubernetes.io/references/kustomize/glossary/#kustomization) file. It is integrated with `kubectl` and allows you to customize raw, template-free YAML files for multiple purposes declaratively. Kustomize relies on the following configuration layering to enable reusability: 1. Base Layer: This layer specifies the most common resources. 2. Overlays: This layer specifies use-case specific resources by utilizing patches to override other kustomization files and Kubernetes manifests. See below an example of a Kustomize file structure including the base and overlays within an application: ``` ~/demoApp ├── base │ ├── configMap.yaml │ ├── deployment.yaml │ ├── kustomization.yaml │ └── service.yaml └── overlays ├── production │ ├── deployment.yaml │ └── kustomization.yaml └── staging ├── kustomization.yaml └── map.yaml ``` The **base** folder holds common resources, such as the `deployment.yaml`, `service.yaml`, and configuration files. The **overlays** folder houses **environment-specific** overlays, which use **patches** to allow YAML files to be defined and overlaid on **top of the base** for any changes. When using Kustomize, there are some powerful commands you will find useful. You can generate a customized YAML file: `kustomize build <name of application>` This file/resources can then be applied to a cluster: `kustomize build <name of application> | kubectl apply -k or --kustomize` This then creates all 3 resources previously mentioned: `ConfigMap`, `Deployment`, and `Service`. Both Kustomize and ArgoCD are declarative tools for Kubernetes that follow the GitOps pattern and **work well together**. [ArgoCD supports Kustomize](https://argo-cd.readthedocs.io/en/stable/user-guide/kustomize/) and has the ability to read a `kustomization.yaml` file. To deploy Kustomize with ArgoCD, ensure the Kubernetes cluster is set up and you are logged into ArgoCD so that these resources are provided and can be deployed. Create a namespace in the cluster: `kubectl create ns kustomize` Reference the Git repository with the Kustomize file to create an ArgoCD app: `argocd app create <name of application> --repo <repo url> --revision kustomize --path <if you have a separate branch for your kustomize files> --dest-server <server url> --dest-namespace kustomize` Sync deployment managed by ArgoCD: `argocd app sync <application name>` Check status of deployment: `argocd app get <application name>` Each time a new `kustomize.yaml` file is created or the file changes, ArgoCD can detect those and make updates to the deployment. Let's do an exercise. The example application is at https://github.com/codefresh-contrib/gitops-certification-examples/tree/main/kustomize-app Mine: `https://github.com/drpaneas/gitops-certification-examples/tree/main/kustomize-app` Take a look at the base and overlays folders to understand what we will deploy. It is a very simple application with 2 environments to deploy to: staging and production. The environments differ in several aspects such as number of replicas, database used, networking etc. Click the "NEW APP" button on the top left and fill the following details: ``` application name : kustomize-gitops-example project: default sync policy: automatic repository URL: https://github.com/codefresh-contrib/gitops-certification-examples path: ./kustomize-app/overlays/staging Cluster: https://kubernetes.default.svc (this is the same cluster where ArgoCD is installed) Namespace: default ``` in YAML: ```yaml= apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kustomize-gitops-example spec: destination: name: '' namespace: default server: 'https://kubernetes.default.svc' source: path: ./kustomize-app/overlays/staging repoURL: 'https://github.com/codefresh-contrib/gitops-certification-examples' targetRevision: HEAD project: default syncPolicy: automated: prune: false selfHeal: false ``` Leave all the other values empty or with default selections. Finally click the Create button. The application entry will appear in the main dashboard. Click on it. Congratulations! Your have setup your first Kustomize application with GitOps. You should see the following. ![](https://i.imgur.com/dVm5JTc.png) Wait some time for the application to start up and then visit the "Deployed App" to see it running. You will see that it has loaded the staging value for the database. It says: > Mysql is at staging-mysql.example.com:3306 #### You can manage your Argo CD installations either from the Web Interface or using the Command Line. Apart from the UI, ArgoCD also has a CLI. We have installed already the cli for you and authenticated against the instance. Try the following commands: ```shell= argocd app list NAME CLUSTER NAMESPACE PROJECT STATUS HEALTH SYNCPOLICY CONDITIONS REPO PATH TARGET kustomize-gitops-example https://kubernetes.default.svc default default Synced Healthy Auto <none> https://github.com/codefresh-contrib/gitops-certification-examples ./kustomize-app/overlays/staging HEAD ``` ```shell= root@kubernetes-vm:~/workdir# argocd app get kustomize-gitops-example Name: kustomize-gitops-example Project: default Server: https://kubernetes.default.svc Namespace: default URL: https://localhost:30443/applications/kustomize-gitops-example Repo: https://github.com/codefresh-contrib/gitops-certification-examples Target: HEAD Path: ./kustomize-app/overlays/staging SyncWindow: Sync Allowed Sync Policy: Automated Sync Status: Synced to HEAD (083f8d3) Health Status: Healthy GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE ConfigMap default staging-the-map Synced configmap/staging-the-map created Service default staging-demo Synced Healthy service/staging-demo created apps Deployment default staging-the-deployment Synced Healthy deployment.apps/staging-the-deployment created ``` ```shell= root@kubernetes-vm:~/workdir# argocd app history kustomize-gitops-example ID DATE REVISION 0 2022-01-05 17:55:59 +0000 UTC HEAD (083f8d3) ``` Let's delete the application and deploy it again but from the CLI this time. First delete the app: ```shell= # argocd app delete kustomize-gitops-example Are you sure you want to delete 'kustomize-gitops-example' and all its resources? [y/n] y ``` Confirm the deletion by answering yes in the terminal. The application will disappear from the Argo CD dashboard after some minutes. Now deploy it again.: ```shell= # argocd app create demo \ > --project default \ > --repo https://github.com/codefresh-contrib/gitops-certification-examples \ > --path ./kustomize-app/overlays/staging \ > --sync-policy auto \ > --dest-namespace default \ > --dest-server https://kubernetes.default.svc application 'demo' created ``` Confirm the deployment: ``` # kubectl get all NAME READY STATUS RESTARTS AGE pod/staging-the-deployment-85c5c7685f-lfzc8 1/1 Running 0 12s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 17m service/staging-demo NodePort 10.43.118.58 <none> 8080:31000/TCP 12s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/staging-the-deployment 1/1 1 1 12s NAME DESIRED CURRENT READY AGE replicaset.apps/staging-the-deployment-85c5c7685f 1 1 1 12s ``` And it's up and running: ![](https://i.imgur.com/uAp4Uy1.png) ### Questions If you have enabled the "auto-sync" option in an ArgoCD application, and something is changed manually in the cluster then ArgoCD will automatically discard the change. A: False. Unless you have also setup the "self-heal" option, ArgoCD will never discard manual changes If you have enabled the "auto-sync" option in an ArgoCD application, and something you delete a resource in Git, then ArgoCD will automatically delete that resource from the cluster as well. A:False. Unless you have also setup the "auto-prune" option, ArgoCD will never remove resources from the live cluster What is the proper way to handle application secrets via GitOps? A: Encrypt them and store them in Git. Then decrypt them during runtime. If you use Bitnami sealed secrets then where does encryption and decryption take place. A: Encryption happens via the kubeseal executable. Decryption happens via the Sealed Secrets controller. You have just logged in the ArgoCD UI and created an application using a Git repository that holds your Helm chart. You sync the application and everything is fine. What is the next step that you should take? A: Create a declarative file of the application and other resources (e.g. ArgoCD project) used and store them in Git. You just created a Helm application using the Argo CD web interface. Now you go the command line and you enter "helm list". To your surprise nothing is printed. A: The helm command will never work no matter what you do in ArgoCD. ArgoCD uses Helm only as a templating mechanism.