# Deployment Documentation

Date: 2022-02-21

## Purpose
The primary objective is to automate the deployment process of the CNC (Computer Numerical Control) application onto a Kubernetes (k8s) cluster. This automation aims to streamline the deployment pipeline, reduce manual intervention, and ensure consistency across environments.

The deployment scripts and configuration files, particularly those within cnc/deployment/, have been updated to reflect the latest software versions and command syntax. This includes ensuring compatibility with current Kubernetes API versions and addressing any deprecated commands or configurations.

## Prerequisites
Before initiating the deployment process, ensure the following environment variables are set to avoid runtime errors and deployment failures.

### Environment Variables

1. **Google Cloud Credentials**
The GOOGLE_APPLICATION_CREDENTIALS environment variable must point to the path of your GCP service account key file. This service account should have the necessary permissions to interact with GCP resources, such as creating clusters, deploying applications, and managing secrets.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/gcp-service-account-key.json

2. Cronjob Execution Flag
The IS_CRONJOB environment variable must be set to 1 to ensure that cronjobs are properly configured and executed within the Kubernetes cluster. This variable affects the content generated in gcp_config and subsequently influences the behavior of `entrypoint.sh`.

export IS_CRONJOB=1

## Updated Files and Comments

This section details the specific updates made to the deployment scripts and configuration files, along with the rationale behind each change.

==`cnc/deployment/`==

- `deploy.sh`
The deployment script `deploy.sh` has been updated to focus on Kubernetes deployment. Services such as MySQL and NFS are not started within this script to simplify the deployment process and focus on container orchestration.

- cnc.tmpl.yaml
- Added deployment.spec.selector
> The selector field is mandatory in Kubernetes versions >= 1.16 and is used to match pods with the deployment.

        selector:
          matchLabels:
            app: cnc

- Warning: It is critical to ensure that nodes within the cluster are appropriately labeled. Specifically, at least one node must have the label `role=static-node`. Without this label, the Kubernetes scheduler will not assign pods to nodes, leading to unscheduled pods and failed deployments.
- To label a node:

        kubectl label nodes <node-name> role=static-node

- Service Exclusions
NFS and Fluentd services are not started during testing. Consequently, related configurations in the deployment files are commented out. Ensure these services are properly configured and uncommented in production environments where they are required.

- Secret Configuration
- Secret Volume: The email configurations are stored in `smtp_auth_file.yaml`. The environment variable `$EMAIL_CONF` should be set to the path of this file.
- `export EMAIL_CONF=/path/to/smtp_auth_file.yaml`
- Creating the Secret: Use the following command to create the secret in Kubernetes: `kubectl create secret generic email-account --from-file="$EMAIL_CONF" --dry-run=client -o yaml | kubectl apply -f -`

- `entrypoint.tmpl.sh`
No updates were made to this file. However, proper configuration of environment variables, particularly `$IS_CRONJOB`, is necessary. Misconfiguration may cause `entrypoint.sh` to fail, disrupting the application startup process.

Ensure that `cnc.tmpl.yaml` is configured to pass the correct environment variables to the container.

==`parent/deployment/gcloudenv/`==`

- `gcloud.sh`
This script is responsible for building and pushing Docker images to Google Container Registry (GCR). The `$VERSION` variable on line 6 is set using git describe to tag the Docker image with the current Git commit hash or tag.
`VERSION= $(git describe --tags --always)`

- `Dockerfile`
Updated to include the installation of the latest versions of kubectl and the Google Cloud SDK. This ensures compatibility with current GCP services and Kubernetes clusters.
- Installing kubectl:

        RUN curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" \
            && chmod +x ./kubectl \
            && mv ./kubectl /usr/local/bin/kubectl

- Installing Google Cloud SDK:

        RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list \
            && apt-get update -y \
            && apt-get install -y google-cloud-sdk

- `authenv.sh`
Updated the command to activate the service account. This script ensures that the environment within the Docker container or CI/CD pipeline has the necessary authentication to interact with GCP resources.

gcloud auth activate-service-account --key-file="$GOOGLE_APPLICATION_CREDENTIALS"

==`parent/deployment/k8s/`==
- `up.sh`
The cluster creation command `gcloud container clusters create` has been updated to reflect the latest syntax and options. This includes specifying the Kubernetes version, node configurations, and network settings.

gcloud container clusters create my-cluster \
  --zone us-central1-a \
  --cluster-version latest \
  --num-nodes 3 \
  --machine-type n1-standard-1

- Kubernetes Manifests Updates
- `default-backend.yaml`
Updated apiVersion from deprecated versions to apps/v1 for the Deployment resource.
- `rbac.yaml`
Updated apiVersion to `rbac.authorization.k8s.io/v1` to use the stable RBAC API.
- `static-ip-assign.yaml`
Updated the apiVersion for CronJob resources to `batch/v1`, as older versions like `batch/v1beta1` are deprecated in newer Kubernetes releases.
- `with-rbac.yaml`
- Updated the apiVersion for Deployment resources to `apps/v1`.
- Code Review Notes: Lines 134-150 have been commented out due to overdesign. These lines include configurations that are not necessary or may complicate the deployment. It is essential to simplify and only include necessary configurations to maintain clarity and reduce potential errors.

## Deployment Insights and Potential Issues

Node Labeling
- Issue: Pods remain in a Pending state due to the absence of nodes matching the specified label selector.
- Analysis:
Kubernetes uses labels and selectors to determine where to schedule pods. If the deployment specifies a node selector (e.g., role=static-node), but no nodes have this label, the pods cannot be scheduled.
- Solution:
Verify node labels:

kubectl get nodes --show-labels

Label the node:

kubectl label nodes <node-name> role=static-node

## Environment Variables
- Issue: Missing or incorrect environment variables can cause authentication failures or prevent cronjobs from running.
- Analysis:
- `GOOGLE_APPLICATION_CREDENTIALS` is required for GCP authentication.
- `IS_CRONJOB` determines whether cronjobs are set up correctly.
- Solution:
- Confirm that all environment variables are exported and accessible within the deployment scripts.

## Secrets Management

- Issue: Applications fail due to missing or incorrect secrets.
- Analysis:
- Sensitive data, such as email configurations, must be stored securely and made available to pods via Kubernetes secrets.
- Solution:
- Ensure the secret is created before deploying the application.
- Verify that the secret is correctly referenced in the deployment manifest.
- Use `kubectl describe secret <secret-name>` to confirm its contents.

## Deprecated Commands and API Versions

- Issue: Deployment scripts fail due to the use of outdated commands or API versions.
- Analysis:
- Kubernetes frequently deprecates older API versions in favor of newer, more stable ones.
- Solution:
- Regularly review and update scripts to use the latest API versions.
- Reference the Kubernetes API deprecation guide for version-specific changes.

## Service Account Activation

- Issue: Authentication errors when interacting with GCP services.
- Analysis:
- The service account may not be properly activated, or the key file may be invalid or inaccessible.
- Solution:
- Ensure the key file exists and the path specified in `GOOGLE_APPLICATION_CREDENTIALS` is correct.
- Activate the service account using `gcloud auth activate-service-account`.
- Test authentication by running a simple gcloud command, such as gcloud projects list.

## Additional Deployment Considerations

### Cluster Configuration

- Ensure the Kubernetes cluster meets the application’s resource requirements, including CPU, memory, and storage.
- Configure autoscaling if necessary to handle varying workloads.

### Networking

- Verify that network policies allow for proper communication between pods and services.
Configure ingress controllers and load balancers for external access.

### Logging and Monitoring

- Integrate logging solutions, such as Fluentd or Stackdriver, to collect logs from pods.
- Set up monitoring tools to track application performance and cluster health.

### Security

- Implement RBAC to control access to Kubernetes resources.
- Use network policies to restrict traffic between namespaces.
- Regularly scan images for vulnerabilities.

# Summary

Automating the Kubernetes deployment process for the CNC application requires careful attention to detail in several areas:

- Environment Preparation
- Set all required environment variables.
- Ensure proper authentication with GCP.
- Script and Configuration Updates
- Use current software versions compatible with Kubernetes and GCP.
- Update deprecated commands and API versions.
- Simplify deployment manifests to include only necessary configurations.
- Testing and Validation
- Perform thorough testing in a staging environment.
- Validate configurations using kubectl commands.
- Monitor logs for errors and warnings during deployment.