# Configuring Let's Encrypt certificate on attached clusters

This is a guide on how to configure attached cluster with Let's Encrypt issued certificate for Traefik, so that the cluster is correctly recognized and configured by the management cluster.

Basic configuration variables:

# Path to management cluster kubeconfig file
export KUBECONFIG=/path/to/the/management-cluster-kubeconfig

# Workspace name where the new cluster will be attached
export WORKSPACE_NAMESPACE=acme

# Name of the newly atttached cluster
export ATTACHED_CLUSTER_NAME=acme-test

# DNS name that will be used for attached cluster
export CLUSTER_DOMAIN_NAME="attached.ksphere-platform.d2iq.cloud"

## Attach the cluster without the Let's Encrypt certificate

**CLUSTER CONTEXT:** `management`

Create new workspacels and attach a cluster:

./dkp create workspace $WORKSPACE_NAMESPACE -n $WORKSPACE_NAMESPACE
./dkp attach cluster -n "$ATTACHED_CLUSTER_NAME" --attached-kubeconfig /path/to/kubeconfig -w $WORKSPACE_NAMESPACE

## Apply a ClusterIssuer to your attached cluster

**CLUSTER CONTEXT:** `attached`

*NOTE:* Reuse the cluster issuer from your management cluster if you are using the same base DNS name.

Create a new ACME `ClusterIssuer` for Let's Encrypt certificate. This particular `ClusterIssuer` is configured to complete ACME challenge `DNS01` challenge.

export ACME_EMAIL=[e-mail]
export AWS_REGION=us-west-2
export EXTERNAL_DNS_ROUTE53_ZONE_ID=Z36CN7WXPDV8VR
export EXTERNAL_DNS_AWS_ASSUME_ROLE=arn:aws:iam::999867407951:role/cert-manager-dns01

cat << EOF | kubectl apply -f -
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: kommander-acme-issuer
spec:
  acme:
    email: "${ACME_EMAIL}"
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: kommander-acme-issuer-account
    solvers:
      - dns01:
          route53:
            region: "${AWS_REGION}"
            hostedZoneID: "${EXTERNAL_DNS_ROUTE53_ZONE_ID}"
            role: "${EXTERNAL_DNS_AWS_ASSUME_ROLE}"
EOF

## Disable Kommander automatically created certificate to update secret containing the certificate.

**CLUSTER:** `attached`

*NOTE:* This is a same operation that `kommander-cli` does on `management` cluster when installed with Let's Encrypt certificate configuration.

cat << EOF | kubectl -n $WORKSPACE_NAMESPACE patch certificate kommander-traefik --type='merge' --patch-file=/dev/stdin
---
spec:
  issuerRef:
    name: no-op
EOF

Create a new traefik `Certificate` issued by the newly created ACME issuer:

cat << EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: kommander-traefik-acme
  namespace: ${WORKSPACE_NAMESPACE}
spec:
  secretName: kommander-traefik-certificate
  duration: 19200h # 800d
  commonName: "${CLUSTER_DOMAIN_NAME}"
  isCA: false
  usages:
    - server auth
  dnsNames:
    - "${CLUSTER_DOMAIN_NAME}"
  issuerRef:
    name: kommander-acme-issuer
    kind: ClusterIssuer
EOF

Configure new domain name and certificate in `konvoyconfig-kubeaddons` on the attached cluster.

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
    name: konvoyconfig-kubeaddons
    namespace: ${WORKSPACE_NAMESPACE}
data:
    clusterHostname: "${CLUSTER_DOMAIN_NAME}"
    caSecretName: kommander-traefik-certificate
EOF

## Trigger update on management cluster

**CLUSTER:** `management`

This will manually trigger the updates on the management cluster that will reconfigure its components to use the newly configured domain on attached cluster.

cat << EOF | kubectl -n $WORKSPACE_NAMESPACE patch kommandercluster $ATTACHED_CLUSTER_NAME --type='merge' --patch-file=/dev/stdin
---
metadata:
  annotations:
    d2iq-update: "`date +%s`"
EOF

DNS per attached cluster

# Per cluster External DNS

`external-dns` is a component that manages DNS records in public DNS providers (like AWS Route53) and creates a DNS records for `LoadBalancer` services based on the configured annotations on the `Service` objects. This feature is useful when provisioning clusters with custom domains that should be automatically managed by the cluster itself.

This document explains on how to create a unique domain configuration per each cluster in a single Kommander Workspace.

## Unique DNS domain for each cluster

**CLUSTER:** `management`

Create Traefik configuration override in the workspace. Having these overrides allows to deploy per cluster configuration for `external-dns` application.

These commands should be executed only once per Workspace.

# Workspace name where the clusters are attached.
export WORKSPACE_NAMESPACE=acme

cat << EOF | kubectl -n $WORKSPACE_NAMESPACE patch appdeployment traefik --type='merge' --patch-file=/dev/stdin
spec:
  configOverrides:
    name: traefik-config-overrides
EOF

export TRAEFIK_UID=$(kubectl -n $WORKSPACE_NAMESPACE get appdeployment traefik -o jsonpath='{.metadata.uid}')
cat << EOF | kubectl apply -f -
apiVersion: v1
data:
  values.yaml: '# NOOP'
kind: ConfigMap
metadata:
  labels:
    apps.kommander.mesosphere.io/federated-config-map-name: traefik-overrides
  name: traefik-config-overrides
  namespace: ${WORKSPACE_NAMESPACE}
  ownerReferences:
  - apiVersion: apps.kommander.d2iq.io/v1alpha2
    kind: AppDeployment
    name: traefik
    uid: ${TRAEFIK_UID}
EOF

Configuring unique domain name per every attached cluster.

export CLUSTER_DOMAIN_NAME="attached.ksphere-platform.d2iq.cloud"
export ATTACHED_CLUSTER_NAME="cluster-01"

cat << EOF | kubectl -n $WORKSPACE_NAMESPACE patch fcm traefik-overrides --type="merge" --patch-file=/dev/stdin
---
spec:
  overrides:
  - clusterName: "${ATTACHED_CLUSTER_NAME}"
    clusterOverrides:
    - path: "/data"
      value:
        values.yaml: |
          service:
            annotations:
              external-dns.alpha.kubernetes.io/hostname: "${CLUSTER_DOMAIN_NAME}"
EOF

## External DNS

**CLUSTER:** `management`

Create `external-dns` `AppDeployment` in the workspace. This will deploy the `external-dns` each cluster in the workspace.

export EXTERNAL_DNS_VERSION=6.1.8

cat << EOF | kubectl apply -f -
apiVersion: apps.kommander.d2iq.io/v1alpha2
kind: AppDeployment
metadata:
  name: external-dns
  namespace: ${WORKSPACE_NAMESPACE}
spec:
  appRef:
    kind: ClusterApp
    name: external-dns-${EXTERNAL_DNS_VERSION}
  configOverrides:
    name: external-dns-config-overrides
EOF

Applying this configuration assumes that every attached cluster can assume given AWS IAM role and will manage DNS records in the shared DNS zone.

export EXTERNAL_DNS_AWS_REGION=us-west-2
export EXTERNAL_DNS_AWS_ASSUME_ROLE=arn:aws:iam::999867407951:role/cert-manager-dns01
export EXTERNAL_DNS_DOMAIN_FILTER=ksphere-platform.d2iq.cloud
export EXTERNAL_DNS_UID=$(kubectl -n $WORKSPACE_NAMESPACE get appdeployment external-dns -o jsonpath='{.metadata.uid}')

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    apps.kommander.mesosphere.io/federated-config-map-name: external-dns-overrides
  name: external-dns-config-overrides
  namespace: ${WORKSPACE_NAMESPACE}
  ownerReferences:
  - apiVersion: apps.kommander.d2iq.io/v1alpha2
    kind: AppDeployment
    name: external-dns
    uid: ${EXTERNAL_DNS_UID}
data:
  values.yaml: |
    aws:
      region: "${EXTERNAL_DNS_AWS_REGION}"
      assumeRoleArn: "${EXTERNAL_DNS_AWS_ASSUME_ROLE}"
      preferCNAME: true
    policy: upsert-only
    txtPrefix: local-
    domainFilters:
      - "${EXTERNAL_DNS_DOMAIN_FILTER}"
EOF