changed 4 years ago  
Linked with GitHub

DEX Certificate Generated by Cert Manager

This document describes the design for supporting the creation of Dex certificates with Cert Manager.

The use of Dex for K8S Authentication requires the API server and Dex pod to share the same certificate i.e., tls.crt.

Approach 1 - CAPI v1alpha3 (Short-term)

Proposal for short-term in case CAPI v1alpha4 is not considered stable to be integrated with Airship 2.1.

Ephemeral Cluster - OIDC flags and Certificate for API Server

In the Airship 2 environment, the API server is deployed in the Target cluster through CAPI Management (a.k.a., Ephemeral) cluster. At this point OIDC flags and Dex certificate MUST be available, meaning cert-manager is invoked to generate the Certificate Secret during the deployment of Target cluster's ControlPlane node(s),i.e., airshipctl phase run controlplane-ephemeral.

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

NOTE: Cert Manager generates a certificate in the default namespace so it can be mounted by the KubeadmControlPlane resources.

Target Cluster - Dex Certificate from Ephemeral Cluster

The (Certificate) Secret used by API server for OIDC integration was created in the Ephemeral cluster and need to be "replicated" in the Target cluster. Cluster API provides a mechanism for that. By adding the label 'clusterctl-cluster.x-k8s.io/move: "true"' to the Secret in the Ephemeral cluster, it tags this object to be moved from one CAPI Management cluster to another, which in this case Ephemeral cluster to Target cluster.

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Now that we have the (Certificate) Secret moved to the Target cluster, this Secret can be mounted by the dex-aio PoDs.

Caveat - Dex MUST be deployed in the default namespace because Secret moved from Ephemeral cluster was created in this namespace.

Approach 2 - CAPI v1alpha3 - Alternative (Short-term)

Hardcode Secrets for KubeadmControlPlane (Ephemeral cluster) and Dex (Target cluster), where certificates are manually copied into their respective Secret objects.

In this scenario Dex can be deployed in any namespace, e.g., dex.

CAPI v1alpha3 - Non Viable Alternative (FYI only)

In this scenario KubeadmControlPlane is "configured" with OIDC flags and dummy certificates. The dummy certificates is necessary so deployment of control plane node succeeds. Without it the deployment waits until Secret expected by KubeadmControlPlane is created.

After moving the CAPI management resources to the Target cluster the "real" Dex certificates (same Secret name as the dummy Secret) is generated by the Cert Manager. At this point the expectation was that KubeadmControlPlane resource would determine that Secret content has changed and load the new values, which didn't happen.

Without KubeadmControlPlane resource mounting the new Secret this approach become non viable. This approach was documented so the reader understands what works and doesn't work.

Approach 3 - CAPI v1alpha4 (Long-term?)

With CAPI v1alpha4 it became possible to patch KubeadmControlPlane resource, meaning that we can add extraArgs and extraVolumes to the API server.

In this scenario, certificates for API server and Dex do not need to be generated in the Ephemeral cluster and API server can be "reconfigured" with OIDC plugin only when Dex is deployed.

Certificates shared by API server and Dex is generated by Cert-Manager in the Target cluster.

Deploying Dex

Target cluster's cert-manager is used to generate the Dex (certifcate) Secret and then API server is configured with OIDC flags and certificate from the Dex Secret.
Then Dex is deployed, which will also use the Dex Secret to access the certificates.

This approach is cleaner and maintainable. API server is only "configured" with OIDC flags and certifcate (kubectl patch kubeadmcontrolplane) when Dex is deployed as opposed to Approach 1/2, which certificates and API server have been configured pre-control plane creation.

In the case of certificate rotation, the API server and Dex Pods should be automatically updated (rolling deployment).

Caveat - As for first approach, Dex MUST be deployed in the default namespace because KubeadmControlPlane resource lives in this namespace.
Alternative - use Kubed to maintain a copy of Dex Secret in a dedicated namespace, e.g., dex

[TODO] Validate last assertion.

As of today, there is no clusterctl command for applying the patch, therefore kubectl patch command must be used.

[QUESTION] Should kubectl patch be "wrapped" with an airshipctl command?

Select a repo