# Ingress Controllers Implementation Differences  The behavior "Can act as a default controller" is when the controller picks up Ingresses that don't have an annotation nor the ingressClassName. The behavior "Ignores ingressClassName when acting as default controller" is when an ingress controller picks up Ingress resources that don't have the annotation kubernetes.io/ingress.class even though an ingressClassName has been set field. The ingress controller is not aware that the newer ingressClassName field won't play well if you intend to use ingressClassName. The following table summarizes each implementation's behavior. This table is based on reading each implementation's documentation and does not come from testing individual implementations. Ideally, the table should come out from the existing project [ingress-controller-conformance](https://github.com/kubernetes-sigs/ingress-controller-conformance/tree/master/features). **Last updated:** 14 March 2023 | Ingress controller | Last checked | Can act as a default controller | Ignores ingressClassName when acting as default controller | Supports ingressClassName matched by "spec.controller" | IngressClass must exist when ingressClassName is used? | |------------------------------------------|-------------------|---------------------------------|------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------| | Istio | 1.17.0 | yes | no | yes | yes | | Traefik | 2.9.8 | no | n/a | yes | yes | | Contour | 1.24.1 | yes (by default) | no | yes | no | | Gloo Edge Ingress Controller | 1.13.7 | yes (by default) | no | yes | no | | ingress-nginx (>v1.0.1) | 1.6.4 | yes (not by default) | no | yes | yes (but helm creates an IngressClass) | | NGINX Ingress (by NGINX Inc.) | 3.0.2 | no | n/a | yes | yes (but helm creates an IngressClass) | | Azure AGIC (>=1.5.0) | 1.5.4 | yes (not by default) | no | yes (not by default, requires an env var) | yes | | ingress-gce | 1.8.0 | yes (always) | no | no ([details][ingress-gce]) | n/a | | skipper | 0.15.26 | yes (by default) | no | yes | no | | AWS Load Balancer Controller | 2.4.0 | yes (by default) | no | yes | yes (but helm creates an IngressClass) | | Ambassador's Emissary Ingress Controller | 3.5.0 | no | n/a | yes | yes | | Apache APISIX Ingress controller | 1.6.0 | yes (by default) | no | yes | no | | Kong Ingress controller | 2.8.1 | no | n/a | yes | yes (but helm creates an IngressClass) | | Pomerium | 0.21.2 | yes (not by default) | no | yes | yes | | Cilium | 1.14.0-snapshot.0 | yes (not by default) | no | yes | yes (but helm creates an IngressClass) | - Istio and Traefik do not use the "default controller" behavior and operate with the "only matching annotation" behavior. The annotation does not need to match an IngressClass resource name. The ingressClassName requires the presence of an IngressClass with that name. Traefik's Helm chart does not create a default IngressClass, and istioctl install doesn't either. Istio does not allow you to change the annotation value, it is fixed to "istio". Traefik allows you to change the default matching annotation with --providers.kubernetesingress.ingressclass. - Contour uses the "default controller" behavior unless the flag [--ingress-class-name](https://projectcontour.io/docs/main/config/ingress/#ingressclass-and-ingressclass-name) is passed in which case the controller changes to the "annotation" behavior. The annotation does not need a matching IngressClass. The ingressClassName field does not need to match an IngressClass resource name either. - Gloo uses the "default controller" behavior, unless REQUIRE_INGRESS_CLASS=true is set in which case the controller has the "only matching annotation" behavior. The annotation does not need a matching IngressClass. The ingressClassName field does not need to match an IngressClass resource name. - ingress-nginx (v1.0.1+) has the "only matching annotation" behavior by default, and the "default controller" behavior can be turned on with [--watch-ingress-without-class=true](https://kubernetes.github.io/ingress-nginx/#what-is-the-flag-watch-ingress-without-class). The annotation does not require an IngressClass to exist. The ingressClassName field requires an IngressClass resource to exist. The Helm chart creates an IngressClass nginx by default. It is possible to customize the annotation used to pick up Ingresses with --ingress-class (this flag does not affect which IngressClass gets picked and doesn't have anything to do with IngressClass). Note that running two instances of ingress-nginx with two different controllerName strings e.g., --controller-class=k8s.io/ingress-nginx-internal and --controller-class=k8s.io/ingress-nginx-external, only works since v1.1.2 (cf [ingress-nginx#7546](https://github.com/kubernetes/ingress-nginx/issues/7546)). - ingress-nginx (< v1.0.0) is used when using Kubernetes 1.18 or lower since it only supports the Ingress v1beta1. It uses the "default controller" behavior. - NGINX Ingress (by NGINX Inc.) has neither the "default controller" nor the "only matching annotation" behavior since it [only supports](https://github.com/nginxinc/kubernetes-ingress/pull/1133) the ingressClassName on Kubernetes 1.18 and above. The ingressClassName field requires the name of an existing IngressClass. The Helm chart creates an IngressClass "nginx" by default. - Azure AGIC 1.4.0 and below does not use the "default controller" behavior. It operates with the "only matching annotation" behavior. It [does not](https://github.com/Azure/application-gateway-kubernetes-ingress/issues/1122) support the ingressClassName field. Another issue with AGIC is that its default class name is azure/application-gateway, which is fine for the annotation but not fine for ingressClassName. - With Azure AGIC 1.5.0 and above, the annotation is supported. The default annotation value is "azure/application-gateway". The field ingressClassName is supported when INGRESS_CLASS_RESOURCE_ENABLED=true. When you install the Helm chart, the default IngressClass is named azure-application-gateway.  You can enable the "default controller" behavior by creating an IngressClass and using INGRESS_CLASS_RESOURCE_ENABLED=true and set INGRESS_CLASS_RESOURCE_CONTROLLER to something. - <a id="ingress-gce"></a> ingress-gce acts as a default even for Ingresses that have the ingressClassName field set. The ingressClassName field is [still unsupported](https://github.com/kubernetes/ingress-gce/issues/1301). Although ingress-gce does not support ingressClassName, it ignores Ingresses that have the field ingressClassName set on them in order to work well with Istio and other ingressClassName-enabled implementations ([source](https://github.com/kubernetes/ingress-gce/pull/1308/files#r513820404)). There is no way to change the annotation value that ingress-gce uses to pick up Ingresses (ie "gce" and "gce-internal"). - Skipper uses the "default controller" behavior unless the flag [--kubernetes-ingress-class](https://opensource.zalando.com/skipper/kubernetes/ingress-controller/#multiple-skipper-deployments) is passed in which case the controller changes to the "only matching annotation" behavior. The annotation does not need a matching IngressClass. The ingressClassName field does not need to match an IngressClass resource name either. - AWS Load Balancer Controller (kube-ingress-aws-controller) acts as a default controller unless ingressClassName is set. If the flag [--ingress-class-filter](https://github.com/zalando-incubator/kube-ingress-aws-controller#running-multiple-instances) is passed, The  ingressClassName field does not need to match an IngressClass resource name. You can use a "default IngressClass" by setting the annotation ingressclass.kubernetes.io/is-default-class ([source](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/ingress_class/)). - Ambassador Edge Stack does not act as a "default controller" and runs with the "only matching annotation" behavior. The annotation does not need a matching IngressClass. The ingressClassName field needs to match an IngressClass resource name. Unrelated, but Ingress v1 is not supported yet. Ambassador doesn't allow you to customize the annotation value used for picking up Ingresses. - [Apache APISIX Ingress controller](https://github.com/apache/apisix-ingress-controller/)  - Kong Ingress Controller doesn't use the "default controller" behavior. It behaves in "only matching annotation". The flag --ingress-class or CONTROLLER_INGRESS_CLASS environment variable lets you change the default "kong" annotation. - Pomerium can act as a "default controller" by setting an annotation on an IngressClass. Pomerium [supports](https://www.pomerium.com/docs/deploying/k8s/ingress#cert-manager-integration) the annotation "kubernetes.io/ingress.class: pomerium", maybe just for working around cert-manager's limitation. It seems the annotation is used the same way as ingressClassName, and requires an IngressClass with that name. Pomerium's Helm chart creates an IngressClass called "pomerium". Pomerium picks up IngressClasses that match the spec.controller which is configured with --name=pomerium.io/ingress-controller-one.
Last changed by  
 
Maël Valais
44

Read more

The slight differences between Unix tools across BSD (incl. macOS), GNU/Linux, and Busybox

This is a list of discrepencies I came across while trying to while cross-systemcompatible makefiles and Bash scripts.

5/13/2024
Fixing "etcdserver: mvcc: database space exceeded" (kind cluster)

docker exec -it kind-control-plane sh -c 'crictl ps -name etcd -o json | jq "."'

4/29/2024
Reproducing cert-manager’s serviceAccountRef with Out-of-Cluster Vault Without Setting token_reviewer_jwt

Tested on macOS with Docker running on a VM created by limactl. It shouldalso work on macOS with Docker Desktop, but also on Linux. The reason we runVault in a container instead of locally is because it needs to be on the sameVM as the kind cluster so that cert-manager can reach Vault, and Vault canreach kube-apiserver.

2/7/2024
Why the Kubernetes Auth Isn't Appropriate when Operating a Central Vault Instance, and Why the Vault JWT Auth Makes More

Why the Kubernetes Auth Isn’t Appropriate when Operating a Central Vault Instance, and Why the Vault OIDC Auth Makes More Sense

2/7/2024
Read more from Maël Valais
Published on HackMD