### where we are at (status-quo)

- we are not able to generate TLS certificate with letsencrypt. How does we handle that? with [CertManager](https://cert-manager.io/) - E.g: [doc](https://www.howtogeek.com/devops/how-to-install-kubernetes-cert-manager-and-configure-lets-encrypt/)

1. we use
- eks 1.24
- jetstack/cert-manager helm chart --version 1.5.3
- ingress-nginx/ingress-nginx 1.7.1

2. We have two ingressClasses:
2.1 - one **welance-develop** with the ELB [DEV](abb8fb0a1078041f38d76b51de9289c7-8e6adbf6767e08d6.elb.eu-central-1.amazonaws.com)
2.2 - one **welance-staging** with the ELB [STAGING](a291aa86e97624c20bf0e84774fa2639-5116e86c535841c7.elb.eu-central-1.amazonaws.com)

we registered dns in cloudflare as ELB hostname we got from EKS so

1. all *.dev.welance.com -> welance-develop nginx ingress controller [2.1]
2. all *.staging.welance.com -> welance-staging nginx ingress controller [2.2]

we deployed
- dev http [ingress](https://gitlab.com/welance/platform/infrastructure/welance-aws/-/blob/develop/kubernetes/test-ingress-generation/ingress_develop_http.yaml)

1) http://http.dev.welance.com/hello

- dev https [ingress](https://gitlab.com/welance/platform/infrastructure/welance-aws/-/blob/develop/kubernetes/test-ingress-generation/ingress_develop_tls.yaml)

1) https://https.dev.welance.com/hello

- staging http [ingress](https://gitlab.com/welance/platform/infrastructure/welance-aws/-/blob/develop/kubernetes/test-ingress-generation/ingress_staging_http.yaml)

1) http://http.staging.welance.com/hello

- staging https [ingress](https://gitlab.com/welance/platform/infrastructure/welance-aws/-/blob/develop/kubernetes/test-ingress-generation/ingress_staging_tls.yaml)

1) https://https.staging.welance.com/hello

**as you can see all http connections go fine - so DNL-LB and so on are ok - HTTPS not.**

### where we think the problem might be

The issue is with CertificateRequest created by CertManager operator which will not end successfully.

NAME APPROVED DENIED READY ISSUER REQUESTOR AGE
https-tls-secret-certificate-62fmw True False letsencrypt-develop system:serviceaccount:cert-manager:cert-manager 37m
https-tls-secret-staging-certificate-v5w7s True False letsencrypt-staging system:serviceaccount:cert-manager:cert-manager 36m

from the logs we noticed the order is created but the challenge to complete the certiicate issuing does not complete

amedeopalopoli@MacBook-Pro-di-Amedeo platform % kubectl get challenge
NAME                                                              STATE     DOMAIN                      AGE
https-tls-secret-certificate-62fmw-522397194-1322428516           pending   https.dev.welance.com       38m
https-tls-secret-staging-certificate-v5w7s-300742362-3816885166   pending   https.staging.welance.com   38m

**this is the cause**

Reason:      Waiting for HTTP-01 challenge propagation: failed to perform self check GET request 'http://https.dev.welance.com/.well-known/acme-challenge/IKi1Jdd2kA2ailCQjMkqx6Z27_fX_ulCEQ992wcPtKY': Get "http://https.dev.welance.com/.well-known/acme-challenge/IKi1Jdd2kA2ailCQjMkqx6Z27_fX_ulCEQ992wcPtKY": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
  State:       pending

### what we have already tried
- checking if possible with editing the certmanager dns to pointing out 8.8.8.8 nameserver as reported [here](https://stackoverflow.com/questions/73607837/cert-manager-did-not-get-expected-response-when-querying-endpoint-expected-tok) but did not work
- as reported in the [FAQ](https://cert-manager.io/docs/troubleshooting/webhook/#error-context-deadline-exceeded) from cert manager the issue seems to be related network trouble somehow but did not get the cause yet unfortunately