# SinoPac 20241018 troubleshoot # 調整方法 查看啟用FIPS後OCP節點上可用的cipher ``` $ openssl ciphers -v TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD TLS_AES_128_CCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESCCM(128) Mac=AEAD ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD ECDHE-ECDSA-AES256-CCM TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESCCM(256) Mac=AEAD ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD ECDHE-ECDSA-AES128-CCM TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESCCM(128) Mac=AEAD DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD DHE-RSA-AES256-CCM TLSv1.2 Kx=DH Au=RSA Enc=AESCCM(256) Mac=AEAD DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD DHE-RSA-AES128-CCM TLSv1.2 Kx=DH Au=RSA Enc=AESCCM(128) Mac=AEAD PSK-AES256-GCM-SHA384 TLSv1.2 Kx=PSK Au=PSK Enc=AESGCM(256) Mac=AEAD PSK-AES256-CCM TLSv1.2 Kx=PSK Au=PSK Enc=AESCCM(256) Mac=AEAD PSK-AES128-GCM-SHA256 TLSv1.2 Kx=PSK Au=PSK Enc=AESGCM(128) Mac=AEAD PSK-AES128-CCM TLSv1.2 Kx=PSK Au=PSK Enc=AESCCM(128) Mac=AEAD DHE-PSK-AES256-GCM-SHA384 TLSv1.2 Kx=DHEPSK Au=PSK Enc=AESGCM(256) Mac=AEAD DHE-PSK-AES256-CCM TLSv1.2 Kx=DHEPSK Au=PSK Enc=AESCCM(256) Mac=AEAD DHE-PSK-AES128-GCM-SHA256 TLSv1.2 Kx=DHEPSK Au=PSK Enc=AESGCM(128) Mac=AEAD DHE-PSK-AES128-CCM TLSv1.2 Kx=DHEPSK Au=PSK Enc=AESCCM(128) Mac=AEAD ``` 比較啟用fips的OCP ingresscontroler 預設cipher ``` # oc -n openshift-ingress-operator get ingresscontroller default -o jsonpath='{.status.tlsProfile}' tlsProfile: ciphers: - ECDHE-ECDSA-AES128-GCM-SHA256 - ECDHE-RSA-AES128-GCM-SHA256 - ECDHE-ECDSA-AES256-GCM-SHA384 - ECDHE-RSA-AES256-GCM-SHA384 - ECDHE-ECDSA-CHACHA20-POLY1305 # 沒有出現在上面清單內,需要移除 - ECDHE-RSA-CHACHA20-POLY1305 # 沒有出現在上面清單內,需要移除 - DHE-RSA-AES128-GCM-SHA256 - DHE-RSA-AES256-GCM-SHA384 - TLS_AES_128_GCM_SHA256 - TLS_AES_256_GCM_SHA384 - TLS_CHACHA20_POLY1305_SHA256 # 沒有出現在上面清單內,需要移除 minTLSVersion: VersionTLS12 ``` 調整ingress controller移除無法使用的Cipher ``` oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{ "spec": { "tlsSecurityProfile": { "type": "Custom", "custom": { "ciphers": [ "ECDHE-ECDSA-AES128-GCM-SHA256", "ECDHE-RSA-AES128-GCM-SHA256", "ECDHE-ECDSA-AES256-GCM-SHA384", "ECDHE-RSA-AES256-GCM-SHA384", "DHE-RSA-AES128-GCM-SHA256", "DHE-RSA-AES256-GCM-SHA384", "TLS_AES_128_GCM_SHA256", "TLS_AES_256_GCM_SHA384" ], "minTLSVersion": "VersionTLS12" } } } }' ``` 如需恢復預設值 ``` oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{"spec":{"tlsSecurityProfile":null}}' ``` ------ # 除錯 ## Case - Edge ok / Firefox Error OS: win10 Edge version: 125 OK Firefox version: n/a 於Windows端開wireshark發現用firefox連接時，完全沒有發送封包 ## Case - Edge Error / Firefox ok 10.11.7.81 Win 10 Edge version: 128.0.2739.42 Firefox version: 131.0.3 ### Log ``` edge 2024-10-18T06:52:55.827961+00:00 ocp-sit-i-rt2 ocp-sit-i-rt2.ocp-sit.test haproxy[54]: 10.11.7.81:16797 [18/Oct/2024:06:52:55.825] fe_sni/1: SSL handshake failure 2024-10-18T06:52:55.831189+00:00 ocp-sit-i-rt2 ocp-sit-i-rt2.ocp-sit.test haproxy[54]: 10.11.7.81:16797 [18/Oct/2024:06:52:55.825] public_ssl be_sni/fe_sni 1/0/5 99 -- 67/34/33/33/0 0/0 firefox 2024-10-18T06:54:19.207411+00:00 ocp-sit-i-rt2 ocp-sit-i-rt2.ocp-sit.test haproxy[54]: 10.11.7.81:15325 [18/Oct/2024:06:54:19.201] fe_sni~ be_edge_http:rd-cardlite:wl-admin-route/pod:wl-admin-ddd9d7ff-w6ssv:wl-admin::10.177.8.128:8058 242 0 GET /frontendConfig HTTP/1.1 ``` ## Case - Edge ok / Firefox Error OS: win11 Edge version: 128.0.2535.92 Firefox version: 10.131.0.3 FireFox error ![image](https://hackmd.io/_uploads/HkMT_FJeye.png) # Haproxy支援之cipher ``` # Configure the TLS versions we support ssl-default-bind-options ssl-min-ver TLSv1.2 # The default cipher suite can be selected from the three sets recommended by https://wiki.mozilla.org/Security/Server_Side_TLS, # or the user can provide one using the ROUTER_CIPHERS environment variable. # By default when a cipher set is not provided, intermediate is used. # user provided list of ciphers (Colon separated list as seen above) # the env default is not used here since we can't get here with empty ROUTER_CIPHERS tune.ssl.default-dh-param 2048 ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 ``` You closed the case earlier and mentioned that issue has been self-solved. You have identified an issue with TLS profile, you were using intermediate and client is 2012. OCP is looking for CIPHERS "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384" "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256" and windows 2012 won't support these. When you updated the TLS profile for ingress to low it started working. https://access.redhat.com/articles/5348961 # 測試 ``` # Haproxy停用 TLS_CHACHA20_POLY1305_SHA256 oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{"spec":{"tlsSecurityProfile":{"type":"Custom","custom":{"ciphers":["TLS_AES_128_GCM_SHA256","TLS_AES_256_GCM_SHA384"],"minTLSVersion":"VersionTLS13"}}}}' curl -kL --tls13-ciphers TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384 https://url curl -kL --tls13-ciphers TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 https://url ```