# TLS
> - **Objective:** Configure TLS in a TiDB Cluster
> - **Prerequisites:**
> - background knowledge of TiDB components
> - AWS account
> - **Optionality:** Optional
> - **Estimated time:** TBD
## TLS Support in TiDB
TiDB Operator 1.1.0 supports Transport Layer Security (TLS) between cluster components and between client and the cluster. This document describes how to enable TLS in TiDB including:
- TLS between cluster components.
- TLS between MySQL client and TiDB cluster.
This document is organized as the follows, we first create secrets in Kubenetes and if the certificates is not already in Kubernetes.
Note that this document assumes the following TiDB cluster name and namespace are used. Please make necessary changes to match the cluster name and namespace of the TiDB cluster.
```
$ export cluster_name=basic
$ export namespace=demo
```
## Issue Certificates
We need to generate certificates for inter and intra component communication. In summary, we need to generate the following certificates:
- PD Server certificate for communicating between PD instances.
- TiKV Server certificate for communicating between TiKV instances.
- TiDB Server certificate for communicating between MySQL client and TiDB Cluster.
- Client certificate for communicating between components and between MySQL client and TiDB Cluster.
TLS certificates can be issued by multiple methods. Please consult your organization on how you can obtain them. For testing purposes, you can follow the steps in [Issue TLS Certificates Using cfssl](https://hackmd.io/gL-13LYlTA2vPRzzMrA22w?both) to issue and verify certificates.
## TLS Configuration
There are two configurations to enable TLS. To enable TLS between clusters, set `spec.tlsCluster.enabled` to `true`. To enable TLS between MySQL client and TiDB cluster, set `spect.tidb.tlsClient.enabled` to `true`.
To enable mutual TLS, for each compoment, we need to config the `config.security.cert-allowed-cn`.
**Note that as the configuration `cert-allowed-cn` is stored PD, we can only set one value. This means that when generating the certificates for cluster components, we need to make sure that the CN field is correctly set and is the same between different certificates. This can be verified by using the openssl tool.**
### Enable TLS
#### Deploy a TiDB Cluster with TLS enabled
One example configuration for TiDB cluster with TLS enabled is as follows:
```
apiVersion: pingcap.com/v1alpha1
kind: TidbCluster
metadata:
name: basic
spec:
tlsCluster:
enabled: true
version: v4.0.0-rc
timezone: UTC
pvReclaimPolicy: Delete
pd:
baseImage: pingcap/pd
replicas: 1
requests:
storage: "1Gi"
config:
security:
cert-allowed-cn:
- TiDB
tikv:
baseImage: pingcap/tikv
replicas: 1
requests:
storage: "1Gi"
config:
security:
cert-allowed-cn:
- TiDB
tidb:
baseImage: pingcap/tidb
replicas: 1
service:
type: ClusterIP
tlsClient:
enabled: true
config:
security:
cert-allowed-cn:
- TiDB
```
**Note that the value `spec.pd.config.security.cert-allowed-cn`,`spec.tikv.config.security.cert-allowed-cn` and `spec.tikv.config.security.cert-allowed-cn` needs to be the same and match the CN field in the corresponding certificates.**
Assuming that the yaml file is stored in the following path (TODO: put the configuration file in the tidb operator repo). Please also set the environment variable to point to the path of the configuration.
Deploy a TiDB cluster with TLS enabled:
```
$ kubectl apply -f ${tidb_cluster_conf} -n ${namespace}
```
#### Enable TLS on an Existing TiDB Cluster
We can edit the TiDB cluster configuration to enable TLS configuraitons using the following command:
```
$ kubectl edit tc ${cluster_name} -n ${namespace}
```
After the prompt, peforming the following changes:
- Set `spec.tlsCluster.enabled` to `true`.
- Set `spec.pd.config.security.cert-allowed-cn`, `spec.tikv.config.security.cert-allowed-cn` and `spec.tikv.config.security.cert-allowed-cn` to be the same and match the CN field in the corresponding certificates.
- Set `spect.tidb.tlsClient.enabled` to `true`.
After the saving the configuration, if there's no issues, TiDB Operator will perform a rolling restart of the TiDB cluster and apply the configuration changes.
### Verify Cluster is Successfully Deployed
```
$ kubectl get po -n ${namespace}
NAME READY STATUS RESTARTS AGE
basic-discovery-69879c5b85-jvz2f 1/1 Running 0 103s
basic-pd-0 1/1 Running 0 103s
basic-tidb-0 1/2 Running 0 22s
basic-tikv-0 1/1 Running 0 82s
```
```
$ kubectl get svc -n ${namespace}
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
basic-discovery ClusterIP 10.101.136.222 <none> 10261/TCP 2m33s
basic-pd ClusterIP 10.100.120.45 <none> 2379/TCP 2m33s
basic-pd-peer ClusterIP None <none> 2380/TCP 2m33s
basic-tidb ClusterIP 10.102.77.101 <none> 4000/TCP,10080/TCP 72s
basic-tidb-peer ClusterIP None <none> 10080/TCP 72s
basic-tikv-peer ClusterIP None <none> 20160/TCP 2m12s
```
```
$ kubectl logs ${cluster_name}-pd-0 -n ${namespace}
[2020/04/29 00:46:13.045 +00:00] [INFO] [etcd.go:576] ["serving peer traffic"] [address="[::]:2380"]
[2020/04/29 00:46:13.045 +00:00] [INFO] [etcd.go:241] ["now serving peer/client/metrics"] [local-member-id=5d5d47e6e68a829f] [initial-advertise-peer-urls="[https://basic-pd-0.basic-pd-peer.demo.svc:2380]"] [listen-peer-urls="[https://0.0.0.0:2380]"] [advertise-client-urls="[https://basic-pd-0.basic-pd-peer.demo.svc:2379]"] [listen-client-urls="[https://0.0.0.0:2379]"] [listen-metrics-urls="[]"]
```
```
$ kubectl logs ${cluster_name}-tikv-0 -n ${namespace}
[2020/04/29 00:46:42.230 +00:00] [INFO] [mod.rs:335] ["starting working thread"] [worker=addr-resolver]
[2020/04/29 00:46:42.230 +00:00] [INFO] [mod.rs:335] ["starting working thread"] [worker=region-collector-worker]
[2020/04/29 00:46:46.912 +00:00] [INFO] [future.rs:136] ["starting working thread"] [worker=gc-worker]
[2020/04/29 00:46:46.912 +00:00] [INFO] [mod.rs:335] ["starting working thread"] [worker=lock-collector]
[2020/04/29 00:46:46.945 +00:00] [INFO] [mod.rs:181] ["Storage started."]
[2020/04/29 00:46:46.948 +00:00] [INFO] [mod.rs:335] ["starting working thread"] [worker=split-check]
[2020/04/29 00:46:46.989 +00:00] [INFO] [node.rs:145] ["trying to bootstrap cluster"] [region="id: 2 region_epoch { conf_ver: 1 version: 1 } peers { id: 3 store_id: 1 }"] [store_id=1]
[2020/04/29 00:46:46.994 +00:00] [INFO] [node.rs:281] ["bootstrap cluster ok"] [cluster_id=6820928503865373278]
[2020/04/29 00:46:46.994 +00:00] [INFO] [node.rs:348] ["start raft store thread"] [store_id=1]
[2020/04/29 00:46:46.994 +00:00] [INFO] [peer.rs:148] ["create peer"] [peer_id=3] [region_id=2]
```
```
$ kubectl logs ${cluster_name}-tidb-0 -c tidb -n ${namespace}
[2020/04/29 00:47:31.029 +00:00] [INFO] [server.go:235] ["server is running MySQL protocol"] [addr=0.0.0.0:4000]
[2020/04/29 00:47:31.030 +00:00] [INFO] [http_status.go:80] ["for status and metrics report"] ["listening on addr"=0.0.0.0:10080]
```
### Verify Security Configuraiton
There are a couple of ways to verify security is correctly configured.
#### Using pd-ctl
```
$ pd-ctl --cacert=${certificate_path}/ca.pem --cert=${certificate_path}/client.pem --key=${certificate_path}/client-key.pem -u https://${cluster_name}-pd.${namespace}.svc:2379 member
```
**Note that you may need to do a port forwarding to access pd.**
The output is:
```
{
"header": {
"cluster_id": 6820928503865373278
},
"members": [
{
"name": "basic-pd-0",
"member_id": 6727612475397276319,
"peer_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2380"
],
"client_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2379"
],
"deploy_path": "/",
"binary_version": "v4.0.0-rc"
}
],
"leader": {
"name": "basic-pd-0",
"member_id": 6727612475397276319,
"peer_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2380"
],
"client_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2379"
]
},
"etcd_leader": {
"name": "basic-pd-0",
"member_id": 6727612475397276319,
"peer_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2380"
],
"client_urls": [
"https://basic-pd-0.basic-pd-peer.demo.svc:2379"
],
"deploy_path": "/",
"binary_version": "v4.0.0-rc"
}
}
```
Note that `https` is used for `client_urls` and `peer_urls`.
#### Using MySQL Client
```
$ mysql -u root -P 4000 -h ${tidb_host} --ssl-cert=${certificate_path}/client.pem --ssl-key=${certificate_path}/client-key.pem --ssl-ca=${certificate_path}/ca.pem --ssl-mode=VERIFY_IDENTITY
```
```
mysql> \s
--------------
mysql Ver 8.0.19 for osx10.15 on x86_64 (Homebrew)
Connection id: 29
Current database:
Current user: root@127.0.0.1
SSL: Cipher in use is TLS_AES_256_GCM_SHA384
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 5.7.25-TiDB-v4.0.0-rc TiDB Server (Apache License 2.0), MySQL 5.7 compatible
Protocol version: 10
Connection: 127.0.0.1 via TCP/IP
Server characterset: utf8mb4
Db characterset: utf8mb4
Client characterset: utf8mb4
Conn. characterset: utf8mb4
TCP port: 4000
Binary data as: Hexadecimal
--------------
```
```
mysql> SHOW STATUS LIKE "%Ssl%";
+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Variable_name | Value |
+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Ssl_cipher | TLS_AES_256_GCM_SHA384 |
| Ssl_cipher_list | RC4-SHA:DES-CBC3-SHA:AES128-SHA:AES256-SHA:AES128-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-ECDSA-RC4-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-RC4-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-CHACHA20-POLY1305:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256: |
| Ssl_verify_mode | 5 |
| Ssl_version | TLSv1.3 |
+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
4 rows in set (0.01 sec)
```
Sign in with Wallet
Connect another wallet