# Deploy a TiDB Cluster
> - **Objective:** Learn to deploy a TiDB cluster on AWS (with Kubernetes)
> - **Prerequisites:**
> - background knowledge of TiDB components
> - AWS account
> - **Optionality:** Required
> - **Estimated time:** TBD
## Prepare tools
> - **Optionality:** Required
To deploy a TiDB Cluster on AWS, some tools are required to interact with AWS and Kubernetes.
**Note: You can continue to manually download and install tools, or [use the all-in-one prepared Docker image](#Use-all-in-one-prepared-Docker-image) directly.**
### Manually download tools
Below is the prerequisites list. Please follow the instruction links to install them.
| Tool | Use | Version | Install instructions |
| :-- | :-- | :-- | :-- |
| awscli | Interact with AWS services in command line | >=1.16.73 | [Installing the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) |
| Terraform | Build, change, and destroy AWS infrastructure | >=0.12 | [Installing Terraform](https://learn.hashicorp.com/terraform/getting-started/install.html) |
|kubectl | Run commands against Kubernetes clusters | >=1.12 | [Install and Set Up kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) |
| Helm | The package manager for Kubernetes | >=2.11.0 <2.16.4 | [Installing Helm](https://helm.sh/docs/intro/install/) |
| jq | Command-line JSON processor | - | [Download jq](https://stedolan.github.io/jq/download/) |
| aws-iam-authenticator | Provide authentication to the Kubernetes cluster | - | [Installing aws-iam-authenticator](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html) |
#### Verify tools
```{.output}
$ aws --version
aws-cli/2.0.7 Python/3.7.3 Linux/4.15.0-66-generic botocore/2.0.0dev11
$ terraform --version
Terraform v0.12.24
$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"7879fc12a63337efff607952a323df90cdc7a335", GitTreeState:"clean", BuildDate:"2020-04-08T17:38:50Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
$ helm version --client
Client: &version.Version{SemVer:"v2.16.3", GitCommit:"1ee0254c86d4ed6887327dabed7aa7da29d7eb0d", GitTreeState:"clean"}
$ jq --version
jq-1.5-1-a5b5cbe
$ aws-iam-authenticator version
{"Version":"v0.5.0","Commit":"1cfe2a90f68381eacd7b6dcfa2bf689e76eb8b4b"}
```
#### Troubleshooting
TODO
### Use all-in-one prepared Docker image
**If you have already downloaded and installed the tools, please [Create Kubernetes Cluster](#create-kubernetes-cluster) directly.**
Run the following command to enter the container environment that contains all required tools:
```{.bash .copyable}
docker run -it disksing/tidb-eks-tools:latest
```
#### Troubleshooting
TODO
## Create Kubernetes Cluster
- **Optionality:** Required
### Configure AWS credentials
Use `aws configure` to setup access keys. Note: The access key must have at least permissions to: create VPC, create EBS, create EC2 and create role.
#### Commands
**Note: following keys are examples, please replace with your own keys.**
```{.output}
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-2
Default output format [None]: json
```
#### Verify Keys
```{.output}
$ aws configure list
Name Value Type Location
---- ----- ---- --------
profile <not set> None None
access_key ******************** shared-credentials-file
secret_key ******************** shared-credentials-file
region us-west-2 config-file ~/.aws/config
```
#### Troubleshooting
TODO
### Clone the `tidb-operator` repo
The `tidb-operator` project provides utilities and configurations for creating an EKS cluster that is suitable for deployment of TiDB components.
#### Commands
```{.bash .copyable}
git clone --branch v1.1.0-rc.4 --depth=1 https://github.com/pingcap/tidb-operator.git
cd tidb-operator/deploy/aws
```
#### Verify clone
```{.output}
$ ls
README.md aws-tutorial.tfvars clusters.tf default-cluster.yaml main.tf manifests outputs.tf variables.tf versions.tf
```
#### Troubleshooting
TODO
### Install Terraform Dependencies
Terraform needs to install some necessary components to work.
#### Commands
```{.bash .copyable}
terraform init
```
#### Verify dependency
After the dependency installation is successful, the following information will be included in the command line output:
```{.output}
Terraform has been successfully initialized!
```
#### Troubleshooting
TODO
### Configure EKS Cluster
Some Terraform configuration files are pre-defined in the AWS directory. You can modify the configuration file to customize deploy region, cluster size, instance type, etc.
Since applying for resources is a time-consuming operation (about 20+ minutes), we recommend that you first create a cluster using the default configuration we provide, and then during the creation process you can take your time to learn how to customize the configurations.
**Note: you can [create EKS cluster](#Create-EKS-Cluster) with default configrations directly.**
#### Configure AWS region
You can customize AWS region by editing the `variables.tf` file. All avaliable values are listed in comments. The default value is `us-west-2`.
```
variable "region" {
description = "AWS region"
# supported regions:
# US: us-east-1, us-east-2, us-west-2
# Asia Pacific: ap-south-1, ap-northeast-2, ap-southeast-1, ap-southeast-2, ap-northeast-1
# Europe: eu-central-1, eu-west-1, eu-west-2, eu-west-3, eu-north-1
default = "us-west-2"
}
```
#### Configure cluster name
You can customize EKS cluster name or TiDB cluster name by editing the `variables.tf` file. **Be aware, if you change the cluster name, in the following steps, some commands may need to be adjusted accordingly.**
```
variable "eks_name" {
description = "Name of the EKS cluster. Also used as a prefix in names of related resources."
default = "my-cluster"
}
variable "default_cluster_name" {
default = "my-cluster"
}
```
#### Configure cluster size
You can customize cluster size by editing the `variables.tf` file. By default, the cluster contains 3 PDs, 3 TiKVs and 2 TiDBs. **Note that if you change the cluster size, in the following steps, the output of some commands and cluster status may be quite different from what we described.**
```
variable "default_cluster_pd_count" {
default = 3
}
variable "default_cluster_tikv_count" {
default = 3
}
variable "default_cluster_tidb_count" {
default = 2
}
```
#### Configure instance type
You can customize instance type by editing the `variables.tf` file. The default confirations are listed below. The default configuration is suitable for basic functional evaluating, not suitable for performance tests.
[AWS EC2 instance pricing](https://aws.amazon.com/ec2/pricing/on-demand/) contains more detailed information about instance types and their prices.
```
variable "default_cluster_pd_instance_type" {
default = "c5d.large"
}
variable "default_cluster_tikv_instance_type" {
default = "c5d.large"
}
variable "default_cluster_tidb_instance_type" {
default = "c4.large"
}
variable "default_cluster_monitor_instance_type" {
default = "c5.large"
}
```
### Create EKS Cluster
We will use Terraform to create AWS resources and create EKS clusters based on the information provided by the configuration file.
#### Commands
```{.bash .copyable}
terraform apply
```
#### Verify
Terraform does not directly perform operations, but first outputs the operation plan in the form of a diff. At the end of the command line output, there will be a line summarizing the operations to be performed and prompting you to confirm:
```
Plan: 66 to add, 0 to change, 0 to destroy.
...
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value:
```
If you confirm the execution, you need to enter `yes` and hit Enter. The running process will last more than 20 minutes, and a status message will be output every 10 seconds during the running process, which includes the total time currently used:
```
module.tidb-operator.null_resource.setup-env: Still creating... [1m10s elapsed]
module.tidb-operator.null_resource.setup-env: Still creating... [1m20s elapsed]
module.tidb-operator.null_resource.setup-env: Still creating... [1m30s elapsed]
module.tidb-operator.null_resource.setup-env: Still creating... [1m40s elapsed]
module.tidb-operator.null_resource.setup-env: Still creating... [1m50s elapsed]
module.tidb-operator.null_resource.setup-env: Still creating... [2m0s elapsed]
```
**If you are creating using default configration, you can take the time to learn about [Configure EKS Cluster](#Configure-EKS-Cluster).**
When it is finally completed, you will see command line output like this:
```
Apply complete! Resources: 66 added, 0 changed, 0 destroyed.
Outputs:
bastion_ip = [
"52.25.125.150",
]
default-cluster_monitor-dns = not_created
default-cluster_tidb-dns = not_created
eks_endpoint = https://F989422954FCA9A08505C4BBE98FA194.sk1.us-west-2.eks.amazonaws.com
eks_version = 1.15
kubeconfig_filename = credentials/kubeconfig_my-cluster
region = us-west-2
```
The output contains the information needed to connect the EKS cluster. The `bastion_ip` is the IP address of an EC2 server that can access the TiDB server. You can use `terraform output` to display the output anytime later.
#### Troubleshooting
##### Error: No valid credential sources found for AWS Provider
You need to [configure AWS credentials](#Configure-AWS-credentials) before creating the cluster.
## Install TiDB Cluster
> **Optionality:** Required
### Prepare yaml file
#### Command
Following command copies the pre-defined `db.yaml.example` file and replace `CLUSTER_NAME` with new cluster name `my-cluster`.
**Note: If you have changed the TiDB cluster name (`default_cluster_name` in `variables.tf`), remember to replace `my-cluster` to your configuration.**
```{.bash .copyable}
cluster_name=my-cluster
sed "s/CLUSTER_NAME/$cluster_name/" manifests/db.yaml.example > poc.yaml
```
### Create TiDB pods
```{.bash .copyable}
namepace=poc
export KUBECONFIG=credentials/kubeconfig
kubectl create namespace "$namespace"
kubectl create -f poc.yaml -n "$namespace"
```
A number of different pods will be created in groups. First, the PD pods, then the TiKV pods, and later the TiDB pods.
#### Verify
You can use `kubectl get pod` to see the progress of the deployment.
```{.bash .copyable}
watch kubectl -n "$namespace" get pod
```
Output:
```
NAME READY STATUS RESTARTS AGE
my-cluster-discovery-d9854558f-gxpsb 1/1 Running 0 7m6s
my-cluster-pd-0 1/1 Running 0 7m6s
my-cluster-pd-1 1/1 Running 2 7m6s
my-cluster-pd-2 1/1 Running 0 7m6s
my-cluster-tidb-0 2/2 Running 0 3m19s
my-cluster-tidb-1 2/2 Running 0 3m19s
my-cluster-tikv-0 1/1 Running 0 3m38s
my-cluster-tikv-1 1/1 Running 0 3m38s
my-cluster-tikv-2 1/1 Running 0 3m38s
```
As soon as you see pods for all 3 pod types and all pods show "Running", you can move on to [connecting to your cluster](<external>).
#### Troubleshooting
TODO
Sign in with Wallet
Connect another wallet