owned this note
owned this note
Published
Linked with GitHub
###### tags: `TiDB` `TiUP` `Google` `GCP` `Tutorial`
# Getting TiDB Up and Running on Google Cloud Compute (GCP) Instance - Part 1
## Overview
The primary goal of this blog post series is to get a simple TiDB cluster up and running on Google GCP. This will allow you to try out TiDB and get familiar with the major components.
This is a two part post. The first part is setting up GCP so that we can run TiDB. The second part is setting up and running TiDB on the GCP environment that we created in the first part.
Here's a Reference Architecture of what we will have the end of the 2nd post.
Table of Contents
[ToC]
In this first part, we will be focused on implementing Prerequisites Operations (see image below). Notice the two Swim Lanes (Local, GCP Instance). Each operation is run on the system associated with its specific Swim Lane
"Local" is refers to your person/company computer. For me, my local computer is a MacBook Pro that is my daily computer. "GCP Instance", is a virtual machine instance that we will create in Google GCP.
In each section, I will reference the image above with an arrow identifying the section we are working on.
### Requirements
* Basic understanding and access to Google GCP
* Google GCP `gcloud` installed and configured on local computer
# The Process
## Validate Google `gcloud` Configuration on Local Computer
Alright, let's start. On your local computer, you should have Google gcloud installed and configured.
On our local computer let's take a look at `gcloud` configuration.
```bash=
gcloud config list
```
Notice that I'm using a User Account; not a Service Account. Usually a service account will end with `gserviceaccount.com`.
In the following image, notice that I have `compute` configuration set for `region` and `zone` and you may also have this set. It shouldn't matter if these values are set or not. In this getting started guide, when we use gcloud commands we will specify where the resource will live.
## Create GCP Instance
Let's create a GCP instance where we will run TiDB.
```bash=
gcloud compute instances create tidb-vm \
--image-family=ubuntu-1804-lts \
--image-project=ubuntu-os-cloud \
--machine-type=e2-medium \
--boot-disk-size=200GB \
--boot-disk-type=pd-standard \
--boot-disk-device-name=tidb-vm \
--zone=us-west1-a
```
This may take a few moments to complete
##### Sanity Check
Let's validate the instance is up and running
```bash=
gcloud compute instances list
```
:::info
If you get an error and not able to create an instance, you probably do not have valid permissions with the GCP account that gcloud is associated with.
:::
## SSH into New GCP Instance
Here we will use `gcloud` ssh feature to access our newly created GCP instance.
```bash=
gcloud compute ssh --zone "us-west1-a" "tidb-vm"
```
Notice that the server we SSH into has a prompt that references the `tidb-vm` instance.
## GCP Instance - Change Service Account to User Account
### gcloud Account
We should now be logged into the GCP instance tidb-vm
Let's do something simple with `gcloud` that should validate our level of access, or the lack of access.
Let's get a list of GCP instances .
```
gcloud compute instances lists
```
Notice that I received an error. This is because the account associated with gcloud is a Service Account that does not have permissions to get a list of instances. Even though you may not get an error, I recommend following the instructions below to change from a Services Account to your User Account.
:::info
A GCP Service Account is a special kind of account used by an application or a virtual machine (VM) instance, not a person. By default, GCP creates a Service Account and adds it to your project and is enabled on all instances.
</br>
Reference:
* [GCP Docs - Account Types](https://cloud.google.com/sdk/docs/authorizing#types_of_accounts)
* [GCP Docs - Service Accounts](https://cloud.google.com/compute/docs/access/service-accounts)
:::
Let's see what account we are running as. Here I run two commands that show similar information. Both of these commands provide similar information
```bash=
gcloud config list
# or
gcloud auth list
```
Note that the account email address and the domain. If it ends in `compute@developer.gserviceaccount.com`, you are running as a service account. In this getting started guide we want to execute gcloud commands with our User Account.
Let's change the account that gcloud is associated with.
### Login with GCP User Account
Here we will change the account that gcloud is associated with from a Service Account our User Account.
It's a multi-step process to change the `gcloud` account.
:::info
Reference: [GCP Docs - Authorizing Cloud SDK tools](https://cloud.google.com/sdk/docs/authorizing)
:::
Let's start the process by using `gcloud` `auth login` command. This will initiate the process
```bash=
gcloud auth login
```
You will see a long URL (see image above). Copy this url and paste it in a browser address bar.
In the browser, select the Google Account that has the permissions needed. For me, this is the same account I used when creating the GCP instance.
Click the "Allow" button
Click the copy button
Paste the verification code from the browser to the `gcloud` CLI.
Our User Account should now be associated with `gcloud`.
##### Sanity Check
Let's confirm the account we are running as.
```bash=
gcloud config list
# Or
gcloud auth list
```
Perfect, gcloud is now running under a User Account.
Let's try again to get a list of machines.
```
gcloud compute instances lists
```
This doesn't prove that we have all the access we need, but it's a good baseline.
## Create SSH Keys for TiUP
In Part 2 of this post, we will use TiUP to create TiDB clusters. TiUP uses SSH. We will need to configure SSH.
Think of TiUP as a package manager that makes it easier to manage different cluster components in the TiDB ecosystem. We will discuss TiUP more in Part 2 of this post.
Let's see what's in the .ssh directory
```
ls -al .ssh
```
There are no actual keys in this directory. We can use `gcloud` to create SSH key that TiUP will need.
:::info
Reference: [Google Docs - gcloud compute config-ssh](https://cloud.google.com/sdk/gcloud/reference/compute/config-ssh)
:::
When running the following command, you will be prompted to enter a passphrase. For simplicity, do not enter a passphrase when prompted. Just hit <return> to provide an empty passphrase.
```bash=
gcloud compute config-ssh
```
##### Sanity Check
Let's see what files were created.
```bash=
ls -al .ssh/
```
Notice we now have 3 new files. The primary file we are interested in is the `google_compute_engine`, which is the private key that TiUP will use for SSH.
## Open TiDB Ports on GCP Network
We will want to access TiDB resources (TiDB Dashboard, Grafana, Prometheus) from a browser on your local computer. To do this, we will need to open ports on the GCP network by creating a GCP Firewall Rule.
Below, we are going to create a firewall rule that will allow only our local computer to access TiDB resources.
:::info
Reference: [Google Docs - VPC firewall rules overview](https://cloud.google.com/vpc/docs/firewalls)
:::
Let's get our local computer public IP Address
:::info
Here's a link that provides different ways of getting your local IP
[How To Find My Public IP Address From Command Line On a Linux](https://www.cyberciti.biz/faq/how-to-find-my-public-ip-address-from-command-line-on-a-linux/)
:::
:::warning
:exclamation: We want our local computer IP address, **NOT** the GCP instance IP address. Therefore, run the following command on your local computer. For me, I'm running the following command on my MacBook. Notice in the image below, that the command prompt does not include `tidb-vm`
:::
```bash=
host myip.opendns.com resolver1.opendns.com
```
Notice my blurred-out IP address. We are going to use this IP Address below.
Let's look at the current firewall rules
```bash=
gcloud compute firewall-rules list
```
Notice, by default, that GCP already has some firewall rules. We will create a new firewall rule.
<br>
Here are the TiDB ports that we will make available to our local computer:
* TiDB SQL Client: 4000
* Grafana: 3000
* Prometheus: 9090
* TiDB Dashboard: 2379
</br>
:::info
[PingCAP Docs - List of All TiDB Ports](https://docs.pingcap.com/tidb/stable/hardware-and-software-requirements#network-requirements)
:::
Let's create a firewall rule with our local IP Address.
In the code example below, for the parameter`--source-ranges`, replace <LOCAL IP> with your own local IP Address. Do keep the `/32`. This will limit it to your specific IP Address.
```bash=
gcloud compute firewall-rules create access-from-home \
--allow tcp:4000,tcp:3000,tcp:9090,tcp:2379 \
--description="Allow traffic from my personal computer" \
--source-ranges="98.245.63.241/32"
```
:::info
I added port 2379 to the above command. It's different than the image shows below.
:::
In part 2, we will access TiDB from our local computer using these ports.
## MySQL Client
TiDB is fully compatible with the MySQL 5.7 protocol and the common features and syntax of MySQL 5.7.
:::info
REFERENCE: [PinCAP Docs - MySQL Compatibility](https://docs.pingcap.com/tidb/stable/mysql-compatibility)
:::
In Part 2, after we installed TiDB, we will access TiDB with a MySQL Client. Here we will install MySQL Client using apt-get
```bash=
sudo apt-get update
sudo apt-get install mysql-client
```
## Wrap Up
You should now have a GCP instance that you can use to install and run a simple TiDB environment.
Next is [Part 2](https://hackmd.io/DUIgo9WhTeyzPqpPlSj7uQ) where we install and run TiDB.
