# Move to AWS

POC by moving `K8s control plane` and `databases` of mixing cluster to AWS region *`Asia Pacific (Hong Kong) ap-east-1`*.

## Infra

- AWS network setup (network team support)
- VPC creation
- VPC Peer with current HK and pop sites
- Server setup (Linux team support)
- K8s nodes - master *3, worker *2
- Other database requirement
- MongoDB, kafka, scylla, ...etc
- Infra service setup
- NTP,DNS, LB, squid proxy
- ES, Kibana
- Basic testing
- Network testing

## Kubernetes Control Plane

- New k8s node installation (as pure node)
- Firewall (Network team support)
- Disable AWS src/dst checks
- MTU check
- selinux config check
- Document references
- [install worker nodes]( https://issuetracking.maaii.com:9443/display/LCC5/Kubernetes+Cluster+Creation+Document#install-worker-nodes)
- deploy control plane to new master nodes
- APIserver, controller manager, scheduler yamls
- APIserver join master LB member or DNS (Linux team support)
- Remove on-prem HK apiserver
- Observe latencies.
- Document references
- [Install master nodes](https://issuetracking.maaii.com:9443/display/LCC5/Kubernetes+Cluster+Creation+Document#install-master-nodes)
- [Master node migration](https://issuetracking.maaii.com:9443/display/LCC5/K8s+CoreOS+migration+to+Ubuntu+1804#K8sCoreOSmigrationtoUbuntu1804-MasterNodeMigration)
- New node join etcd cluster, add one and then remove one.
- Two etcd clusters - calico-etcd and kube-etcd
- Firewall
- ETCD ←→ ETCD
- APIserver → kube-etcd
- K8s nodes, OSS nodes → calico-etcd
- Observe latencies.
- Document references
- [Join etcd cluster](https://issuetracking.maaii.com:9443/display/LCC5/K8s+CoreOS+migration+to+Ubuntu+1804#K8sCoreOSmigrationtoUbuntu1804-JointoETCDCluster)
- [Remove node from etcd cluster](https://issuetracking.maaii.com:9443/display/LCC5/K8s+CoreOS+migration+to+Ubuntu+1804#K8sCoreOSmigrationtoUbuntu1804-RemovenodefromETCDcluster)
- K8s function testing
- node to pod, pod to pod ...etc
- kube-dns resolve

## Deployment

- Monitoring
- Fluent-bit with local ES
- Local prometheus
- Jaeger
- Deploy application on AWS
- DB migration to AWS (DBA, Evan support)
- Mongo, Kafka, redis, scylla
- Deployment
- Install voice services
- Prepare app-configs
- Prepare certificate for application (if needed)
- Deploy apps
- Setup LB, Firewall, FQDN
- Target namespaces
- `dev-hk-03` → `int-hk-02` → `tb-hk-02`
- Others
- Office NCR to
- New APIserver
- Local Kibana
- Jaeger tracing

## Fallback Plan
### Infra
For infra leve setup, there's no influence of production live workloads, just shutdown and remove the configurations will be good.

### Kubernetes Control Plane
For kubernetes control plane migration, there's HA for the control plane. We will do the migration one by one. If there's something wrong during the migration, fallback could be done by starting and rejoin the master nodes back to the cluster.

### Applications
- DB fallback
- datas sync back the data to original DBs.
- Voice services fallback
- Application services fallback
- After worker node is ready, redeploy the applications to the original worker nodes and point back to original DB.