# Upgrading Presto with EMR This describes an upgrade procedure with running two clusters at the same time to switch between them. The current running one is called Blue and the new one is called Green. If your running cluster is already Green, exchange the colors in this document accordingly. ## Step 2: Change terraform, add Green (new) Cluster In essence you have to add all needed ressources you want to change. At least you need a new cluster (task nodes and emr cluster). Search for comments (`#deployment`) in the terraform `provision/presto-emr/main.tf` file. ## Step 3: Change configfiles If needed. ## Step 4: Deploy Green (new) Cluster ### Prepare and check the plan ``` terraform init -backend-config="bucket=dl-iac-state-XXXXXXXX-eu-central-1-prod" -backend-config="key=i-presto.json" ``` TODO: build tfvars from pod.yaml file!! Until then build your own `terraform.tfvars` from openshift/deployment/pod.yaml. Add username and password for RDS. Here how you get the current secrets from AWS: ``` rds_access=`aws secretsmanager get-secret-value --secret-id rds_access | jq '.SecretString' | jq -r` RDS_USERNAME=`echo $rds_access | jq -r '.username'` RDS_PASSWORD=`echo $rds_access | jq -r '.password'` echo $RDS_USERNAME echo $RDS_PASSWORD ``` If you feel fine, commit and push your changes to the master branch and continue with the next step. ### Deploy w/ jenkins You can trigger a deployment for Presto here by clicking the Build Now button on the left navigation: ***REMOVED confidential link*** Lets refer to the README file for the Jenkins Link :) Once the build starts, it will pause on the *Apply Terraform* stage. Review the Terraform Plan by looking at the output of the *Terraform Plan* stage. Things to look out for: * The existing Presto cluster should **not** be replaced or redeployed * A second Presto cluster is being created. Accept the prompt for the *Terraform Apply* stage to let the infrastructure deployment to continue. ## Step 5: Test Blue Cluster run test queries + hue is working Hue URL: [master-public-dns]:8888 ## Step6: Switch to the Blue Cluster ### Check load on Green Cluster TODO: - What to check before (to compare w/ after the switch) ### Announce the short interruption of the service TODO: Text ... TODO: Where to publish? TODO: When to publish where? ### Switch switch elastic IP - how to get the public IP - How to get the instance id to associate with the IP ## Verify Check that everything works as expected. - Checklist here! - Event Listener ## Step 7: Tear down the Green Cluster Removing all Green Resource from terraform files. Commit & push to master branch and check the `terraform plan` in jenkins. If everything looks good, apply.