# IaC <!-- Put the link to this slide here so people can follow --> slide: https://hackmd.io/@q9cU-jkbS6WVfnxVnHOzVw/HyRKCrZyj --- ## Overview * Concept * Terraform * HCL * State * Alternatives --- ## IaC - Cost, speed and risk: - The cost of infrastructure configuration relate to mitigation of repetative tasks - Automation enables speed of faster execution - Risk is mitigated trough automation --- # Terraform - 2014 - Ansible 2012, Chef 2009 - Push based - In what way the configuration is propagated - Pull - servers fetch the configuration and changed the setup on its own - Written in GO - Day 0 Provisioning --- # Terraform - One of many IaC implementation - Cloud agnostic - Modular - Stateful - Declarative - Standardized configuration - Somewhat colaborative --- ## HCL * Configuration language common in Hashicorp products * Something entirely new and different * Sysadmins do not need to learn howto golang, nodejs, ... --- ### Hashicorp Configuration Language (HCL & HCL 2) - created by resources ```hcl <BLOCK TYPE> "<BLOCK LABEL>" "<BLOCK LABEL>" { # Block body <IDENTIFIER> = <EXPRESSION> # Argument } ``` --- ## Resource ```hcl resource "aws_vpc" "main" { cidr_block = var.base_cidr_block } ``` --- ## Variable ```hcl variable "base_cidr_block" { description = "A description" default = "10.1.0.0/16" } ``` --- ## Output ```hcl output "test" { description = "A description" value = "test" } ``` --- ## State * Mapping of real world resources to your configuration --- ### State example ```hcl resource "random_password" "password" { length = 16 special = true override_special = "!#$%&*()-_=+[]{}<>:?" } output "password" { value = nonsensitive(random_password.password.result) } ``` --- ### State example ```bash terraform apply ``` --- ### State example ```bash Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # random_password.password will be created + resource "random_password" "password" { + bcrypt_hash = (sensitive value) + id = (known after apply) + length = 16 + lower = true + min_lower = 0 + min_numeric = 0 + min_special = 0 + min_upper = 0 + number = true + numeric = true + override_special = "!#$%&*()-_=+[]{}<>:?" + result = (sensitive value) + special = true + upper = true } Plan: 1 to add, 0 to change, 0 to destroy. Changes to Outputs: + password = (known after apply) Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes random_password.password: Creating... random_password.password: Creation complete after 0s [id=none] Apply complete! Resources: 1 added, 0 changed, 0 destroyed. Outputs: password = "{(-_92bg-:D8:W1C" ``` --- ## State management issues * Locking has to be used * Cooperation is very limited * Generally a larger file, 2 MB is ok * Secrets are placed as string in the file * `terraform refresh` can take minutes --- # Expressions * Expressions are used to refer to or compute values within a configuration. * An exception to the declarative approach For local debuging, open random directory and type `terraform console` --- ## Type & values * string * number * bool * list/tuples * map/object * null --- ## String and templates * JSON from map: ```hcl example = jsonencode({ a = 1 b = "hello" }) ``` --- ## Heredoc standard: ``` locals { test = <<EOT test test EOT } ``` --- ## Heredoc indented: ``` locals { test = <<-EOT test test EOT } ``` `local.test` : ``` test test ``` --- ## Interpolation & directives ```hcl "Test, ${var.test}!" ``` ```hcl "Hello, %{ if var.name != "" }${var.name}%{ else }unnamed%{ endif }!" ``` --- ## Variable references ```hcl locals { test = "test" test_two = local.test } ``` Refering by tree structure with dots. --- ## Operators Standard, supports also ternary operator ```hcl locals { is_friday = formatdate("EEEE", timestamp()) == "Friday" ? true : false } output "friday" { value = local.is_friday } output "day" { value = formatdate("EEEE", timestamp()) } ``` Outputs: ``` day = "Thursday" friday = false ``` --- ## For expression ```hcl > [for s in ["a", "b", "c"] : upper(s)] [ "A", "B", "C", ] > {for s in ["a", "b", "c"] : s => upper(s)} { "a" = "A" "b" = "B" "c" = "C" } > [for s in ["a", "b", "c"] : upper(s) if s != "b"] [ "A", "C", ] ``` --- ## Splat expression ```hcl locals { a_list = [{"id": 1}] } > [for o in local.a_list : o.id] [ 1, ] # splat expression: > local.a_list[*].id [ 1, ] ``` Old and confusing --- # Resources Description of one or more object in infrastructure. ```hcl resource "aws_instance" "web" { ami = "ami-a1b2c3d4" instance_type = "t2.micro" } ``` [Arguments](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) given by provider. --- ## Resources meta-arguments * depends_on - explicit dependency * count - multiple resource instance * for_each - multple resources according to a map or set * provider - in case there are non-default providers * lifecycle - adjusting object lifecycle handling - when to destroy/create * provisioner - extra actions after reource creation --- ## Resources dependency Implicit ```hcl resource "google_compute_address" "static" { name = "jumper-ipv4-address" } resource "google_compute_instance" "jumper" { ... network_interface { network = "default" access_config { nat_ip = google_compute_address.static.address } } } ``` --- ## Resources dependency Explicit ```hcl resource "google_compute_instance" "jumper_a" { ... } resource "google_compute_instance" "jumper_b" { ... depends_on = [google_compute_instance.jumper_a] } ``` --- ## Local-only resources SSH, TLS keys, Random numbers, ... ```hcl resource "random_password" "password" { length = 16 special = true override_special = "!#$%&*()-_=+[]{}<>:?" } output "password" { value = random_password.password.result } ``` -- --- ## Count and for_each * Creating multiple resources * Creating resource once condition is met --- ### Count ```hcl resource "google_compute_instance" "server" { count = 10 name = "server_${count.index}" ... } ``` Each instance refered with index `google_compute_instance.server[0-9]` --- ### Count ```hcl resource "google_compute_instance" "server" { count = var.enable_server ? 1 : 0 name = "server_${count.index}" ... } ``` Creates a server once variable `enable_server` is set to true. --- ### for_each ```hcl resource "random_password" "password" { for_each = { user_1: { lower: false, special: true, } user_2: { lower: true, special: false, } } length = 16 special = each.value["special"] lower = each.value["lower"] override_special = "!#$%&*()-_=+[]{}<>:?" } output "passwords" { value = {for k, v in random_password.password: k => nonsensitive(v.result)} } ``` --- ### for_each ```hcl output "passwords" { value = {for k, v in random_password.password: k => nonsensitive(v.result)} } Outputs: passwords = { "user_1" = "*J4R>FKAOE0I4P-]" "user_2" = "Pi4Bkogxk0tvqpBq" } ``` --- ## Dynamic expressions ```hcl resource "aws_elastic_beanstalk_environment" "_" { ... setting { namespace = "dev" name = "test_1" value = "test_1" } setting { namespace = "dev" name = "test_2" value = "test_2" } } ``` --- ## Dynamic expressions ```hcl resource "aws_elastic_beanstalk_environment" "_" { ... dynamic "setting" { for_each = [ { namespace: "dev", name: "test_1" value: "test_1" }, { namespace: "dev", name: "test_2" value: "test_2" }, ] content { namespace = setting.value["namespace"] name = setting.value["name"] value = setting.value["value"] } } } ``` --- ## Useful for conditional blocks ```hcl resource "google_container_cluster" "primary" { ... dynamic "network_policy" { for_each = var.network_policy == null ? [] : [1] content { enabled = true provider = var.network_policy } } ... ``` --- # Modules - Recycable code - Version control - Modules registry at https://registry.terraform.io/browse/modules --- ## Modules folder layout ``` project │ main.tf │ └─── module_1 │ main.tf │ variables.tf │ outputs.tf │ ... ``` --- ## Modules example main.tf ```hcl module "servers" { source = "./module_1" servers = 5 } ``` --- ## Modules example `module_1/main.tf` ```hcl resource "digitalocean_droplet" "server" { count = var.servers image = "ubuntu-18-04-x64" name = "web-${count.index}" region = "nyc2" size = "s-1vcpu-1gb" } ``` --- ## Modules example `module_1/variables.tf` ```hcl variable "server" { default = 0 description = "Count of servers" } ``` --- # Providers * Binaries used for communication with upstream api * Devided in official, partner, community, archived * Anyone can create a provider * Each provider is versioned * Typically, version matches the API version * Providers are build with different architecture --- ## Providers ``` .terraform/providers/registry.terraform.io/cloudflare/cloudflare/3.1.0/linux_amd64$ ./terraform-provider-cloudflare_v3.1.0 This binary is a plugin. These are not meant to be executed directly. Please execute the program that consumes these plugins, which will load any plugins automatically ``` Providers are used in the code --- ## Providers `providers.tf`: ```hcl provider "kubernetes" { cluster_ca_certificate = var.gke_cluster_ca_certificate token = var.token host = "https://${var.gke_endpoint}" } ``` --- ## Providers `versions.tf`: ```hcl terraform { required_version = "1.2.9" required_providers { kubernetes = { source = "hashicorp/kubernetes" version = "~> 2.13.0" } } } ``` --- ## Providers `.terraform.lock.hcl`: ```hcl # This file is maintained automatically by "terraform init". # Manual edits may be lost in future updates. provider "registry.terraform.io/hashicorp/kubernetes" { version = "2.13.1" constraints = "~> 2.13.0" hashes = [ "h1:cN3OwZvhtn/y3XfnGQ4hi+7oZp1gU2zVYhznRv2C7Qg=", "zh:061f6ecbbf9a3c6345b56c28ebc2966a05d8eb02f3ba56beedd66e4ea308e332", "zh:2119beeccb35bc5d1392b169f9fc748865261b45fb75fc8f57200e91658837c6", "zh:26c29083d0d84fbc2e356e3dd1db3e2dc4139e943acf7a318d3c98f954ac6bd6", "zh:2fb5823345ab05b3df74bb5c51c61072637d01b3cddffe3ad36a73b7d5b749e6", "zh:3475b4422fffaf58584c4d877f98bfeff075e4a746f13e985d2cb20adc873a6c", "zh:366b4bef49932d1d71b12849c1878c254a887962ff915f37982299c1185dd48a", "zh:589f9358e4a4bd74a83b97ccc64df455ddfa64c4c4e099aef30fa29080497a8a", "zh:7a0d75e0e4fee6cc5599ac9d5e91de563ce9ea7bd8137480c7abd09642a9e72c", "zh:a297a42aefe0650e3d9fbe55a3ee48b14bb8bb5edb7068c09512d72afc3d9ca5", "zh:b7f83a89b646542d02b733d464e45d6d0739a9dbb921305e7b8347e9fc98a149", "zh:d4c721174a598b66bd1b29c40fa7cffafe90bb58186cd7506d792a6b04161103", "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", ] } ``` --- ## Providers `.terraform.lock.hcl`: * Each hash represent different build platform * Semver is not enough --- ## Version constraints * Semver vA.B.C from https://semver.org/spec/v2.0.0.html * ~> update rightmost component, e.g. ~> 1.0.4 allows 1.0.5, but not 1.1.0 * <=, >=, =, != --- # Example project ``` project │ main.tf │ variables.tf │ outputs.tf │ versions.tf │ providers.tf │ terraform.tfvars │ .terraform │ .terraform.lock.hcl │ └─── module_1 │ main.tf │ variables.tf │ outputs.tf │ ... ``` --- ## Example project * Terraform works with current directory * not with single files * You can put all tf code in one file and it will work --- ## `main.tf` * Usually contains resources for configuration * `locals` block * A few comments --- ## `variables.tf` * Variables used for the configuration * If not filled in `terraform.tfvars` * values can be inserted from stdin --- ## `outputs.tf` * Display values usually needed for further configuration * `terraform refresh` & `terraform output` ```hcl output "passwords" { value = ... } ``` --- ## `versions.tf` ```hcl terraform { required_version = "1.2.2" required_providers { cloudflare = { source = "cloudflare/cloudflare" version = "~> 3.0" # configuration_aliases = [cloudflare.bg, cloudflare.hr, cloudflare.gr] } google = { source = "hashicorp/google" version = "4.24.0" } postgresql = { source = "cyrilgdn/postgresql" version = "1.16.0" } } } ``` --- ## `providers.tf` ```hcl provider "google" { project = var.project region = var.zone } provider "postgresql" { scheme = "gcppostgres" host = module.postgresql.instance_connection_name port = 5432 database = replace(var.project, "-", "_") username = "admin" password = random_password.development.result sslmode = "require" connect_timeout = 15 } provider "cloudflare" { api_token = data.vault_generic_secret.cloudflare_credentials_gr.data["API_TOKEN"] } ``` --- ## `terraform.tfvars` Let's have `variables.tf`: ``` variable "region" { default = "europe-west3" } ``` `terraform.tfvars` can contain: ``` region = "europe-west3" ``` --- ## `.terraform` * Contains providers, modules and local state ``` .terraform │ modules │ providers │ terraform.tfstate ``` --- ## `.terraform.lock.hcl` * Lock file `.terraform.lock.hcl`: ```hcl # This file is maintained automatically by "terraform init". # Manual edits may be lost in future updates. provider "registry.terraform.io/hashicorp/google" { version = "4.35.0" constraints = ">= 3.53.0, ~> 4.35.0, < 5.0.0" hashes = [ "h1:ybilPdQgRamexNcj8YpH74jIVGqiBIGaMULs3wx249g=", "zh:122318ed6ba4253fca3fd0f75462c58337f1d86c34f20f2918e3fd12b2f0a31c", "zh:15eb57c0c54002d04610c7682393407b23fde732993bf023536ff33a7c43212d", "zh:29f7833338f778c1bf68d663a22c38ff92d642bf05a001005c570be7f55b6fae", "zh:4f61d7cc0b02ae9e456b5547fe617c880a6396727ef9126c6ec60fab2ee6cc9e", "zh:6b84cc5b571e453afc604899748954677211ad7d97c1eda0ac0ca27764169f7f", "zh:7171bab4ac33a7852e8a7813d0742f5772666d132f8688507ecca209b19f5b3e", "zh:802ba225215490587cdffe25dd180c7af9f0ba8cca9cbb0d8b31612cb0a594b4", "zh:af6aee1833ba6a0dce980fd2b61183496e2c3ae030bb6350bd6e8a3b033fe291", "zh:d146ef63698b53bf9dd22313250f1eab196c2f0aa84c3a4dce5a70b73c55a363", "zh:dd4d00ec04d119c33ec63ab3f422a52404cefa9c8075ccc177949d119ceae16f", "zh:e93cefc27b2ec7af10f35abc9e7c9fc6163af8f9dd2e028719b7280497113605", "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", ] } ``` --- # Alternatives * AWS CDK * Pulumi * CDK for Terraform --- ## Alternatives * No extra language, just use what ever you are use to: * Python * NodeJs * Golang * Imperative approach mixed with declaratives trends * Once code is erased, it have to be erased from to [cloud] * State is always present * Unit testing is done by the tests in the language --- ## CDK * npm install -g aws-cdk * cdk list * cdk synthetize - to cloudformation * cdk init app --language python|typescript * cdk diff - like terraform plan * cdk deploy & destroy - like terraform apply & destroy --- ## Pulumi * pulumi new * pulumi up - preview * pulumi stack output * pulumi console - UI hosted by Pulumi --- ## CDK for Terraform * Same principle * No HCL --- # Summary * IaC * State * Terraform * HCL * Alternatives