# Google Cloud Training
### TOC
- [GCP Training Day 1](#GCP-Training-Day-1)
- [Topology](#Topology-(Regions-and-Zones))
- [VMs](#VMs)
- [VPC and Networking](#VPCs-and-networking)
- [VPC Peering](#VPC-Peering)
- [DNS](#DNS)
- [Responsibility models](#Responsibility-models)
- [Hierarcy level model](#Hierachy-level-model)
- [Labs - day 1](#Labs---day-1)
- [Questions](#Questions)
- [GCP Training Day 2](#GCP-Training-Day-2)
- [VMs - compute](#VMs---compute)
- [Labs - day 2](#Labs---day-2)
-
- [GCP Training Day 3](#GCP-Training-Day-3)
- [GCP Training Day 4](#GCP-Training-Day-4)
- [Notes](#Notes)
# GCP Training Day 3
### Plan
Google cloud Storage: Google Git style storage
## Storage types
Google Cloud offers multiple types and layered tiers for storage. Some are block storage (Cloud storage), some are relational (sql type), some are non-relational (document style), and there was one more.
Some of these storage types also contain options for layered tiers (hot, semi hot, cold, archieve), and may have options for retention policies. Many of the options also have encryption options. All options have full access control systems to restrict access on a granular level.
# GCP Training Day 1
### Notes
Hadoop stemmed from Cloud
ML/AI: Tensorflow
Google OpenSourced Kubernetes after use internally
GCP creates GLOBAL VPC resources, contrary to other providers that make regional resources
Google is leader in NLP with BERT??
Big Query: scalable datawarehouse which can manage PB of data
Anthos: Multi cloud strategy
Gartner report: Google APIgee as the leader in this cloud strategy space
GCP pricing strategy is transparent, and the lowest of all providers
### Hacks
To switch off all resources within a "project" in one click, disable billing (NB! this could delete certain resources...).
If you want to retain **all** resources, shut them down individually in GCP
## Topology (Regions and Zones)
Different regions exists across each "continent" (US, EU, Asia). Within each region, there will be one or more data centers. Depending on the Regions, it is possible to set up granular definition where resources will run. Within larger regions, there will also be multiple zones (3 zones, consisting of 3 different data centers seperated by 10-15 miles).
Regions: Large blue have 3 zones within, large white are future planned with 3 zones.
Network:
For data transfers, topology will matter in terms of communication!
VPCs can span multiple regions and zones in GCP, unlike other providers.
Resources that can reach each other on internal IPs/network, will cost considerable less than those VPCs and networks that need to communicate through egress to the web.
## VMs
Once created in a VPC/network, they cannot be moved outside of their zone. They will need to be moved by creation of 'copies', i.e. making a machine image, deploying image, and copying data over
## VPCs and networking
VPCs have a firewall that links all resources togther. It is important to note, that there are default firewall rules with lowest possible priority, 65 535. Any new firewall rules made will have higher priority to override the "deny all ingress/allow all egress". It is important to set up allowance for TCP/UDP traffic if needed, and SSH to be able to remotely connect to hosts.
Generally VPCs are a resource within a project/subscription, but an organization entity can set up a 'host VPC', which would encompass all resources/VPCs within child projects, and allow them to cross communicate on internal IPs. Hence it is **very** important to preplan IP subnet ranges across VPCs and projects that potentially must cross communicate!
## VPC Peering
It is possible to set up a secure tunnel between two VPCs. This is done through a VPC peering connection. **Do bear in mind, that an established VPC peering needs to be set up *both* ways!** And this PVC peering connection **does not override any firewall rules that may block communication traffic in and out of each individual VPC**. Hence, creating a peering does not automatically allow all communication, but it does allow communication to occurr over **internal IPs**.
## DNS
Google provides a powerful DNS for within the resources used. Hosts can be given full FQDN names, instead of just hostnames/IPs. This enabled ingress from "public" web, and can be used to provide together with LoadBalancing ingress for webservices, microservices and more
## Responsibility models
On-premise is 100% client's responsibility
IaS means that everything beneath "operations" becomes managed (infra, access, data, network)
PaS adds identity authentication and authorization
Managed services also takes care of web app security and deployment
## Hierarchy level model
## Labs - day 1
### Lab 1
VM set up, VPC configuration, Firewall configurations, VPC peering and ping tests for connectivity
## Questions
- How to put unnecessary resources to sleep in GCP
- Does internal IP communication mean no network costs, even if VPC spans regions and zones?
# GCP Training Day 2
## Plan
compute types, LBs, webservices,
## VMs - compute
Regular VMs are available, and "high" prices. These VMs are stable and won't easily go down.
Pre-emptive VMs however, are meant as a cost saving type of VM, **that can maximum run 24 hours(!). There is also a 30 second graceful shutdown window that Google can trigger at any time.** This means that pre-emptive VMs should only be used for ephemeral workloads in a distributed compute set up. Masters should delegate work, and consiste of regular VMs. If work delegated isn't returned, the master can reschedule the work to another node. As such, there is no danger if the worker node is taken down during the compute work. But the rental of pre-emptive VMs **can be up to 80% cheaper** than regular VM prices.
Instance groups under Compute tab in GCP is a nice way to always ensure that a set number of worker nodes are available, and if a pre-emptive VM is taken down, the instance group will spin up a new worker under the specifications given to the group.
## IAM and access
If there is a conflict between roles and access through IAM, Google uses the pattern of ***least restrictive permission will get applied***. Say Mark is allowed all services access on an organizational level. But on a resource level, Mark is listed as being denied for BigQuery. The result is, that Mark will have access to BigQuery.
Google recommends that users are granted permissions only on a need basis, at the lowest level possible. Higher levels should be as restrictive as possible, otherwise the *higher* level of permission will be applied.
- Organization
- Project
- Service
- Resource
Google recommends to use Audit logs regularly to monitor policies.
## Labs - day 2
### 2 - Loadbalancer and 'siege' testing
1 VM as loadbalancer per approx 100k conns
- Remove all previous configurations in our GCP project
- Create new VPC, with auto mode and default FW rules. Convert to manual mode, and add custom rules
- Create instance templates **(these are valid for specific regions! Make sure to enable correct network within template)**
- Be sure to select correct network and subnet + adding network tags for ingress `http-server`
- Set up either optional startup script, or point to a startup-script-url location with metadata KV
- startup-script-url = gs://cloud-training/gcpnet/httplb/startup.sh
- Create health checks **(be sure to set a high enough interval, lowish timeout, and a high count of failure threshold, so that VMs don't get swapped too often)**
- Configure Instance groups
- Configure LoadBalancer
startup-script:
```
#! /bin/bash
apt-get update
apt-get install -y apache2 php
apt-get install -y wget
cd /var/www/html
rm index.html -f
rm index.php -f
wget https://storage.googleapis.com/cloud-training/gcpnet/httplb/index.php
META_REGION_STRING=$(curl "http://metadata.google.internal/computeMetadata/v1/instance/zone" -H "Metadata-Flavor: Google")
REGION=`echo "$META_REGION_STRING" | awk -F/ '{print $4}'`
sed -i "s|region-here|$REGION|" index.php
```
# GCP Training Day 4
Sign in with Wallet
Connect another wallet