Done: Core Infrastructure - 8 April 2021

# Done: Core Infrastructure - 8 April 2021 > Notes: https://is.gd/huzifi or https://hackmd.io/@ajinkyakolhe112/ryJdwxhH_ > Slides: https://1drv.ms/b/s!Aq6hYeVV5o6Dh4BJ15DGit_UFMSSzw?e=qh0ir2 --- ## Notes General GCP Resources - Online copies of the the course today - https://www.coursera.org/learn/gcp-fundamentals ((Audit the course then you can do it for free. https://www.classcentral.com/report/coursera-signup-for-free/)) - https://cloudonair.withgoogle.com/events/apac-gcp-fundamentals-series - https://github.com/gregsramblings/google-cloud-4-words - Compare Cloud Providers 1. http://comparecloud.in/ 2. https://cloud.google.com/docs/compare/azure 3. https://cloud.google.com/docs/compare/aws - https://cloudwars.co/special-report-2020/ - https://go.acloudguru.com/rs/194-UHP-609/images/State_of_Cloud_Learning_Report_A_Cloud_Guru.pdf - https://www.gcpweekly.com - Check the learning paths: https://cloud.google.com/training#learning-paths - Today’s class is part of Infrastructure modernization | Google Cloud Training - Either or of following - Cloud Engineer - https://cloud.google.com/training/cloud-infrastructure#cloud-engineer-learning-path - Cloud Architect - https://cloud.google.com/training/cloud-infrastructure#cloud-architect-learning-path - But both have common starting point - Google Cloud Fundamentals: Core Infrastructure - Other Certifications: https://www.evernote.com/shard/s295/sh/ab8acf7b-98b0-46b3-afbd-3756b46a825e/ffb53c4f70d0fe7fb85f56a9a80bad2f ### Module 1 OnPrem vs Cloud vs Cloud Native / Serverless Cloud - OnPrem - User configured, user managed and user maintained - Cloud - User configured, provider managed and provider maintained - Different Ways of Using Cloud - (on prem is user bought, user configured & user maintained) - Infrastructure as a Service: User configured, user maintained & Provider provided - Platform as a Service / Managed Product: User configured, provider managed & maintained but partial work still is needed by user - Fully Managed / Serverless: Everything is done by the provider. User just codes - Example restaurant - On Prem: We cooking in Our own house - IaaS: You cook in the restaurant. Ingredients & kitchen is given by the restaurant & we cook with it. - PaaS: Buffet self service - Serverless: Waiter serves you prepared food - https://www.episerver.com/articles/pizza-as-a-service & https://www.bmc.com/blogs/saas-vs-paas-vs-iaas-whats-the-difference-and-how-to-choose/ - Cloud Native / Serverless Cloud - Fully automated and no configuration required Types of Jobs on Cloud 1. Migration. We move the codebase’s place of execution from on prem to on cloud - Google Cloud has partner ecosystem and patterns are heavily involved in migration. https://cloud.withgoogle.com/partners/ - Many of the technical guides related to migration: Find them here https://cloud.google.com/docs/tutorials 2. Cloud Native Development or New application: Generally startups do this or MNCs for a new application 3. Feature Addition on prexisting cloud env product 4. Maintenance Migration to Cloud 1. Lift & Shift. Move as is 2. Optimize for Cloud. Tiny rewrite to improve the performance 3. Complete Rewrite for Cloud Native or Restructure for Hybrid cloud via Container Evolution of Migration to Cloud 1. First On Prem either server or data center - My Infra, I buy it, I manage it 2. Then On Cloud. (First migration called lift & shift) - Someone else’s infra - Renting not Buying. But renting gives flexibility - House Rent vs Buy? - Renting gives us ability to change whenever 3. Then Cloud Native / Serverless / Fully Managed - Someone else’s Infra, but completely automatically managed - Iron Man? Mark I vs Mark 85 in End Game. nano tech Identity Access Management found in Navigation Menu -> IAM & Admin 1. Who 2. Identity verification of the User. 3. Can either be a Google Account or be a part of Google Group or be part of a domain or be a service account 2. Can do What 4. Roles a user can Have. Roles can be basic or fine grained. 3. on Which Resources ### Module 2 Understanding Scale & Auto Scale - Scale - Scale means can change either up or down - Scale up making it more powerful. More demand can be handled but it takes more cost - Scale up making it less powerful. Less demand is handled and it takes less cost - it doesn’t change automatically. We have to do it by executing commands or using UI - e.g. Ant Man is Scale - e.g. editing Compute engine disk - change it either via UI or CloudShell - Auto Scale - It can change automatically without user intervention - Harry Potter Room of Requirement is AutoScale - SQL storage - How do you scale or auto scale? One of two approaches - Vertical Scaling or Scale Up - Increased demand is met by increasing the power of the same machine - Decreased demand is met by decreasing the power of the same machine. That way bill reduces. - Single machine computing - Horizontal Scaling or Scale Out - Increased demand is met by increasing the number of machines working together - Reduced demand is met by decreasing the number of machines in the group. That way bill reduces - Distributed computing - This is a sustainable way because there is a upper limit to vertical scaling and it’s very expensive to build more powerful machines Scale & AutoScale List of Products - Scale (Need to manually call resize) - Compute Engine - BigTable - SQL Compute - Spanner - AutoScale (Resize happens automatically) - Compute Engine Instance Group - Google Cloud Storage - SQL Storage - DataStore - Horizontal Scale (Distributed Computing hence infinitely scalable) - Compute Engine Instance Group - Google Cloud Storage - BigTable - Spanner - DataStore - BigQuery - Vertial Scale - SQL Compute - Different kinds of Data. Every data is going to have 3 properties associated with it from below - Batch Data or Streaming Data: Data which is constantly arriving. - Structured or unstructured Data: (RDBMS or Non RDBMS Data) - (Non Big) Data or Big Data - Whenever Data is Big Data, cloud becomes necessary to deal with the volume of the Data - SQL, NoSQL, New SQL - Database is for transactional queries, datawarehouse is only for read hence optimized for large volume of read queries - NOSQl for Big Data - NewSQL for Structured Big Data Storage Products (https://cloud.google.com/products/storage & https://cloud.google.com/products/databases) 1. Cloud Storage - File System - Auto Scale - Fully Managed - https://cloud.google.com/storage 2. Bigtable - NoSQL wide columnar DB - Scale (horizontal) - (Partially) Managed - https://cloud.google.com/bigtable 3. SQL - SQL DB - Scale (vertical) - (Partially) Managed - https://cloud.google.com/sql 4. Spanner - New SQL DB - Scale (horizontal) - (Partially) Managed - https://cloud.google.com/spanner - https://quizlet.com/blog/quizlet-cloud-spanner 5. DataStore - NoSQL Document DB - NoSQL DB, format is key value pair - Autoscale (horitzontal) - Fully Managed - https://cloud.google.com/datastore 6. BigQuery - Relational SQL - For OLAP Operations - https://cloud.google.com/datastore https://db-engines.com/en/ranking ### Module 3 Docker Container is the (Code + Libraries + Operating System) to run We run the container by command docker run xzy. The container is running on the hardware. - Container First Approach. (Hybrid is part of Container based Approach) - Instead of OS, think in terms of virtualized (light weight OS. It is not a complete OS, it transferes all the work to the host os) called as Container - https://www.youtube.com/watch?v=4ht22ReBjno | The Illustrated Children’s Guide to Kubernetes - YouTube - https://www.youtube.com/watch?v=R9-SOzep73w | A Kubernetes story: Phippy goes to the zoo - YouTube - https://www.katacoda.com/courses/kubernetes | Learn Kubernetes using Interactive Browser-Based Labs | Katacoda - Examples of Containers in companies - https://kubernetes.io/case-studies/pinterest/ - https://www.youtube.com/watch?v=z7TIzCAEo0M&ab_channel=CNCF[CloudNativeComputingFoundation] - https://medium.com/tinder-engineering/tinders-move-to-kubernetes-cda2a6372f44 Components of Application Development * Application Code using Libraries * Databases with the Application (SQL or NoSQL or NewSQL) * Continuous Integration * Continuous Deployment * Infrastructure Creation Tools * Configuration Management on Infrastructure * API creation and maintainance and monteization and security * Business Intelligence. Data Analytics * Logging Components of Cloud * Compute: Where Application Code Runs * Storage: Where are files stored * Database * Networking * Security * Operations & Tools * API Creation & Management via Endpoint & Apigee * Cloud Scheduler * Creating the Infrastructure via Deployment Manager * Cloud Monitoring * Big Data * Artificial Intelligence All components of Application Development - Coding using Libraries - Databases with the Application - Databases have changed with Cloud - Different types of Databases - SQL, NoSQL, NewSQL - Different types of Databases for Batch vs Streaming Data - Compute Infrastructure. Place where the Code is going to Run - Dependancies related to Code - When we containerize the Code, container is (Code + Dependencies + Mini OS for the Code) - Version Control - Infrastructure Creation Tools: Deployment Manager - Create infrastructure to run the code - eg deployment manager - Don’t need in Serverless Products - Configuration Management on Infrastructure: Deployment Manager - Configure the created infrastructure so that code can be run - Optional: Containerization - Code can be containerized. If containerized, it can run anywhere. - Kubernetes manages containers and you manage kubernetes - App Engine is automatic to run your code. It runs in container format in background - API - Creation of API: Cloud Endpoint - Secure the API & Monetize API: APIgee - Logging & Monitoring: Stackdriver / Cloud Monitoring Compute Products (https://cloud.google.com/products/compute) 1. App Engine 1. Two Types of App Engine 1. App Engine Standard 1. Auto Scale 2. Fully Managed 3. From zero to full deployment takes less than a few seconds 2. App Engine Flexible (With great power comes great responsibilty.Similarly With great power comes great latency for cloud) 1. Auto Scale 2. Fully Managed but with more features, hence slower than app engine standard 3. More features but slower because of them 4. From zero to full deployment takes around 5 minutes 5. Not as snappy and quick to scale as App Engine Standard 2. Compute Engine 1. Infrastructure as a Service 3. Kubernetes Engine 1. Kubernetes decides where & how many Docker Container App 2. e.g. when installing an app in normal scenario, we need to downlaod the app specific to the OS version e.g. browse specific to windows 8, windows 10 or windows server 2007, 64 bit or 32 bit. But with container, an application can be run on any OS irrespective of what its version. Application will work on all the OS and versions. 4. Cloud Functions 1. Light weight temp event processing 5. Cloud Run 1. https://dev.to/pcraig3/cloud-run-vs-app-engine-a-head-to-head-comparison-using-facts-and-science-1225 --- ## Questions - [x] test question - [ ] https://is.gd/huzifi - we can open this URL in our system itself right? - [x] So is quota only for no of CPU limit? - [ ] Quotas are for every product - [x] On organisational level do we have access to create the project in google cloud. What kind of liscensing is required? - [ ] Licence on Google Cloud doesn't work like licence of Microsoft - [ ] Hope we have the option for mass loads - [ ] How Google cloud is doing in SAP workload ?? - [ ] does google cloud supports SAP HANA, if answer is yes, what is biggest machine supported in SAP HANA space. - [x] Do you mean Google Cloud doesn't support apps with Oracle DB? - [ ] Gooogle Cloud doesn't have native oracle DB, but it can be used via IaaS. If you check technical tutorial on gcp you will find how to do it. - [ ]