# Architecting on AWS --- ### ########################### ### Architecting on AWS [Class Notes] ### ########################### --- **Important Links** :::success [AWS Builder Labs](https://us-east-1.student.classrooms.aws.training/class/3kx9YZjiCKEqPp8AqchhiA) [Ebooks / Student Guide](http://online.vitalsource.com) [Training&Feedback](https://aws.training) [Online Course Supplement](https://explore.skillbuilder.aws/learn/course/8319/play/38904/online-course-supplement-architecting-on-aws) [AWS SkillBuilder](https://explore.skillbuilder.aws/learn/signin) [AWS Ramp Up Guides](https://aws.amazon.com/training/ramp-up-guides/) ::: --- ### Keywords --- :::info - AWS Unique Identifiers --> Resource id / ARN - CIDR Notation - RFC1918 - Agile Manifesto (2001) - Hypervisors (ESXi / HyperV / KVM / Xen Server / Nitro system) - HCI --> Hyper Converged Infrastructure (Cloud in a Box) - Storage Types --> SAN / NAS / DAS - SDDC - Server Virtualization (ESXi / HyperV / KVM) - Network Virtualization (SDN / NSX) - Storage Virtualization (SAN) - Submarine / Undersea cables - Configuration Drift - Out-of-band Changes - Idempotence - VM vs Container - VM --> hardware level Virtualization - Container --> OS / Kernel level Virtualization ::: --- ### AWS General Concepts --- #### AWS Global Infrastructure :::success - Regions --> Multiple AZ - Availability Zones --> Cluster of DC - Edge Locations --> CDN | Web content Caching | CloudFront - AWS Outposts --> AWS infrastructure and services on premises for consistent hybrid experience - AWS Local Zones --> Run latency sensitive applications closer to end users - AWS Backbone Network --> High Speed, low latency physical cables connecting all AWS DCs ::: --- #### AWS services can be :::success - Managed - S3, DynamoDB, Internet Gateway, NAT Gateway - Unmanaged - EC2, NAT Instance, EBS ::: --- #### AWS service scope :::success - Global - Route53, IAM - Regional - VPC, DynamoDB, - AZ specific - EBS, EC2 ::: --- #### AWS Services can have Quotas / Limits: :::success - Soft Limits - can be changed (S3 - 100 buckets per account) - Hard Limits - cannot be changed (S3 - 5TB per object) ::: :mag: You can check your account specific limits by going to "Service Quota". --- ### Access Management on AWS --- **3 A's of Security** :::info - A - Authentication (2FA/MFA) - What you know --> Credentials - What you have --> OTP, Soft/hard tokens - What you are --> Biometrics - A - Authorization - A - Accountability or Accounting ::: **Identity and Access Management** :::info - IAM - Users - Groups - Policies - Roles - Temporary elevated priviliges - Federated access - Services/applications trying to access a AWS resource ::: --- ### Compute on AWS --- #### Compute Types :::warning - Virtual Machines - EC2 - Containers - ECS - EKS - Serverless - Lambda ::: --- #### EC2 Nomenclature :::warning m5.xlarge m --> Family 5 --> Generation xlarge --> Size (vCPU / Memory / Network Bandwidth) ::: --- #### EC2 Pricing Options :::warning - On demand - Spot Instances (Upto 90% discount) - can be reclaimed by giving a 120 seconds notice - Works great with ECS, EKS, Batch - Spot advisor (https://aws.amazon.com/ec2/spot/instance-advisor/) - Integration with ASG - Commitment Based | 70% discounts over Ondemand | 1/3 year contract | PUP/NUP/FUP - Reserved - Savings Plan - Hardware Isolation - Dedicated Instance - Dedicated Hardware ::: --- ### Networking on AWS --- #### Networking Part 1 [Components of a single network] :::info - VPC **Routing** - Subnets (Public or Private) - Route Tables - Internet Gateway - NAT Gateway (NAT Instance) - ENI (Elastic Network Interface) (Ethernet Card --> vNIC ---> ENI) - Elastic IP --> Static IPs in AWS **Security** - Security Groups - Firewall at the instance level - Default: Allow all outgoing; block all incoming - Stateful - Explicit Deny not supported, only allow - NACLs - Firewall at the subnet level - Default: Allow all incoming / outgoing traffic - Stateless - Allow/deny rules are supported ::: #### Networking Part 2 [Connecting Multiple Networks] :::info - Hybrid / Connectivity - VPC Peering - VPN - Direct Connect - Transit Gateway - Virtual Private Gateway (VGW) - VPC Endpoints - Gateway --> S3 and DynamoDB - Interface --> Almost all the other services ::: #### CIDR Notation :::info 10.0.0.0/8 --> 10.{0-255}.{0-255}.{0-255} --> 256x256x256 --> 16,777,216 172.31.0.0/16 --> 172.31.{0-255}.{0-255} --> 256x256 --> 65,536 192.168.1.0/24 192.168.1.{0-255} --> 256 0.0.0.0/0 --> Internet 1.2.3.4/32 --> CIDR Notation for 1.2.3.4 ::: #### 5 IPs are reserved by AWS :::info - 10.0.0.0: Network - 10.0.0.1: Gateway (router) - 10.0.0.2: DNS - 10.0.0.3: Unknown or future use - 10.0.0.255: Broadcast ::: --- ### AWS Storage Portfolio :::success - Object Storage - S3 | Internet accessible Object storage (Pay for what you use) - Storage classes - 99.999999999% durability (replication in >= AZs) - Versioning - Unlimited storage (only limit of 5TB for individual file size) - Cross-region replication - Access control on S3 - Bucket Policies - IAM policies - Bucket ACL (Legacy way) - Block Storage | Attach the volumes - Elastic Block Store (EBS) | Persistent block Storage | SAN (Pay for what you provision) - SSD Based - Provisioned IOPS SSD (io2 Block Express, io2 and io1) - General Purpose SSD (gp2 and gp3) - HDD Based - Throughput Optimized HDD (st1) - Cold HDD (sc1) - Instance Store | Ephemeral Storage | DAS (Complimentary) - File/Network Storage | Mount the file systems (Pay for what you use) - EFS | NFS 4.0 and 4.1 | Linux workloads - FSx | NTFS, CIFS, SMB | Windows workloads ::: --- ### Databases on AWS --- #### Relational :::info - RDS DB Engines - MySQL - MS SQL - Oracle - PosgreSQL - Maria DB - Amazon Aurora - RDS Features - Multi-AZ (Synchronous Replication) --> High Availablity - Read replicas (Asynchronous Replication) --> Performance Benefit - Automated backups (Upto 35 Days retention) - Database Sharding (For Performance benefits) ::: #### Non Relational :::info - DynamoDB - Fully Managed non-relational DB service - Global Tables - Eventually and Strongly consistency models - Extreme horizontal scaling capacity - DocumentDB [MongoDB_compatible] - Keyspaces [Managed_Apache_Cassandra] ::: #### Other Purpose-Built Databases :::info - Elasticache [Database_Caching] - Redis - Memcached - Neptune [Graph_Database] - Amazon QLDB [Ledger/Blockchain] - Amazon Timestream [Time_Series] - Redshift [OLAP] [Data_Warehousing] ::: #### DB Migration :::info MySQL (on Prem) ----> AWS DMS ----> AWS RDS MySQL MySQL (on Prem) ----> AWS DMS + AWS SCT ----> AWS RDS Aurora ::: --- ### Monitoring on AWS --- :::warning - CloudWatch - Basic Monitoring (Default with 5 minutes granularity) - Detailed Monitoring (Additional Charges, 1 minute granularity) - CloudTrail - VPC Flow logs - VPC - Subnet - ENI (EC2) ::: --- ### Load Balancing on AWS --- :::warning - ELB (Elastic Load Balancer) - ALB | L7 | http and https | Content / path based routing - NLB | L4 | tcp, udp, tls | IP based routing - GLB | L3/L4 | IP filtering | Third party security applicances - CLB | L4/L7 | Legacy LB | works with EC2-Classic network ::: :::warning Triad of services for Self Healing: CloudWatch + ELB + ASG ::: --- ### Automation on AWS --- :::warning - Infrastructure-as-Code (Provisioning) - Cloudformation - Terraform - CDK - HEAT template - Configuration Management (Maintenance of the State) - AWS Systems Manager - Chef - Puppet - Ansible - AWS OpsWorks ::: --- ### Containerization --- #### Docker Architecture :::warning - Docker Host - Docker Daemon - Docker Images (~~ AMIs) - Docker Containers (~~ EC2 Instances) - Docker Registry - Public --> https://hub.docker.com/ - Private --> ECR ::: #### Container Runtime (VM) :::warning - Docker - rkt - runc - containerd - podman ::: #### Container Orchestration Engines (CoE) :::warning **On-Prem** - Kubernetes - Docker Swarm - Apache Mesos - RedHat Open Shift **CloudBased** - ECS - EKS - AKS - GKE ::: #### Container Services on AWS :::warning - ECS (Amazon's Proprietory Software) - EC2 mode - Fargate mode - EKS (AWS managed Kubernetes service) - EC2 Mode - Fargate Mode - ECR (Elastic Container Registry) ::: --- ### Decoupling on AWS --- :::success - SQS **Queue Types** - Standard queues - At-least once delivery - Unlimited TPS - Order not guaranteed - FIFO queues - Exactly Once delivery - Order maintained - upto 300 messages per second (can be increased with Batching) **Features** - Dead letter queue support - Visibility Timeout - Long polling - Polling Mechanism - Producer/Consumer model - SNS - Publisher/subscriber model - Supported subscribers - Email - http/https - SMS - SQS queus - Lambda functions - One to Many - Push mechanism - Amazon MQ - based on Apache ActiveMQ and RabbitMQ - Active/Standby model for High Availability - Supports industry standard protocols such as JMS, NMS, AMQP etc. ::: --- ### Serverless Architecture on AWS --- :::info - AWS Lambda - Amazon API Gateway - AWS Step Functions ::: --- ### Edge Services (Caching) on AWS --- :::success - AWS Route 53 --> DNS on AWS - Amazon CloudFront --> CDN service on AWS - DDoS Protection - WAF - Shield - Standard - Free - enabled by default - Adavnced - USD 3000/Month - 24x7 Access to SRT (Shield Response Team) - Needs to be subscribed ::: --- ### Additional Services --- :::info - Macie --> Discover and protect your sensitive data using ML - Guardduty --> Automated Intelligent Threat detection using ML - Inspector --> Automated Vulnarability detection and Management - Kinesis --> Collect, process, and analyze video and data streams in real time ::: --- ### References --- - https://aws.amazon.com/architecture/well-architected/ - https://www.finops.org/introduction/what-is-finops/ - https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_examples.html - https://aws.amazon.com/s3/transfer-acceleration/ - https://aws.amazon.com/quickstart/ - https://wellarchitectedlabs.com/ - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html - https://aws.amazon.com/ec2/instance-types/ - https://instances.vantage.sh/ - https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf - https://docs.aws.amazon.com/vpc/latest/tgw/TGW_Scenarios.html - https://d1.awsstatic.com/whitepapers/Security/DDoS_White_Paper.pdf - https://aws.amazon.com/fis/ - https://www.12factor.net/ - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/mappings-section-structure.html - https://d0.awsstatic.com/whitepapers/Security/AWS_Security_Whitepaper.pdf - https://docs.aws.amazon.com/whitepapers/latest/how-aws-pricing-works/key-principles.html - https://aws.amazon.com/certification/certification-prep/ **Docker Beginner** - https://docs.docker.com/get-started/overview/ - https://docker-curriculum.com - https://www.docker.com/play-with-docker **Docker Intermediate** - https://ecsworkshop.com/ - https://eksworkshop.com/ :::danger :warning: *Disclaimer: The content provided here is for informational purposes only and is based purely on my own understanding, knowledge and experience. This is **not** an official AWS documentation.* :::