# Architecting on AWS ![Screenshot 2024-11-21 at 11.53.12](https://hackmd.io/_uploads/B1XrJshMkx.png) :::info [toc] ::: **Important Links** :::success [AWS SkillBuilder](https://explore.skillbuilder.aws/learn/signin) [AWS Ramp Up Guides](https://aws.amazon.com/training/ramp-up-guides/) [Online Course Supplement](https://explore.skillbuilder.aws/learn/course/8319/play/38904/online-course-supplement-architecting-on-aws) [Ebooks / Student Guide](http://online.vitalsource.com) [Training & Feedback](https://aws.training) [AWS Certification home page [All certifications]](https://aws.amazon.com/certification/) [AWS Solutions Architect - Associate [certification home page]](https://aws.amazon.com/certification/certified-solutions-architect-associate/) [AWS SA Associate Practice Exam](https://explore.skillbuilder.aws/learn/course/internal/view/elearning/13593/exam-prep-official-practice-exam-aws-certified-solutions-architect-associate-saa-c03-english) ::: **Keywords** :::info - AWS Unique Identifiers --> Resource id / ARN - CIDR Notation - RFC1918 - Agile Manifesto (2001) - Hypervisors (ESXi / HyperV / KVM / Xen Server / Nitro system) - HCI --> Hyper Converged Infrastructure (Cloud in a Box) - Storage Types --> SAN / NAS / DAS - SDDC - Server Virtualization (ESXi / HyperV / KVM) - Network Virtualization (SDN / NSX) - Storage Virtualization (SAN) - Submarine / Undersea cables - Configuration Drift - Out-of-band Changes - Idempotence - VM vs Container - VM --> hardware level Virtualization - Container --> OS / Kernel level Virtualization ::: --- ### AWS General Concepts --- #### [AWS Global Infrastructure](https://aws.amazon.com/about-aws/global-infrastructure/) --- :::success - Regions --> Multiple AZ - Availability Zones --> Cluster of DC - Edge Locations --> CDN | Web content Caching | CloudFront - AWS Outposts --> AWS infrastructure and services on premises for consistent hybrid experience - AWS Local Zones --> Run latency sensitive applications closer to end users - AWS Backbone Network --> High Speed, low latency physical cables connecting all AWS DCs ::: --- #### AWS services can be :::success - Managed - S3, DynamoDB, Internet Gateway, NAT Gateway - Unmanaged - EC2, NAT Instance, EBS ::: --- #### [AWS service scope](https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/aws-service-types.html) --- :::success - Global - Route53, IAM - Regional - VPC, DynamoDB, - AZ specific - EBS, EC2 ::: --- #### AWS Services can have [Quotas / Limits](https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html): --- :::success - Soft Limits - can be changed - S3 | 10,000 buckets per account - Lambda | 1000 parallele invocations - Hard Limits - cannot be changed - S3 | 5TB per object - Lambda | 10 GB max memory allocation per function - Lambda | 15 minutes execution time ::: :mag: *You can check your account specific limits by going to "**Service Quota**".* --- ### [Access Management on AWS](https://aws.amazon.com/identity/) --- **3 A's of Security** :::info - A - Authentication (2FA/MFA) - What you know --> Credentials - What you have --> OTP, Soft/hard tokens - What you are --> Biometrics - A - Authorization - A - Accountability or Accounting ::: **Identity and Access Management** :::info - IAM - Users - Groups - Policies - Roles - Temporary elevated priviliges - Federated access - Services/applications trying to access a AWS resource ::: --- ### [Compute on AWS](https://aws.amazon.com/products/compute/) --- #### Compute Types :::warning - Virtual Machines - EC2 - Containers - ECS - EKS - Serverless - Lambda ::: --- #### EC2 Nomenclature :::warning **m5.xlarge** m --> Family 5 --> Generation xlarge --> Size (vCPU / Memory / Network Bandwidth) ::: --- #### EC2 Pricing Options :::warning - On demand - Spot Instances (Upto 90% discount) - can be reclaimed by AWS (2 minutes notice) - Works great with ECS, EKS, Batch - [AWS Spot advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) - Integration with ASG - Commitment Based | 70% discounts over Ondemand | 1/3 year contract | PUP/NUP/FUP - Reserved Instance - Savings Plan - Hardware Isolation - Dedicated Instance - Dedicated Hardware ::: --- ### [Networking on AWS](https://aws.amazon.com/products/networking/) --- #### Networking Part 1 [Components of a single network] :::info - VPC **Routing** - Subnets (Public or Private) - Route Tables - Internet Gateway - NAT Gateway (NAT Instance) - ENI (Elastic Network Interface) (Ethernet Card --> vNIC ---> ENI) - Elastic IP --> Static IPs in AWS **Security** - Security Groups - Firewall at the instance level - Default: Allow all outgoing; block all incoming - Stateful - Explicit Deny not supported, only allow - NACLs - Firewall at the subnet level - Default: Allow all incoming / outgoing traffic - Stateless - Allow/deny rules are supported ::: #### Networking Part 2 [Connecting Multiple Networks] :::info - Hybrid / Connectivity - VPC Peering - VPN - Direct Connect - Transit Gateway - Virtual Private Gateway (VGW) - VPC Endpoints - Gateway --> S3 and DynamoDB - Interface --> Almost all the other services ::: #### CIDR Notation :::info 10.0.0.0/8 --> 10.{0-255}.{0-255}.{0-255} --> 256x256x256 --> 16,777,216 172.31.0.0/16 --> 172.31.{0-255}.{0-255} --> 256x256 --> 65,536 192.168.1.0/24 192.168.1.{0-255} --> 256 0.0.0.0/0 --> Internet 1.2.3.4/32 --> CIDR Notation for 1.2.3.4 ::: #### 5 IPs are reserved by AWS :::info - 10.0.0.0: Network - 10.0.0.1: Gateway (router) - 10.0.0.2: DNS - 10.0.0.3: Unknown or future use - 10.0.0.255: Broadcast ::: --- ### [AWS Storage Portfolio](https://aws.amazon.com/products/storage/) --- :::warning - Object Storage - S3 | Internet accessible Object storage (Pay for what you use) - Storage classes - 99.999999999% durability (replication in >= AZs) - Versioning - Cross-region replication - Access control on S3 - Bucket Policies - IAM policies - Bucket ACL (Legacy way) - Replication - SRR - CRR - Event Notification - Transfer acceleration - Multipart upload - Block Storage | Attach the volumes - Elastic Block Store (EBS) | Persistent block Storage | SAN (Pay for what you provision) - SSD Based - Provisioned IOPS SSD (io2 Block Express, io2 and io1) - General Purpose SSD (gp2 and gp3) - HDD Based - Throughput Optimized HDD (st1) - Cold HDD (sc1) - Instance Store | Ephemeral Storage | DAS (Complimentary) - File/Network Storage | Mount the file systems (Pay for what you use) - EFS | NFS 4.0 and 4.1 | Linux workloads - FSx | NTFS, CIFS, SMB | Windows workloads ::: --- ### [Databases on AWS](https://aws.amazon.com/products/databases/) --- #### Relational :::warning - RDS - DB Engines - Amazon Aurora - PostgreSQL compatible - MySQL Compatible - MySQL - MS SQL - Oracle - PosgreSQL - Maria DB - IBM Db2 - RDS Features - Multi-AZ (Synchronous Replication) --> High Availablity - Read replicas (Asynchronous Replication) --> Performance Benefit - Automated backups (Upto 35 Days retention) ::: #### Non Relational :::warning - DynamoDB - Fully Managed non-relational DB service - Global Tables - Eventually and Strongly consistency models - Extreme horizontal scaling capacity - DocumentDB [MongoDB_compatible] - Keyspaces [Managed_Apache_Cassandra] ::: #### Other Purpose-Built Databases :::warning - Elasticache [Database_Caching] - Redis - Memcached - Neptune [Graph_Database] - Amazon QLDB [Ledger/Blockchain] - Amazon Timestream [Time_Series] - Redshift [OLAP] [Data_Warehousing] ::: #### DB Migration :::warning MySQL (on Prem) ----> AWS DMS ----> AWS RDS MySQL [Homogeneous Migration] MySQL (on Prem) ----> AWS DMS + AWS SCT ----> AWS RDS Aurora [Heterogeneous Migration] ::: --- ### [Monitoring on AWS](https://aws.amazon.com/cloudops/monitoring-and-observability/) --- :::warning - CloudWatch - Basic Monitoring (Default with 5 minutes granularity) - Detailed Monitoring (Additional Charges, 1 minute granularity) - CloudTrail - VPC Flow logs - VPC - Subnet - ENI (EC2) ::: --- ### [Load Balancing on AWS](https://aws.amazon.com/elasticloadbalancing/) --- :::warning - ELB (Elastic Load Balancer) - ALB | L7 | http and https | Content / path based routing - NLB | L4 | tcp, udp, tls | IP based routing - GLB | L3/L4 | IP filtering | Third party security applicances - CLB | L4/L7 | Legacy LB | works with EC2-Classic network ::: :::warning Triad of services for Self Healing: CloudWatch + ELB + ASG ::: --- ### [Automation on AWS](https://docs.aws.amazon.com/prescriptive-guidance/latest/migration-operations-integration/aws-services-for-automation.html) --- :::warning - Infrastructure-as-Code (Provisioning) - Cloudformation - Terraform - CDK - HEAT template - Configuration Management (Maintenance of the State) - AWS Systems Manager - Chef - Puppet - Ansible - AWS OpsWorks ::: --- ### [Container Services on AWS](https://aws.amazon.com/containers/services/) --- #### Docker Architecture :::warning - Docker Host - Docker Daemon - Docker Images (~~ AMIs) - Docker Containers (~~ EC2 Instances) - Docker Registry - Public --> https://hub.docker.com/ - Private --> ECR ::: #### Container Runtime (VM) :::warning - Docker - rkt - runc - containerd - podman ::: #### Container Orchestration Engines (CoE) :::warning **On-Prem** - Kubernetes - Docker Swarm - Apache Mesos - RedHat Open Shift **CloudBased** - ECS - EKS - AKS - GKE ::: #### AWS Container Services :::warning - ECS (Amazon's Proprietory Software) - EC2 mode - Fargate mode - EKS (AWS managed/hosted Kubernetes) - EC2 Mode - Fargate Mode - ECR (Elastic Container Registry) ::: --- ### [Distributed architecture on AWS](https://aws.amazon.com/microservices/) --- :::success - SQS **Queue Types** - Standard queues - At-least once delivery - Unlimited TPS - Order not guaranteed - FIFO queues - Exactly Once delivery - Order maintained - upto 300 messages per second (can be increased with Batching) **Features** - Dead letter queue support - Visibility Timeout - Long polling - Polling Mechanism - Producer/Consumer model - SNS - Publisher/subscriber model - Supported subscribers - Email - http/https - SMS - SQS queus - Lambda functions - One to Many - Push mechanism - Amazon MQ - based on Apache ActiveMQ and RabbitMQ - Active/Standby model for High Availability - Supports industry standard protocols such as JMS, NMS, AMQP etc. ::: --- ### [Serverless on AWS](https://aws.amazon.com/serverless/) --- :::info - AWS Lambda - Amazon API Gateway - AWS Step Functions ::: --- ### [Edge Services (Caching) on AWS](https://aws.amazon.com/edge/services/) --- :::success - AWS Route 53 --> DNS on AWS - Amazon CloudFront --> CDN service on AWS - DDoS Protection - WAF - Shield - Standard - Free - enabled by default - Adavnced - USD 3000/Month - 24x7 Access to SRT (Shield Response Team) - Needs to be subscribed ::: --- ### [Additional Services](https://aws.amazon.com/products/security/detection-and-response/) --- :::info - Macie --> Discover and protect your sensitive data using ML - Guardduty --> Automated Intelligent Threat detection using ML - Inspector --> Automated Vulnarability detection and Management - Kinesis --> Collect, process, and analyze video and data streams in real time ::: --- ### References --- :::success **General** - https://aws.amazon.com/architecture/well-architected/ - https://www.finops.org/introduction/what-is-finops/ - https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_examples.html - https://aws.amazon.com/s3/transfer-acceleration/ - https://aws.amazon.com/quickstart/ - https://wellarchitectedlabs.com/ - https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_boundaries.html - https://datatracker.ietf.org/doc/html/rfc1918 - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html - https://aws.amazon.com/ec2/instance-types/ - https://instances.vantage.sh/ - https://aws.amazon.com/fis/ - https://www.12factor.net/ - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/mappings-section-structure.html - https://aws.amazon.com/certification/certification-prep/ - https://aws.amazon.com/blogs/architecture/disaster-recovery-dr-architecture-on-aws-part-i-strategies-for-recovery-in-the-cloud/ **Multi-region Architectures** - AWS re:Invent 2022 - Multi-Region design patterns and best practices (ARC306) --> https://www.youtube.com/watch?v=ilgpzlE7Hds - AWS re:Invent 2019: [REPEAT 2] Architecture patterns for multi-region active-active (ARC213-R2) --> https://www.youtube.com/watch?v=3K9AzSrCmiQ - AWS re:Invent 2018: Architecture Patterns for Multi-Region Active-Active Applications (ARC209-R2) --> https://www.youtube.com/watch?v=2e29I3dA8o4 **Spot Instances** - https://aws.amazon.com/ec2/spot/instance-advisor/ - https://github.com/awslabs/ec2-spot-labs - https://ec2spotworkshops.com/ **Networking** - https://tgw.networking-workshop.com/#/README - https://catalog.workshops.aws/networking/en-US - https://docs.aws.amazon.com/vpc/latest/tgw/TGW_Scenarios.html **Whitepapers** - https://d0.awsstatic.com/whitepapers/Security/AWS_Security_Whitepaper.pdf - https://docs.aws.amazon.com/whitepapers/latest/how-aws-pricing-works/key-principles.html - https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf - https://d1.awsstatic.com/whitepapers/Security/DDoS_White_Paper.pdf - https://docs.aws.amazon.com/pdfs/whitepapers/latest/introduction-devops-aws/introduction-devops-aws.pdf **Container Resources - Beginner** - https://docs.docker.com/get-started/overview/ - https://docker-curriculum.com - https://www.docker.com/play-with-docker **Container Resources - Intermediate** - https://ecsworkshop.com/ - https://eksworkshop.com/ ::: --- ### Workshops --- :::info **Generic** - [12 Weeks Workshop Challenge](https://12weeksworkshops.com/) **IaC** - [AWS CloudFormation Workshop](https://catalog.workshops.aws/cfn101/en-US) - [AWS Terraform Workshop](https://aws-quickstart.github.io/workshop-terraform-modules/) - [AWS CDK Workshop](https://cdkworkshop.com/) **Serverless** - [AWS State Function Workshop](https://catalog.workshops.aws/stepfunctions/en-US) - [Build a Serverless Web Application using Generative AI](https://aws.amazon.com/getting-started/hands-on/build-serverless-web-app-lambda-amplify-bedrock-cognito-gen-ai/) **Containerization** - [AWS ECS Workshop](https://ecsworkshop.com/) - [AWS EKS Workshop](https://eksworkshop.com/) ::: :::danger :warning: *Disclaimer: The content provided here is for informational purposes only and is based purely on my own understanding, knowledge and experience. This is **NOT** an official AWS documentation.* :::