AWS Certified Solutions Architect Associate
SAA-C02 :desktop_computer:
===
###### tags: `AWS`
IAM -- Identity and Access Management
===
## What's IAM?

**IAM allows you to manage users and their level of access to the AWS console.**
### IAM offers:
* Centrallised control of your AWS account
* Shared Acccess to your AWS account
* Granular Permissions
* Identity Federation
* Multifactor Authentication
* Privide temporary access for users/devices and services where necessary
* Allows you to set up your own password rotation policy
* Integrates with many different AWS services
* Supports PCI DSS Compliance

### Exam tips
* IAM is==universal==
* The==root account==is simply the account created when first setup your AWS account. It had complete Admin access.
* New Users have==NO permissions==when first created.
* Nwe Users are assigned==Access Key ID & Secret Access Keys== when first created.
* You==cannot== use the Access key ID & Secrect Key to login into the console. You can use this to access AWS via the APIs and CLI.
* ==You only get to view these once.== If you lose them, you have to regenerate.
* **Always setup Multifactor Authentication on root account.**
* You can create and customise your own **password rotation policies.**

# S3 -- Simple Storage Service

## S3 101
### What Is S3?
* S3 is **Object-based** -- i.e. allows you to upload files.
* Files can be from 0 to 5 TB.
* There is unlimited storage.
* Files are stored in Buckets.

### S3 -- Objects

* Think of Objects just as files.
* Objects consist:
* Key (name)
* Value (data)
* Version ID (Important for versioning)
* Metadata (data about data you are storing)
* Subresources;
* ACL -- Access Control Lists
* Torrent
### Guarantees
* Built for 99.99% availability for the S3 platform.
* Guarantee ==99.9% availability==.
* Guarantee ==99.999999999% durability== for S3 information. (11x9s)

### Features
* Tiered Storage Available
* Lifecycle Management
* Versioning
* Encryption
* MFA Delete
* Secure data using **Access Control Lists** and **Bucket Policies**

### S3 Storage Classes

1. Standard: 99.99% availability
2. Infrequently Accessed(IA): Lower fee than S3, but charged a retrieval fee.
3. One Zone - IA: Lower-cost option for IA
4. Intelligent Tiering: Optimize costs by automatically moving data.
5. Glacier: Data archiving, super cheap.
6. Glacier Deep Archive: Retrieval time up to 12 hours.


## S3 Security & Encryption
### 3 Types Encryptions
1. **Encryption in transit is achieved by:**
- SSL/TLS

2. **Encryption at rest (server side) is achieved by:**

- S3 Managed Keys - SSE -S3
- AWS Key Management Service, Managed Keys - SSE-KMS
- Server Side Encryption With Customer Provided Keys - SSE-C

3. **Client side encryption**

## S3 Versioning
### Features
* Stores all versions of an object(writes and deletes)
* Great backup tool
* Once enabled, **Versioning cannot be disabled**, only suspended.
* Integrates with **Lifecycle** rules
* Versioning's MFA Delete capability, which uses multi-factor authentication, can be used to provide an additional layer of security.
* If you want to delete a bucket, you have to delete the delete marker.

## S3 Object Lock

### What Is S3 Object Lock?
* Store objects using a **write once, read many(WORM)** model.
* Prevent objects from being deleted or modified for a fixed amount of time or indefinitely.
* Meet regulatory requirements that require WORM storage, or add an extra layer of protection against object changes and deletion.
* Can be on individual objects or **applied across the bucket** as a whole.

### S3 Object Lock Mode:

1. Governance mode: 要有permission才能動
* Users can't overwrite or delete an object version or alter its lock settings unless they have special permissions.
2. Compliance Mode: 誰都不准動，包含root
* A protected object version can't be overwritten or deleted by any user, including the root user in AWS account.

### Retention Periods
* 保護一段時間，之後就可以覆寫or刪除
* Protects an object version for ==a fixed amount of time==
* After the retention period expires, the object versino can be ==overwritten or deleted==, unless you placed a **legal hold** on the object version.

### Legal Holds
* 保護版本被覆寫or刪除，除非是有permission 才能動
* **Prevents an object version from being overwritten or deleted.**
* Unless you have the **s3:PutObjectLegalHold** permission.

### Glacier Vault Lock
* 控制S3 WORM policy
* Easily deploy and enforce compliance controls for individual S3 Glacier vaults with a Vault Lock policy.
* Once locked, the policy can **no longer** be changed.

## S3 Performance

### S3 Prefixs:

* The more prefix the better performance and requests per second.


### KMS limits:
* If you are using **SSE-KMS** to encrypt your objects in S3, you must keep in mind the **KMS limits**.
* When you upload a file, will call **GenerateDataKey** in the KMS API.
* When you **download** a file, you will call **Decrypt** in the KMS API.

### Multipart Uploads:

1. 檔案超過100MB時建議使用，超過5GB一定要用
3. Improved throughput - upload parts in parallel
4. Quick recovery from any network issues - Smaller part size minimizes the impact of restarting a failed upload due to a network error.
5. Pause and resume object uploads - You can upload object parts over time. After you initiate a multipart upload, there is no expiry; you must explicitly complete or stop the multipart upload.
6. Begin an upload before you know the final object size - You can upload an object as you are creating it.
7. If you're uploading large objects over a stable high-bandwidth network, use multipart upload to maximize the use of your available bandwidth by uploading object parts in parallel for multi-threaded performance.
8. If you're uploading over a spotty network, use multipart upload to increase resiliency to network errors by avoiding upload restarts. When using multipart upload, you need to retry uploading only parts that are interrupted during the upload. You don't need to restart uploading your object from the beginning.

### Byte-Range Fetches:

1. 平行下載
2. 僅對特定byte range（可選要下載多少）

## Share Buckets Cross Accounts

1. Using bucket policies & IAM (applies across the entire bucket).

* Programmatic access only.

2. Using bucket ACLs & IAM (individual objects).

* Programmatic access only.

3. Cross-account IAM roles.

* **Programmatic and console access.**

## Datasync

1. Used to move **large amounts** of data from on-premises to AWS.
2. Used with **NFS**- and **SMB**-compatible file systems.
3. **Replication** can be done hourly, daily, or weekly.
4. Install the **DataSync agent** to start the replication
5. Can be used to replicate **EFS** to **EFS**

## CloudFront
### What Is CloudFront?

Deliver entire website using a global network of edge locations.
其實就是CDN的概念

1. Edge Location - Where content will be cached. 跟AWS Region/AZ不一樣
2. Origin - 指會被CDN傳送的所有檔案，包含S3, EC2, Elastic Load Balancer or Route53.
3. Distribution - This is the name given the CDN (Edge Locations 的集合)
4. Web Distribution - Typically used for Websites.
5. RTMP - Used for Media Streaming.

### Signed URL or Signed cookie

* The policy can include:

- URL expiration
- IP ranges
- Trusted signers (which AWS accounts can create **signed URLs** )

### How signed URLs work:

### CloudFront signed url:

* If your origin is EC2, then use CloudFront.
* Can have different origins. Doesn't have to be EC2.
* Key-pair is account wide and managed by the root user.
* Can utilize **caching** features
* Can filter by data, path, IP address, expiration, etc.

### S3 signed url:

* 差別在於能不能進入S3 bucket
* Issues a request as the **IAM user** who creates the presigned URL.
* Limited **lifetime**

## Snowball

移入/移出大量的資料，會寄一箱很酷的東西到你家
存完檔案之後再寄回去

## Storage Gateway

是一種混合雲儲存服務，希望可以繼續存取地端儲存空間並同步到AWS

- **FIle Gateway (NFS & SMB)一般檔案**
- For flat files, stored directly on S3.
- 存取以local為主，File Gateway 會慢慢將資料同步到S3
- 本地端可透過 NFS 的方式進行儲存 (POSIX compitable)
- 與檔案相關的 ownership, permission, timestamp 等資訊，則會存放在 S3 object user-metadata 中

- **Volume Gateway (iSCSI)跟硬碟很像**

Volume Gateway 提供了 iSCSI 的方式，讓資料可透過 單一磁碟(volume) 為單位的角度來進行資料的儲存與管理；寫入 volume 的資料會以非同步的方式透過 volume snapshot 的方式進行儲存，也因為是 block device 的關係，因此每次的變更備份都是只有處理變更的 block 部份而已。

iSCSI: 主要是透過 TCP/IP 的技術，將儲存設備端透過 iSCSI target (iSCSI 標的) 功能，做成可以提供磁碟的伺服器端，再透過 iSCSI initiator (iSCSI 初始化用戶) 功能，做成能夠掛載使用 iSCSI target 的用戶端

- **Stored Volumes**
- Entire Dataset is stored on site and is asynchronously backed up to S3.
- 資料主要儲存在本地端(對本地端的應用程式相對延遲低)，非同步備份到 AWS
- 實際寫進 volume 的資料會先存在於本地端的儲存設備中
- 資料備份會以非同步的方式，並以 EBS(Elastic Block Volume) snapshot 的形式存放於 S3 中
- Low-latency access to your entire dataset

Snapshots: Incremental backups that capture only changed blocks. All snapshot storage is also compressed to minimize your storage charges.

- **Cached Volumes**
- Entire Dataset is stored on S3 and the most frequently accessed data is cached on site.
- 資料其實是存放在 S3 中，但為了讓本地端使用上可以更為快速，會將常存取的資料 cache 在本地端，透過此方式，可以讓地端設備不需要購置大容量的儲存設備


- **Tape Gateway (VTL)封存於磁帶中**
- Virtual Tape Library
- 用來解決磁帶備份 & 保存的問題
- 以 iSCSI device 的形式提供給使用者進行資料存放
- 支援將資料直接存入 S3 Glacier & S3 Glacier Deep Archive


## Athena vs Macie

### What is Athena?

Interactive query service which enables you to analyze and query data located in S3 using standard SQL.

- **Serverless**, nothing to provision, pay per query / per TB scanned
- No need to set up complex Extract/Transform/Load (ETL)
- Works directly with data stored in S3

### What Can Athena Be Used For?

- Can be used to query log files stored in S3, e.g. ELB logs, S3 access logs etc
- Generate business reports on data stored in S3
- Analyze AWS cost and usage reports
- Run queries on click-stream data

### What is Macie?

Security service which uses Machine Learning and NLP to discover, classify and protect sensitive data stored in S3.

- Uses AI to recognize if your S3 objects contain sensitive data such as PII (Personal Identifiable Information)
- Dashboards, reporting and alerts
- Works directly with data stored in S3
- Can also analyze CloudTrail logs
- Great for PCI-DSS and preventing ID theft

EC2 -- Elastic Compute Cloud
===
## What is EC2?

Elastic Compute Cloud is a web service that provides resizable compute capacity in the cloud.

### Instance metadata and userdata
* Metadata(提供):
* Configure or manage the running instance.
* like key/value
* e.g. host name, events, IP, Security Group
* Userdata(配置):
* Can be used to perform common automated configuration tasks and even run scripts after the instance starts.
* e.g. shell scripts and cloud-init directives.
## EC2 Pricing Models

- On Demand
- Low cost and flexibility
- Can for short-term
- Reserved
- Steady or predictable usage
- Required reserved capacity
- For 1-year or 3-year terms, reserved Instance Marketplace sellers also offer RIs often with shorter terms.
- 3 Types:
1. **Standard Reserved Instances**:
* Up to 75% off on demand instances.
3. **Convertible Reserved Instances**:
* Up to 54% off on demand capacity.
* 可以讓你去切換你的VM types（增加效能）
3. **Scheduled Reserved Instances**:
* Match your capacity reservation to a predicable recurring schedule.

- Spot
- For applications that have flexible start or end times.
- Users with urgent computing needs for large amounts of additional capacity.
- Dedicated Hosts
- A physical server with EC2 instance capacity fully dedicated to your use.
- Great for licensing which does not support multi-tenancy or cloud deployments.
- Dedicated Instances: Runs in a virtual private cloud (VPC) on hardware that's dedicated to a single customer.
只要記英文字母就好，後面數字會變 (SAP才會考)

## EBSio

### Elastic Block Store (EBS):

* Provides persistent block storage volumes for use with EC2 instance.
* You can create an encrypted volume from a snapshot of another encrypted volume.
* 5 Types
- General Purpose (SSD)
- Provisioned IOPS (SSD)
- Throughput Optimized Hard Disk Drive (HDD)
- Cold Hard Disk Drive (HDD)
- Magnetic (HDD)


### Exam Tips
* Volumes exist on EBS. Think of EBS as a virtual hard disk.
* Snapshots exist on S3. Think of snapshots as a photograph of the disk.
* Snapshots are point in time copies of Volumes.
* Snapshots are incremental -- means that only the blocks that have changed since your last snapshot are moved to S3.
* To create a snapshot for EBS volumes that serve as root devices, you should stop the instance before taking the snapshot.
* Can take a snap while the instance is running.
* Can create AMI's from Snapshots.
* Can change EBS volume sizes on the fly, including changing the size and storage type.
* Volumes will **AWAYS** bt in the same AZ as the EC2.
* To move an EC2 volume from one AZ to another, take a snapshot of it, create an AMI from the snapshot and then use the AMI to launch the EC2 instance in a new AZ.
* To move an EC2 volume from one region to another, take a snapshot of it, create an AMI from the snapshot and then copy the AMI from one region to the other. Then use the copied AMI to launch the new EC2 instance in the new region.

## AMI Types

An Amazon Machine Image (AMI) provides the information required to launch an instance.

- **Instance Store (Ephemeral Storage):**
- The root device for an instance launched from the AMI is an instance store volume created from a template stored in Amazon S3.
- The reason why instance store also called ephemeral storage is that if you stop the instance store, you'll lose all of the data.
- Instance store volumes can't be stopped. If the underlying host fails, you'll lose your data.
- **EBS Backed Volumes:**
- The root device for an instance launched from the AMI is an Amazon EBS volume created from an Amazon EBS snapshot.
- EBS Backed instance can be stopped. You won't lose the data on this instance if it is stopped.
- By default, both ROOT volumes will be deleted on termination. However, with EBS volumes, you can tell AWS to keep the root device volume.

## ENI vs EN vs EFA

### Elastic Network Interface (ENI)

Essentially a virtual network card.

- **An ENI is simply a virtual network card for your EC2 instances. It allows:**
- A primary private IPv4 address from the IPv4 address range of your VPC.
- One Elastic IP address (IPv4) per private IPv4 address
- One public IPv4 address
- One or more IPv6 addresses
- One or more security groups
- A MAC address
- **Scenarios for Network Interfaces:**
- Create a management network.
- Use network and security appliances in your VPC.
- Create dual-homed instances with workload/roles on distinct subnets.
- Create a low-budget, high-availability solution.

### Enhanced Networking (EN)

Using single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on supported instance types.

- **What is Enhanced Networking?**
- It uses **single root I/O virtualization (SR-IOV)** to provide high-performance networking capabilities on supported instance types. SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization when compared to traditional virtualization network interfaces.
- Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies. There is no additional charge for using enhanced networking.
- Use where you want good network performance.
- **Depending on your instance type, enhanced networking can be enabled using:**
- **Elastic Network Adapter (ENA)**, which supports network speeds of up to **100Gbps** for supported instance types.
OR
- Intel 82599 **Virtual Function (VF)** interface, which supports network speeds of up to **10 Gbps** for supported instance type. This is typically used on older instances.

In any scenario question, you probably want to **choose ENA over VF** if given the option.

### Elastic Fabric Adapter (EFA)

* What Is Elastic Fabric Adapter?
- An Elastic Fabric Adapter (EFA) is a network device that you can attach to your Amazon EC2 instance to accelerate High Performance Computing (HPC) and machine learning applications.
- EFA can use OS-bypass. OS-bypass enables HPC and machine learning applications to bypass the operating system kernel and to communicate directly with the EFA device. It makes it a lot faster with a lot lower latency. Not supported with Windows currently, only Linux.

### **Exam Tips:**

- **Elastic Network Interface**
For basic networking. Perhaps you need a separate management network to your production network or a separate logging network and you need to do this at low cost. In this scenario use multiple ENIs for each network.
- **Enhanced Network**
For when you need speeds between 10Gbps and 100Gbps. Anywhere you need reliable, high throughput.
- **Elastic Fabric Adaptor**
For when you need to accelerate High Performance Computing (HPC) and machine learning applications or if you need to do an OS by-pass. If you see a scenario question mentioning HPC or ML and asking what network adaptor you want, choose EFA.

## Volume & Snapshots

- Snapshots of encrypted volumes are encrypted automatically.
- Volumes restored from encrypted snapshots are encrypted automatically.
- You can share snapshots, but only if they are unencrypted.
- These snapshots can be shared with other AWS accounts or made public.
- You can now encrypt root device volumes upon creation of the EC2 instance.
- Create a Snapshot of the unencrypted root device volume.
- Create a copy of the Snapshot and select the encrypt option.
- Create an AMI from the encrypted Snapshot.
- Use that AMI to launch new encrypted instances.

## Spot Instances & Spot Fleets

### What Is an EC2 Spot Instance?

**To use Spot Instances, you must first decide on your maximum Spot price. The instance will be provisioned so long as the Spot price is BELOW your maximum Spot price.**


### **Spot Instances are useful for the following tasks:**

- Big data and analytics
- Containerized workloads
- CI/CD and testing
- Web Services
- Image and media rendering
- High-performance computing

### Spot Instances are not good for the following tasks:

- Persistent workloads
- Critical jobs
- Databases

## EC2 Hibernate

Hibernation **saves the contents** form the instance memory (RAM) to your Amazon EBS root volume. We persist the instance's Amazon EBS root volume and any attached Amazon EBS data volumes.

### When you start your instance out of hibernation:

- The **Amazon EBS** root volume is restored to its previous state.
- The **RAM** contents are reloaded.
- The processes that were previously running on the instance are resumed.
- Previously attached data volumes are **reattached and the instance retains its instance ID**.
- No need to restart the operating system or applications, etc.


### Useful for :

1. **Long-running processes**
2. **Services that take time to initialize.**

### Exam Tips:

- EC2 Hibernate preserves the in-memory RAM on persistent storage (EBS).
- Much faster to boot up because you don't need to reload the operating system.
- Instance RAM must be less than 150GB.
- Available for Windows, Amazon Linux 2 AMI, and Ubuntu.
- Instances can't be hibernated for more than **60 days**.
- Available for **On-Demand instances** and **Reserved Instances.**

## CloudWatch vs CloudTrail

### CloudWatch: Monitors Performance

Amazon CouldWatch is a monitoring service to monitor your AWS resources, as well as the applications that you run on AWS.

### CloudTrail: Monitors API calls (auditing)

CloudTrail increases visibility into your user and resource activity by recording AWS Management Console actions and API calls.

## EFS vs Amazon Fix for Windows vs Amazon Fix for Lustre

- **EFS**
When you need distributed, highly resilient storage for Linux instances and Linux-based applications.
- **Amazon Fix for Windows**
When you need centralized storage for Windows-based applications such as Sharepoint, Microsoft SQL Server, Workspaces, IIS Web Server or any other native Microsoft Application.
- **Amazon Fix for Lustre**
When you need high-speed, high-capacity distributed storage. This will be for applications that do High Performance Compute (HPC), financial modeling Technology. Remember that FSx for Lustre can store data directly on S3.

## EC2 Placement Groups

- **Clustered Placement Group: Low Network Latency / High Network Throughput**
- A Grouping of instances within a single Availability Zone.
- Placement groups are recommended for applications that need **low network latency**, **high network throughput**, or both.
- Only certain instances can be launched into a Clustered Placement Group.
- **Spread Placement Group: Individual Critical EC2 instances**
- A Group of instances that are each placed on distinct underlying hardware.
- Recommended for applications that have a small number of critical instances that should be kept separate from each other.
- Think **individual** instances.
- **Partitioned Placement Group: Multiple EC2 instances HDFS, HBase, and Cassandra**
- EC2 divides each group into logical segments called partitions. EC2 ensures that each partition within a placement group has its own set of racks.
- Each rack has its own network and power [source.](http://source.No) No two partitions within a placement group share the same racks, allowing you to isolate the impact of hardware failure within your applications.
- Think **multiple** instances.

## HPC

High Performance Computing (HPC) is used for industries such as genomic, finance and financial risk modeling, machine learning, whether prediction, and even autonomous driving.

### Achieve HPC on AWS

1. **Data Transfer**
- Snowball, Snowmobile (terabytes/petabytes worth of data)
- AWS DataSync to store on S3, EFS FSx for Windows, etc.
- Direct Connect: A cloud service solution that makes it easy to establish a dedicated network connection from your premises to AWS.
- Reduce network costs
- Increase bandwidth throughput
- Provide a more consistent network experience than internet-based connections
2. **Compute & Networking**
- EC2 fleets (Spot Instances or Spot Fleets)
- Placement groups (cluster placement groups)
- Enhanced Networking:
- **Using Single Root I/O Virtualization (SR-IOV) provides high-performance networking capabilities on supported instance types.**
- SR-IOV: A method of device virtualization that provides higher I/O performance and lower CPU utilization when compared to traditional virtualized network interfaces.
- ENA: Elastic Network Adapter (Suggest)
- VF: Virtual Function
- Elastic Fabric Adapter (EFA): Lower, more consistent latency and higher throughput
3. **Storage**
- Instance-attached storage:
- EBS: Scale up to 64,000 IOPS with Provisioned IOPS (PIOPS).
IOPS: Input/Output Operations Per Second
- Instance Store: Scale to millions of IOPS; low latency.
- Network storage:
- S3: Distributed object-based storage; not a file system.
- EFS: Scale IOPS based on total size, or use Provisioned IOPS
- FSx for Lustre: HPC-optimized distributed file system; million of IOPs, which is also backed by S3.
4. **Orchestration & Automation**
- Batch
- Run hundreds of thousands of batch computing jobs on AWS.
- Run a single job that spans multiple EC2 instances.
- Schedule jobs and launch EC2 instances according to your needs.
- ParellelCluster
- Open-source cluster management tool that easy to deploy and manage HPC clusters.
- Use a simple text file to model and provision all the resources.
- Automate creation of VPC, subnet, cluster type, and instance types.

## WAF (Web Application Firewall)

Monitor the HTTP or HTTPS requests that are forwarded to Amazon CloudFront, an Application Load Balancer or API Gateway, also lets you control access to account.

### 3 behaviors:

1. **Allow all requests except the ones you specify.**
2. **Block all requests except the ones you specify.**
3. **Count the requests that match the properties you specify.**

Databases
===

## RDS 101:
### OLAP vs OLTP
* Online Analytics Processing (OLAP):

* 一般稱為聯機事務處理，是針對具體的業務在資料庫中的聯機操作，具有數據量較少的特點，通常對少量的數據記錄進行查詢、修改。

* Online Transaction Processing (OLTP):
* 一般稱為聯機分析處理，是針對某些主題（綜合數據）的歷史數據進行分析，支持管理決策。

### RDS (OLTP)

* RDS runs on virtual machines and is NOT Serverless.
* **Patching of the RDS Operating System and DB is Amazon's responsibility.**
* Incluing:
- SQL
- MySQL
- PostgreSQL
- Oracle
- Aurora (Aurora Serverless is Serverless)
- MariaDB

## Back Ups, Multi-AZ & Read Replicas

### Back Ups

The restored version of the database will be a new RDS instance with a new DNS endpoint.

1. **Automated Backups:**
- Recover your database to any point in time within a "retention period".
- The backup data is stored in S3 and you get free storage space equal to the size of your database. (If you have an RDS instance of 10Gb, you'll get 10Gb worth of storage.)
2. **Database Snapshots:**
- Done manually and stored even after you delete the original RDS instance, unlike automated backups.

### Multi-AZ

Allows you to have exact copy of your production database in another Availability Zone.

- **For Disaster Recovery Only**
- For performance improvement you need Read Replicas.
- Failover conditions:
- Loss of availability in primary Availability Zone
- Loss of network connectivity to primary
- Compute unit failure on primary
- Storage failure on primary
### Read Replica

Copy of your production database (reads-only).

- Has its own DNS end point.
- You can have read replicas that have Multi-AZ.
- Read replicas are updated asynchronously(非同步) not synchronously.
- Read replicas can be promoted to be their own databases. This breaks the replication.
- You can have a read replica in a second region.

## DynamoDB

NoSQL Solution, fully managed database and supports both document and key-value data models, great fit for mobile, web, gaming, ad-tech, IoT, and many other applications.

- Stored on SSD storage.
- Spread across 3 geographically distinct data centers.
- Eventual Consistent Reads (Default)
- Strongly Consistent Reads
- **If you need to read data in one second or less, then use Strongly Consistent Reads.**

## RedShift (OLAP)

Data Warehousing or Business Intelligence solution, only available in 1 AZ.

### Massively Parallel Processing (MPP):

Redshift automatically distributes data and query load across all nodes.

### Backups:

- Default with a 1 day retention period, max with 35 days.
- Redshift always attempts to maintain at least three copies of your data (the original and replica on the compute nodes and a backup in Amazon S3).
- Asynchronously replicate your snapshots to S3 in another region for disaster recovery.

## Aurora

A MySQL and PostgreSQL-compatible relational database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases.

1. Start with 10GB, Scales in 10GB increments to 64TB (Storage Autoscaling).
2. Compute resources can scale up to 32vCPUs and 244GB of memory.
3. 2 copies of your data is contained in each availability zone, with minimum of 3 availability zones. 6 copies of your data.
4. Aurora has automated backups turned on by default, still can also take snapshots with Aurora. And share with other AWS accounts.

## ElastiCache

Speed up performance of existing databases (frequent identical queries).

### Memcached vs Redis (Professional)

Remember Redis is Multi-AZ, can do backups and restores.

## Database Migration Service (DMS)

Migrate relational databases, data warehouse, NoSQL databases, and other types of data stores.

### How Does DMS Work?

### Homogenous Migration


### Heterogeneous Migration

**You don't need Schema Conversion Tool (SCT) if you are migrating to identical databases.**


## Caching Services

### Services have caching capabilities:

Balancing up-to-date, accurate information and latency.

1. **CloudFront**
* Reduce the number of requests that your origin server must respond to directly.
* With CloudFront caching, more objects are served from CloudFront edge locations which are closer to your users.
* Edge caches can improve overall performance, but not specific to DynamoDB.
2. **API Gateway**
3. **ElastiCache — Memcached and Redis**
4. **DynamoDB Accelerator (DAX)**
* Reduce DynamoDB response times from milliseconds to microseconds, even at millions of requests per second.
* With DAX, your applications remain fast and responsive, even when a popular event or news story drives unprecedented request volumes your way.

### Typical Architecture:

## Elastic Map Reduce (EMR)

The central component of Amazon EMR is the cluster. A cluster is a collection of Amazon Elastic Compute Cloud (Amazon EC2) instances. Each instance in the cluster is called a node. Each node has a role within the cluster, referred to as the node type. Amazon EMR also installs different software components on each node type, giving each node a role in a distributed application like Apache Hadoop.

* Can exporting data from DynamoDB and importing data into DynamoDB
* Exporting data stored in DynamoDB to Amazon S3
* Importing data stored in Amazon S3 to DynamoDB

### Node types in EMR:

- **Master node:**
1. Managing, tracking the status of tasks and monitoring the health of the cluster.
2. Log data is stored on the master node (by default).
3. Can configure replication to S3 on five-minute intervals for all log data (when creating the cluster for the first time).
- **Core node:**
Runs tasks and stores data in the Hadoop Distributed File System (HDFS) on cluster.
- **Task node:**
Only runs tasks and doesn't store data in HDFS. Task nodes are **optional**.

Advanced IAM
===

## Directory Service

### What is AWS Directory Service?

- Family of managed services
- Connect AWS resources with on-premises AD
- Standalone directory in the cloud
- Use existing corporate credentials
- Single sign on (SSO) to any domain-joined EC2 instance

### What is Active Directory?

- On-premises directory service
- Hierarchical database of users, groups, computers — trees and forests
- Group policies
- Lightweight Directory Across Protocol (LDAP) and DNS
- Kerberos, LDAP, and NTLM authentication

### AD Connector

- Directory gateway (proxy) for on-premises AD
- Avoid caching information in the cloud
- Allow on-premises users to log in to AWS using AD
- Join EC2 instances to your existing AD domain
- Scale across multiple AD Connectors

### Cloud Directory

- Directory-based store for **developers**
- Multiple hierarchies with hundreds of millions of objects
- Use cases: org charts, course catalogs, device registries
- Fully managed service

### Cognito User Pools

- Managed user directory for SaaS applications
- Sign-up and sign-in for web or mobile
- Works with social media identities

### AD vs Not AD

|AD Compatible|Not AD Compatible|
|---|---|
|Managed Microsoft AD | Cloud Directory|
|AD Connector|Cognito user pools|
|Simple AD||

## IAM Policies

### Amazon Resource Name (ARM)

### IAM Policies

- JSON document that defines permissions
- Identity policy
- Resource policy
- No effect until attached
- List of statements

### Permission Boundaries

- Use to **delegate** administration to other users
- Prevent **privilege escalation** or **unnecessarily broad permissions**
- Control **maximum** permissions an IAM policy can grant
- Use cases:
- Developers creating roles for Lambda functions
- Application owners creating roles for EC2 instances
- Admins creating ad hoc users

### Exam Tips

- Not explicitly allowed == **implicitly denied**
- Explicit deny > everything else
- Only attached policies have effect
- AWS joins all applicable policies

## Resource Access Manager (RAM)

Allows resource sharing between accounts**

### Which AWS resources can share using RAM?
- App Mesh
- Aurora
- CodeBuild
- EC2
- EC2 Image Builder
- License Manager
- Resource Groups
- Route 53

## Single Sign-On (SSO)

Helps centrally manage access to AWS accounts and business applications.

### SAML

A standard for logging users into applicatio ns based on their session in another context.

- Security
- Assertion
- Markup
- Language

### Exam Tips

* When you see SAML 2.0 in question, look for SSO in one of the answers.

Route 53
===
## DNS 101

### What is DNS?

DNS is used to convert human friendly domain names into an Internet Protocol (IP) address.

- IPv4: 32 bit field and has over 4 billion different addresses
- IPv6: 128 bit and was created to solve the depletion issue.

Problem: A lot of ISPs are just not switching over to IPv6, so using both IPv4 and IPv6.

### Start Of Authority Record (SOA)

When bought a domain, every DNS address begin with a SOA.

**Stores Information:**

- The name of the server that supplied the data for the zone.
- The default number of seconds for the time-to-live file on resource records.
- The administrator of the zone.
- Has all DNS records.

### Name Server Record (NS)

Used by Top Level Domain servers to direct traffic to the Content DNS server which contains the authoritative DNS records.

Request → Top Level Domain server → NS Records → SOA

### DNS Record

- **A record**:
- The fundamental type of DNS record.
- The "A" stand for "Address".
- The A record is used by computer to translate the name of the domain to an IP address.
- **CName**:
- A Canonical Name (CName) can be used to resolve one domain name to another.
- For example, **http://m.acloud.guru** equals to **http://mobile.acloud.guru.**
- ==Can't== be used for naked domain names (zone apex record).
- **Alias Record**:
- Used to map resource record sets in hosted zone to Elastic Load Balancers, CloudFront distributions, or S3 buckets that are configured as websites.
- Unlike a CNAME record, you==can==create an alias record at the top node of a DNS namespace, also known as the zone apex.
- For example, if you register the DNS name KingJack.com, the zone apex is KingJack.com. You can't create a CNAME record for KingJack.com, but you can create an alias record for KingJack.com that routes traffic to www.KingJack.com (as long as www.KingJack.com doesn't already have a CNAME record).

## Routing Policy

### Simple Routing Policy:

- Can only have one record with multiple IP addresses.
- If you specify multiple values in a record, Route 53 returns all values to the user in a **random** order.

### Weighted Routing Policy:

- Allows you split your traffic based on different weights assigned.
- For example, can set 10% of your traffic to go to US-EAST-1 and 90% to go to EU-WEST-1.

### Latency-Based Routing Policy:

- Route your traffic based on the lowest network latency for your end user.
- When Route53 receives a query for the site, it selects the latency resource record set for the region that gives the user the lowest latency.

### Failover Routing Policy:

- Used when you want to create an active/passive set up.
- For example, to make primary site to be in EU-WEST-2 and your secondary DR Site in AP-SOUTHEAST-2.
- Route53 will monitor the health of your primary site using a health check.
- A health check monitors the health of your end points.

### Geolocation Routing Policy:

- Lets you choose where your traffic will be sent based on the geographic location of your users.
- For example, you might want all queries from Europe to be routed to a fleet of EC2 that are specifically configured for your European customers.

### Geoproximity Routing Policy (Traffic Flow Only):

- Let Route53 route traffic to your resources based on the geographic location of your users and resources.
- Can also optionally choose to route more traffic or less to a given resource by specifying a value, known as a bias.
- A bias expands or shrinks the size of the geographic region from which traffic is routed to a resource.
- To use geoproximity routing, must use Route53 traffic flow.

### Multivalue Answer Policy:

- Lets you configure Route53 to return multiple values, such as IP addresses for your web servers, in response to DNS queries.
- Can specify multiple values for almost any record, but multivalue answer routing also lets you check the health of each resource, so Route53 returns only values for health resource.
- **This is similar to simple routing however it allows you to put health checks on each record set.(就是simple routing 新增health check功能)**

VPCs
===
HA Architecture
===
## Elastic Load Balancer

### Load Balancer Types

1. **Application Load Balancer**
- Best suited for load balancing of **HTTP and HTTPS traffic**. They operate at Layer 7 and are application-aware. They are intelligent, and you can create advanced request routing, sending specified requests to specific web servers.
- Three HTTP Routing Patterns
- Host-based:
- The old standard.
- Host-based routing is what enables virtual servers on web servers. It’s also used by application services like load balancing and ingress controllers to achieve the same thing. One IP address, many hosts.
- Host-based routing allows you to send a request for api.example.com and for web.example.com to the same endpoint with the certainty it will be delivered to the correct back-end application.
- Path-based:
- Increasingly common – particularly in the realm of scaling containers using ingress controllers – is path-based routing.
- Path-based routing requires visibility into the URI portion of an HTTP request.
- Header-based:
- Header-based is a broad category that includes some familiar routing patterns such as persistence (sticky sessions).
- Header-based routing simply means that you use an arbitrary HTTP header as the basis for determining how to route a request.
- Support for path-based routing. You can configure rules for your listener that forward requests based on the URL in the request. This enables you to structure your application as smaller services, and route requests to the correct service based on the content of the URL.
- Support for host-based routing. You can configure rules for your listener that forward requests based on the host field in the HTTP header. This enables you to route requests to multiple domains using a single load balancer.
- Support for routing based on fields in the request, such as standard and custom HTTP headers and methods, query parameters, and source IP addresses.
2. **Network Load Balancer (Extreme Performance)**
- Best suited for load balancing of **TCP traffic** where extreme performance is required. Operating at the connection level (Layer 4), Network Load Balancer are capable of handing millions of requests per second, while maintaining ultra-low latencies.
3. **Classic Load Balancer**
- The Legacy Elastic Load Balancers. You can load balance HTTP/HTTPS applications and use Layer 7-specific features, such as X-Forwarded and ==sticky sessions==. You can also use strict Layer 4 load balancing for applications that rely purely on the TCP protocol.
- If your application stops responding, the ELB responds with a 504 error. This means that the application is having issues. This could be either at the Web Server layer or at the Database Layer. Identify where the application is failing, and scale it up or out where possible.

### Exam Tips

- If you need the IPv4 address of your end user, look for the **X-Forwarded-For** header.
- Instances monitored by ELB are reported as; InService, or OutofService.
- Health Checks check the instance health by talking to it.
- Load Balances have their own DNS name. **You are never given an IP address.**
- Read the ELB FAQ for Classic Load Balancers.

## Advance Load Balancer Theory

### What Are Sticky Sessions?

- Classic Load Balancer routes each request independently to the registered EC2 instance with the smallest load.
- Sticky sessions allow you to bind a use's session to a **specific EC2 instance**. This ensures that all requests from the user during the session are sent to the same instance.
- You can enable Sticky Sessions for Application Load Balancers as well, but the traffic will be sent at the Target Group Level.

### What Are Path Patterns?

- You can create a listener with rules to forward requests based on the URL path. This is known as path-based routing.
- If you are running micro services, you can route traffic to multiple back-end services using path-based routing.
- For example, you can route general requests to one target group and requests to render images to another target group.


## Auto Scaling

### 3 Components

1. **Groups**
- Logical component. Web server group or Application group or Database group etc.
2. **Configuration Templates**
- Groups uses a launch template or a launch configuration as a configuration template for its EC2 instances.
- You can specify information such as the AMI ID, instance type, key pair, security groups, and block device mapping for your instances.
- You can create a default template that defines common configuration parameters and allow the other parameters to be specified as part of another version of the same template.
3. **Scaling Options**
- Scaling Options provides several ways for you to scale your Auto Scaling groups.
- For example, you can configure a group to scale based on the occurrence of specified conditions (dynamic scaling) or on a schedule.
### Cooldown period

- The cooldown period helps you prevent your Auto Scaling group from launching or terminating additional instances before the effects of previous activities are visible.
- If a value for the default cooldown period is not provided, its default value is 300 seconds.
### What are Scaling Options?

- **Maintain current instance levels at all times**
- **Scale manually**
- **Scale based on a schedule**
- **Scale based on demand**
- **Use predictive scaling**

## HA Architecture

**Always design for failure.**

### Exam Tips

- Use Multiple AZ's and Multiple Regions where ever you can.
- Know the difference between Multi-AZ and Read Replicas for RDS.
- Know the difference between scaling out and scaling up.
- Scaling Out: Use Auto Scaling groups and add additional EC2 instances
- Scaling Up: Increase the resources inside EC2 instances.
- Know the different S3 storage classes.

## Elastic Beanstalk and CloudFormation
### What is Elastic Beanstalk?

* AWS Elastic Beanstalk provides an environment to deploy and run applications in the cloud.
* With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS Cloud without worrying about the infrastructure that runs those applications.
* It is integrated with developer tools and provides a one-stop experience for you to manage the lifecycle of your applications.
### What is CloudFormation?
* CloudFormation is infrastructure as code, and the CloudFormation feature of templates allows this infrastructure as code to be version controlled.
* AWS CloudFormation is a convenient provisioning mechanism for a broad range of AWS and third-party resources.
* It supports the infrastructure needs of many different types of applications, such as existing enterprise applications, legacy applications, applications built using a variety of AWS resources, and container-based solutions (including those built using AWS Elastic Beanstalk).
### How is differnt from them?
* These services are designed to complement each other.
* AWS CloudFormation supports Elastic Beanstalk application environments as one of the AWS resource types.
* This allows you, for example, to create and manage an AWS Elastic Beanstalk–hosted application along with an RDS database to store the application data. In addition to RDS instances, any other supported AWS resource can be added to the group as well.
### Exam Tips

- With Elastic Beanstalk, you can quickly deploy and manage applications in the AWS Cloud without worrying about the infrastructure that runs those applications.
- You simply upload your application, and Elastic Beanstalk automatically handles the details of capacity provisioning, load balancing, scaling, and application health monitoring.

## Highly Availability with Bastion Hosts

### Bastion in Action

Scenario 1: Expensive

Scenario 2: Cheaper

## Simple Queue Service

### What Is SQS?

**A web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.**

### Exam Tips

- Two hosts in two separate AZs. Use a Network Load Balancer with static IP addresses an health checks to fail over from one host to the other.
- Can't use an Application Load Balancer, as it is layer 7 and you need to use layer 4.
- One host in one AZ behind an Auto Scaling group with health checks and a fixed EIP. If the host fails, the health check will fall and the Auto Scaling group will provision a new EC2 instance in a separate AZ. You can use a user data script to provision the same EIP to the new host. This is the cheapest option, but it's not 100% fault tolerant.

## On-Premises Services with AWS

### 1. Database Migration Service (DMS)

**Allows you to move database to and from AWS**

- Might have your DR environment in AWS and your on-premises environment as your primary.
- Works with most popular database technologies, such as Oracle, MySQL, DynamoDB, etc.
- Supports homogenous migrations:

Oracle → Oracle

- Supports heterogeneous migrations:

SQL Server → Amazon Aurora

### 2. Server Migration Services (SMS)

**Supports incremental replication of your on-premises service in to AWS.**

- Can be used as a backup tool, multi-site strategy (on and off premises), and a DR tool.

### 3. AWS Application Discovery Service

**Helps enterprise customers plan migration projects by gathering information about their on-premises data centers.**

- You install the AWS Application Discovery Agentless Connector as a virtual appliance on VMware vCenter.
- It will then build a server utilization map and dependence map of your on-premises environmen.
- The collected data is retained in encrypted format in an AWS Application Discovery Service data store. You can export this data as a CSV file and use it to estimate the Total Coset of Ownership (TCO) of running on AWS and to plan your migration to AWS.
- This data is also available in AWS Migration Hub, where you can migrate the discovered servers and track their progress as they get migrated to AWS.

### 4. VM Import/Export

**Migrate existing applications in to EC2**

- Can be used to create a DR strategy on AWS or use AWS as a second site.
- You can also use it to export your AWS VMs to your on-premises data center.

### 5. Download Amazon Linux 2 as an ISO

- Works with all major virtualization providers, such as VMware, Hyper-V, KVM, VirtualBox (Oracle), etc.

Applications
===
## Simple Queue Service

### What Is SQS?

**A web service that gives you access to a message queue that can be used to store messages while waiting for a computer to process them.**

- It's a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component.
- A queue is a temporary repository for messages that are awaiting processing.
- Decouple the components of an application so they run independently, easing message management between components.
- Any component of a distributed application can store messages in a fail-safe queue.
- Messages can contain up to 256KB of text in any format.
- Any component can later retrieve the messages programmatically using the Amazon SQS API
- The queue acts as a buffer between the component producing and saving data, and the component receiving the data for processing.
- This means the queue resolves issues that arise if the producer is producing it, or if the producer or consumer are only intermittently connected to the network.

### Queue Type

1. **Standard queue**
- Nearly-unlimited number of transactions per second.
- Guarantee that a message is delivered **at least once**.
2. **FIFO**
- **Exactly-once processing.**
- Duplicates are not introduced into the queue.
- Support message groups that allow multiple ordered message groups within a single queue.
- Are limited to 300 transactions per second (TPS), but have all the capabilities of standard queues.

### Short Polling vs Long Polling
* Short polling
The ReceiveMessage request queries only a subset of the servers (based on a weighted random distribution) to find messages that are available to include in the response. Amazon SQS sends the response right away, even if the query found no messages.
* Long polling
The ReceiveMessage request queries all of the servers for messages. Amazon SQS sends a response after it collects at least one available message, up to the maximum number of messages specified in the request. Amazon SQS sends an empty response only if the polling wait time expires.

### Delay Queues vs Visibility Timeouts
* Delay Queues
A message is hidden when it is first added to queue.
* Visibility Timeouts
A message is hidden only after it is consumed from the queue.
### Exam Tips

- SQS is pull-based, not push-based, you have to have an EC2 instance pulling the message out of the queue.
- Message are 256 KB in size.
- **Visibility timeout** is the amount of time that the message is invisible in the SQS queue after a reader picks up that massage. Providing the job is processed before the visibility timeout expires, the message will then be deleted form the queue. If the job is not processed within that time, the message will become visible again and another reader will process it. This could result in the same message being delivered twice. (Maximum 12 hours)
- Guarantee the message will be processed at least once.
- SQS long polling is a way to retrieve messages from your SQS queues. While the regular short polling returns immediately (even if the message queue being polled is empty), long polling doesn't return a response until a message arrives in the message queue, or the long poll times out.
- Any time you see a scenario based question about "decoupling" your infrastructure → SQS
## Simple Workflow Service (SWF)

### What Is SWF?

**A web service that makes it easy to coordinate work across distributed application components.**

- SWF enables applications for a range of use cases, including media processing, web application backends, business process workflows, and analytics pipelines, to be designed as a coordination of tasks.
- Task represent invocations of various processing steps in an application which can be performed by executable code, web service calls, human actions, and scripts.
- SWF ensures that a task is assigned only once and is **never duplicated**.

### SWF vs SQS
1. SWF workflow executions can last up to 1 year
SQS has a retention period of up to 14 days.
2. SWF presents a task-oriented API
SQS offers a message-oriented API.
3. SWF ensures that a task is assigned only once and is never duplicated.
SQS, you need to handle duplicated messages and may also need to ensure that a message is processed only once.
4. SWF keeps track of all the tasks and events in an application.
SQS, you need to implement your own application-level tracking, especially if your application uses multiple queues.
### SWF Actors
- Workflow Starters — An application that can initiate a workflow. Cloud be your e-commerce website following the placement of an order, or a mobile app searching for bus times.
- Deciders — Control the flow of activity tasks in a workflow execution. If something has finished in a workflow, a Decider decides what to do next.
- Activity Workers — Carry out the activity tasks.

## Simple Notification Service (SNS)

### What Is SNS?

**A web service that makes it easy to set up, operate, and send notifications from the cloud.**

- It provides developers with a highly scalable, flexible, and cost-effective capability to publish messages from an application and immediately deliver them to subscribers or other applications.
- Pushing cloud notifications directly to mobile devices, SNS can also deliver notifications by SMS text message or email to SQS queues, or to any HTTP endpoint.

### What Is A Topic?

**SNS allows you to group multiple recipients using topics. A topic is an "access point" for allowing recipients to dynamically subscribe for identical copies of the same notification.**

- One topic can support deliveries to multiple endpoint types.
- You can group together iOS, Android and SMS recipients.
- When you publish once to a topic, SNS delivers appropriately formatted copies of your message to each subscriber.

### SNS Benefits

- Instantaneous, push-based delivery (no polling).
- Simple APIs and easy integration with applications.
- Flexible message delivery over multiple transport protocols.
- Inexpensive, pay-as-you-go model with no up-front costs.
- Web-based AWS Management Console offers the simplicity of a point- and-click interface.

### SNS vs SQS

1. Both Messaging Services in AWS
2. SNS — Push, SQS — Polls (Pulls)

## Elastic Transcoder

### What Is Elastic Transcoder?

**Convert media files from their original source format in to different formats that will play on smartphones, tablets, PCs, etc.**

- Media Transcoder in the cloud.
- Provides transcoding presets for popular output formats, which means that you don't need to guess about which settings work best on particular devices.
- Pay based on the minutes that you transcode and the resolution at which you transcode.

## API Gateway

### What Is API Gateway?

A fully managed service that makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale.

### What Can API Gateway Do?

- Expose HTTPS endpoints to define a RESTful API.
- Serverless-ly connect to services like Lambda & DynamoDB.
- Send each API endpoint to a different target.
- Run efficiently with low cost.
- Scale effortlessly.
- Track and control usage by API key.
- Throttle requests to prevent attacks.
- Connect to CloudWatch to log all requests for monitoring.
- Maintain multiple versions of your API.

### How to Configure API Gateway?

- Define an API (container).
- Define Resources and nested Resources (URL paths).
- For each Resource:
- Select supported HTTP methods.
- Set security.
- Choose target (EC2, Lambda, DynamoDB, etc.)
- Set request and response transformations

### How to Deploy API Gateway?

- Deploy API to a stage:
- Uses API Gateway domain, by default.
- Can use custom domain.
- Now supports AWS Certificate Manager: free SSL/TLS certs.

### API Gateway Caching

**You can enable API caching in API Gateway to cache your endpoint's response.**

- With caching, you can reduce the number of calls made to your endpoint and also improve the latency of the requests to your API.
- When you enable caching for a stage, API Gateway caches responses from your endpoint for a specified time-to-live (TTL) period, in seconds. API Gateway then responds to the request by looking up the endpoint response from the cache instead of making a request to your endpoint.

### Same Origin Policy

- In computing, the same-origin policy is an important concept in the web application security model. Under the policy, a web browser permits scripts contained in a first web page to access data in a second web page, but only if both web pages have the same origin.
- Prevent Cross-Site Scripting (XSS) attacks:
- Enforced by web browsers.
- Ignored by tools like PostMan and curl.

### Cross-Origin Resource Sharing (CORS)

**CORS is one way the server at the other end (not the client code in the browser) can relax the same-origin policy.**

- Cross-origin resource sharing (CORS) is a mechanism that allows restricted resources (e.g. fonts) on a web page to be requested from another domain outside the domain from which the first resource was served.

### CORS in Action

- Browser makes an HTTP OPTIONS call for a URL (OPTIONS is an HTTP method like GET, PUT, and POST).
- Server returns a response that says:
"These other domains are approved to GET this URL."
- Error - "Origin policy cannot be read at the remote resource?"
You need to enable CORS on API Gateway.

### Exam Tips

- Remember what API Gateway is at a high level.
- API Gateway has caching capabilities to increase performance.
- API Gateway is low cost and Scales automatically.
- You can throttle API Gateway to prevent attacks.
- You can log results to CloudWatch.
- If you are using AJAX that uses multiple domains with API Gateway, ensure that you have ena
- CORS is enforce
## Kinesis 101
### What Is Streaming Data?
**Is data that is generated continuously by thousands of data sources, which typically send in the data records simultaneously, and in small sizes (order of Kilobytes.)**
* Perchases from online stores (Amazon.com)
* Stock Prices
* Game data (as the gamer plays)
* Socail network data
* Geospatial data (uber.com)
* IoT sensor data

### What Is Kinesis?

**Is a platform on AWS to send your streaming data to. Kinesis makes it easy to load and analyze streaming data, and also providing the ability for you to build your own custom applications for your business needs.**
### Three types of Kinesis
1. **Kinesis Streams**
* Kinesis Streams: Store data. (24hrs to 7days)
* Shard: Where data being contained.
* The data capacity of the stream is a function of the number of shards that you specify for the stream.
* The total capacity of the stream is the sum of the capacities of its shards.
* EC2: Data Comsumers

2. **Kinesis Firehose**
* Kinesis Firehorse: `No persistent storage`, have to do sth with coming data.(e.g. Lambda)
* Output: S3, Redshift, Elastic Search Cluster, 除了Athena
* Key components:
1. Delivery streams
2. Records of data
3. Destinations


3. **Kinesis Analytics**
* Contains both streams and firehorse.
* Analyze data on the fly.

## Web Identity Federation & Cognito
### What Is Web Identity Federation?
Web Identity Federation lets you givw your users access to AWS resources after they have successfully authenticated with a web-based identity provider like Amazon, Facebook, or Google. Following successful authentication, the user receives an authentication code from the Web ID provider, which they can trade for temporary AWS security credentials.

### Features of Cognito
**Provides Web Identity Federation with:**
* Sign-up and sign-in to your apps.
* Access for guest users.
* Acts as an Identity Broker between your application and Web ID providers, so you don't need to write any additional code.
* Synchronizes user data for multiple devices.
* Recommended for all mobile applications AWS services.

### Cognito Use Cases
* Cognito borkers between the app and Facebook or Google to provide temporary credentials which map to an IAM role allowing access to the required resources.
* No need for the application to embed or store AWS credentials locally on the device and it gives users a seamless experience across all mobile devices.


### Cognito User Pools
* User Pools are user directories used to manage sign-up and sigh-in functionality for mobile and web applicaions.
* Users can sign-in directly to the User Pool, or using Facebook, between the identity provider and AWS.
* Successful authentication generates a JSON Web token (JWTs).

### Cognito Identity Pools
* Enable provide temporary AWS credentials to access AWS services like S3 or DynamoDB.

### Cognito in Action
* User Pools: usernames, passwords, etc.
* Identity Pools: actual granting of authentication to AWS resources.

### Cognito Synchronisation
* Cognito tracks the association between user identity and the various different devices they sign-in from.
* In order to provide a seamless user experience for your application, Cognito uses Push Synchronization to push updates and synchronize user data across multiple devices.
* Cognito uses SNS to send a notification to all the devices associated with a given user identity whenever data stored in the cloud changes.

## Event Processing Patterns
### Event-Driven Architechture
* The pub/sub model allows messages to be broadcast to different parts of a system asynchronously.
* SNS Topic: broadcasts asynchronous event notifications and endpoints that allow other AWS services to connect to the topic in order to send and receive messages, to broadcast a message.
* Publisher: pushes a message to the Topic
* Subscriber: performs different functions when accepting broadcast messages.

### Dead-Letter Queue (DLQ)
* SNS
* Messages published to a topic that fail to deliver are sent to an **SQS queue**; held for further analysis or reprocessing.
* SQS
* Messages sent to SQS that exceed the queue's **maxReceiveCount** are sent to a DLQ (another SQS queue)
* Lambda
* Result from failed **asynchronous** invocations; will retry twice and send to ==either== an SQS queue or SNS topic.


### Fanout Pattern
* Situation 1
* If system crushed or loses network connectivity before sending a message to queue number two, data warehouse won't receive that message.
* Situation 2 (better way)
* Subscribe another queue to the same Topic for new coming data.

### S3 Event Notification
1. Object uploaded to S3 Bucket.
2. Sent to multiple subscribers or destination simultaneously.
3. Can filt the file to dicide whether to send notifications.
4. Sometimes will miss notifications -> enable versioning.


* Different Events for S3 Notifications
1. Object Created:
* Like the S3 put object event
2. Object Removed:
* Supports deletes of versioned and unversioned objects
3. Object Restored:
* Restoration of objects in Glacier
4. RRS Object Lost:
* Detection of a reduved-redundancy storage object is lost.
5. Replication:
* Replication failed
* Replication exceeds 15 mins
* Object no longer tracked by replication metrics

Security
===
## Reducing Security Threats
### Bad Actors
* Typically automated processes
* Content scrapers
* Bad bots
* Fake user agent
* Denial of service (DoS)

### Benefits of Preventing Bas Actors
* Reduce security threats
* Lower overall costs
### NACL -- Network Access Control Lists
1. Using Inbound & Outbound rules:
* Reject IPs
2. Using Host-based firewall
* Directly on EC2
3. Operates on layer 4
4. ==Block a range of IP==
### ABL -- Application Load Balancer
1. The incoming bad actor will ==terminate at ALB== itself.
2. The EC2 will be unaware of that origin IP.
3. A host-based firewall will be ineffective in this case.
4. Allow ABL security group access to EC2 security group.
5. Still have to use NACL in this case.


### WAF -- Web Application Firewall
1. Monitor web requests and protect web applications from malicious requests from bad actors.
2. Operates on layer 7.
3. ==Block SQL injection or cross-site scripting attacks.==


### WAF + CloudFront
1. Similar to ALB, the connection terminates at CloudFront
2. The client IP is not visible to NACL, only the CloudFront IP pass along to the NACL.
3. NACL is no use behind the CloudFront.
4. Attach a WAF to CloudFront distribution and use the IP blocking and filtering options.
5. Geo match features to block a certain country.


## KMS -- Key Management Service
### What Is KMS?
* ==Regional== secure key management and encryption and decryption.
* Manages ==customer master keys== (CMKs)
* A logical representation of a key.
* A pointer or reference to some underlying cryptographic material.
* Idea for S3 objects, database passwords and API keys stored in Systems Manager Parameter Store.
* Encrypt and decrypt data up to ==4KB== in size.
* Integrated with most AWS services.
* Pay per API call.
* Listing your keys.
* Encrypting, decrypting, reencrypting data.
* Audit capability using CloudTrail -- logs delivered to S3.
* Easier to satisfy compliance requirements.
* ==FIPS 140-2 Level 2==
* US government computer security standard used to approve crytographic modules.
* Level 2 means you just have to show evidence of tampering.
* Level 3 is CloudSHM -- more stringent security mechanisms.

### 3 Types of CMKs
1. AWS Managed CMK
* Free
* Used by default if you pick encryption in most AWS services.
* Only that service can use them directly.
2. Customer Managed CMK
* Allows key ==rotation==
* Controlled via key policies and can be enabled/disabled
3. AWS Owned CMK
* Used by AWS on a shared basis across many accounts
* Typically won't see these


### Symmetric vs Asymmetric CMKs

| Symmetric | Asymmetric |
|-------|------------------------|
|==Same== key used for encryption and deceyption|Mathematically related public/private key pair|
|==AWS-256==|==RSA==and==elliptic-curve cryptopraphy (ECC)==|
|Never leaves AWS unencrypted|==Private==key never leaves AWS unencrypted|
|Must call the KMS APIs to use|Must call the KMS APIs to use==private==key|
|AWS services integrated with KMS use symmetric CMKs|==Download==the piblic key and use outside AWS|
|Encrypt, decrypt, and re-enceypt data|Used outside AWS by users who can't call KMS APIs|
|Generate data keys, data key pairs, and random byte strings|AWS services integrated with KMS==do not support==asmmetric CMKs|
|==Import==your own key material|Sign messages and verify signatures|

## CloudHSM
### What Is CloudHSM?
* Provides a tamper resistent environment for managing keys.
* Dedicated hardware security module (HSM)
* ==FIPS 140-2 Level 3==
* KMS is level 2
* Mangage your own keys
* ==No access==to the AWS-managed component
* Runs within a VPC in your account
* Single tenant, dedicated hardware, multi-AZ cluster
* Industry-standard APIs -- ==no AWS APIs==
* PKCS#11
* Java Cryptography Extensions (JCE)
* Microsoft CryptoNG (CNG)
* Keep your keys safe --==irretrievable==if lost!

### CloudHSM Architecture
1. Create a cluster in VPC, CloudHSM will operate inside its own VPC.
2. Projects ENI (Elastic Network Interfaces) into the choosing VPC (This is how the applications communicate with the CloudHSM cluster).
3. Inside the cluster, creating specific instances of HSMs, need to explicitly provision HSMs across AZs.
4. Idealy, place one HSM subnet per one AZ


## Systems Manager Parameter Store

An essential tool for caching and distributing secrets securely to AWS resources.

### What Is Parameter Store?
* Component of AWS System Manager (SSM)
* Secure==serverless==storage for configuration and secrets:
* Passwords
* Database connection strings
* License codes
* API keys
* Values can be stored encrypted (KMS) or plaintext
* Seperate data from source control
* Store parameters in ==hierarchies==
* Track versions
* Set TTL to expire valus such as passwords

### Organizing Parameters into Hierarchies

* GetParametersByPath (API call) to retrieve all parameters in a hierarchy:
* /dev
* /dev/db
* /prod/app


### Lauching Latest Amazon Linux AMI in CloudFormation
* No need to create any complex mappings.


## Secrets Manager

Helps rotate, manage, and retrieve all kind of secrets, like database credentials, API keys, etc.

### What Is Secrets Manager?

* Similar to Systems Manager Parameter Store.
* Different to Systems Manager Parameter Store:
* ==Charge==per secret stored and per 10,000 API calls.
* ==Rotate the keys and automatically apply the new credentials in RDS==.
* Generate==random secrets==

## AWS Shield

### What Is AWS Shield?
* Protects against distributed denail-of-service (DDoS) attacks.
* Sits at the edge of AWS's perimeter network.

|Standard|Advanced|
|----|---|
|Automatically enabled for all customers at==no cost==|$3,000 per month, per org|
|Protects against common layer 3 and 4 attacks(SYN/UDP floods, Reflection attacks）|Enhanced protection for EC2, ELB, CloudFront, Global Accelerator, Route 53|
|Stopped a 2.3 Tbps DDoS attack for three days in Feb 2020|Business and Enterprise support customers get 24x7 access to the DDoS Response Team (DRT)|

## WAF -- Web Application Firewall
### What Is WAF?
* Lets you monitor HTTP(S) requests to==CloudFront==,==ALB==or==API Gateway==
* Control access to content
* Configure==filtering rules==to allow/deny traffic:
* IP addresses
* Query string parameters
* SQL query injection
* Blocked traffic returns HTTP 403 Forbidden


### How Does WAF Work?
* 3 Behaviors:
1. ==Allow==all requests, except the ones you specify
2. ==Block== all requests, except the ones you specify
3. ==Count==the requests that match the properties you specify
* Request properties:
* Originating IP address
* Originating Country
* Request size
* Values in request headers
* Strings in request matching regular expressions (regex) patterns
* SQL code (injection)
* Cross-site scripting (XSS)

### Firewall Manager

* Centrally configure and manage firewall rules across an==AWS Organization==
* WAF rules:
* ALB
* API Gateway
* CloudFront distributions
* AWS Shield Advanced protections:
* ALB
* ELB Classic
* EIP
* CloudFront distributinos
*
* Enable security groups for EC2 and ENIs

## Disaster Recovery
### DR objectives
* RTO -- Recovery time objective:
The maximum acceptable delay between the interruption of service and restoration of service. This determines an acceptable length of time for service downtime.
* RPO -- Recovery point objective:
The maximum acceptable amount of time since the last data recovery point. This determines what is considered an acceptable loss of data.
### Scope of impact for a disaster event
* Multi-AZ strategy
* If you’re designing a DR strategy to withstand events such as power outages, flooding, and other other localized disruptions, then using a Multi-AZ DR strategy within an AWS Region can provide the protection you need.
* Multi-Region strategy
* AWS provides multiple resources to enable a multi-Region approach for your workload. This provides business assurance against events of sufficient scope that can impact multiple data centers across separate and distinct locations.

### DR strategies
1. Backup and restore:
- This simple and low cost DR approach backs up your data and applications from anywhere to the AWS cloud for use during recovery from a disaster. Unlike conventional backup methods, data is not backed up to tape.
2. Pilot light:
- The idea of the pilot light is an analogy that comes from gas heating. In that scenario, a small flame that’s always on can quickly ignite the entire furnace to heat up a house. In this DR approach, you simply replicate part of your IT structure for a limited set of core services so that the AWS cloud environment seamlessly takes over in the event of a disaster. A small part of your infrastructure is always running simultaneously syncing mutable data (as databases or documents), while other parts of your infrastructure are switched off and used only during testing. Unlike a backup and recovery approach, you must ensure that your most critical core elements are already configured and running in AWS (the pilot light). When the time comes for recovery, you can rapidly provision a full-scale production environment around the critical core.
3. Warm standby:
- The term warm standby is used to describe a DR scenario in which a scaled-down version of a fully functional environment is always running in the cloud. A warm standby solution extends the pilot light elements and preparation. It further decreases the recovery time because some services are always running. By identifying your business-critical systems, you can fully duplicate these systems on AWS and have them always on.
4. Multi-site:
- A multi-site solution runs on AWS as well as on your existing on-site infrastructure in an active-active configuration. The data replication method that you employ will be determined by the recovery point that you choose, either Recovery Time Objective (the maximum allowable downtime before degraded operations are restored) or Recovery Point Objective (the maximum allowable time window whereby you will accept the loss of transactions during the DR process).


# Serverless
## Lambda

### What Is Lambda?
* A compute service where you can upload your code, run your code in response to HTTP requests using Amazon API Gateway or API calls made using AWS SDKs.
* Takes care of provisioning and managing the servers that you use to run the code.
* As an event-friven compute service where Lambda runs your code in response to events. These events could be changes to data in an Amazon S3 bucket or an Amazon DynamoDB table.
* Use X-ray to debug

### How Does Lambda Work?
* Case 1
1. Upload meme to S3
2. Trigger Lambda_1 function
3. Lambda_1 triggers Lambda_2 to tell the user the meme
4. Lambda_1 triggers Lambds_3 to store the meme back to the S3 bucket
5. Then use cross region replication to store that meme somewhere else in the world
* Case 2
1. User sends a HTTP request to API Gateway
2. API Gateway proxies that to Lambda
3. Lambda will run the code in response to the HTTP request, and then send it back to API Gateway
4. API Gateway sends it back to the user
5. If having more users, will trigger more Lambda functions, that how well itscales.

### Traditional vs Serverless
* Traditional:
Requests->Route 53->ELB->Web Server->Database->Back to user
- Relying on physical (VM, OS)
* Serverless
API Gateway->Lambda->DynamoDB/Aurora->Back to user
- Can scale instantly, don'y worry about auto scaling

### What Languagues Does Lambda Support?
* Node.js
* Java
* Python
* C#
* Go
* PowerShell

### How is Lambda Priced?
1. Number of requests
2. Duration

## SAM -- Serverless Applicatino Model
### What Is SAM?

* CloudFormation extension optimized for serverless applications
* New types: functions, APIs, tables
* Supports anything CloudFormation supports
* Run serverless applications locally (Docker)
* Package and deploy using CodeDeploy

### Anatomy of a SAM Temlate

## ECS -- Elastic Container Service
### What Are Containers and Docker?
* A==container==is a package that contains an application, libraries, runtime, and tools required to run it.
* Run on a container engine like==Docker==
* Provides the==isolation==benefits of virtualization with less oberhead and faster starts than VMs
* Containerized applications are==protable==and offer a consistent environment


### What Is ECS?
* Managed container==orchestration service==
* Create clusters to manage fleets of container deployments
* ECS manages EC2 or Fargate instances
* Schedules containers for optimal placement
* Defines rules for CPU and memory requirements
* Monitors resource utilization
* Quickly deploy, update, roll back
* ==FREE==, but still need to pay EC2.
* Integrates with VPC, security groups, EBS volumes, ELB
* Can see any of changes to clusters, all logs in CloudTrail


### 6 Components
1. Cluster
* Logical collection of ECS resources -- either ECS, EC2 instances or Fargate instances
2. Task Definition
* Defines your application.
* Similar to a Dockerfile but for running containers in ECS.
* Can contain multiple containers
* If you have two containers that always need to run together, you would place both inside the same task definition.
3. Container Definition
* Defines the individual containers a task uses
* Contrils CPU and memory allocation and port mappings
4. Task
* Single running copy of any containers defined by a task definition
* One working copy of an application (e.g. DB and web containers)
5. Service
* Allows task definitions to be sacled by adding tasks.
* Defines minimum and maximum values
* Depends on your auto scaling rules
6. Registry
* Storage for container images (e.g. Elastic Container Registry (ECR) or DOcker Hub)
* Used to download images to create containers

### Fargate
* A serverless compute engine
* Works with both==ECS and EKS==
* Eliminates need to provision and manage servers
* Each workload runs in its own kernel
* Isolation and security, no risk for provisioning
* Choose EC2 instead if:
* Compliance requirements
* Require broader costomization
* Require GPUs

### EKS -- Elastic Kubernetes Service
* K8s is==open-source==software that lets you deploy and manage containerized applications at sacle
* Same toolset on-premises and in cloud
* Containers are grouped in==pods==
* Like ECS, supports both EC2 and Fargate
* Why use EKS?
* Already using K8s
* Want to migrate to AWS

### ECR -- Elastic Container Registry
* Managed Docker container registry
* Store, manage, and deploy images
* Integrated with ECS and EKS
* Works with on-premises depllyments
* Highly available
* Integrated with==IAM==
* Pay for storage and data transfer (similar to S3)

### ECS + ELB = :heart:
* Distribute traffic evenly across tasks in your service
* Supports ALB, NLB, CLB (application, network, classic load balancer)
* Use ALB to route HTTP/HTTPS (layer 7) traffic
* Use NLB or CLB to route TCP (layer 4) traffic
* Supported by both EC2 and Fargate launch types
* ALB allows:
* Dynamic host port mapping
* Path-based routing
* Priority rules
* **ALB is recommended over NLB or CLB**
(除非你要的功能只有NLB或CLB才有)

### ECS Security
* Instance roles:
* All EC2 intances can access S3 bucket, that's not we want
*
* Task roles:
* Allows to have much more granular control over the permissions
* Apply policy on a per task basis, each task is signed a role


### Exam Tips
* Lambda scales out (not up) automatically
* Lambda functions are independent, 1 event = 1 function
* RDS is not serverless, except Aurora Serverless
* ALB, Cognito, Lex, Alexa, API Gateway, CloudFront, and Kinesis Data Firehose are all valid direct (synchronous) triggers for Lambda functions.
* S3 and EC2 are the valid asynchronous triggers.
* Serverless computing is not about eliminating servers, but shifting most of the responsibility for infrastructure and operation of the infrastructure to a vendor.
* Like any services in AWS, Lambda needs to have a Role associated with it that provide credentials with rights to other services. This is exactly the same as needing a Role on an EC2 instance to access S3 or DDB.
* Lambda billing is based on both The MB of RAM reserved and the execution duration in 100ms units.
* AWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a microservices & serverless architectures.