Learning Notes for AWS Certified Solutions Architect

# Learning Notes for AWS Certified Solutions Architect – Associate (SAA-C03) This is the learning notes for AWS Certified Solutions Architect – Associate (SAA-C03). The notes are for AWS services key points, and are written based on my learning journal. ## <img src="https://icon.icepanel.io/AWS/svg/Security-Identity-Compliance/Identity-and-Access-Management.svg" style="min-width:15px; max-width:30px;" /> Identity and Access Management (IAM) Identity and Access Management is the key for managing multiple users in AWS account. In IAM, it is easy to set different policies for different user (groups). The basic form of IAM policy looks like the following: #### IAM Policy ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::example-bucket" }, { "Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::example-bucket/*" } ] } ``` The policy allows users to perform two specific actions in Amazon S3: ListBucket on the bucket itself and GetObject on the objects within that bucket. > [!Tip] > Using "*" represents everything. It can be used in either Action or Resource label, if both, it means that given the user full access to everything. (Which is exactly what the AdministratorAccess doing) #### IAM Role IAM also provides the Role feature to control who can access what resources. Differences between **Role** and **Group** is that **Role** can allow users in it to access target resources, while **Group** are used to manage multiple users together and assign common policies to them. All users or services that assume the role will receive temporary access permissions defined by the role. This makes it easier to manage temporary identities. #### IAM Security Head on to `IAM` → `Credential report` can download the credentils report for all users in the AWS account. It provides an easy way to view the potential problems. ![Screenshot 2025-09-23 at 5.27.03 PM](https://hackmd.io/_uploads/rJoMuke3xx.png) > [!Important] Important notes for IAM > 1. Don't use root account except for AWS account setup. > 2. If others want to use our AWS, create another user for him/her. > 3. Create a strong pssword policy. > 4. Never give others the access key. ## <img src="https://icon.icepanel.io/AWS/svg/Cloud-Financial-Management/Budgets.svg" style="min-width:15px; max-width:30px;" /> AWS Budget Set up the AWS Budget can help notifying any budget exceed on the way. If a budget exceeded, it will send a notification email to the target mail account assigned in budget setup page. #### AWS Budget alert ![Screenshot 2025-09-24 at 5.08.25 PM](https://hackmd.io/_uploads/SkvMHNb3gg.png) ## <img src="https://icon.icepanel.io/AWS/svg/Compute/EC2.svg" style="min-width:15px; max-width:30px;" /> Amazon EC2 Amazon EC2 offers several services including renting virtual machines, storing data on virtual drives, distributing load across machines and auto scaling. #### Bootstraping Bootstraping means the launch commands when a machine starts. EC2 provides this kind of feature to let users initialize the machine with actions like install update, install software, download necessary files and others. #### Create instance To create an instance, head on to EC2 page and click `Launch Instance`. The important points is that make sure to generate key pair if not having one, it allows us to login using SSH. <img style="width: 70%" src="https://hackmd.io/_uploads/HyTIQPMnlx.png"> If the instance is for web server, `Allow HTTP(s) traffic from the internet` should also be checked. ![Screenshot 2025-09-25 at 2.40.31 PM](https://hackmd.io/_uploads/rJs1EwMnlx.png) After creating the instance, the access route can be set at the `Network & Security` tab. > [!Warning] > When an instance is not needed, it can be stopped, if it is attached to a volume the state will be remembered. > **Restarting the instance** will **cause the public IP to be changed**. #### IAM role for EC2 If we want to control things in EC2 instance, never enter the (Secret) Access Key in the aws configure in EC2 instances, as it could be accessed by any other users in the account that has the permission to EC2. Instead, use IAM role. Select the instance, go to `Actions` → `Security` → `Modify IAM role` at top right corner: ![Screenshot 2025-09-25 at 4.47.18 PM](https://hackmd.io/_uploads/BkX5GtM3gx.png) #### IAM Pricing options There are some pricing options for Amazon EC2 showing as follow: - On Demand: Basically pay as what we use, it has the highest cost but no upfront cost. - Reserved Instances: Saved about 72% than On Demand, specify the reservation period to use this service (1 or 3 years). - Saving Plans: Get a discount based on long-term usage. - Spot Instances: The most efficient instances in AWS. Spot Instances run as long as spare AWS capacity is available, but can be interrupted by AWS when the capacity is needed back or the bid price is exceeded. - Dedicated Hosts: A physical server with EC2 instance capacity fully dedicated by ourselves. It allow ourselves to address our own licenses. - Dedicated Instances: Instances runs on hardware that is dedicated to ourselves. - Capacity Reservtions: Reserve On-Demand instances capacity in a specific AZ for any duration. #### Placement Groups Options Controling multiple EC2 instances with placement group is a way of management. The main ways of placement groups contain Cluster, Spread and Partition. #### Placement Groups - Cluster Cluster is basically connecting all instances together. The benefit of cluster is that instances in it connect with each others with great network. However, the cons of it is if the AZ fails, all instances fails at the same time. **Use cases**: Work that contains huge data set and needs to be completed fast with low latency. #### Placement Groups - Spread Spread is the concept that dividing the instances into multiple AZs to prevent simultansouly failure. The cons of it is that instances in one AZ is limited to 7. **Use cases**: Applications that needs maximize availability. #### Placement Groups - Partition Partition is the way that group some instances as a partition, and can have multiple partitions in one AZ. Partitions prevent rack(hardware) failure, and each AZ can have up to 7 partitions. **Use cases**: Big data and partition aware applications. #### EC2 Hibernate EC2 Hibernate is a method to store the current state of a EC2 instance. It allows the instance to start up again fast by the stored RAM data. Applications like large data analysis would use this kind of mechanism. #### Elastic Network Interface (ENI) An ENI can be thought of as a virtual network card that can be attached or detached from an EC2 instance to connect it to a VPC. The main benefit is its flexibility, as it allows users to quickly switch network configurations or move network interfaces between instances when an error occurs. #### Amazon Machine Image (AMI) AMI are the customization of an EC2 instance that we can add our own software, configuration, operating system, offering, etc... Right click on instance → `Image and templates` → `Create image` to create a image for specific image, and can be restored from AMI when creating new instances. #### EC2 Instance Store Temporary block storage that is physically attached to the host machine. It offers very high I/O performance but data is lost when the instance is stopped, terminated, or fails. Best suited for caching, buffers, and temporary data. ## <img src="https://icon.icepanel.io/AWS/svg/Storage/Elastic-Block-Store.svg" style="min-width:15px; max-width:30px;" /> Amazon EBS Amazon Elastic Block Store(EBS) is the storage system designed to attached on running instances. It allows the instance to persist data even after the termination. It has a provisioned capacity. #### Snapshot For EBS, it can be snapshot to save the resources or transfering it to other AZs (or even regions). Head on to `Actions` → `Create snapshot` to get a snapshot for EBS volume. In snapshot page, we can copy the snapshot into specific AZ, to restore the volume. #### EBS Volumes Type - General Purpose SSD (`gp2`/`gp3`): Cost effective storage, low-latency. - Provisioned IOPs (PIOPS) SSD (`io1`/`io2`): Critical bussiness applications with sustained IOPS performance. - Hrad Disk Drives (`stl`/`scl`): Cnnot be a boot volume. > [!Tip] > EBS io1/io2 volumes can be attached to multiple instances in the same AZ at the same time, the maximum is 16 instances at a time. ## <img src="https://icon.icepanel.io/AWS/svg/Storage/EFS.svg" style="min-width:15px; max-width:30px;" /> Amazon EFS Amazone Elastic File System is the managed network file system that can be mounted on many EC2 instances. It is highly available and scalable, but only compatible for Linux-based AMI. #### Storage Classes EFS provides several storage classes for different usage: - Standard: Designed for frequently accessed files. - Infrequent Access(EFS-IA): Cost to retrieve files, lower price to store. - Archive: Designed for rarely accessed data (few times each year), and is cheaper 50%. EFS can automatically change the storage class based on the access frequency. ![Screenshot 2025-10-01 at 1.47.14 PM](https://hackmd.io/_uploads/B1mOxSqnee.png) ## <img src="https://icon.icepanel.io/AWS/svg/Networking-Content-Delivery/Elastic-Load-Balancing.svg" style="min-width:15px; max-width:30px;" /> AWS ELB AWS Elastic Load Balancer(ELB) is the mechanism that balance the connections into multiple instances. Currently there are four generation load bakancers: Classic Load Balancer(Deprecated), Application Load Balancer, Network Load Balancer and Gateway Load Balancer. It is best to use newer generation as they provide more features. #### Application Load Balancer(ALB) Application Load Balancer to multiple HTTP application across machines (target groups) / at the same machine. It support HTTP/2 and WebSocket, and support redirect feature. Application Load Balancer supports routing on path in URL, hostname in URL or Query String, Headers. ALB are good for micro services and container-based application. > [!Note] > Before using thew load balancer, we should first add the resources in target group to let load balancer recognize them. > [!Note] > For a load balancer, it also allow us to do custom definition for different responces when reaching specific paths or redirect. #### Network Load Balancer(NLB) Network Load Balancer supports one static IP per AZ, and is supported to assigned Elastic IP. It allows TCP/UDP trafic to instances and has Ultra-low latency. #### Gateway Load Balancer(GWLB) Deploy, scale and manage a fleet of 3rd party network virtual applicances in AWS. It operates at Layer 3(Network Layer) and uses the GENEVE protocol on port 6081. #### Sticky Sessions Stick Session is the mechanism that makes the same client always redirect to the same instance behind a load balancer. Usually there will be a cookie to set the expiration date, so that the client can be reroute to another instaance after a period of time. <img style="width: 50%" src="https://hackmd.io/_uploads/BywXwR3nlg.png"> There are Application-Based Cookies, which are stored in the browser for the current session, and Duration-Based Cookies, which persist for a specified period of time. In AWS, sticky sessions can be configured through the load balancer’s attribute settings. #### Cross-Zone Load Balancing Each load balancer instance distributes evenly across all registered instances in all AZ. For example, if 2 instances in AZ1 and 8 instances in AZ2, than each instances in both AZ will handle 10% of the incoming traffic. #### SSL - Server Name Indication(SNI) SNI solves the problem of loading multiple SSL certificates onto one web server (in order to serve multiple websites), and the settings can be set at the security listener in the load balancer page. ## <img src="https://icon.icepanel.io/AWS/svg/Compute/EC2-Auto-Scaling.svg" style="min-width:15px; max-width:30px;" /> EC2 ASG Auto Scaling Group(ASG) is the scaling mechanism for EC2 instances. In real life load in websites and applications might change, auto scaling helps scale up and in when facing different situations. #### Scaling Policies There are three main scaling policies for auto scaling: - Dynamic Scaling: There are two advanced choices in dynamic scaling, target tracking scaling for specific matrix and value, and simple / step scaling when a specific point(service) is triggered. - Scheduled Scaling: Anticipate a scaling based on known usage patterns. - Predictive Scaling: Continuously forecast load and schedule scaling ahead. ## <img src="https://icon.icepanel.io/AWS/svg/Database/RDS.svg" style="min-width:15px; max-width:30px;" /> Amazon RDS Relational Database Service(RDS) is managed using SQL as query language. It supports Postgres, MySQL, MariaDB, Orcale, Microsoft SQL Server, IBM DB2 and Aurora. RDS managed OS patching, automatic provisioning, multi AZ setup and many other features that is better than directly deploying DB on EC2 instances. > [!Note] > RDS provides the auto scaling machanism that does not need us to stop the database and change the storage space by ourselves. We can set the maximum storage threshold to let it configured automatically. #### Read Replicas RDS provides read replicas to help reduce the query load for a single database. The replicas will be read-only and async. The use case might be another team or work needs to read the data in the existing database, but we don't want the extra query for the current one, than the read replicas will be a good choice. > [!Important] > Read replicas will not cost if the target is in the same region(eu-east-1a → eu-east-1b), but will cost money for cross-region(eu-east-1a → eu-west-1b). Read replicas can be setup as multi-AZ for disaster recovery. #### Create Database RDS provides the setup options for multi-AZ and single instance in a clear manner. ![Screenshot 2025-10-08 at 3.56.12 PM](https://hackmd.io/_uploads/BkY7t97alx.png) #### Delete Database To delete the database, head on to `Modify` and scroll down to the very bottom, there's an enable deletion protection checkbot that needs to be unchecked before deleting it. ![Screenshot 2025-10-08 at 4.47.24 PM](https://hackmd.io/_uploads/BJ97Sompxg.png) Click `Next`, we can select to apply the modification immediately if we want. After that, it is good to go for deletion process. ![Screenshot 2025-10-08 at 4.49.41 PM](https://hackmd.io/_uploads/Hy72HsmTge.png) > [!Tip] > RDS Custom is the alternative option for Oracle and Microsoft SQL Server. It allows us to access and customize legacy applications or specialized workloads that require custom database settings. ## <img src="https://icon.icepanel.io/AWS/svg/Database/Aurora.svg" style="min-width:15px; max-width:30px;" /> Amazon Aurora Aurora is the database that will automatically grows up to 128TB if needed. Also, it is 5x times faster than MySQL on RDS, and 3x timed faster than Postgres on RDS. Aurora copy the data into 3 pairs. They are stored in 3 different AZs and each AZ has 2 copies. The concept is that: - If 4 out of 6 copies successfully written, it would be considered as action completed successfully. - If 3 out of 6 copies works, than the data can be successfully read. Aurora provides a Writer Endpoint for us to communicate with, in case the current main database fails, it will automatically switch to another available one. Same, it provides Reader Endpoint for us, while it will automatically trying to read from one database. Aurora also provides Custom Endpoints in case we want to use analytic queries on instances that is set to have higher performance. #### Aurora Global Database Nowadays it is recommended to use the global database tht we will have to select 1 primary region, and select up to 10 secondary regions. Read replicas can be set up to 16 per secondary region, which highly reduce the load for database. > [!Note] > Babelfish allows Aurora PostgreSQL to understand commands targeted for Microsoft SQL Server. #### RDS Backups RDS automatically backup the database daily, and backup the transaction logs every 5 minutes. The benefit is that it does not need us to manually set it, but the backup state can only be the last 35 days, so if specific moment is required, snapshot will be a better choice. #### Aurora Backups Similar to RDS backups, Aurora backups can also save up to 35 days, and can be restored using snapshot or from S3 just like RDS backup. #### Encryption At launch time, it automatically encrypt the master database. If somehow it didn't, to create an encrypted database with the same data, it must be snapshotted and change the config from the image. ## <img src="https://icon.icepanel.io/AWS/svg/Database/ElastiCache.svg" style="min-width:15px; max-width:30px;" /> Amazon ElastiCache ElastiCache is in-memory databases supports Redis and Memcached with really high performance. If the application can fetch the dataa from cache, it can reduce the database query load, if not, it will read the data and write to ElastiCache. <img style="width: 70%" src="https://hackmd.io/_uploads/SJECpxBTgg.png"> > [!Note] > ElastiCache is lazy loading, write through(add or update data in cache when written to db) and session store. ## <img src="https://icon.icepanel.io/AWS/svg/Networking-Content-Delivery/Route-53.svg" style="min-width:15px; max-width:30px;" /> Amazon Route 53 Route 53 is a highly available, scalable, fully managed and Authoritative DNS. Route 53 is also a domain registrar which allow us to register our own domain name on it. Route 53 supports the DNS record types: A, AAAA, CNAME and NS. Below are the explanation of them: - A: maps a hostname to IPv4 - AAAA: maps a hostname to IPv4 - CNAME: maps a hostname to another hostname, the target domain must have an A or AAAA record - NS: Name Servers for the Hosted Zone > [!Note] > A hosted zone is a container for records/definitions to decide how to route traffic to domain and its subdomains. ### Time To Live(TTL) For records in Route 53, we can determine their TTL. A short TTL will result in a large traffic in Route 53 and costs more money, but can respond to easy change when needed. In the opposite side, a long TTL can efficiently reduce the load of DNS request, but we will have to wait a long time to make sure all users getting the newest data after change. ### CNAME v.s. Alias CNAME and alias in Route 53 are both used for pointing hostname to another hostname. The difference is that alias can be used on a root domain. Alias cannot set the TTL, and it must be A/AAAA type. ### Routing Policies Here are the routing policies used in Route 53, basically following the rules like below: - Simple: Route traffic to a single source. If multiple source provided, client will choose a random one. - Weighted: Control the % of requests that go to each specific resources. - Latency: Redirect the traffic to the least latency resource. - Failover: Automatically switch to secondary route if the primary fails. - Geolocation: Route based on the user physical location. - Geoproximity: Provides the ability to shift more trafic to resources based on defined bias. - IP-based: Routing based on client's IP address. - Multi-value: For routing traffic to multiple resources, and can be associated with health checks. ### Health checks The health check mechanism verifies whether a resource is healthy. Fifteen global health checkers send requests to the resource, and if more than 80% of them receive a healthy response, the resource is considered healthy. There is also a Calculated Health Check, which is used to combine the results of multiple health checks. We can define custom rules to determine the overall health status, such as specifying how many individual checks must pass, or by using logical operators like AND, OR, and NOT to create complex evaluation logic. > [!Note] > We can choose to buy a 3rd party domain and use Route 53 as DNS service provider with the following steps: > 1. Create a Hosted Zone in Route 53 > 2. Update NS records on 3rd party websites to use Route 53 name servers ## Solutions Architecture ### Stateless services Assume that we're building a service on AWS, we sould consider all possible cases as we can. The origin form might look something like this(assigning an elastic IP to the instance): <img style="width: 30%" src="https://hackmd.io/_uploads/ryilGwI0xe.png"> However, when users getting more and more, we might need to scale up vertically, and also horizontally to match the requirements: <img style="width: 30%" src="https://hackmd.io/_uploads/HJZ-zvLAlg.png"> We then want the user not to remember that many IP addresses, so implementing the Route 53: <img style="width: 40%" src="https://hackmd.io/_uploads/H107vP8Ale.png"> To prevent some users experiencing the down time of some specific instances, we can instead change the instances to be private, and use an Elastic Load Balancer and enable Health Check to prevent the issue: <img style="width: 70%" src="https://hackmd.io/_uploads/r12E6wIAxx.png"> Then, we can provide some auto scaling mechanism to help increase the instances when needed. <img style="width: 75%" src="https://hackmd.io/_uploads/H1H_TPLCeg.png"> In the current situation when an AZ went down, the service will no longer available. So instead, we can enable multi-AZ: <img style="width: 80%" src="https://hackmd.io/_uploads/H19lCwLRlg.png"> Finally, reduce the initial scale to reduce cost: <img style="width: 80%" src="https://hackmd.io/_uploads/rJabCP8Agl.png"> ### Stateful services For services that needs to remember some state like shopping carts, login status, we can consider enabling sticky session or using Cookies. <img style="width: 80%" src="https://hackmd.io/_uploads/BkgqX_IAxl.png"> Or using the elastic cache to remember those information. <img style="width: 80%" src="https://hackmd.io/_uploads/SJGQE_U0ex.png"> For those information that needs to be stored permanently, consider using RDS, and maybe create read replicas if needed. <img style="width: 85%" src="https://hackmd.io/_uploads/SJ4nEdL0ge.png"> For some cases, we can also read the data from RDS and store it in elastic cache, and other instances can just fetch the data from cache. <img style="width: 85%" src="https://hackmd.io/_uploads/HJJVUuIRxg.png"> > [!Note] > When using alternative databases or when storing data in Amazon EFS that is accessed by multiple ENIs, ensure that the connection mechanism is correctly configured. For example, avoid using Amazon EBS in this scenario, as it does not support concurrent connections from multiple instances. ## <img src="https://icon.icepanel.io/AWS/svg/Compute/Elastic-Beanstalk.svg" style="min-width:15px; max-width:30px;" /> AWS Elastic Beanstalk Elastic Beanstalk is a developer centric view of deploying an application on AWS. There are two types of deployment modes in Beanstalk, single instance is great for development stage, and high availability with load balancer is great for production stage. ### Create Elastic Beanstalk When creating elastic beanstalk, we can go to CloudFormation to see the events it created, and check the current creation status.![Screenshot 2025-10-23 at 9.14.15 PM](https://hackmd.io/_uploads/H1UEqiPCex.png) ## <img src="https://icon.icepanel.io/AWS/svg/Storage/Simple-Storage-Service.svg" style="min-width:15px; max-width:30px;" /> Amazon S3 Amazon Simple Storage Service(Amazon S3) is a service that provides infinite scaling storage. Many website and service in AWS uses it as backbone. S3 can be used at most of the cases, including backup, storage, disaster recovery, archive, application hosting, data lakes and so on. Amazon S3 allows data to be stored in buckets, the buckets should have a globally unique name. ### Objects Objects is the main components in Amazon S3. Here's some definition: - Objects uses a file path as a key. - Max object size is 5TB. ### Security Security in Amazon S3 also provides many security options: - IAM policies for allowing specifc users. - Bucket policies that can build up wide rules from console. - Object Access Control List - finer gran. - Bucket Access Control List - less common. To modify the permission of objects for public to see, we can use the [policy generator](https://awspolicygen.s3.amazonaws.com/policygen.html) to get the required policy and apply to S3 bucket. > [!Tip] > When deleting an object in the S3 bucket, it will actually apply a Delete Marker on it. When we turn on the versioning tab, we can still see the object there. To restore it, we can delete the Delete Marker, that will make the object available again. ### Replication Replication for S3 bucket has two types: Cross-Regin Replication(CRR) and Same-Region Replication(SRR). For deletions between replicated buckets, we can turn on the Replicate Delete Marker option, but it will not affect the actual deletion operation(permantely delete will not be replicated). ### S3 Storage Classes S3 provides many storage classes: - Standard - General Perpose: 99.99 availablity, used for frequently access data. - Standard - Infrequent Access: 99.9 availablity, for les frequent access, but requires rapid access when needed. - One Zone - Infrequent Access: 99.5 availablity, 99.999999999 durability, in a single AZ. - Glacier Instant Retrieval: Low cost storage meant for archive or backup. Milisecond retreival, great for data accessed once a quater. - Glacier Flexible Retrieval: Expedited (1 to 5 minutes), Standard (3 to 5 hours), Bulk (5 to 12 hours) - free. - Glacier Deep Archive: Standard (12 hours), Bulk (48 hours). - Intelligent Tiering(Frequent Access Tier, automatically): Move object automatically between tiers based on usage. This is the default tier. - Intelligent Tiering(Infrequent Access Tier, automatically): Object not accessed for 30 days. - Intelligent Tiering(Archive Instant Access Tier, automatically): Object not accessed for 90 days. - Intelligent Tiering(Archive Access Tier, optional): Object not accessed for 90 days to 700 days+. - Intelligent Tiering(Deep Archive Access Tier, optional): Object not accessed for 180 days to 700 days+. ### S3 Express One Zone This is the explicit storage class that is high performance, sigle Availability Zone storage class. Objects are stored in Direct Bucket(bucket in a single AZ). It is designed for co-locate storage and compute resources in the same AZ. ### S3 Lifecycle Rules We can define our own lifecycle rules in Amazon S3. ![Screenshot 2025-10-27 at 8.48.25 PM](https://hackmd.io/_uploads/r1LQ51pAge.png) The option is in Management tab, and we got many actions to choose. <img style="width: 70%" src="https://hackmd.io/_uploads/HyT1sJTRgl.png"> After selecting an action, we can set the transition time for each tier. ![Screenshot 2025-10-27 at 8.56.09 PM](https://hackmd.io/_uploads/BkrlnkTCgx.png) > [!Note] > There's a thing called requester pay. It is designed for some user that might provide heavy files to be able to reduce the cost for other users' download actions. ### S3 Event Notification Amazon S3 provides several event notification, that can be then attached to services like SNS, SQS or Lambda function. In AWS, we can speed up the upload actions using multi-part load, transfer acceleration or byte-range fetches. ### S3 Storage Lens Another service is Storage Lens that can understand, analyze and optimize storage across entire organization. Storage Lens contains metrics: - Summary Metrics: General insights about the S3 storage. - Cost-Optimization Metrics: Provide insights to manage and optimize the storage cost. - Data Protection Metrics: Provide insights for data protection features. - Access-Management Metrics: Provide insights for S3 Object Ownership. - Event Metrics: Provide insights for S3 event notifications. - Performance Metrics: Provide insights for S3 transfer acceleration. - Activity Metrics: Provide insights about how storage is requested. - Detailed Status Code Metrics: Provide insights for HTTP status codes. ### Amazon S3 Encryption Amazon S3 envryption has both SSE(Server Side Encryption) and CSE(Client Side Encryption). #### SSE - For SSE, we uses the encryption key handled, managed and owned by AWS. - SSE-KMS let us uses keys from Key Management Service, help us handle and manage them. - SSE-C allow us to provide our own keys, but required https encryption. #### CSE - For CSE, clients must encrypt the data before sending it do Amazon S3, and decrypted by themselves when receiving data. > [!Note] > When we're making requests to S3 buckets in a different origin, remember to set the `CORS` in `Permission` tab. ### S3 Access Logs We can set a bucket as log storage for recording actions taken on other buckets. This allow us to later on uses analytic tools to analyze those actions. > [!Caution] > **Never** use the bucket that been monitored as the storage bucket, or it will create an infinite loop that costs tons of money. The logging setting can be triggered in Permission tab: ![Screenshot 2025-10-28 at 10.12.35 PM](https://hackmd.io/_uploads/S1XD1UR0ge.png) ### S3 Access Points Access points is a way to simplify security management for S3 buckets. We can define them like bucket policy, and it is also possible to set an access point that accessible only in the VPC(but we need to set up a VPC endpoint). ### S3 Object Lambda We can setup AWS Lambda function to modify the data before sending it out to the requester. That allows us to use the same bucket for datas that only needs a few changes. ## <img src="https://icon.icepanel.io/AWS/svg/Networking-Content-Delivery/CloudFront.svg" style="min-width:15px; max-width:30px;" /> AWS CloudFront CloudFront is a Content Delivery Network(CDN) that improves read performance by caching content at edge locations around the world. Unlike S3 replication, which copies data across AWS regions, CloudFront uses a global edge network to serve content closer to users. To use it, we simply create a distribution and point it to an origin such as an S3 bucket. Even if the bucket is private, CloudFront can still access it through Origin Access Control (OAC), ensuring that the content is only reachable through CloudFront and not directly from S3. The benefit is that CloudFront acts as a public access layer, while the actual resources (S3, ALB, EC2, etc...) remain private and secure. > [!Note] > In the security tab of distribution, we can set the geographic restriction allow/block list. When updating datas in S3 bucket, we should do cache invalidate to force CloudFront refersh it's memorization, and get the new file when users asking for it. ## <img src="https://icon.icepanel.io/AWS/svg/Networking-Content-Delivery/Global-Accelerator.svg" style="min-width:15px; max-width:30px;" /> AWS Global Accelerator AWS Global Accelerator uses the concept of anycast to route client connection to the nearest edge location, and then directly goes into services like ALB. Global accelerator did not cache anything, it is a way to reduce the connection path. ## AWS storage extras Here are some extra storage services that would work better on specific cases: - AWS snowball: It is highly secure, portable devices to collect and process data at the edge, and can help migrate up to Petabytes data into AWS. - Amazon FSx: Allow us to launch 3rd party high performance file systems on AWS such as Lustre, NetApp ONTAP, Windows File Server, OpenZFS, etc... ## AWS Storsge Gateway A beidge between AWS cloud data and on-premises data. Depend on the needs, there are many storage gateway options, here are some of them: - S3 File Gateway: Used for S3 family(except Glacier). - Volume Gateway: Blocked storage using iSCSI portocol backed by S3. Cached volumes is for low latency access to most recent data and Storage Volumes that the entire dataset is on premises, scheduled backups to S3. - Tape Gateway: Support the companies that have backup processes using physical tapes. ## <img src="https://icon.icepanel.io/AWS/svg/Migration-Transfer/Transfer-Family.svg" style="min-width:15px; max-width:30px;" /> AWS Transfer Family Transfer Family is a fully-managed service for the file transfers into and out of Amazon S3 or Amazon EFS using FTP portocol. It supports FTP, SFTP and FTPS, and can be milti-AZ. ## <img src="https://icon.icepanel.io/AWS/svg/Migration-Transfer/DataSync.svg" style="min-width:15px; max-width:30px;" /> AWS DataSync DataSync allows us to move large amount of data to and from on-premises or other cloud to AWS. One agent task can use 10 Gbps, and it is possible to setup a bandwidth limit. > [!Note] > DataSync is not continues, it is scheduled task. ## <img src="https://icon.icepanel.io/AWS/svg/App-Integration/Simple-Queue-Service.svg" style="min-width:15px; max-width:30px;" /> Amazon SQS Simple Queue Service(SQS) is used for decoupleing services. It allows multiple input and output, and is fully managed. <img style="width: 50%" src="https://hackmd.io/_uploads/BJgQTQIJ-e.png"> ### Consumer Messages Consumer might be an EC2 instance, server, AWS Lambda..., it will try to poll SQS for messages and process them. After that, it will call a DeleteMessage API so that the message processing is complete. > [!Tip] > We can setup an auto scaling group and monitor the SQS for alarm to increase the group size, in order to handle more messages when the application is on heavy load period. ### SQS Security In SQS, it has in-flight encryption using HTTPS API, at-rest encryption using KMS keys, and client side encryption. For access control, it allows IAM control or SQS access policies. ### Message Visibility Timeout When a message being requested, it will invisible for others. However, if it did not been processed in a period of time, it will be mark as visible, allowing others to poll it again. Of course we can call the ChangeMessageVisibility API to tell SQS that it just need more time to process, and the message will not be opened to others. <img style="width: 80%" src="https://hackmd.io/_uploads/r1-tXNLy-g.png"> ### Long Polling When a consumer requested for messages and can wait until it gets one if the queue is currently empty, it is called long polling. ## <img src="https://icon.icepanel.io/AWS/svg/App-Integration/Simple-Notification-Service.svg" style="min-width:15px; max-width:30px;" /> Amazon SNS Same as Amazon SQS, Simple Notification Service(SNS) is a kind of message process service. Instead of storing it and polled by others, it can have many topics for subscribers to listen on them. <img style="width: 50%" src="https://hackmd.io/_uploads/HJp5CdDkZe.png"> > [!Note] > Instead of sending multiple messages into many SQS queue, we should consider sending it into SNS, and let those SQS queues become the subscribers of that topic. In SNS, we can also apply filter to determine which kind of messages should be sent in what SQS queue. ## <img src="https://icon.icepanel.io/AWS/svg/Analytics/Kinesis-Data-Streams.svg" style="min-width:15px; max-width:30px;" /> Amazon Kinesis Data Streams Amazon Kinesis Data Streams is used to collect and store streaming data in real-time. Data in Kinesis will disappear when it expires, and cannot be deleted manually. Kinesis has two capacity modes: - Provisioned mode: Select how many shards we need, each shard gets 1MB in (or 1000 records per second) and 2MB out. - On-Demand mode: Default capacity of 4MB/s, and scale automatically based on observed throughput peak during the last 30 days. Kinese Data Steam require us to write the producer & consumer code by ourselves. ## <img src="https://icon.icepanel.io/AWS/svg/Analytics/Kinesis-Firehose.svg" style="min-width:15px; max-width:30px;" /> Amazon Data Firehose Amazon Kinesis Data Firehose is a fully managed, near real-time data delivery service. It can automatically receive, transform, and load streaming data into destinations such as S3, Redshift, or Elasticsearch without requiring you to write data processing applications. > [!Note] > Another service similar is Amazon MQ. It is a managed message broker service that supports open-source messaging protocols like ActiveMQ and RabbitMQ, making it easy to migrate existing message-based applications to AWS without rewriting code. ## <img src="https://icon.icepanel.io/AWS/svg/Containers/Elastic-Container-Service.svg" style="min-width:15px; max-width:30px;" /> Amazon ECS Launching Docker on AWS equals launching ECS Tasks on ECS clusters. ECS agent will register the actual container in AWS ECS clusters. If it is being launched as EC2 Launch Type, it means we need to maintain and provision the infrastracture by ourselves. If It is launched as Fargate Launch Type, it is serverless, we don't need to provision and maintain them. ### IAM Roles for ECS In ECS, we can set EC2 Instance Profile(for EC2 launch type) to manage the container, or use ECS Task Role to specify each tasks. > [!Tip] > We can launch the ECS and use EFS to store data. ## <img src="https://icon.icepanel.io/AWS/svg/Containers/Elastic-Container-Registry.svg" style="min-width:15px; max-width:30px;" /> Amazon ECR Amazon ECR(Elastic Container Registry) is used to store and manage Docker images in AWS. The access is controlled through IAM. ECR supports vulnerability scanning, versioning, image tags, image lifecycles, etc... ## <img src="https://icon.icepanel.io/AWS/svg/Containers/Elastic-Kubernetes-Service.svg" style="min-width:15px; max-width:30px;" /> Amazon EKS Amazon EKS(Elastic Kubernetes Service) is to launch managed Kubernetes cluster on AWS, it's similar to ECS but with different API. Amazon EKS provides several Node Types: - Managed Node Groups: Creates and managed Nodes for us, it supports on-demand or spot instances. - Self-Managed Nodes: Node created by ourselves and register to EKS. Also supports on-demand or spot instances. - AWS Fargate: No maintenance required, no nodes managed. We need to specify StorageClass manifest on the EKS cluster. ## <img src="https://icon.icepanel.io/AWS/svg/Compute/App-Runner.svg" style="min-width:15px; max-width:30px;" /> AWS App Runner It is a fully managed service that makes it easy to deploy web applications and APIs at scale. We can just configure settings like vCPU, RAM, Auto Scaling, Health Check and so on directly, and just run the builds and deploy the web app with App Runner. It also help automatic scaling, high available, load balancer and encryption. ## AWS App2Container(A2C) It's a CLI tool for migrating and modernizing Java and .NET web apps into Docker Containers. We can easily Lift-and-shift our apps running in from many different places to AWS. ## <img src="https://icon.icepanel.io/AWS/svg/Compute/Lambda.svg" style="min-width:15px; max-width:30px;" /> AWS Lambda AWS Lambda is a serverless way to provide services. We only pay for what we use, it is designed for on-damand, and automatically scale when needed. Lambda is good for responding to events. For example we can upload some files to S3 bucket and trigger Lambda to make actions, maybe upload it to another database after some modification to the file. Or we can use CloudWatch Events EventBridge to trigger Lambda per hour to do some serverless jobs. After creating a function, we can test it using the tab below: ![Screenshot 2025-11-12 at 9.18.13 PM](https://hackmd.io/_uploads/SymXtZMx-l.png) > [!Note] > AWS Lambda has the memory allocation around 128MB to 10GB, and the maximum execution time is 15 mins. Environment variables can take up to 4KB, disk capacity in the function container can be 512MB to 10GB, and the concurrency executions can be up to 1000. > > The Lambda function deployment size can be 50MB for compressed, 250 MB for uncompressed. > [!Warning] > We can set the Lambda function concurrency, and we must be careful of it since concurrency limit are shared between all functions, so as soon as one of them reaches the limit, it will let other functions been throttled. We can also set up Provisioned Concurrency, asking Lambda to allocate before the function is invoked in advance, so that users will not need to wait for too long. ### Lambda SnapStart For Java, Python and .NET Lambda functions, SnapStart can help pre-initialized, improves the performance up to 10 times. ### CloudFront Function CloudFront Function is a lightweight function written in JavaScript. It is used for high-scale, latency-sensitive CDN customizations. ### Lambda@Edge Lambda@Edge is Lambda functions written in NodeJS or Python. It is used to change CloudFront requests and responses. ### Lambda by default Lambda function is launched outside our own VPC by default, so we need to define the VPC ID, the subnet and security groups to create an ENI, which allows it to reach other services in AWS. ## <img src="https://icon.icepanel.io/AWS/svg/Database/DynamoDB.svg" style="min-width:15px; max-width:30px;" /> Amazon DynamoDB Amazon DynamoDB is fuylly managed, high availability with replication across multiple AZs. It's a NoSQL database. For DynamoDB, we don't need to create a database, it is made of Tables. For each of them, we give it the primary key, and can have an infinite number of items(=rows). For each item, the maximum size is 400KB. It has two types of read/wrtie mode: - Provisioned Mode: Specify the number of reads/writes per second, which requires us to plan capacity beforehand. We pay for provisioned Read Capacity Units(RCU) & Write Capacity Units(WCU), which is also possible to enable auto-scaling. - On-Demand Mode: Read/writes automatically scale up/down with the workloads. We just pay for what we use. It is more expensive, but great for handling unpredictable sudden spikes in workload. ### DynamoDB Accelerator(DAX) DAS is a fully managed, highly available, seamless in-memory cache for DynamoDB. It gets microseconds latency for cached data. ## <img src="https://icon.icepanel.io/AWS/svg/App-Integration/API-Gateway.svg" style="min-width:15px; max-width:30px;" /> API Gateways We can enable versioning on API Gateways, handling security and use Swagger, Open API import to quickly define APIs. There are three ways of endpoint types: - Edge-Optimized (default): Routed through the CloudFront Edge locations, but in only one region. - Regional: For clients within thr same region, and could manually combined with CloudFront. - Private: Can only be accessed from VPC using and VPC endpoint. ## <img src="https://icon.icepanel.io/AWS/svg/Security-Identity-Compliance/Cognito.svg" style="min-width:15px; max-width:30px;" /> Amazon Cognito Give users identity to interact with web or mobile application. ### Cognito Users Pools(CUP) It creates a serverless database of users for web & mobile apps, we can define a simple login mechanism, and also set password reset, MFA, email & phone verfication and so on. Cognito can also applies to get temporary IAM policy credentials. ## <img src="https://icon.icepanel.io/AWS/svg/Database/DocumentDB.svg" style="min-width:15px; max-width:30px;" /> Amazon DocumentDB DocumentDB storage automatically grows in increments of 10GB, and it is the same for MonogoDB. It is fully managed and used to store, query and index JSON data. ## <img src="https://icon.icepanel.io/AWS/svg/Database/Neptune.svg" style="min-width:15px; max-width:30px;" /> Amazon Neptune Amazon Neptune is a fully managed graph database. A common example is social network, which has users, friends, posts, comments and so on. It is high availability across 3 AZ, and up to 15 read replicas. ### Streams A real-time ordered sequence of every change to the graph data, which is no duplicates, strict order, and is accessible with REST API. ## <img src="https://icon.icepanel.io/AWS/svg/Database/Keyspaces.svg" style="min-width:15px; max-width:30px;" /> Amazon Keyspaces Amazon Keyspaces is a managed Apache Cassandra-compatible database service, using the Cassandra Query Language(CQL). ## <img src="https://icon.icepanel.io/AWS/svg/Database/Timestream.svg" style="min-width:15px; max-width:30px;" /> Amazon Timestream Amazon Timestream is a fully managed, fast, scalable, serverless time series database. It is 1000s times faster & 1/10th the cost of relational databases. It is good for use cases like IoT apps, operational applications, real-time analytics, etc... ## <img src="https://icon.icepanel.io/AWS/svg/Analytics/Athena.svg" style="min-width:15px; max-width:30px;" /> Amazon Athena Amazon Athena is a serverless query service to analyze data stored in Amazon S3. We can use the m on business intelligence, analytics, reporting, ana;yze, etc... It is recommanded to use Apache Parquet or ORC as they are columnar data, which is cost-saving. We can partition datasets in S3 for easy querying on virtual columns. > [!Note] > We can use larger files to minimize overhead.