DevOps Engineering on AWS, v3.5.2

Instructor: Scott Jones
Email: [e-mail]
Based on a page developed by Victor Okunev

Bookmark these links:

• Path to this page:
  https://bit.ly/scojoe-on-devops
• Path to our lab classroom:
  https://us-east-1.student.classrooms.aws.training/class/s6pXDaBkyK5obq3EGuE3x1

---

Essential Resources

Documents/Articles

• DevOps on AWS: https://aws.amazon.com/devops/
• Developer Tools on AWS: https://aws.amazon.com/products/developer-tools/?intClick=dev-center-2023_main
• Amazon DevOps Prescriptive Guidance: https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/devops-pattern-list.html
• Amazon Builders Library: https://aws.amazon.com/builders-library/
• Amazon DevOps, Container, and IaC Workshops: https://workshops.aws

---

Introduction to DevOps on AWS

Important facts

• The Three Ways - The Principles Underpinning DevOps: https://itrevolution.com/the-three-ways-principles-underpinning-devops/
• As of 2014, Amazon ran more than 50M production deployments a year, once every 0.63 second
• Number of people in a two-pizza team: 8

Recommended

• What Led Amazon to its Own Microservices Architecture: https://thenewstack.io/led-amazon-microservices-architecture/
• Bezos’ API Mandate, 2002: https://api-university.com/blog/the-api-mandate/
• DevOps at Amazon:
  • AWS Online Tech Talks, Feb 2019 | DevOps at Amazon with Ken Exner, GM of AWS Developer Tools: https://www.youtube.com/watch?v=FlZm3nFMIAM
  • AWS Public Sector Summit 2018 | DevOps Culture at Amazon: https://www.youtube.com/watch?v=mBU3AJ3j1rg
  • AWS re:Invent 2015 | DevOps at Amazon: A Look at Our Tools and Processes: https://www.youtube.com/watch?v=esEFaY0FDKc
  • How Amazon deploys new software to production every 11.6 seconds. Velocity 2011: Jon Jenkins, “Velocity Culture”: https://www.youtube.com/watch?v=dxk8b9rSKOo
• What Team Structure is Right for DevOps to Flourish? https://web.devopstopologies.com/
• What Can Development Teams Learn from Conway’s Law? https://www.jamasoftware.com/blog/what-can-development-teams-learn-from-conways-law/
• The Science Behind Why Small Teams Work More Productively: Jeff Bezos’ 2 Pizza Rule: https://buffer.com/resources/small-teams-why-startups-often-win-against-google-and-facebook-the-science-behind-why-smaller-teams-get-more-done
• 9 Metrics DevOps Teams Should be Tracking: http://www.datical.com/blog/9-metrics-devops-teams-tracking/

The milestones of DevOps culture

• The History of DevOps by Damon Edwards: : http://itrevolution.com/the-history-of-devops/
• 2009 - John Allspaw and Paul Hammond discuss what later becomes known as the DevOps culture as well as the best practices like one-step deploy, feature switches, dark launches, shared metrics, etc. in their groundbreaking talk “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr” at Velocity 2009 O’Reilly Conference: https://youtu.be/LdOe18KhtT4?t=205
• 2009 - Patrick Debois kicks off first Devopsdays conference in Ghent, Belgium: https://www.devopsdays.org
• 2013 - Gene Kim, Kevin Behr and George Spafford publish “The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win”: https://www.amazon.com/Phoenix-Project-DevOps-Helping-Business/dp/0988262592
• 2016 - Gene Kim, Patrick Debois, John Willis and Jez Humble publish “The DevOps Handbook - How to Create World-Class Agility, Reliability, and Security in Technology Organizations”: https://www.amazon.com/DevOps-Handbook-World-Class-Reliability-Organizations/dp/1942788002

See Also

• AWS DevOps Blog: https://aws.amazon.com/blogs/devops/
• Mark Schwartz, the head of Citizenship and Immigration Services with the US Department of Homeland Security discusses how CIS initially fit DevOps inside of a strict waterfall model mandated by DHS regulations:: https://www.youtube.com/watch?v=QwHVlJtqhaI
• Let’s deploy to production (or why we need DevOps): https://youtu.be/5p8wTOr8AbU

---

Infrastructure Automation

Recommended

• Infrastructure as Code: https://d1.awsstatic.com/whitepapers/DevOps/infrastructure-as-code.pdf?trk=wp_c
• Constructs Hub (CDKs for Cloudformation, Kubernetes, and Terraform): https://constructs.dev/
• Why we use Terraform and not Chef, Puppet, Ansible, Pulumi, or CloudFormation: https://blog.gruntwork.io/why-we-use-terraform-and-not-chef-puppet-ansible-saltstack-or-cloudformation-7989dad2865c

---

AWS Cloudformation

What’s new?

• Import entire applications into AWS CloudFormation: https://aws.amazon.com/blogs/devops/import-entire-applications-into-aws-cloudformation/
• Announcing CDK Migrate: A single command to migrate to the AWS CDK: https://aws.amazon.com/blogs/devops/announcing-cdk-migrate-a-single-command-to-migrate-to-the-aws-cdk/
• Introducing a Public Registry for AWS CloudFormation: https://aws.amazon.com/blogs/aws/introducing-a-public-registry-for-aws-cloudformation/
• Introducing AWS CloudFormation Guard 2.0: https://aws.amazon.com/blogs/mt/introducing-aws-cloudformation-guard-2-0/
• Migrating CloudFormation templates to the AWS Cloud Development Kit: https://aws.amazon.com/blogs/developer/migrating-cloudformation-templates-to-the-aws-cloud-development-kit/
• Amazon CodeGuru Profiler now includes support for AWS CloudFormation: https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-codeguru-profiler-now-includes-support-for-aws-cloudformation/

Recommended

• Sample templates: https://aws.amazon.com/cloudformation/aws-cloudformation-templates/
• Service Catalog Reference Templates: https://github.com/aws-samples/aws-service-catalog-reference-architectures
• Template anatomy, Mappings: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/mappings-section-structure.html
• To share information between stacks, export a stack’s output values: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-stack-exports.html
• CloudFormation Helper Scripts Reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-helper-scripts-reference.html
• AWS CloudFormation Conditions: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html
• AWS CloudFormation Deletion Policy: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html
• AWS Resource and Property Types Reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html
• Intrinsic Function Reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference.html
• Pseudo Parameters Reference: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html
• SSM Parameters as the CloudFormation template parameters:
  • Using SSM Parameters with CloudFormation Templates and Terraform Projects: https://start.jcolemorrison.com/using-ssm-parameters-with-cloudformation-templates-and-terraform-projects/
  • Note, that AWS CloudFormation does not support defining template parameters as SecureString Systems Manager parameter types: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-ssm-parameter-types
• Custom Resources:
  • AWS Lambda-backed Custom Resources: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-lambda.html
  • Amazon Simple Notification Service-backed Custom Resources: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources-sns.html
  • Amazon S3 presigned URL: https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html
  • Generate Passwords in AWS CloudFormation Template: https://www.itonaut.com/2018/01/03/generate-passwords-in-aws-cloudformation-template/
  • Source code for my Table Name Generator demo: https://github.com/realokun/aws/tree/master/cloudformation
• Template quality control:
  • TaskCat - An open-source tool that tests AWS CloudFormation templates. It creates stacks in multiple AWS Regions simultaneously and generates a report with a pass/fail grade for each region:
  • AWS CloudFormation Guard - Validate cloud environments with policy-as-code:
  • cfn-lint - An open-source CloudFormation Linter validates yaml/json templates against the CloudFormation spec and additional checks:
    • Stack Policy Example:

    { 
        "Statement" : [
        {    
          "Effect" : "Deny",
          "Action" : "Update:*",
          "Principal": "*",
          "Resource" : "*"
        }
      ]
    }

Best practices:

• AWS CloudFormation Best Practices: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/best-practices.html

Hands-on:

• Learn AWS CloudFormation best practices and how to build a Quick Start, the right way: https://workshop.quickstart.awspartner.com/

See Also

• Idempotency in Infrastructure as Code: https://skundunotes.com/2019/04/19/idempotency-in-infrastructure-as-code/
• TerraForm is a popular cloud-agnostic Infrastructure as Code product from HashiCorp: https://www.terraform.io/
• CloudFormation extension for Visual Studio Code: https://marketplace.visualstudio.com/items?itemName=aws-scripting-guy.cform

---

AWS Cloud Development Kit (AWS CDK)

What’s new?

• AWS SAM CLI announces local testing and debugging support on Terraform projects: https://aws.amazon.com/about-aws/whats-new/2023/09/aws-sam-cli-local-testing-debugging-terraform-projects/
• AWS Cloud Development Kit (CDK) announces CDK Construct tree view in the AWS CloudFormation console: https://aws.amazon.com/about-aws/whats-new/2022/09/aws-cloud-development-kit-cdk-announces-cdk-construct-tree-view-cloudformation-console/

Recommended

• The AWS CDK code is open source and is available through GitHub: https://github.com/awslabs/aws-cdk.
• Constructs Hub: https://constructs.dev/
• Infrastructure Is Code with the AWS Cloud Development Kit: https://www.infoq.com/presentations/aws-cdk/
• AWS CDK Constructs: https://docs.aws.amazon.com/cdk/v2/guide/constructs.html
• AWS CDK API Reference: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-construct-library.html
• A curated list of awesome projects related to the AWS Cloud Development Kit: https://github.com/eladb/awesome-cdk
• AWS CDK Patterns: https://cdkpatterns.com/
• AWS CDK Examples:
  https://github.com/aws-samples/aws-cdk-examples
  https://docs.aws.amazon.com/cdk/latest/guide/examples.html
• The AWS Serverless Application Model - Build serverless applications in simple and clean syntax:
  • Getting Started: https://aws.amazon.com/serverless/sam/
  • SAM Acclerate: https://aws.amazon.com/blogs/compute/accelerating-serverless-development-with-aws-sam-accelerate/
  • SAM Pipeline: https://aws.amazon.com/blogs/compute/introducing-aws-sam-pipelines-automatically-generate-deployment-pipelines-for-serverless-applications/

CDK for Terraform

• Announcing CDK for Terraform on AWS: https://aws.amazon.com/blogs/opensource/announcing-cdk-for-terraform-on-aws/
• Terraform CDK Tutorials: https://developer.hashicorp.com/terraform/tutorials/cdktf
• Hashicorp CDKTF GitHub: https://github.com/hashicorp/terraform-cdk
• Examples: https://developer.hashicorp.com/terraform/cdktf/examples

Hands-on:

• AWS CDK Intro Workshop: https://cdkworkshop.com/
• AWS CDK Advanced Workshop: https://catalog.us-east-1.prod.workshops.aws/workshops/d93fec4c-fb0f-4813-ac90-758cb5527f2f/en-US
• AWS CDK Python Workshop: https://cdkworkshop.com/30-python.html

---

Developer’s toolset

Important facts

• API throttling is the process of limiting the number of API requests a user can make in a certain period. The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time.

What’s new?

• Introducing Amazon CodeWhisperer, the ML-powered coding companion: https://aws.amazon.com/blogs/machine-learning/introducing-amazon-codewhisperer-the-ml-powered-coding-companion/
• Introducing AWS Copilot: https://aws.amazon.com/blogs/containers/introducing-aws-copilot/
• AWS Copilot CLI: https://aws.github.io/copilot-cli/
• AWS Cloud Control API, a Uniform API to Access AWS & Third-Party Services: https://aws.amazon.com/blogs/aws/announcing-aws-cloud-control-api/
• AWS Resource types that support Cloud Control API: https://docs.aws.amazon.com/cloudcontrolapi/latest/userguide/supported-resources.html
• Announcing General Availability of Construct Hub and AWS Cloud Development Kit Version 2: https://aws.amazon.com/blogs/aws/announcing-general-availability-of-construct-hub-and-aws-cloud-development-kit-version-2/

Recommended

• Tools to Build on AWS: https://aws.amazon.com/tools/
• Types of AWS APIs, Amazon S3 example:
  • Amazon S3 REST API: https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html
  • AWS Java SDK, low-level API: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3Client.html#putObject-com.amazonaws.services.s3.model.PutObjectRequest-
  • AWS Java SDK, high-level API: https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html
  • AWS Cloud Control API: https://docs.aws.amazon.com/cloudcontrolapi/index.html
• Signature Version 4 Signing Process: https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
• Language Support in the AWS Cloud9 IDE: https://docs.aws.amazon.com/cloud9/latest/user-guide/language-support.html
  • Tutorials and samples for AWS Cloud9: https://docs.aws.amazon.com/cloud9/latest/user-guide/tutorials.html
• Automate code reviews with Amazon CodeGuru Reviewer: https://aws.amazon.com/blogs/devops/automate-code-reviews-with-amazon-codeguru-reviewer/
• Locking API Versions: https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/locking-api-versions.html
• Enabling Metrics for the AWS SDK: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/generating-sdk-metrics.html
• How to filter an AWS CLI command output with the –query option: https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#cli-usage-output-filter
• JMESPath is a query language for JSON, see it is action: http://jmespath.org/
• 15 Essential Amazon AWS EC2 CLI Command Examples: https://www.thegeekstuff.com/2016/04/aws-ec2-cli-examples/
• Use AWS services locally for testing:
  • LocalStack - Develop and test your cloud apps offline: https://localstack.cloud/
  • Setting Up DynamoDB Local: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html
  • AWS SAM Local – Build and Test Serverless Applications Locally: https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/
• Tools for creating AWS diagrams:
  • AWS Architecture Icons | The official AWS icon set for building architecture diagrams: https://aws.amazon.com/architecture/icons/
  • Draw.io, a free online diagram tool: https://app.diagrams.net/?splash=0&libs=aws4

See Also

• AWS Developer Tools Blog: https://aws.amazon.com/blogs/developer/

Hands-on:

• AWS CDK Intro Workshop: https://cdkworkshop.com/
• Amazon Rekognition CLI demo: https://github.com/realokun/aws/tree/master/rekognition

---

CI/CD with AWS Developer Tools

Recommended

• The Twelve-Factor App methodology can be applied to apps written in any programming language, and which use any combination of backing services (database, queue, memory cache, etc): https://12factor.net/

• Code

  • A Trunk-Based Development practice explained in details: https://trunkbaseddevelopment.com/
  • Git Tutorial – Atlassian (a great tutorial on Git – well written, concise, and comprehensive): https://www.atlassian.com/git/tutorials
  • Git Cheatsheet : http://ndpsoftware.com/git-cheatsheet.html
  • Integrate AWS Cloud9 with AWS CodeCommit: https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-ide-c9.html
  • Create an AWS CodeCommit Repository: https://docs.aws.amazon.com/codecommit/latest/userguide/how-to-create-repository.html
  • Setup steps for HTTPS connections to AWS CodeCommit repositories with the AWS CLI credential helper: https://docs.aws.amazon.com/codecommit/latest/userguide/setting-up-https-unixes.html

• Build

  • Working with test reporting in AWS CodeBuild: https://docs.aws.amazon.com/codebuild/latest/userguide/test-reporting.html
  • Practicing Continuous Integration and Continuous Delivery on AWS: https://d0.awsstatic.com/whitepapers/DevOps/practicing-continuous-integration-continuous-delivery-on-AWS.pdf
  • Continuous Integration Best Practices for Software Development: https://www.youtube.com/watch?v=GEPJ7Lo346A
  • How to Enable Caching for AWS CodeBuild: https://aws.amazon.com/blogs/devops/how-to-enable-caching-for-aws-codebuild/
  • AWS Elastic Beanstalk Sample for AWS CodeBuild: https://docs.aws.amazon.com/codebuild/latest/userguide/sample-elastic-beanstalk.html
  • Announcing Local Build Support for AWS CodeBuild: https://aws.amazon.com/blogs/devops/announcing-local-build-support-for-aws-codebuild/
  • CodeBuild BuildSpec file reference: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

• Deploy

  • CodeDeploy Tutorials: https://docs.aws.amazon.com/codedeploy/latest/userguide/tutorials.html
  • Overview of CodeDeploy Deployment Types: https://docs.aws.amazon.com/codedeploy/latest/userguide/welcome.html#welcome-deployment-overview
  • Working with Deployment Configurations in CodeDeploy: https://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-configurations.html
  • Deploying Applications on Amazon EC2 with AWS CloudFormation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/deploying.applications.html
  • Often when running a web service, you’ll have your instances behind a load balancer. But when deploying new code to these instances, you don’t want the load balancer to continue sending customer traffic to an instance while the deployment is in progress. The lifecycle event scripts give you the ability to integrate your AWS CodeDeploy deployments with instances that are behind an Elastic Load Balancer or in an Auto Scaling group. Simply set the name (or names) of the Elastic Load Balancer your instances are a part of, set the scripts in the appropriate lifecycle events, and the scripts will take care of deregistering the instance, waiting for connection draining, and re-registering after the deployment finishes: https://github.com/awslabs/aws-codedeploy-samples/tree/master/load-balancing/elb
  • CodeDeploy AppSpec File Reference: https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file.html
  • How can I use launch configurations to automatically install the CodeDeploy agent on an Amazon EC2 instance running Amazon Linux or Ubuntu? https://aws.amazon.com/premiumsupport/knowledge-center/codedeploy-agent-launch-configuration/
  • How to check if CodeDeploy Agent service is running? Run the following command: sudo service codedeploy-agent status

• Test

  • Natively integrated 3-rd party test action providers for AWS CodePipeline:
  • Jenkins: https://jenkins.io
  • BlazeMeter: https://www.blazemeter.com
  • GhostInspector UI Testing: https://ghostinspector.com/
  • MicroFocus StormRunner Load: https://www.microfocus.com/
  • Runscope API Monitoring: https://www.runscope.com/

• Pipeline

  • How pipeline executions work: https://docs.aws.amazon.com/codepipeline/latest/userguide/concepts-how-it-works.html
  • CI/CD pipelines on AWS - Builders Day Israel: https://www.slideshare.net/AmazonWebServices/cicd-pipelines-on-aws-builders-day-israel?qid=ff5193a5-ba96-4049-b478-5f13f81aada1&v=&b=&from_search=21
  • Accelerating DevOps Pipelines with AWS: https://www.youtube.com/watch?v=7hxe_o6493s
  • Tutorial | Create a Simple Pipeline (CodeCommit Repository): https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html#codecommit-add-code
  • Detect and React to Changes in Pipeline State with Amazon CloudWatch Events: https://docs.aws.amazon.com/codepipeline/latest/userguide/detect-state-changes-cloudwatch-events.html

• Jenkins integration

  • Jenkins build with AWS CodeBuild: https://aws.amazon.com/blogs/devops/simplify-your-jenkins-builds-with-aws-codebuild/
  • Jenkins Amazon EC2 Plugin: https://wiki.jenkins.io/display/JENKINS/Amazon+EC2+Plugin
  • On-Demand Jenkins Slaves With Amazon EC2: http://artsy.github.io/blog/2012/07/10/on-demand-jenkins-slaves-with-amazon-ec2/
  • AWS CodePipeline plugin for Jenkins: https://plugins.jenkins.io/aws-codepipeline/

---

Microservices

Recommended

• Martin Fowler, Microservices, a definition of this new architectural term: https://www.martinfowler.com/articles/microservices.html
• Microservices on AWS: https://aws.amazon.com/microservices/
• Sam Newman, Principles of Microservices: https://samnewman.io/talks/principles-of-microservices/
• Analyzing Polyglot Microservices | An Exploration for Enterprise: https://medium.com/capital-one-tech/analyzing-polyglot-microservices-f6f159a1a3e7
• Implementing Microservices on AWS: https://d1.awsstatic.com/whitepapers/microservices-on-aws.pdf
• Distributed systems components
  • Service Discovery: https://docs.aws.amazon.com/whitepapers/latest/microservices-on-aws/service-discovery.html
  • AWS Cloud Map is a cloud resource discovery service: https://aws.amazon.com/cloud-map/
  • AWS App Mesh is a service mesh that provides application-level networking: https://aws.amazon.com/app-mesh/
• AWS re:Invent 2018: Using DevOps, Microservices, & Serverless to Accelerate Innovation: https://www.youtube.com/watch?v=eXl6Bumksnk&t=562

Best practices:

• Distributed systems
  • Availability and Beyond: Understanding and Improving the Resilience of Distributed Systems on AWS: https://docs.aws.amazon.com/whitepapers/latest/availability-and-beyond-improving-resilience/availability-and-beyond-improving-resilience.html
  • CAP theorem: https://docs.aws.amazon.com/whitepapers/latest/availability-and-beyond-improving-resilience/cap-theorem.html
• Patterns
  • A pattern language for microservices: https://microservices.io/patterns/
  • The Strangler pattern in practice: https://www.michielrook.nl/2016/11/strangler-pattern-practice/
  • StranglerFigApplication: https://martinfowler.com/bliki/StranglerFigApplication.html
  • Implement the serverless saga pattern by using AWS Step Functions: https://docs.aws.amazon.com/prescriptive-guidance/latest/patterns/implement-the-serverless-saga-pattern-by-using-aws-step-functions.html
  • Using the circuit breaker pattern with AWS Step Functions and Amazon DynamoDB: https://aws.amazon.com/blogs/compute/using-the-circuit-breaker-pattern-with-aws-step-functions-and-amazon-dynamodb/
• The Amazon Builders’ Library | Challenges with distributed systems: https://aws.amazon.com/builders-library/challenges-with-distributed-systems/
• What is AWS Migration Hub Refactor Spaces? https://docs.aws.amazon.com/migrationhub-refactor-spaces/latest/userguide/what-is-mhub-refactor-spaces.html
• CI/CD best practices for building modern applications: https://www.slideshare.net/AmazonWebServices/cicd-best-practices-for-building-modern-applications-mad304-chicago-aws-summit
• Semantic versioning: http://www.semver.org

See Also

• What is Cell-Based Architecture? https://nordicapis.com/what-is-cell-based-architecture/
• Cell-Based Architecture | A Decentralized Reference Architecture for Cloud-native Applications: https://github.com/wso2/reference-architecture/blob/master/reference-architecture-cell-based.md

Hands-on:

• Workshop – Mythical Mysfits: https://github.com/aws-samples/amazon-ecs-mythicalmysfits-workshop/tree/master/workshop-1#lets-begin
• Tutorial | How to deploy a monolithic Node.js application to a Docker container. The tutorial guides you on how to decouple the application into microservices without incurring any downtime: https://aws.amazon.com/getting-started/projects/break-monolith-app-microservices-ecs-docker-ec2/.

---

Containers

Important facts

• 81% of all containers in the cloud are running on AWS*.
• 84% of all Kubernetes workloads in the cloud are running on AWS*.
• 40% of customers adopting an AWS container service choose Fargate*.
  *Source: Andy Jassy, AWS CEO, re:Invent 2019 keynote.

Recommended

• Learn by analogy - VMs are houses, containers are apartments: https://blog.docker.com/2016/03/containers-are-not-vms/
• What Is A Container Scheduler?https://technologyconversations.com/2017/12/14/what-is-a-container-scheduler/
• CI/CD for Serverless and Containerized Applications: https://www.youtube.com/watch?v=01ewawuL-IY
• CI/CD for Containers: A Way Forward for Your DevOps Pipeline: https://www.slideshare.net/AmazonWebServices/cicd-for-containers-a-way-forward-for-your-devops-pipeline?qid=e659617f-abb4-4729-8dcc-5be52963e771&v=&b=&from_sea
• Amazon ECS:
  • Amazon Elastic Container Service Developer Guide: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
  • Amazon ECS task networking: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-networking.html
  • A curated list of awesome ECS guides, development tools, and resources: https://github.com/nathanpeck/awesome-ecs
  • Amazon ECS Task Definition Parameters: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html
  • Docker Basics for Amazon ECS: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html
  • How to create a custom scheduler for Amazon ECS: https://aws.amazon.com/blogs/compute/how-to-create-a-custom-scheduler-for-amazon-ecs/
  • Continuous Deployment to Amazon ECS using AWS CodePipeline, AWS CodeBuild, Amazon ECR, and AWS CloudFormation: https://aws.amazon.com/blogs/compute/continuous-deployment-to-amazon-ecs-using-aws-codepipeline-aws-codebuild-amazon-ecr-and-aws-cloudformation/
• Amazon EKS:
  • Kubernetes concepts: https://kubernetes.io/docs/concepts/

Hands-on:

• Amazon ECS Workshop for AWS Fargate: https://ecsworkshop.com/
• Amazon EKS Workshop: https://eksworkshop.com/
• Mythical Mysfits | Monolith to Microservices with Docker and AWS Fargate: https://github.com/aws-samples/amazon-ecs-mythicalmysfits-workshop/tree/master/workshop-1

---

AWS Lambda

Important facts

• AWS Lambda integrates with 47 AWS services. No other cloud provider integrates with more than 20.
• After a Lambda function is executed, AWS Lambda maintains the execution context for some time in anticipation of another Lambda function invocation.
• Each Lambda function receives 500MB of non-persistent disk space in its own /tmp directory.
• Lambda Deployment package size limits: 50 MB (zipped, for direct upload), 250 MB (unzipped, including layers), 3 MB (console editor).
• Lambda restrictions: Inbound network connections are blocked by AWS Lambda, and for outbound connections only TCP/IP and UDP/IP sockets are supported, and ptrace (debugging) system calls are blocked. TCP port 25 traffic is also blocked as an anti-spam measure.
• Lambda concurrency is subject to a Regional quota that is shared by all functions in a Region.
  • Default quota of concurrent executions is 1,000 and it can be increased to the tens of thousands.
• Types of Lambda concurrency:
  • Reserved concurrency – an optional value set per function that both reserves a subset of the Regional quota for the function and also establishes the maximum concurrent instances allowed for the function. Comes at no charge.
  • Provisioned concurrency - initializes a requested number of execution environments so that they are prepared to respond to your function’s invocations. Incurs charges.
  • Burst concurrency - for an initial burst of traffic, your functions’ cumulative concurrency in a Region can reach an initial level of between 500 and 3000 (varies per Region). After that, your functions’ concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached.

What’s new?

• AWS Fargate enables faster container startup using Seekable OCI: https://aws.amazon.com/about-aws/whats-new/2023/07/aws-fargate-container-startup-seekable-oci/
• Announcing AWS Lambda Function URLs: Built-in HTTPS Endpoints for Single-Function Microservices: https://aws.amazon.com/blogs/aws/announcing-aws-lambda-function-urls-built-in-https-endpoints-for-single-function-microservices/
• AWS Lambda Now Supports Up to 10 GB Ephemeral Storage: https://aws.amazon.com/blogs/aws/aws-lambda-now-supports-up-to-10-gb-ephemeral-storage/
• AWS Lambda Functions Powered by AWS Graviton2 Processor – Run Your Functions on Arm and Get Up to 34% Better Price Performance: https://aws.amazon.com/blogs/aws/aws-lambda-functions-powered-by-aws-graviton2-processor-run-your-functions-on-arm-and-get-up-to-34-better-price-performance/
• AWS Lambda Extensions Are Now Generally Available: https://aws.amazon.com/blogs/aws/getting-started-with-using-your-favorite-operational-tools-on-aws-lambda-extensions-are-now-generally-available/
• Container Image Support for AWS Lambda: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/

Recommended

• AWS Lambda Pricing Calculator: https://s3.amazonaws.com/lambda-tools/pricing-calculator.html
• Lambda Power Tuning: https://github.com/alexcasalboni/aws-lambda-power-tuning
• Optimizing AWS Lambda cost and performance using AWS Compute Optimizer: https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-cost-and-performance-using-aws-compute-optimizer/
• Lambda function scaling: https://docs.aws.amazon.com/lambda/latest/dg/invocation-scaling.html
• Lambda’s Concurrency:
  • Managing concurrency for a Lambda function: https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html
  • For details on concurrency and how Lambda scales your function concurrency in response to traffic: https://docs.aws.amazon.com/lambda/latest/dg/invocation-scaling.html
  • Provisioned Concurrency for Lambda Functions: https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/
  • AWS Lambda Reserved Concurrency: https://blog.symphonia.io/posts/2017-12-13_aws-lambda-reserved-concurrency
• AWS IoT Button, Cloud Programmable Dash Button: https://aws.amazon.com/iotbutton/
• Comparing AWS SAM with the Serverless framework: https://sanderknape.com/2018/02/comparing-aws-sam-with-serverless-framework/
• Use Application Auto Scaling to automatically configure provisioned concurrency for Lambda function: https://docs.aws.amazon.com/autoscaling/application/userguide/what-is-application-auto-scaling.html
• The SQS batch processing utility: https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/batch/
• Optimizing batch processing with custom checkpoints in AWS Lambda: https://aws.amazon.com/blogs/compute/optimizing-batch-processing-with-custom-checkpoints-in-aws-lambda/
• Announcing Code Signing, a trust and integrity control for AWS Lambda: https://aws.amazon.com/about-aws/whats-new/2020/11/announcing-code-signing-a-trust-and-integrity-control-for-aws-lambda/
• Amazon CodeGuru Profiler simplifies profiling for AWS Lambda functions: https://aws.amazon.com/about-aws/whats-new/2020/11/amazon-codeguru-profiler-simplifies-profiling-for-aws-lambda-functions/
• AWS Lambda now supports AWS PrivateLink: https://aws.amazon.com/about-aws/whats-new/2020/10/aws-lambda-now-supports-aws-privatelink/

See Also

• Custom AWS Lambda runtimes: https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html
• Running COBOL on AWS Lambda: https://paika.tech/blog/2020/09/27/running-cobol-on-aws-lambda.html
• AWS Compute Blog: https://aws.amazon.com/blogs/compute/

Hands-on

• Building Event-Driven Architectures on AWS: https://event-driven-architecture.workshop.aws
• Wild Rydes Serverless Workshops | This repository contains a collection of workshops and other hands on content that will guide you through building various serverless applications using AWS Lambda, Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, Amazon Kinesis, and other AWS services: https://github.com/aws-samples/aws-serverless-workshops
• Build a Serverless Web Application with AWS Lambda, Amazon API Gateway, Amazon S3, Amazon DynamoDB, and Amazon Cognito: https://aws.amazon.com/getting-started/hands-on/build-serverless-web-app-lambda-apigateway-s3-dynamodb-cognito/
• AWS Alien Attack Workshop | The purpose of AWS Alien Attack is to create a fun environment where you can test and think about different aspects of serverless architectures for (near) real-time ingesting and processing of data at AWS: https://alienattack.workshop.aws/

Lambda humour

• A Serverless Musical: https://www.youtube.com/watch?v=zMua0cuhFnc

---

AWS Step Functions

Important facts

• With Step Functions AWS SDK Service Integrations, developers can integrate their state machines directly to AWS service that has AWS SDK support:
  • Number of supported AWS services: 200+
  • AWS API Actions: 9,000+
• Standard Workflows:
  • Use for long-running, durable, and auditable workflows where repeating workflow steps is expensive (e.g., restarting a long-running media transcode) or harmful (e.g., charging a credit card twice)
  • Guarantee exactly once execution of each workflow step
  • Maximum duration - 1 year
  • Track and store detailed step-by-step information about each workflow that you may inspect during and after the workflow execution
  • Support all service integrations, activities, and design patterns
• Express Workflows:
  • Use for workloads with high event rates and short durations
  • Event rates of more than 100,000 per second
  • Guarantee “at least once” execution of each workflow step
  • Maximum duration - 5 min
  • Failed workflows must be re-run from the beginning
  • Support all service integrations
  • Do not support activities, Job-run (.sync), and Callback patterns

Recommended

• Standard vs. Express Workflows: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-standard-vs-express.html
• Amazon States Language: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html
• Tim Bray introduces newly launched AWS Step Functions service at re:Invent 2016: https://www.youtube.com/watch?v=75MRve4nv8s
• The concept of Activities: https://docs.aws.amazon.com/step-functions/latest/dg/concepts-activities.html
• Hitchhiker’s Guide to AWS Step Functions: http://blog.epsagon.com/hitchhikers-guide-to-aws-step-functions
• Things Go Better With Step Functions (a real-world use case): https://aws.amazon.com/blogs/aws/things-go-better-with-step-functions/
• Sample Projects for Step Functions: https://docs.aws.amazon.com/step-functions/latest/dg/create-sample-projects.html
• Source code for my Greedy Approver demo: https://github.com/realokun/aws/tree/master/stepfunctions

Hands-on:

• Intro to service coordination using AWS StepFunctions: https://step-functions-workshop.go-aws.com/

See Also

• With Amazon Simple Workflow Service (Amazon SWF), instead of writing state machines in declarative JSON, you write a decider program to separate activity steps from decision steps. This provides you complete control over your orchestration logic, but increases the complexity of developing applications. You may write decider programs in the programming language of your choice, or you may use the Flow framework to use programming constructs that structure asynchronous interactions for you: https://aws.amazon.com/swf/

---

AWS Serverless Application Model (SAM)

Important facts

• You cannot specify a custom deployment configuration in an AWS SAM template.

Recommended

• AWS SAM GitHub repo: https://github.com/awslabs/serverless-application-model
• AWS SAM CLI Command Reference: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-command-reference.html
• Safe Lambda deployments: https://github.com/awslabs/serverless-application-model/blob/master/docs/safe_lambda_deployments.rst
• AWS SAM Local – Build and Test Serverless Applications Locally: https://aws.amazon.com/blogs/aws/new-aws-sam-local-beta-build-and-test-serverless-applications-locally/
• Implementing safe AWS Lambda deployments with AWS CodeDeploy: https://aws.amazon.com/blogs/compute/implementing-safe-aws-lambda-deployments-with-aws-codedeploy/
• Deploying Serverless Applications Gradually: https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/automating-updates-to-serverless-apps.html

Hands-on:

• Deploy an Updated Lambda Function with CodeDeploy and the AWS Serverless Application Model: https://docs.aws.amazon.com/codedeploy/latest/userguide/tutorial-lambda-sam.html

See Also

• OpenAPI Specification: https://swagger.io/resources/open-api/
• LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. It spins up a testing environment on your local machine that provides the same functionality and APIs as the real AWS cloud environment: https://localstack.cloud/
• Setting Up DynamoDB Local: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html

---

Deployment Strategies

Recommended

• Six Strategies for Application Deployment: https://thenewstack.io/deployment-strategies/
• Martin Fowler on Blue/Green Deployment: https://martinfowler.com/bliki/BlueGreenDeployment.html
• Why Leading Companies Dark Launch: https://dzone.com/articles/why-leading-companies-dark-launch
• Dynamic application configuration:
• Dynamic configuration with AWS AppConfig: https://aws.amazon.com/podcasts/501-using-aws-appconfig/
• Safe Deployment of Application Configuration Settings With AWS AppConfig: https://aws.amazon.com/blogs/aws/safe-deployment-of-application-configuration-settings-with-aws-appconfig/
• Automating Feature Release using AWS AppConfig Integration with AWS Codepipeline: https://aws.amazon.com/blogs/mt/automating-feature-release-using-aws-appconfig-integration-with-aws-codepipeline/
• Deploying application configuration to serverless: Introducing the AWS AppConfig Lambda extension: https://aws.amazon.com/blogs/mt/proactive-monitoring-of-application-configuration-deployment-using-aws-appconfig-and-amazon-cloudwatch/
• Application Load Balancer simplifies deployments with support for weighted target groups: https://aws.amazon.com/about-aws/whats-new/2019/11/application-load-balancer-simplifies-deployments-support-for-weighted-target-groups/
• Application configuration deployment to container workloads using AWS AppConfig: https://aws.amazon.com/blogs/mt/application-configuration-deployment-to-container-workloads-using-aws-appconfig/
• Proactive monitoring of application configuration deployment using AWS AppConfig and Amazon CloudWatch: https://aws.amazon.com/blogs/mt/proactive-monitoring-of-application-configuration-deployment-using-aws-appconfig-and-amazon-cloudwatch/
• Working with AWS AppConfig: https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-working.html
• AWS Elastic Beanstalk deployment policies:
• How rolling deployments work: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.rolling-version-deploy.html#environments-cfg-rollingdeployments-method
• How traffic-splitting deployments work: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.rolling-version-deploy.html#environments-cfg-trafficsplitting-method
• Immutable environment updates: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentmgmt-updates-immutable.html

See Also

• English idiom “Canary in the coal mine”: https://share.america.gov/english-idiom-canary-coal-mine/

Hands-on:

• Tutorial: Create a pipeline that uses AWS AppConfig as a deployment provider: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-AppConfig.html
• Weighted Target Groups on ALB Demo: https://www.exampleloadbalancer.net/albwtg_demo.html

---

Automated Testing on AWS

What’s new?

• New digital course: Advanced Testing Practices using AWS DevOps Tools: https://aws.amazon.com/about-aws/whats-new/2020/10/new-digital-course-advanced-testing-practices-using-aws-devops-tools/
• Amazon CloudWatch Synthetics is now generally available: https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-cloudwatch-synthetics-generally-available/

Recommended

• Amazon EC2 Testing Policy: https://aws.amazon.com/ec2/testing/
• Failing Fast:
  • The Fail-Fast Principle in Software Development: https://dzone.com/articles/fail-fast-principle-in-software-development
  • Fail Fast by Martin Fowler: https://martinfowler.com/ieeeSoftware/failFast.pdf
• The Third Way of DevOps in action:
  • Principles of Chaos Engineering: http://principlesofchaos.org/?lang=ENcontent
  • AWS re:Invent 2017 | Performing Chaos at Netflix Scale: Chaos Engineering is described as “the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” With unit tests and integration tests we test a known to increase confidence in software. With chaos we experiment with the unknown: https://www.youtube.com/watch?v=LaKGx0dAUlo
  • Chaos Monkey randomly terminates virtual machine instances and containers that run inside of your production environment. Exposing engineers to failures more frequently incentivizes them to build resilient services: https://github.com/netflix/chaosmonkey
• How one bad algorithm cost traders $440m, a look at the worst software testing day ever: https://www.theregister.co.uk/2012/08/03/bad_algorithm_lost_440_million_dollars/
• WireMock - a simulator (mock server) for HTTP-based APIs. It supports testing of edge cases and failure modes that the real API won’t reliably produce.
• Using AWS CodePipeline, AWS CodeBuild, and AWS Lambda for Serverless Automated UI Testing: https://aws.amazon.com/blogs/devops/using-aws-codepipeline-aws-codebuild-and-aws-lambda-for-serverless-automated-ui-testing/
• How to Bake Open Source Security into Your AWS CodeBuild Pipeline: https://aws.amazon.com/blogs/apn/how-to-bake-open-source-security-into-your-aws-codebuild-pipeline/
• Integrating Ghost Inspector with AWS CodePipeline: https://ghostinspector.com/docs/integration/aws-codepipeline
• Synthetic monitoring with Amazon CloudWatch Synthetics Canaries: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Canaries.html
• AWS Fault Injection Simulator: https://aws.amazon.com/fis/
• 10 Fascinating A/B Testing Examples: https://www.designforfounders.com/ab-testing-examples/
• Static code scanners:
  • AWS CloudFormation Guard - an open-source, policy-as-code evaluation tool that provides developers with a simple-to-use, yet powerful language (DSL) to define policies used to validate JSON- or YAML- formatted templates.
  • awslabs/git-secrets - scans commits, commit messages, and –no-ff merges to prevent adding secrets into your git repositories.
  • Amazon CodeGuru - Amazon CodeGuru is a machine learning service for automated code reviews and application performance recommendations. When you associate your AWS CodeCommit repository with Amazon CodeGuru Reviewer, you will receive recommendations to help improve Java code in all pull requests.
  • Bandit - an open-source tool designed to find common security issues in Python code.
  • Clair - an open source project for the static analysis of vulnerabilities in appc and docker containers.
  • Contrast CE (Contrast Security) - a free and full-strength application security platform that provides “always on” IAST, RASP, and SCA for Java applications and APIs (other languages coming soon). Contrast isn’t a scanner or firewall, instead it works from inside the running application.
  • Checkmarx – application security testing and static code analysis software.
  • OWASP Dependency-Check - an open source Software Composition Analysis (SCA) tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies.
  • NPM Audit - a command tool that performs a moment-in-time security review of a Node.js project’s dependency tree.
  • Qualys - an automated full spectrum auditing, compliance and protection of IT systems and web applications.
  • Retire.js - a web app scanner for use of vulnerable JavaScript libraries. The goal of retire.js is to help you detect use of version with known vulnerabilities.
  • Trufflehog - an open source tool that searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accidentally committed.
  • Visual Code Grepper - an open source automated code security review tool that handles C/C++, Java, C#, VB and PL/SQL.
  • Veracode Platform - a solution that can provide visibility into application status across all testing types, including Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), Source Code Analysis (SCA), and manual penetration testing, in one centralized view.
• Infrastructure scanners
  • Amazon Inspector - an automated security assessment service that helps improve the security and compliance of applications deployed on AWS. Amazon Inspector automatically assesses applications for exposure, vulnerabilities, and deviations from best practices. After performing an assessment, Amazon Inspector produces a detailed list of security findings prioritized by level of severity. These findings can be reviewed directly or as part of detailed assessment reports which are available via the Amazon Inspector console or API.
  • Nikto - an open source web server scanner which performs comprehensive tests against web servers for multiple items, including over 6700 potentially dangerous files/programs, checks for outdated versions of over 1250 servers, and version specific problems on over 270 servers. It also checks for server configuration items such as the presence of multiple index files, HTTP server options, and will attempt to identify installed web servers and software. Scan items and plugins are frequently updated and can be automatically updated.
  • Nessus (Tenable) - vulnerability analysis, patch confirmation, configuration assessment, and sensitive data identification for EC2 environments and instances.
  • Nmap - short for Network Mapper, is an open-source tool for vulnerability scanning and network discovery. Network administrators use Nmap to identify what devices are running on their systems, discovering hosts that are available and the services they offer, finding open ports and detecting security risks.
  • Snyk - finds vulnerabilities in your repos and remediate risks with automated updates and patches. Block vulnerable libraries in CI/CD and monitor PaaS/Serverless apps for dependency flaws.
• Web UI test automation tools
  • Selenium - automates browsers. That’s it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that.
  • Ghost Inspector - provides a test recorder tool for both Chrome and Firefox that allows you to record operations and assertions right in your web browser. If you’re familiar with Selenium IDE, this is similar but much more streamlined.
  • Cypress Fast, easy and reliable testing for anything that runs in a browser.
• Web application vulnerability scanners
  • Arachni Scanner - a free/public-source Web application security scanner aimed towards helping users evaluate the security of web applications.
  • AppSpider (Rapid7) - a dynamic application security testing solution that allows scanning web and mobile applications for vulnerabilities.
  • Burp Suite (PortSwigger) - a graphical tool written in Java for testing Web application security.
  • OpenVAS - a full-featured open source vulnerability scanner. Its capabilities include unauthenticated testing, authenticated testing, various high level and low level Internet and industrial protocols, performance tuning for large-scale scans and a powerful internal programming language to implement any type of vulnerability test.
  • OWASP Zed Attack Proxy (ZAP) - an open-source web application security scanner.
• Load testing tools
  • Apache JMeter is designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.
  • Bees with Machine Guns - an open source utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications).

The fun way to learn

• A real-world analogy to the syntetic canary: https://www.youtube.com/watch?v=6lJRM9pPOf4

Hands-on:

• Advanced Testing Practices using AWS DevOps Tools: https://aws.amazon.com/about-aws/whats-new/2020/10/new-digital-course-advanced-testing-practices-using-aws-devops-tools/

---

DevSecOps

What’s new?

• AWS Shield Advanced Update – Automatic Application Layer DDoS Mitigation: https://aws.amazon.com/blogs/aws/aws-shield-advanced-update-automatic-application-layer-ddos-mitigation/
• Amazon CodeGuru Reviewer Introduces Secrets Detector to Identify Hardcoded Secrets and Secure Them with AWS Secrets Manager: https://aws.amazon.com/blogs/aws/codeguru-reviewer-secrets-detector-identify-hardcoded-secrets/
• Introducing AWS CloudFormation Guard 2.0: https://aws.amazon.com/blogs/mt/introducing-aws-cloudformation-guard-2-0/

Recommended

• AWS Cloud Adoption Framework (AWS CAF): https://aws.amazon.com/professional-services/CAF/
• AWS Shared Responsibility Model: https://aws.amazon.com/compliance/shared-responsibility-model/
• Baseline Environment on AWS (BLEA) - a set of reference CDK templates to establish secure baseline on standalone-account or multi-account AWS environment: https://github.com/aws-samples/baseline-environment-on-aws
• This is why you need an MFA: https://www.theregister.co.uk/AMP/2019/03/20/steffan_needham_aws_rampage_prison_sentence_voova/
• WS-compatible MFA devices: https://aws.amazon.com/iam/details/mfa/
• Configuring MFA-Protected API Access: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_configure-api-require.html
• How do I use an MFA token to authenticate access to my AWS resources through the AWS CLI? https://aws.amazon.com/premiumsupport/knowledge-center/authenticate-mfa-cli/
• AWS IAM Policy Simulator: https://policysim.aws.amazon.com
• Requesting Temporary Security Credentials from AWS STS: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html
• Service control policies (SCPs): https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scp.html
• Referencing AWS Secrets Manager Secrets from Parameter Store Parameters: https://docs.aws.amazon.com/systems-manager/latest/userguide/integration-ps-secretsmanager.html
• AWS Key Management Service Concepts: https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html
• Classify sensitive data in your environment using Amazon Macie: https://aws.amazon.com/blogs/security/classify-sensitive-data-in-your-environment-using-amazon-macie/
• Goodbye SSH, use AWS Session Manager instead: https://cloudonaut.io/goodbye-ssh-use-aws-session-manager-instead/
• Nine AWS Security Hub best practices: https://aws.amazon.com/blogs/security/nine-aws-security-hub-best-practices/
• The Rule Development Kit is designed to support a “Compliance-as-Code” workflow that is intuitive and productive. It abstracts away much of the undifferentiated heavy lifting associated with deploying AWS Config rules backed by custom lambda functions, and provides a streamlined develop-deploy-monitor iterative process: https://github.com/awslabs/aws-config-rdk
• WhiteSource integrations with CodeBuild: https://aws.amazon.com/blogs/apn/how-to-bake-open-source-security-into-your-aws-codebuild-pipeline/
• Scanning images with Trivy in an AWS CodePipeline: https://aws.amazon.com/blogs/containers/scanning-images-with-trivy-in-an-aws-codepipeline/
• Scanning Docker Images for Vulnerabilities using Clair, Amazon ECS, ECR, and AWS CodePipeline: https://aws.amazon.com/blogs/compute/scanning-docker-images-for-vulnerabilities-using-clair-amazon-ecs-ecr-aws-codepipeline/

Testing Policies

• Penetration Testing: https://aws.amazon.com/security/penetration-testing/
• DDoS Simulation Testing Policy: https://aws.amazon.com/security/ddos-simulation-testing/
• Network Stress Test: https://aws.amazon.com/ec2/testing/

Hands-on:

• Building end-to-end AWS DevSecOps CI/CD pipeline with open source SCA, SAST and DAST tools: https://aws-blogs-prod.amazon.com/devops/building-end-to-end-aws-devsecops-ci-cd-pipeline-with-open-source-sca-sast-and-dast-tools/

---

Configuration Management

What’s new?

• AWS Systems Manager announces support for port forwarding to remote hosts using Session Manager: https://aws.amazon.com/about-aws/whats-new/2022/05/aws-systems-manager-support-port-forwarding-remote-hosts-using-session-manager/
• AWS Systems Manager Fleet Manager: https://aws.amazon.com/blogs/aws/new-aws-systems-manager-fleet-manager/
• Announcing the Golden AMI Pipeline: https://aws.amazon.com/blogs/awsmarketplace/announcing-the-golden-ami-pipeline/
• Introducing EC2 Image Builder: https://aws.amazon.com/about-aws/whats-new/2019/12/introducing-ec2-image-builder/

Recommended

• Automate OS Image Build Pipelines with EC2 Image Builder: https://aws.amazon.com/blogs/aws/automate-os-image-build-pipelines-with-ec2-image-builder/
• Immutable infrastructure as a DevOps practice:
  https://blog.codeship.com/immutable-infrastructure/
  https://www.fpcomplete.com/blog/2016/11/devops-best-practices-immutability
  https://www.digitalocean.com/community/tutorials/what-is-immutable-infrastructure
• EC2 Image Builder | Build and maintain secure images: https://aws.amazon.com/image-builder/

See Also

• HashiCorp Packer | Automates the creation of any type of machine image: https://www.packer.io/

---

Observability

Important facts

• The minimum metric resolution supported by CloudWatch is 1-second data points, which is a high-resolution metric. If you do not specify that a metric is high resolution, then by default CloudWatch will aggregate and store the metrics at 1-minute resolution.
• Custom CloudWatch metrics follow the same retention schedule and resolution options as AWS-managed metrics.
• The CloudWatch Logs Agent will send log data every five seconds by default and is configurable by the user.

What’s new?

• New – Real-User Monitoring for Amazon CloudWatch: https://aws.amazon.com/blogs/aws/cloudwatch-rum/
• New – Amazon CloudWatch Evidently – Experiments and Feature Management: https://aws.amazon.com/blogs/aws/cloudwatch-evidently/
• Amazon CloudWatch now allows you to combine multiple alarms: https://aws.amazon.com/about-aws/whats-new/2020/03/amazon-cloudwatch-now-allows-you-to-combine-multiple-alarms/
• Announcing Amazon CloudWatch ServiceLens: https://aws.amazon.com/about-aws/whats-new/2019/11/announcing-amazon-cloudwatch-servicelens/
• For the complete list of announcements: https://aws.amazon.com/new/?awsf.whats-new-products=general-products%23amazon-cloudwatch|general-products%23aws-x-ray

Recommended

• AWS Podcast | Deep Dive into Observability: https://aws.amazon.com/podcasts/357-deep-dive-into-observability
• Sending CloudWatch Custom Metrics From Lambda: https://stackify.com/custom-metrics-aws-lambda/
• CloudWatch Logs Insights Query Syntax: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html
• Tracing serverless application with AWS X-Ray: https://medium.com/nordcloud-engineering/tracing-serverless-application-with-aws-x-ray-2b5e1a9e9447
• Running the X-Ray Daemon Locally: https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-local.html

Hands-on:

• AWS Management Tools Workshop: https://workshop.aws-management.tools/

---

After the course

Provide your feedback for the course (2 min):

1. Navigate to:
   • Customers: https://www.aws.training/Account/Transcript/Archived
   • Amazonians: https://kiku.aws.training/Account/Transcript/Archived
2. Find this course.
3. On the right, click Evaluate.
4. Provide your feedback.

Continue learning

• Study with AWS:
  • AWS Training and Certification learning library: https://www.aws.training/LearningLibrary
  • AWS Training and Certification offers free live and on-demand training on Twitch: https://aws.amazon.com/training/twitch/
• Study with AWS partners:
  • https://acloud.guru/
  • https://www.whizlabs.com/
  • https://linuxacademy.com/
  • https://www.udemy.com/
• Follow AWS blogs for the best practices in implementing solutions on AWS:
  • AWS Compute Blog: https://aws.amazon.com/blogs/compute/
  • AWS Architecture Blog: https://aws.amazon.com/blogs/architecture/
  • AWS Management & Governance Blog: https://aws.amazon.com/blogs/mt/
  • AWS Infrastructure & Automation Blog: https://aws.amazon.com/blogs/infrastructure-and-automation/
  • AWS DevOps Blog: https://aws.amazon.com/blogs/devops/
  • AWS Front-End Web & Mobile Blog: https://aws.amazon.com/blogs/mobile/
  • and many others: https://aws.amazon.com/blogs/
• Listen to AWS podcasts
  • The Official weekly AWS Podcast: https://aws.amazon.com/podcasts/aws-podcast/
  • AWS TechChat: https://aws.amazon.com/podcasts/aws-techchat/
  • Stay up-to-date with AWS news:
  • What’s New with AWS - learn about the latest product, service, and feature announcements from AWS. https://aws.amazon.com/new/
  • Read AWS news blog: https://aws.amazon.com/blogs/aws
  • Read AWS Training and Certification Blog: https://aws.amazon.com/blogs/training-and-certification/
• Beginning on Thursday, January 28, 2021 the AWS Dev Hour: Building Modern Applications series will offer developers a hands-on approach to deepen their AWS knowledge in a fun, live, and interactive environment: https://pages.awscloud.com/traincert-twitch-dev-hour.html
• AWS Stash, searchable historical database of the videos released by AWS at public events: https://awsstash.com/
• AWS on Twitch: https://www.twitch.tv/aws

---

Get AWS Certification

• 15 Top-Paying IT Certifications, updated annually: https://www.globalknowledge.com/us-en/resources/resource-library/articles/top-paying-certifications/
• Prepare for Your AWS Certification Exam: https://aws.amazon.com/certification/certification-prep/
• Take free digital self-paced AWS Certification Exam Readiness training: https://www.aws.training/LearningLibrary?filters=digital%3A1&filters=language%3A1&search=Exam Readiness&tab=view_all

Learn from the Best Practices, Reference Architectures and Case Studies

• AWS Well-Architected | Learn, measure, and build using architectural best practices: https://aws.amazon.com/architecture/well-architected/
• AWS Prescriptive Guidance | Resources from AWS and APN Partners to help accelerate cloud adoption and modernization: https://aws.amazon.com/prescriptive-guidance/
• The Amazon Builders’ Library | How Amazon builds and operates software: https://aws.amazon.com/builders-library/
• This Is My Architecture | Innovative cloud architectures from AWS partners and customers: https://aws.amazon.com/this-is-my-architecture/
• AWS Quick Starts | Automated, gold-standard deployments in the AWS Cloud: https://aws.amazon.com/quickstart/
• AWS Customer Success Stories: https://aws.amazon.com/solutions/case-studies/
• How Netflix works: https://media.netflix.com/en/company-blog/how-netflix-works-with-isps-around-the-globe-to-deliver-a-great-viewing-experience

Get hands-on experience

• Get started with simple step-by-step tutorials to launch your first AWS workload: https://aws.amazon.com/getting-started/hands-on/
• Workshops - hands-on events designed to teach or introduce practical skills, techniques, or concepts which you can use to solve business problems:
  • Collection of AWS workshops maintained by AWS: https://workshops.aws/
  • Collection of AWS security workshops maintained by AWS: https://awssecworkshops.com/workshops/
  • Collection of AWS workshops found around in the internet: https://awesome-aws-workshops.com/
  • Mythical Mysfits: https://www.mythicalmysfits.com/
• Practice with the Qwiklabs, requires to purchase credits: https://amazon.qwiklabs.com/

Get expert guidance and assistance

• AWS IQ - hire the AWS Certified third-party experts for the on-demand project work: https://iq.aws.amazon.com/
• AWS Support - choose an AWS Support plan: https://aws.amazon.com/premiumsupport/plans/