Try   HackMD

DevOps Engineering on AWS, v3.5.2

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

Instructor: Scott Jones
Email: scojoe@amazon.com
Based on a page developed by Victor Okunev

:pushpin: Bookmark these links:

:books: Essential Resources

Documents/Articles

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Introduction to DevOps on AWS

:pushpin: Important facts

:sparkles: Recommended

:scroll: The milestones of DevOps culture

:bulb: See Also

:gear: Infrastructure Automation

:sparkles: Recommended

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
AWS Cloudformation

:fire: What’s new?

:sparkles: Recommended

    { 
        "Statement" : [
        {    
          "Effect" : "Deny",
          "Action" : "Update:*",
          "Principal": "*",
          "Resource" : "*"
        }
      ]
    }

:thumbsup: Best practices:

:keyboard: Hands-on:

:bulb: See Also

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
AWS Cloud Development Kit (AWS CDK)

:fire: What’s new?

:sparkles: Recommended

:sparkles: CDK for Terraform

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Developer’s toolset

:pushpin: Important facts

  • API throttling is the process of limiting the number of API requests a user can make in a certain period. The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time.

:fire: What’s new?

:sparkles: Recommended

:bulb: See Also

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
CI/CD with AWS Developer Tools

:sparkles: Recommended

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Microservices

:sparkles: Recommended

:thumbsup: Best practices:

:bulb: See Also

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Containers

:pushpin: Important facts

  • 81% of all containers in the cloud are running on AWS*.
  • 84% of all Kubernetes workloads in the cloud are running on AWS*.
  • 40% of customers adopting an AWS container service choose Fargate*.
    *Source: Andy Jassy, AWS CEO, re:Invent 2019 keynote.

:sparkles: Recommended

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
AWS Lambda

:pushpin: Important facts

  • AWS Lambda integrates with 47 AWS services. No other cloud provider integrates with more than 20.
  • After a Lambda function is executed, AWS Lambda maintains the execution context for some time in anticipation of another Lambda function invocation.
  • Each Lambda function receives 500MB of non-persistent disk space in its own /tmp directory.
  • Lambda Deployment package size limits: 50 MB (zipped, for direct upload), 250 MB (unzipped, including layers), 3 MB (console editor).
  • Lambda restrictions: Inbound network connections are blocked by AWS Lambda, and for outbound connections only TCP/IP and UDP/IP sockets are supported, and ptrace (debugging) system calls are blocked. TCP port 25 traffic is also blocked as an anti-spam measure.
  • Lambda concurrency is subject to a Regional quota that is shared by all functions in a Region.
    • Default quota of concurrent executions is 1,000 and it can be increased to the tens of thousands.
  • Types of Lambda concurrency:
    • Reserved concurrency – an optional value set per function that both reserves a subset of the Regional quota for the function and also establishes the maximum concurrent instances allowed for the function. Comes at no charge.
    • Provisioned concurrency - initializes a requested number of execution environments so that they are prepared to respond to your function’s invocations. Incurs charges.
    • Burst concurrency - for an initial burst of traffic, your functions’ cumulative concurrency in a Region can reach an initial level of between 500 and 3000 (varies per Region). After that, your functions’ concurrency can scale by an additional 500 instances each minute. This continues until there are enough instances to serve all requests, or until a concurrency limit is reached.

:fire: What’s new?

:sparkles: Recommended

:bulb: See Also

:keyboard: Hands-on

:smiley: Lambda humour

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
AWS Step Functions

:pushpin: Important facts

  • With Step Functions AWS SDK Service Integrations, developers can integrate their state machines directly to AWS service that has AWS SDK support:
    • Number of supported AWS services: 200+
    • AWS API Actions: 9,000+
  • Standard Workflows:
    • Use for long-running, durable, and auditable workflows where repeating workflow steps is expensive (e.g., restarting a long-running media transcode) or harmful (e.g., charging a credit card twice)
    • Guarantee exactly once execution of each workflow step
    • Maximum duration - 1 year
    • Track and store detailed step-by-step information about each workflow that you may inspect during and after the workflow execution
    • Support all service integrations, activities, and design patterns
  • Express Workflows:
    • Use for workloads with high event rates and short durations
    • Event rates of more than 100,000 per second
    • Guarantee “at least once” execution of each workflow step
    • Maximum duration - 5 min
    • Failed workflows must be re-run from the beginning
    • Support all service integrations
    • Do not support activities, Job-run (.sync), and Callback patterns

:sparkles: Recommended

:keyboard: Hands-on:

:bulb: See Also

  • With Amazon Simple Workflow Service (Amazon SWF), instead of writing state machines in declarative JSON, you write a decider program to separate activity steps from decision steps. This provides you complete control over your orchestration logic, but increases the complexity of developing applications. You may write decider programs in the programming language of your choice, or you may use the Flow framework to use programming constructs that structure asynchronous interactions for you: https://aws.amazon.com/swf/

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
AWS Serverless Application Model (SAM)

:pushpin: Important facts

  • You cannot specify a custom deployment configuration in an AWS SAM template.

:sparkles: Recommended

:keyboard: Hands-on:

:bulb: See Also

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Deployment Strategies

:sparkles: Recommended

:bulb: See Also

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Automated Testing on AWS

:fire: What’s new?

:sparkles: Recommended

  • Amazon EC2 Testing Policy: https://aws.amazon.com/ec2/testing/
  • Failing Fast:
  • The Third Way of DevOps in action:
    • Principles of Chaos Engineering: http://principlesofchaos.org/?lang=ENcontent
    • AWS re:Invent 2017 | Performing Chaos at Netflix Scale: Chaos Engineering is described as “the discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” With unit tests and integration tests we test a known to increase confidence in software. With chaos we experiment with the unknown: https://www.youtube.com/watch?v=LaKGx0dAUlo
    • Chaos Monkey randomly terminates virtual machine instances and containers that run inside of your production environment. Exposing engineers to failures more frequently incentivizes them to build resilient services: https://github.com/netflix/chaosmonkey
  • How one bad algorithm cost traders $440m, a look at the worst software testing day ever: https://www.theregister.co.uk/2012/08/03/bad_algorithm_lost_440_million_dollars/
  • WireMock - a simulator (mock server) for HTTP-based APIs. It supports testing of edge cases and failure modes that the real API won’t reliably produce.
  • Using AWS CodePipeline, AWS CodeBuild, and AWS Lambda for Serverless Automated UI Testing: https://aws.amazon.com/blogs/devops/using-aws-codepipeline-aws-codebuild-and-aws-lambda-for-serverless-automated-ui-testing/
  • How to Bake Open Source Security into Your AWS CodeBuild Pipeline: https://aws.amazon.com/blogs/apn/how-to-bake-open-source-security-into-your-aws-codebuild-pipeline/
  • Integrating Ghost Inspector with AWS CodePipeline: https://ghostinspector.com/docs/integration/aws-codepipeline
  • Synthetic monitoring with Amazon CloudWatch Synthetics Canaries: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Synthetics_Canaries.html
  • AWS Fault Injection Simulator: https://aws.amazon.com/fis/
  • 10 Fascinating A/B Testing Examples: https://www.designforfounders.com/ab-testing-examples/
  • Static code scanners:
    • AWS CloudFormation Guard - an open-source, policy-as-code evaluation tool that provides developers with a simple-to-use, yet powerful language (DSL) to define policies used to validate JSON- or YAML- formatted templates.
    • awslabs/git-secrets - scans commits, commit messages, and –no-ff merges to prevent adding secrets into your git repositories.
    • Amazon CodeGuru - Amazon CodeGuru is a machine learning service for automated code reviews and application performance recommendations. When you associate your AWS CodeCommit repository with Amazon CodeGuru Reviewer, you will receive recommendations to help improve Java code in all pull requests.
    • Bandit - an open-source tool designed to find common security issues in Python code.
    • Clair - an open source project for the static analysis of vulnerabilities in appc and docker containers.
    • Contrast CE (Contrast Security) - a free and full-strength application security platform that provides “always on” IAST, RASP, and SCA for Java applications and APIs (other languages coming soon). Contrast isn’t a scanner or firewall, instead it works from inside the running application.
    • Checkmarx – application security testing and static code analysis software.
    • OWASP Dependency-Check - an open source Software Composition Analysis (SCA) tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies.
    • NPM Audit - a command tool that performs a moment-in-time security review of a Node.js project’s dependency tree.
    • Qualys - an automated full spectrum auditing, compliance and protection of IT systems and web applications.
    • Retire.js - a web app scanner for use of vulnerable JavaScript libraries. The goal of retire.js is to help you detect use of version with known vulnerabilities.
    • Trufflehog - an open source tool that searches through git repositories for secrets, digging deep into commit history and branches. This is effective at finding secrets accidentally committed.
    • Visual Code Grepper - an open source automated code security review tool that handles C/C++, Java, C#, VB and PL/SQL.
    • Veracode Platform - a solution that can provide visibility into application status across all testing types, including Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), Source Code Analysis (SCA), and manual penetration testing, in one centralized view.
  • Infrastructure scanners
    • Amazon Inspector - an automated security assessment service that helps improve the security and compliance of applications deployed on AWS. Amazon Inspector automatically assesses applications for exposure, vulnerabilities, and deviations from best practices. After performing an assessment, Amazon Inspector produces a detailed list of security findings prioritized by level of severity. These findings can be reviewed directly or as part of detailed assessment reports which are available via the Amazon Inspector console or API.
    • Nikto - an open source web server scanner which performs comprehensive tests against web servers for multiple items, including over 6700 potentially dangerous files/programs, checks for outdated versions of over 1250 servers, and version specific problems on over 270 servers. It also checks for server configuration items such as the presence of multiple index files, HTTP server options, and will attempt to identify installed web servers and software. Scan items and plugins are frequently updated and can be automatically updated.
    • Nessus (Tenable) - vulnerability analysis, patch confirmation, configuration assessment, and sensitive data identification for EC2 environments and instances.
    • Nmap - short for Network Mapper, is an open-source tool for vulnerability scanning and network discovery. Network administrators use Nmap to identify what devices are running on their systems, discovering hosts that are available and the services they offer, finding open ports and detecting security risks.
    • Snyk - finds vulnerabilities in your repos and remediate risks with automated updates and patches. Block vulnerable libraries in CI/CD and monitor PaaS/Serverless apps for dependency flaws.
  • Web UI test automation tools
    • Selenium - automates browsers. That’s it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that.
    • Ghost Inspector - provides a test recorder tool for both Chrome and Firefox that allows you to record operations and assertions right in your web browser. If you’re familiar with Selenium IDE, this is similar but much more streamlined.
    • Cypress Fast, easy and reliable testing for anything that runs in a browser.
  • Web application vulnerability scanners
    • Arachni Scanner - a free/public-source Web application security scanner aimed towards helping users evaluate the security of web applications.
    • AppSpider (Rapid7) - a dynamic application security testing solution that allows scanning web and mobile applications for vulnerabilities.
    • Burp Suite (PortSwigger) - a graphical tool written in Java for testing Web application security.
    • OpenVAS - a full-featured open source vulnerability scanner. Its capabilities include unauthenticated testing, authenticated testing, various high level and low level Internet and industrial protocols, performance tuning for large-scale scans and a powerful internal programming language to implement any type of vulnerability test.
    • OWASP Zed Attack Proxy (ZAP) - an open-source web application security scanner.
  • Load testing tools
    • Apache JMeter is designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.
    • Bees with Machine Guns - an open source utility for arming (creating) many bees (micro EC2 instances) to attack (load test) targets (web applications).

:smiley: The fun way to learn

:keyboard: Hands-on:
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
DevSecOps

:fire: What’s new?

:sparkles: Recommended

:cop: Testing Policies

:keyboard: Hands-on:

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Configuration Management

:fire: What’s new?

:sparkles: Recommended

:bulb: See Also

:eye: Observability

:pushpin: Important facts

  • The minimum metric resolution supported by CloudWatch is 1-second data points, which is a high-resolution metric. If you do not specify that a metric is high resolution, then by default CloudWatch will aggregate and store the metrics at 1-minute resolution.
  • Custom CloudWatch metrics follow the same retention schedule and resolution options as AWS-managed metrics.
  • The CloudWatch Logs Agent will send log data every five seconds by default and is configurable by the user.

:fire: What’s new?

:sparkles: Recommended

:keyboard: Hands-on:

After the course
Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

:trophy: Provide your feedback for the course (2 min):

  1. Navigate to:
  2. Find this course.
  3. On the right, click Evaluate.
  4. Provide your feedback.

:books: Continue learning

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →
Get AWS Certification

:thumbsup: Learn from the Best Practices, Reference Architectures and Case Studies

:keyboard: Get hands-on experience

:wave: Get expert guidance and assistance

Last changed by 
 
scojoe
0
2509

Read more

AWS Technical Essentials

Resources to support the Architecting on AWS course

Mar 31, 2024
Architecting on AWS v7.7 - Additional Resources

Resources to support the Architecting on AWS course

Mar 28, 2024
AAI Delivery Skills Standards - Resources page

Resources to support the Architecting on AWS course

Mar 7, 2024
Running Containers on Amazon EKS

Resources to support the Running Containers on Amazon EKS course from AWS

Feb 22, 2024
Read more from scojoe
Published on HackMD