AWS - HackMD

<style> .slides h1, .slides h2, .slides h3, .slides h4, .slides h5, .slides h6 { color: #FF9900 !important; } .slides p { line-height: 1em; } .slides { color: #ddd; } .slides code { font-size: 0.65em; line-height: 1.35em; letter-spacing: 0.03em; } .slides .text-white { color: white !important; } .slides img { border: none !important; background: none !important; } </style> --- # AWS <h2 class="text-white">Step Functions</h2> ![](https://d1.awsstatic.com/product-marketing/Step%20Functions/workflow.183110b44a1b1981ae30e563ab2733073a3139da.png) Note: AWS Step Functions lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. Using Step Functions, you can design and run workflows that stitch together services such as AWS Lambda and Amazon ECS into feature-rich applications. Workflows are made up of a series of steps, with the output of one step acting as input into the next. Application development is simpler and more intuitive using Step Functions, because it translates your workflow into a state machine diagram that is easy to understand, easy to explain to others, and easy to change. You can monitor each step of execution as it happens, which means you can identify and fix problems quickly. --- ## UIB WhatsApp State Machine ![](https://eb7-eb7-stable-media-upload.s3.eu-central-1.amazonaws.com/uib-state-machine.png =x500) --- ## State Machine Structure State machines are defined using JSON text that represents a structure containing the following fields. ---- ### State Machine fields: <span>Comment (Optional)</span> <span>StartAt (Required)</span> <span>TimeoutSeconds (Optional)</span> <span>Version (Optional)</span> <span>States (Required)</span> <span> ```json= { "State1" : { }, "State2" : { }, ... } ``` </span> Note: Comment - A human-readable description of the state machine. StartAt - A string that must exactly match (is case sensitive) the name of one of the state objects. TimeoutSeconds - The maximum number of seconds an execution of the state machine can run. If it runs longer than the specified time, the execution fails with a States.Timeout Error Name. Version (Optional) - The version of the Amazon States Language used in the state machine (default is "1.0"). States (Required) - An object containing a comma-delimited set of states. ---- ### Common state fields <span>Type (Required)</span> <span>Next</span> <span>End</span> <span>Comment (Optional)</span> <span>InputPath</span> <span>OutputPath</span> Note: Type (Required) The state's type. Next The name of the next state that is run when the current state finishes. Some state types, such as Choice, allow multiple transition states. End Designates this state as a terminal state (ends the execution) if set to true. There can be any number of terminal states per state machine. Only one of Next or End can be used in a state. Some state types, such as Choice, don't support or use the End field. Comment (Optional) Holds a human-readable description of the state. InputPath (Optional) A path that selects a portion of the state's input to be passed to the state's task for processing. If omitted, it has the value $ which designates the entire input. For more information, see Input and Output Processing). OutputPath (Optional) A path that selects a portion of the state's input to be passed to the state's output. If omitted, it has the value $ which designates the entire input. For more information, see Input and Output Processing. --- ## State types <span>Pass</span> <span>Task</span> <span>Choice</span> <span>Parallel</span> <span>Map</span> <span>Wait</span> <span>Succeed</span> <span>Fail</span> --- ### Pass A Pass state `("Type": "Pass")` passes its input to its output, without performing work. ```json= { "georefOf": "Home" } ``` ```json= { "No-op": { "Type": "Pass", "Result": { "x-datum": 0.381018, "y-datum": 622.2269926397355 }, "ResultPath": "$.coords", "Next": "End" } } ``` ```json= { "georefOf": "Home", "coords": { "x-datum": 0.381018, "y-datum": 622.2269926397355 } } ``` Note: Pass states are useful when constructing and debugging state machines. --- ### Task A Task `("Type": "Task")` state represents a single unit of work performed by a state machine. Note: All work in your state machine is done by tasks. A task performs work by using an activity or an AWS Lambda function, or by passing parameters to the API actions of other services. ---- #### Task state fields In addition to the common state fields, Task states have the following fields: <span>Resource (Required)</span> <span>Parameters (Optional)</span> <span>ResultPath (Optional)</span> <span>Retry (Optional)</span> <span>Catch (Optional)</span> <span>TimeoutSeconds (Optional)</span> <span>HeartbeatSeconds (Optional)</span> Note: Resource (Required) A URI, especially an Amazon Resource Name (ARN) that uniquely identifies the specific task to execute. Parameters (Optional) Used to pass information to the API actions of connected resources. The parameters can use a mix of static JSON and JsonPath. For more information, see Pass Parameters to a Service API. ResultPath (Optional) Specifies where (in the input) to place the results of executing the task that's specified in Resource. The input is then filtered as specified by the OutputPath field (if present) before being used as the state's output. For more information, see path. Retry (Optional) An array of objects, called Retriers, that define a retry policy if the state encounters runtime errors. For more information, see Examples Using Retry and Using Catch. Catch (Optional) An array of objects, called Catchers, that define a fallback state. This state is executed if the state encounters runtime errors and its retry policy is exhausted or isn't defined. For more information, see Fallback States. TimeoutSeconds (Optional) If the task runs longer than the specified seconds, this state fails with a States.Timeout error name. Must be a positive, non-zero integer. If not provided, the default value is 99999999. The count begins after the task has been started, for example, when ActivityStarted or LambdaFunctionStarted are logged in the Execution event history. HeartbeatSeconds (Optional) If more time than the specified seconds elapses between heartbeats from the task, this state fails with a States.Timeout error name. Must be a positive, non-zero integer less than the number of seconds specified in the TimeoutSeconds field. If not provided, the default value is 99999999. For Activities, the count begins when GetActivityTask receives a token and ActivityStarted is logged in the Execution event history. ---- #### Task state example ```json= "ActivityState": { "Type": "Task", "Resource": "arn:aws:states:us-east-1:123456789012:activity:HelloWorld", "TimeoutSeconds": 300, "HeartbeatSeconds": 60, "Next": "NextState" } ``` ---- ### Activities Activities are an AWS Step Functions feature that enables you to have a task in your state machine where the work is performed by a worker that can be hosted on Amazon EC2, Amazon ECS, mobile devices — basically anywhere. Note: In AWS Step Functions, activities are a way to associate code running somewhere (known as an activity worker) with a specific task in a state machine. You can create an activity using the Step Functions console, or by calling `CreateActivity`. This provides an Amazon Resource Name (ARN) for your task state. Use this ARN to poll the task state for work in your activity worker. ---- #### Step 1: Create New Activity ````bash arn:aws:states:us-east-1:123456789012:activity:get-greeting ```` Note: In the Step Functions console, choose Activities in the left navigation panel. Choose Create activity. Enter an Activity Name, for example, get-greeting, and then choose Create Activity. When your activity task is created, note its Amazon Resource Name (ARN), for example: ---- #### Step 2: Create a State Machine ```json= { "Comment": "An example using a Task state.", "StartAt": "getGreeting", "Version": "1.0", "TimeoutSeconds": 300, "States": { "getGreeting": { "Type": "Task", "Resource": "arn:aws:states:us-east-1:123456789012:activity:get-greeting", "End": true } } } ``` ![](https://docs.aws.amazon.com/step-functions/latest/dg/images/tutorial-create-state-machine-custom-preview.png) ---- #### Activity Worker ```python= import boto3 import json from botocore.config import Config as BotoCoreConfig boto_config = BotoCoreConfig(read_timeout=60, region_name=<REGION>) sf_client = boto3.client( "stepfunctions", config=boto_config, aws_access_key_id=<AWS_ACCESS_KEY_ID>, aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>, ) arn = "arn:aws:states:us-east-1:123456789012:activity:get-greeting" while True: response = sf_client.get_activity_task(activityArn=arn, workerName=<NAME>) if "taskToken" in response: activity_token = response["taskToken"] sf_client.send_task_success( taskToken=activity_token, output=json.dumps("Hello from Activity worker!")) ``` Note: An activity worker can be an application running on an Amazon EC2 instance, an AWS Lambda function, a mobile device: any application that can make an HTTP connection, hosted anywhere. When Step Functions reaches an activity task state, the workflow waits for an activity worker to poll for a task. --- ### Choice A Choice `("Type": "Choice")` state adds branching logic to a state machine. ---- #### Choice state fields In addition to the common state fields, Choice states introduce the following additional fields. <span>Choices (Required)</span> <span>Default (Optional, Recommended)</span> <span>Important</span> <span>Choice states don't support the End field. In addition, they use Next only inside their Choices field.</span> Note: Choices (Required) An array of Choice Rules that determines which state the state machine transitions to next. Default (Optional, Recommended) The name of the state to transition to if none of the transitions in Choices is taken. ---- #### Choice state example ```json= { "ChoiceStateX": { "Type": "Choice", "Choices": [ { "Not": { "Variable": "$.type", "StringEquals": "Private" }, "Next": "Public" }, { "Variable": "$.value", "NumericEquals": 0, "Next": "ValueIsZero" }, { "And": [ { "Variable": "$.value", "NumericGreaterThanEquals": 20 }, { "Variable": "$.value", "NumericLessThan": 30 } ], "Next": "ValueInTwenties" } ], "Default": "DefaultState" }, "Public": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Foo", "Next": "NextState" }, "ValueIsZero": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Zero", "Next": "NextState" }, "ValueInTwenties": { "Type" : "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:Bar", "Next": "NextState" }, "DefaultState": { "Type": "Fail", "Cause": "No Matches!" } } ``` #### Input parameters: ```json= { "type": "Private", "value": 22 } ``` --- ### Parallel The Parallel `("Type": "Parallel")` state can be used to create parallel branches of execution in your state machine. ---- #### Parallel state fields In addition to the common state fields, Parallel states include these additional fields. <span>Branches (Required)</span> <span>ResultPath (Optional)</span> <span>Retry (Optional)</span> <span>Catch (Optional)</span> Note: Branches (Required) An array of objects that specify state machines to execute in parallel. Each such state machine object must have fields named States and StartAt, whose meanings are exactly like those in the top level of a state machine. ResultPath (Optional) Specifies where (in the input) to place the output of the branches. The input is then filtered as specified by the OutputPath field (if present) before being used as the state's output. For more information, see Input and Output Processing. Retry (Optional) An array of objects, called Retriers, that define a retry policy in case the state encounters runtime errors. For more information, see Examples Using Retry and Using Catch. Catch (Optional) An array of objects, called Catchers, that define a fallback state that is executed if the state encounters runtime errors and its retry policy is exhausted or isn't defined. For more information, see Fallback States. ---- A Parallel state causes AWS Step Functions to execute each branch, starting with the state named in that branch's StartAt field, as concurrently as possible, and wait until all branches terminate (reach a terminal state) before processing the Parallel state's Next field. ---- #### Parallel State Example ````json= { "Comment": "Parallel Example.", "StartAt": "LookupCustomerInfo", "States": { "LookupCustomerInfo": { "Type": "Parallel", "End": true, "Branches": [ { "StartAt": "LookupAddress", "States": { "LookupAddress": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:AddressFinder", "End": true } } }, { "StartAt": "LookupPhone", "States": { "LookupPhone": { "Type": "Task", "Resource": "arn:aws:lambda:us-east-1:123456789012:function:PhoneFinder", "End": true } } } ] } } } ```` In this example, the LookupAddress and LookupPhone branches are executed in parallel. ---- Here is how the visual workflow looks in the Step Functions console. ![Parallel](https://eb7-uib-whatsapp.s3.eu-central-1.amazonaws.com/aws_sm_parallel_state.png =350x300) <div style="font-size:0.65em;">Each branch must be self-contained. A state in one branch of a Parallel state must not have a Next field that targets a field outside of that branch, nor can any other state outside the branch transition into that branch.</div> --- ### Map The Map `("Type": "Map")` state can be used to run a set of steps for each element of an input array. Note: While the Parallel state executes multiple branches of steps using the same input, a Map state will execute the same steps for multiple entries of an array in the state input. ---- #### Map State fields In addition to the common state fields, Map states include these additional fields. <span>ItemsPath (Optional)</span> <span>Iterator (Required)</span> <span>ResultPath (Optional)</span> <span>Retry (Optional)</span> <span>Catch (Optional)</span> <span>MaxConcurrency (Optional)</span> Note: ItemsPath (Optional) The “ItemsPath” field’s value is a reference path identifying where in the effective input the array field is found. For more information, see ItemsPath. Iterator (Required) The “Iterator” field’s value is an object that defines a state machine which will process each element of the array. States within an "Iterator" field can only transition to each other, and no state outside the “ItemsPath” field can transition to a state within it. If any iteration fails, entire Map state fails, and all iterations are terminated. ResultPath (Optional) Specifies where (in the input) to place the output of the branches. The input is then filtered as specified by the OutputPath field (if present) before being used as the state's output. For more information, see Input and Output Processing. Retry (Optional) An array of objects, called Retriers, that define a retry policy in case the state encounters runtime errors. For more information, see Examples Using Retry and Using Catch. Catch (Optional) An array of objects, called Catchers, that define a fallback state that is executed if the state encounters runtime errors and its retry policy is exhausted or isn't defined. For more information, see Fallback States. MaxConcurrency (Optional) The “MaxConcurrency”field’s value is an integer that provides an upper bound on how many invocations of the Iterator may run in parallel. For instance, a "MaxConcurrency" value of 10 will limit your Map state to 10 concurrent iterations running at one time. Note The “MaxConcurrency” value is an upper bound limit, and not a guarantee that it will run that many concurrent iterations. The default value is "0", which places no limit on parallelism and iterations are invoked as concurrently as possible. A "MaxConcurrency" value of "1" invokes the "Iterator" once for each array element in the order of their appearance in the input, and will not start a new iteration until the previous has completed. --- ## Show Time