--- tags: ECS, Containers, AWS, Blog --- # Interactive Session for troubleshooting an ECS Container Instance ![alt text](https://raw.githubusercontent.com/awslabs/aws-icons-for-plantuml/main/source/unofficial/AWS-Architecture-Icons_SVG_20200430/SVG%20Light/_Group%20Icons/AWS-Cloud-alt_light-bg.svg "Tech Series" =20x20) When running capacity providers or launch-type EC2 in your ECS Task Definitions you may have the need to access the underlying ECS Container Instances (EC2) for rudimentary troubleshooting, do this via AWS System Manager. The System Manager settings need to be applied via the EC2 IAM Instance Profile to your Container Instances. Once the policy is applied you can use AWS System Manager for ECS Container Instance troubleshooting in the following ways: 1. Interactive Shell 2. Remote Commands 3. Automation Runbooks supplied by AWS for common tasks ## Attach the System Manager policies to the ecsInstanceRole 1. Open the IAM console at https://console.aws.amazon.com/iam/. 2. In the navigation pane, choose **Roles**. 3. Choose `ecsInstanceRole`. If the role does not exist, follow the procedures in Amazon ECS container instance IAM role to create the role. 4. Choose the **Permissions** tab. 5. Choose **Attach policies**. 6. To narrow the available policies to attach, for Filter, type *SSM*. 7. In the list of policies, select the box next `AmazonSSMManagedInstanceCore`. Use this policy to provide the minimum permissions that are necessary to use Systems Manager. 8. Choose **Attach Policy**. ![](https://hackmd.io/_uploads/SyAIuTGk6.png) ## Interactively troubleshooting ECS Container Instances Through the ECS Console you will be able to find the Service and/or Task with its corresponding EC2 instance. Click through to the EC2 console, select the EC2 instance and choose **Connect**. From the options available use **Session Manager**. ![](https://hackmd.io/_uploads/r1qHFpzk6.png) A new tab will open to give you a secure channel into the EC2 Container Instance to perform troubleshooting. ![](https://hackmd.io/_uploads/rkdFtaz1a.png) :::danger :warning: **Warning** troubleshooting with other production tasks running isn't preferred. Follow the guide to draining a container instance before running any other steps is preferred. https://docs.aws.amazon.com/AmazonECS/latest/developerguide/container-instance-draining.html ::: To review log locations see: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/logs.html ## Manage container instances remotely using Run Command capability You can use the Run Command capability in AWS Systems Manager (Systems Manager) to securely and remotely manage the configuration of your Amazon ECS container instances. Run Command provides a simple way to perform common administrative tasks without logging on locally to the instance. You can manage configuration changes across your clusters by simultaneously executing commands on multiple container instances. Run Command reports the status and results of each command. Here are some examples of the types of tasks you can perform with Run Command: * Install or uninstall packages. * Perform security updates. * Clean up Docker images. * Stop or start services. * View system resources. * View log files. * Perform file operations. ### Using Run Command **Example**: To update container instance software with Run Command A common use case for Run Command is to update the instance software on your entire fleet of container instances at one time. 1. Open the Systems Manager console at https://console.aws.amazon.com/systems-manager. 2. In the left navigation pane, choose Run Command, and then choose Run command. 3. For Command document, choose AWS-RunShellScript. 4. In the Commands section, enter the command or commands to send to your container instances. In this example, the following command updates the instance software: ```bash= yum update -y ``` 7. In the Target instances section, select the boxes next to the container instances where you want to run the update command. 8. Choose Run to send the command to the specified instances. 9. (Optional) Choose the refresh icon to monitor the command status. 10. (Optional) In Targets and output, choose the button next to the instance ID, and then choose View output. ## Remotely troubleshooting ECS Container Instances with System Manager automation runbooks AWS Systems Manager Automation provides predefined runbooks for Amazon Elastic Container Service. The current runbooks include: * AWSSupport-CollectECSInstanceLogs * AWS-InstallAmazonECSAgent * AWSSupport-TroubleshootECSContainerInstance * AWSSupport-TroubleshootECSTaskFailedToStart * AWS-UpdateAmazonECSAgent https://docs.aws.amazon.com/systems-manager-automation-runbooks/latest/userguide/automation-ref-ecs.html