---
tags: NCTU-Deepracer
---
# Amazon CloudWatch auto download

## install asw cli

pip install awscli

### 新增金鑰
https://console.aws.amazon.com/iam/home?#/security_credentials
記得存下來，只會顯示一次

### 設定 aws cli

$ aws configure
AWS Access Key ID [None]: <設定的金鑰>
AWS Secret Access Key [None]: <設定的金鑰>
Default region name [None]: us-east-1 // 看deepracer是在哪個區
Default output format [None]: json

## 懶人包

pip install awslogs
pip install awscli-cwlogs
aws configure set plugins.cwlogs cwlogs

安裝完後可以直接跳到最下面**結合 awscli log , awslog**的部份，有我寫好的shell script

## awslog
### install

pip install awslogs

### 使用

awslogs streams /aws/deepracer/leaderboard/SimulationJobs -s 2020-05-30T00:00:00 -e 2020-12-30T23:59:59

https://stackoverflow.com/questions/27804342/how-do-i-filter-and-extract-raw-log-event-data-from-amazon-cloudwatch

## awscli log
### 下載

pip install awscli-cwlogs
aws configure set plugins.cwlogs cwlogs

https://pypi.org/project/awscli-cwlogs/

### 使用

aws logs pull \
--log-group-name /aws/deepracer/leaderboard/SimulationJobs \
--log-stream-name sim-68v2q6fw8g5n/2020-05-30T02-42-42.132Z_4ceb4dab-9b9d-4800-a7bf-39ea3723ce3b/SimulationApplicationLogs

也可以設時間

--start-time 2014-01-23T00:00:00Z --end-time 2014-01-23T01:00:00Z

## 結合 awscli log , awslog
### shell script
將下面內容存成 `download_log.sh`

#!/bin/bash

temp_file=$(mktemp)
temp_file2=$(mktemp)
mkdir -p deepracer_log
cd deepracer_log
awslogs streams /aws/deepracer/leaderboard/SimulationJobs -s 2020-05-10T23:59:59 -e 2020-12-30T23:59:59 > $temp_file
cat $temp_file | sort -t '/' -k 2 -r | head -n $1 > $temp_file2
cat $temp_file2 | xargs -I{} sh -c 'mkdir -p "$1"; aws logs pull --log-group-name /aws/deepracer/leaderboard/SimulationJobs --log-stream-name "$1" > "$1.txt"' -- {}

之後給權限
`chmod +x download_log.sh`
使用方法
`./download_log.sh <number>`
`<number>` ：表示你要最新的n筆資料，這個**參數一定要加**

### 下載最新9筆

mkdir -p deepracer_log
cd deepracer_log
awslogs streams /aws/deepracer/leaderboard/SimulationJobs | xargs -I{} sh -c 'mkdir -p "$1"; aws logs pull --log-group-name /aws/deepracer/leaderboard/SimulationJobs --log-stream-name "$1" > "$1.txt"' -- {}

### 下載 2020-5-10 ~ 2020-12-30

mkdir -p deepracer_log
cd deepracer_log
awslogs streams /aws/deepracer/leaderboard/SimulationJobs -s 2020-05-10T23:59:59 -e 2020-12-30T23:59:59 | xargs -I{} sh -c 'mkdir -p "$1"; aws logs pull --log-group-name /aws/deepracer/leaderboard/SimulationJobs --log-stream-name "$1" > "$1.txt"' -- {}