AWS DeepRacer
=============================
###### tags: `AWS`, `ドキュメント`

AWS DeepRacerに関するメモです。

注意点
-----------------------------

1. クローンしたモデルを学習させるとき、ActionSpaceの設定を変更しない.
2. RewarGraphは途中までの結果しか表示しないため、常にEvaluationをする.
3. １つのモデルにつき複数のコースは走れない。
4. 負報酬を与えない。また、ゼロ報酬は1e-3など小さな値を報酬とする。
5. 作りたいモデルがネットに落ちている場合があるので、一度探してみる。
6. 状態値の "progress" は定義域が 0~100 である.
7. Evaluation中の動画は後から見れないので録画をする.
8. バーチャル大会にモデルを提出できない場合がある.

報酬関数
-----------------------------

### 定義

def reward_function(params):
    reward = ...
    return float(reward)

### 状態パラメータ

{
    "all_wheels_on_track": Boolean,    # flag to indicate if the vehicle is on the track
    "x": float,                        # vehicle's x-coordinate in meters
    "y": float,                        # vehicle's y-coordinate in meters
    "distance_from_center": float,     # distance in meters from the track center 
    "is_left_of_center": Boolean,      # Flag to indicate if the vehicle is on the left side to the track center or not. 
    "heading": float,                  # vehicle's yaw in degrees
    "progress": float,                 # percentage of track completed
    "steps": int,                      # number steps completed
    "speed": float,                    # vehicle's speed in meters per second (m/s)
    "steering_angle": float,           # vehicle's steering angle in degrees
    "track_width": float,              # width of the track
    "waypoints": [[float, float], … ], # list of [x,y] as milestones along the track center
    "closest_waypoints": [int, int]    # indices of the two nearest waypoints.
}

#### all_wheels_on_track
トラックの内側にいるかどうか

define reward_function(params):

    all_wheels_on_track = params['all_wheels_on_track']
    speed = params['speed']

    SPEED_THRESHOLD = 1.0 

    if not all_wheels_on_track:
        reward = 1e-3
    elif speed < SPEED_THRESHOLD:
        reward = 0.5
    else:
        reward = 1.0

    return reward

#### closest_waypoints
最も近い2つのWayPointのindex

0 : 現在のwaypoint
1 : 次のwaypoint

def reward_function(params):
    import math

    waypoints = params['waypoints']
    closest_waypoints = params['closest_waypoints']
    heading = params['heading']

    reward = 1.0

    next_point = waypoints[closest_waypoints[1]]
    prev_point = waypoints[closest_waypoints[0]]

    track_direction = math.atan2(next_point[1] - prev_point[1], next_point[0] - prev_point[0]) 

    track_direction = math.degrees(track_direction)

    direction_diff = abs(track_direction - heading)
    if direction_diff > 180:
        direction_diff = 360 - direction_diff

    DIRECTION_THRESHOLD = 10.0
    if direction_diff > DIRECTION_THRESHOLD:
        reward *= 0.5

    return reward

#### distance_from_center
中心線から車両中心までの距離

範囲 : [0, tranck_width/2]

def reward_function(params):

    track_width = params['track_width']
    distance_from_center = params['distance_from_center']

    marker_1 = 0.1 * track_width
    marker_2 = 0.5 * track_width

    if distance_from_center <= marker_1:
        reward = 1.0
    elif distance_from_center <= marker_2:
        reward = 0.5
    else:
        reward = 1e-3

    return reward

#### heading
x軸から車両の向きベクトルの角度

範囲 : -180 ~ +180

#### is_left_of_center
トラックの中心より左にいる場合True, 右側にいたらFalse

#### progress
トラック完走の割合

範囲 : 0 ~ 100

#### speed
速度

範囲 : 0.0 ~ 5.0

#### steering_angle
タイヤの曲がり具合

範囲 : -30 ~ 30

def reward_function(params):

    steering = abs(params['steering_angle'])

    reward = 1.0

    STEERING_THRESHOLD = 20.0
    if steering > ABS_STEERING_THRESHOLD:
        reward *= 0.8

    return reward

#### steps
ステップ数

範囲 : 0 ~

def reward_function(params):

    steps = params['steps']
    progress = params['progress']

    TOTAL_NUM_STEPS = 300

    reward = 1.0

    if (steps % 100) == 0 and progress > (steps / TOTAL_NUM_STEPS) * 100 :
        reward += 10.0

    return reward

#### track_width
トラック幅

#### x, y
環境のx, y座標値

#### waypoints
waypointの座標リスト.二次元リストで定義