# LLM karel attempts

## Target: Using LLMs to produce Karel programs


The goal is to climb the stairs to reach where the marker is located. The reward is defined as a sparse reward: 1 if the agent reaches the goal in the environment rollout, -1 if the agent moves to a position off of the stairs during the rollout, and 0 otherwise. This is on a 12 × 12 grid, and the marker location and agent’s initial location are randomized between rollouts.


The goal is to place a marker at each corner of the Karel environment grid. The reward is defined as sum of corners having a marker divided by four. If the Karel state has a marker placed in wrong location, the reward will be 0. This is on a 12 × 12 grid.


The goal is to place a marker wherever there is already a marker in the last row of the environment, and end up in the rightmost square on the bottom row at the end of the rollout. The reward is defined as the number of consecutive places until the agent either forgets to place a marker where the marker is already present or places a marker at an empty location in last row, with a bonus for ending up on the last square. This is on a 12 × 12 grid, and the marker locations in the last row are randomized between rollouts.


The goal is to find a marker in randomly generated maze. The reward is defined as a sparse reward: 1 if the agent finds the marker in the environment rollout, 0 otherwise. This is on a 8 × 8 grid, and the marker location, agent’s initial location, and the maze configuration itself are randomized between rollouts.


We design a complex 14×22 Karel environment grid that resembles an apartment. The goal is to pick up the garbage (markers) placed at 10 different locations and reach the location where there is a dustbin (2 markers in 1 location). To make the task simpler, we place the markers adjacent to any wall in the environment. The reward is defined as total locations cleaned (markers picked) out of the total number of markers placed in initial Karel environment state (10). The agent’s initial location is fixed but the marker locations are randomized between rollouts.


The goal is to pickup a marker from each location in the Karel environment. The final reward is defined as the number of markers picked up divided the total markers present in the initial Karel environment state. This is on a 8 × 8 grid. We run both MAZE and HARVESTER on smaller Karel environment grids to save time and compute resources because these are long horizon tasks.

## Domain specific language

#actions
move()
turn_right()
turn_left()
pickMarker()
putMarker()

#perceptions
frontisClear()
leftisClear()
rightisClear()
markersPresent()
noMarkersPresent()

## Attempts
(The following attempts are all generated by ChatGPT)
## ChatGPT knows what is Karel
user:Do you know what is karel?
assistant:Yes, Karel is a programming language commonly used in education, particularly for teaching programming concepts and basics of computer science to beginners....
## How to encode environment information
### Grid based map:

eg.
1 1 1 1 1
1 0 0 0 1
1 0 N 0 1
1 0 0 0 1
1 1 1 1 1

May be the most intuitive way to represent an environment, but after doing some simple tests, it seems that ChatGPT can not understand the map.

#### Start of conversation:

[User]:Here I'm going to give you a map indicating karel environment, you goal is to tell me the result after excuting certain programs.

11111
10001
10N01
10001
11111
Where 1 is wall, 0 is empty, N means that karel is facing north(which is also up in this map).

programs:
move()

[Assistant]:After executing the program "move()", Karel will move one step forward and end up in the empty space above its starting position. The new map will be:

11111
10001
10001
01001
11111
Where Karel's position is now indicated by a zero (0).
#### End of conversation

#### One interesting found

##### Start of conversation:
[User]:Here I'm going to give you a Karel map, your goal is to tell me the position of the Karel agent.
Map:
11111
10001
10001
10N01
11111
0 represent empty, 1 represents wall, N represent the position of Karel agent and its direction, which is facing North.

[Assistant]:The position of the Karel agent is at row 3, column 2 facing North.

[User]:What will the map look like when executing move()?

[Assistant]:If Karel executes the move() command while standing at the initial position (i.e. row 3, column 2 facing North) in the given map, the updated map will look like this:

11111
10001
10N01
10001
11111

Karel will move one step forward in the direction it's facing (i.e. North), leaving a clean spot where it was previously standing (represented by 0). The final position of Karel is at row 2, column 2 facing North.

[User]:What will the map look like when executing turnright(), move()?

[Assistant]:If Karel first executes turnright() and then move() while standing at the initial position (i.e. row 3, column 2 facing North) in the given map, the updated map will look like this:

**11111
10001
10N01
10001
11111**

**Karel will first turn to face East and then move one step forward in that direction, leaving a clean spot where it was previously standing (represented by 0). The final position of Karel is at row 3, column 3 facing East.**
##### End of conversation

In this example, we can see that although ChatGPT still make mistakes when drawing the map, it actually produce correct answer using text.

#### Some observations and assumptions about grid based world:
- It cannot correctly draw the map using grid based world
- It can produce correct answer using text, though still draw the wrong map.
- Maybe it can actually understand the map but just can't draw it out?

### Text-based(state-based) map:
eg.
state={
"height":5,
"width":5,
"robot_position":(2,2),
"direction":north,
"walls":((1,2), (2,3)),
"beepers"((1,4), (4,5))
}

Represent map using its state, seems that the model can better understand the meaning of map, but still lack the ability to correctly execute the program.

#### Start of example:
[User]:Now I'm going to give you a map and a program, your goal is to tell me the map after executing the program.
Map:
state={
"height":5,
"width":5,
"robot_x":2,
"robot_y":2,
"direction":north
}

program:
turn_left()
move()
turn_right()
move()

[Assistant]:The map after executing the program would be:

state={
"height":5,
"width":5,
"robot_x":3,
"robot_y":3,
"direction":east
}

The robot executes the turn_left() instruction and faces the west. It then executes move() and moves to position (1, 2) while facing the west. After that it executes turn_right() instruction and faces the north. Finally the robot executes the move() instruction and moves to position (1, 3) while facing the north.
#### End of example

It also has the trend to produce correct answer through text, but not the map it states.

## Comparison between GPT3.5 and GPT4
### Given a map and a program, print the map afterward
[User]:Now I'm going to give you a map and a program, your goal is to tell me the map after executing the program.
Map:
state={
"height":5,
"width":5,
"robot_position":(2,2),
"direction":north
}

program:

turn_left()
move()
turn_right()
move()
#### GPT3.5
[Assistant]:The map after executing the program will be:
state={
"height":5,
"width":5,
"robot_position":(2,3),
"direction":east
}

Explanation:
The program first turns the robot left, so it will be facing west. Then it moves one step to the west, ending up at position (1,2). After that, the program turns the robot right, so it will be facing north. Finally, the robot moves one step to the north, ending up at position (1,3).
#### GPT4
[Assistant]:After executing the given program on the provided map, the updated map will be:

state={
"height": 5,
"width": 5,
"robot_position": (1, 3),
"direction": north
}

- GPT4 produces correct and more succinctly answer.

### Given two maps, give the program that can transfer one to another
[User]:What's the program that can transfer Map1 to Map2?
Map1:
state={
"height":5,
"width":5,
"robot_position":(0,5),
"direction":north
}

Map2:
state={
"height":5,
"width":5,
"robot_position":(4,1),
"direction":west
}
#### GPT3.5
[Assistant]:Here is the program that can transfer Map1 to Map2:

turn_left()
move()
move()
move()
turn_left()
turn_left()
move()
move()
move()
turn_left()

#### GPT4
[Assistant]:To transfer Map1 to Map2, follow the given program:

1. turn_right() # Change direction to east
2. move() # Move to (1, 5)
3. move() # Move to (2, 5)
4. move() # Move to (3, 5)
5. move() # Move to (4, 5)
6. turn_right() # Change direction to south
7. move() # Move to (4, 4)
8. move() # Move to (4, 3)
9. move() # Move to (4, 2)
10. move() # Move to (4, 1)
11. turn_right() # Change direction to west

After executing the above program, the Map1 will become Map2.

- GPT4 is much more powerful when it comes to reasoning.

### Some random notes

I use (N,S,E,W) to represent the position and direction of the robot.
N means robot is facing north, which is up in the map.
S means robot is facing south, which is down in the map.
E means robot is facing east, which is right in the map.
W means robot is facing west, which is left in the map.
#means the walls.
. means empty spaces.
G means for goal.
5 | . G . . .
4 | . . # . .
3 | . . # . .
2 | # # # . #
1 | E . . . .
1 2 3 4 5

Can you show me the karel program that can solve this map?

======================================================

[User]:The goal is to place a marker at each corner of the Karel environment grid. The reward is defined
as sum of corners having a marker divided by four. If the Karel state has a marker placed in wrong
location, the reward will be 0. This is on a 12 × 12 grid.

[Assistant]:To achieve the goal, Karel needs to follow these steps:

1. Start at the bottom-left corner (1, 1).
2. Place a marker at the current position.
3. Move to the bottom-right corner (12, 1).
4. Place a marker at the current position.
5. Move to the top-right corner (12, 12).
6. Place a marker at the current position.
7. Move to the top-left corner (1, 12).
8. Place a marker at the current position.

Here's a possible implementation of this plan:

def main():
    place_marker()
    move_to_bottom_right_corner()
    place_marker()
    move_to_top_right_corner()
    place_marker()
    move_to_top_left_corner()
    place_marker()

def move_to_bottom_right_corner():
    while front_is_clear():
        move()
    turn_left()

def move_to_top_right_corner():
    while front_is_clear():
        move()
    turn_left()

def move_to_top_left_corner():
    while front_is_clear():
        move()
    turn_right()
    while front_is_clear():
        move()
    turn_right()

main()

The reward function can be defined as follows:

def reward_function(state):
    bottom_left = state.has_marker_at(1, 1)
    bottom_right = state.has_marker_at(12, 1)
    top_right = state.has_marker_at(12, 12)
    top_left = state.has_marker_at(1, 12)

    if state.has_wrong_marker():
        return 0

    return (bottom_left + bottom_right + top_right + top_left) / 4

This reward function checks if there are markers at each corner and calculates the reward based on the number of corners with markers. If there's a marker placed in the wrong location, the reward will be 0.

============================================================

### Can ChatGPT and GPT-4 write programs with correct syntax?
#### Prompt

The domain specific language for writing program in Karel environment include these tokens:
DEF
run
m(
m)
move
turnRight
turnLeft
pickMarker
putMarker
r(
r)
R=2
R=3
R=4
R=5
REPEAT
c(
c)
i(
i)
e(
e)
IF
IFELSE
ELSE
frontIsClear
leftIsClear
rightIsClear
markersPresent
noMarkersPresent
not
w(
w)
WHILE

You can only use the tokens above.
You should use w( w) for WHILE loop, i( i) for IF, e( e) for ELSE, c( c) for describing condition, r( r) for REPEAT, and m( m) for the function.

Here are example programs:
(1) DEF run m( WHILE c( frontIsClear c) w( turnRight move pickMarker turnRight w) m)
(2) DEF run m( IFELSE c( markersPresent c) i( move turnRight i) ELSE e( move e) move move WHILE c( leftIsClear c) w( turnLeft w) m)
(3) DEF run m( IF c( frontIsClear c) i( putMarker i) move IF c( rightIsClear c) i( move i) IFELSE c( frontIsClear c) i( move i) ELSE e( move e) m)
(4) DEF run m( WHILE c( leftIsClear c) w( turnLeft w) IF c( frontIsClear c) i( putMarker move i) move IF c( rightIsClear c) i( turnRight move i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) m)
(5) DEF run m( REPEAT R=5 r( turnRight move pickMarker turnRight r) m)

Please randomly generate a program:

#### ChatGPT

DEF run m( WHILE c( frontIsClear c) w( turnLeft IF c( markersPresent c) i( pickMarker i) ELSE e( move e) turnRight move w) m)

Please randomly generate a program to solve stairClimber:
> Here's a randomly generated program to solve the stairClimber problem using the given domain specific language:

DEF run m(
    WHILE c(frontIsClear c)
    w(
        climbStairs
        turnRight
        climbStairs
        turnLeft
    )
    move
    WHILE c(leftIsClear c)
    w(
        turnLeft
        climbStairs
        turnRight
        climbStairs
        move
    )
    m
)

DEF climbStairs m(
    REPEAT R=2 r(
        move
        turnLeft
        move
        turnRight
    ) r
)

> This program uses a combination of loops and conditionals to climb a staircase. It first moves forward until it encounters a staircase, then repeatedly climbs up two steps and turns to the right until it reaches the top. Then it moves forward and turns left to start descending the staircase. It repeats this process until there are no more stairs to climb. The `climbStairs` function is called twice in each loop iteration to climb up two steps at a time.

Please randomly generate a program to solve seeder:
> Here's a randomly generated program to solve the Seeder problem using the given domain specific language:

DEF run m(
    WHILE c(frontIsClear c)
    w(
        plantRow
        move
    )
    turnLeft
    move
    WHILE c(frontIsClear c)
    w(
        plantRow
        move
    )
    turnRight
    WHILE c(frontIsClear c)
    w(
        plantRow
        move
    )
    turnRight
    move
    WHILE c(frontIsClear c)
    w(
        plantRow
        move
    )
    turnLeft
    m
)

DEF plantRow m(
    REPEAT R=3 r(
        putMarker
        move
    ) r
    turnLeft
    turnLeft
    WHILE c(frontIsClear c)
    w(
        move
    )
    turnLeft
)

> This program uses a combination of loops and conditionals to plant seeds in four rows around the Karel world. It first plants a row in front of Karel, then turns left and moves forward to plant another row. It repeats this process until there are no more rows to plant in that direction, then turns right and repeats the process in the opposite direction. Finally, it moves forward and turns left to plant the last row. The `plantRow` function is called three times in each loop iteration to plant a row of seeds, and it includes a nested loop to move Karel to the start of the next row.

#### GPT-4
##### Attempt1
GPT-4 can generate a correct program.

DEF run m( WHILE c( rightIsClear c) w( move pickMarker turnLeft w) IF c( frontIsClear c) i( move i) IFELSE c( leftIsClear c) i( turnRight move i) ELSE e( putMarker move e) REPEAT R=3 r( turnLeft move r) m)

Please randomly generate a program to solve stairClimber:

DEF run m( WHILE c( frontIsClear c) w( move w) IF c( leftIsClear c) i( turnLeft move turnRight i) m)

You should modify the program since there are more than one stair.

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move w) IF c( leftIsClear c) i( turnLeft move turnRight i) r) m)

Please randomly generate a program to solve seeder:

DEF run m( REPEAT R=5 r( putMarker move move r) turnRight turnRight WHILE c( frontIsClear c) w( move w) turnLeft move move turnRight m)

The previous program get reward 0.1389. Please randomly generate a program to try to maximize the reward.

I apologize for the previous response, as I am an AI language model and I cannot generate programs based on rewards. However, I can provide another randomly generated program for the seeder problem:

DEF run m( REPEAT R=5 r( putMarker move move r) turnRight turnRight WHILE c( frontIsClear c) w( move w) turnLeft move move turnRight IF c( frontIsClear c) i( putMarker move move i) m)

##### Another attempt: Do not use score, use text to represent the score.

DEF run m( IFELSE c( rightIsClear c) i( turnRight move pickMarker i) ELSE e( turnLeft move e) REPEAT R=3 r( move r) WHILE c( frontIsClear c) w( putMarker w) m)

Please randomly generate a program to solve seeder:

DEF run m( WHILE c( noMarkersPresent c) w( pickMarker move w) WHILE c( markersPresent c) w( putMarker move w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft m)

The program performs poorly. Please generate a program to try to solve the task better.

I apologize for the poor performance of the previous program. Here's a new program to try:

DEF run m( REPEAT R=4 r( WHILE c( noMarkersPresent c) w( move w) pickMarker turnLeft turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight REPEAT R=2 r( move r) turnLeft r) m)

This program attempts to move through the grid, picking up markers, and returning them to the starting point. Please note that the performance of the program may still vary depending on the specific layout of the grid and the markers.

The program performs even worse. Please generate a program to try to solve the task better.

My apologies for the previous programs' poor performance. Here is another program that should perform better for the Seeder task:

DEF run m( REPEAT R=4 r( WHILE c( frontIsClear c) w( IF c( noMarkersPresent c) i( pickMarker i) move w) turnRight r) m)

This program attempts to move through the grid, picking up markers while the front is clear, and turning right after each row. Please note that the performance of the program may still vary depending on the specific layout of the grid and the markers.

The program performs even worse. Please generate a program to try to solve the task better.

I apologize for the confusion. The Seeder task requires placing markers on the grid rather than picking them up. Here is a new program designed for the Seeder task:

DEF run m( REPEAT R=4 r( WHILE c( frontIsClear c) w( IF c( noMarkersPresent c) i( putMarker i) move w) turnRight r) m)

This program attempts to move through the grid, placing markers while the front is clear, and turning right after each row. The performance of the program may still vary depending on the specific layout of the grid and the markers.

The program performs better. However, it only accomplished one-ninth of the task. Please generate a program to try to solve the task better.

I apologize for the previous program's partial success. Here's an improved program for the Seeder task:

DEF run m( REPEAT R=2 r( REPEAT R=4 r2( IF c( noMarkersPresent c) i( putMarker i) move r2) turnRight REPEAT R=4 r3( IF c( noMarkersPresent c) i( putMarker i) move r3) turnLeft r) m)

This program attempts to move through the grid in a zigzag pattern, placing markers while the front is clear, and turning appropriately at the end of each row. The performance of the program may still vary depending on the specific layout of the grid and the markers.

The program performs better. However, it only completes a third of the task. Please generate a program to try to solve the task better.

I apologize for the previous program's partial success. Here's an improved program for the Seeder task:

DEF run m( REPEAT R=3 r( REPEAT R=4 r1( IF c( noMarkersPresent c) i( putMarker i) move r1) turnRight move turnRight REPEAT R=4 r2( IF c( noMarkersPresent c) i( putMarker i) move r2) turnLeft move turnLeft r) m)

This program attempts to move through the grid in a modified zigzag pattern, placing markers while the front is clear, and turning appropriately at the end of each row. The performance of the program may still vary depending on the specific layout of the grid and the markers.

## Binary

### Prompt

Init

The domain specific language for writing program in Karel environment include these tokens:
DEF
run
m(
m)
move
turnRight
turnLeft
pickMarker
putMarker
r(
r)
R=2
R=3
R=4
R=5
REPEAT
c(
c)
i(
i)
e(
e)
IF
IFELSE
ELSE
frontIsClear
leftIsClear
rightIsClear
markersPresent
noMarkersPresent
not
w(
w)
WHILE

You can only use the tokens above.
You should use w( w) for WHILE loop, i( i) for IF, e( e) for ELSE, c( c) for describing condition, r( r) for REPEAT, and m( m) for the function.

Here are example programs:
(1) DEF run m( WHILE c( frontIsClear c) w( turnRight move pickMarker turnRight w) m)
(2) DEF run m( IFELSE c( markersPresent c) i( move turnRight i) ELSE e( move e) move move WHILE c( leftIsClear c) w( turnLeft w) m)
(3) DEF run m( IF c( frontIsClear c) i( putMarker i) move IF c( rightIsClear c) i( move i) IFELSE c( frontIsClear c) i( move i) ELSE e( move e) m)
(4) DEF run m( WHILE c( leftIsClear c) w( turnLeft w) IF c( frontIsClear c) i( putMarker move i) move IF c( rightIsClear c) i( turnRight move i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) m)
(5) DEF run m( REPEAT R=5 r( turnRight move pickMarker turnRight r) m)

Please randomly generate a program to solve a hidden task:

The program performs better. Please generate a program to try to solve the task better.

The program performs worse. Please generate a program to try to solve the task better.

The program performance has not changed. Please generate a program to try to solve the task better.

> Deleted
> ```
> There is a syntax error. Please generate a program to try to solve the task better.
>```

### Test (GPT-4)

#### harvester

##### turn1

DEF run m( WHILE c( frontIsClear c) w( move pickMarker turnRight w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnRight move e) REPEAT R=3 r( putMarker move turnLeft r) m)

Reward: 0.027778

##### turn2

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) ELSE e( turnRight move e) REPEAT R=4 r( move turnRight putMarker r) m)

Reward: -0.001
> Here we use "There is a syntax error". And the program generated would not change.

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) ELSE e( turnRight move e) REPEAT R=4 r( move turnRight putMarker r) m)

Reward: -0.001
> So we would use "The program performs worse." instead

##### turn3

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( turnLeft move e) REPEAT R=3 r( move turnLeft putMarker r) m)

Reward: 0.138889

##### turn4

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( turnLeft move e) REPEAT R=2 r( move turnLeft putMarker r) IF c( leftIsClear c) i( turnLeft move putMarker i) m)

Reward: 0.138889

##### turn5

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( rightIsClear c) i( turnRight move putMarker i) IFELSE c( leftIsClear c) i( turnLeft move putMarker i) ELSE e( move e) REPEAT R=2 r( turnLeft move putMarker r) m)

Reward: 0.138889

##### turn6

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( turnLeft move e) REPEAT R=4 r( move turnLeft putMarker r) IF c( leftIsClear c) i( turnLeft move putMarker i) m)

Reward: 0.138889

##### turn7

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( turnLeft move e) IF c( leftIsClear c) i( turnLeft move putMarker i) REPEAT R=3 r( move turnRight putMarker r) m)

Reward: 0.138889

##### turn8

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( rightIsClear c) i( turnRight move putMarker i) IFELSE c( leftIsClear c) i( turnLeft move putMarker i) ELSE e( move e) REPEAT R=4 r( turnRight move putMarker r) m)

Reward: 0.138889

##### turn9

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( move e) REPEAT R=3 r( move turnLeft putMarker r) m)

Reward: 0.138889
##### turn10

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IFELSE c( rightIsClear c) i( turnRight move putMarker i) ELSE e( turnLeft move e) IF c( leftIsClear c) i( turnLeft move putMarker i) REPEAT R=4 r( move turnRight putMarker r) m)

Reward: 0.138889

###### Attempt: change 'a hidden task' to 'harvester'

DEF run m( WHILE c( frontIsClear c) w( move pickMarker turnRight move turnLeft w) REPEAT R=4 r( move r) turnRight WHILE c( frontIsClear c) w( move w) putMarker turnLeft m)

Reward: 0.055556

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move w) putMarker turnLeft REPEAT R=4 r( move r) turnRight m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move w) putMarker turnLeft move REPEAT R=4 r( move r) turnRight m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft move WHILE c( frontIsClear c) w( move w) putMarker turnLeft REPEAT R=4 r( move r) turnRight m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft REPEAT R=4 r( move putMarker r) turnRight m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft REPEAT R=4 r( move putMarker r) turnRight turnLeft m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnRight WHILE c( frontIsClear c) w( move w) putMarker turnLeft move turnLeft REPEAT R=4 r( move r) turnRight m)

Reward: 0.111111

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft REPEAT R=4 r( move putMarker r) turnLeft turnRight m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft REPEAT R=4 r( move putMarker r) turnLeft turnRight turnLeft m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( move pickMarker r) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft REPEAT R=4 r( move putMarker r) turnLeft turnRight turnRight m)

Reward: 0.138889

## Score

### Prompt

Init

The domain specific language for writing program in Karel environment include these tokens:
DEF
run
m(
m)
move
turnRight
turnLeft
pickMarker
putMarker
r(
r)
R=2
R=3
R=4
R=5
REPEAT
c(
c)
i(
i)
e(
e)
IF
IFELSE
ELSE
frontIsClear
leftIsClear
rightIsClear
markersPresent
noMarkersPresent
not
w(
w)
WHILE

You can only use the tokens above.
You should use w( w) for WHILE loop, i( i) for IF, e( e) for ELSE, c( c) for describing condition, r( r) for REPEAT, and m( m) for the function.

Here are example programs:
(1) DEF run m( WHILE c( frontIsClear c) w( turnRight move pickMarker turnRight w) m)
(2) DEF run m( IFELSE c( markersPresent c) i( move turnRight i) ELSE e( move e) move move WHILE c( leftIsClear c) w( turnLeft w) m)
(3) DEF run m( IF c( frontIsClear c) i( putMarker i) move IF c( rightIsClear c) i( move i) IFELSE c( frontIsClear c) i( move i) ELSE e( move e) m)
(4) DEF run m( WHILE c( leftIsClear c) w( turnLeft w) IF c( frontIsClear c) i( putMarker move i) move IF c( rightIsClear c) i( turnRight move i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) m)
(5) DEF run m( REPEAT R=5 r( turnRight move pickMarker turnRight r) m)

Please randomly generate a program to solve a hidden task:

The reward is []. Please generate a program to try to solve the task better.

### Test (GPT-4)

#### harvester

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) IF c( rightIsClear c) i( turnRight i) ELSE e( turnLeft e) w) IFELSE c( leftIsClear c) i( move turnLeft i) ELSE e( move turnRight e) m)

Reward: -0.001

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker turnRight i) IF c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) w) IFELSE c( leftIsClear c) i( move turnLeft i) ELSE e( move turnRight e) m)

Reward: -0.001

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker turnLeft i) IF c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) w) IFELSE c( leftIsClear c) i( move turnLeft i) ELSE e( move turnRight e) m)

Reward: -0.001

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) IF c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) w) IFELSE c( rightIsClear c) i( move turnRight i) ELSE e( move turnLeft e) m)

Reward: -0.001

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) IF c( leftIsClear c) i( turnLeft move i) ELSE e( turnRight move e) w) IFELSE c( leftIsClear c) i( move turnLeft i) ELSE e( move turnRight e) m)

Reward: -0.001
...
Cannot change in 10 turns

###### Another attempt

DEF run m( IF c( frontIsClear c) i( move i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) WHILE c( markersPresent c) w( pickMarker move w) REPEAT R=3 r( turnLeft r) putMarker m)

Reward: 0.138889

DEF run m( IFELSE c( frontIsClear c) i( move putMarker turnRight i) ELSE e( turnLeft e) WHILE c( markersPresent c) w( pickMarker move w) REPEAT R=2 r( turnRight move r) IF c( leftIsClear c) i( move i) IF c( rightIsClear c) i( turnRight putMarker i) m)

Reward: 0.027778

DEF run m( WHILE c( frontIsClear c) w( move putMarker w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnRight e) REPEAT R=3 r( pickMarker move r) IF c( rightIsClear c) i( turnRight move i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) m)

Reward: 0.083333

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) IF c( rightIsClear c) i( turnRight move i) IFELSE c( frontIsClear c) i( move pickMarker i) ELSE e( turnLeft move e) REPEAT R=4 r( turnRight move r) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move putMarker w) IF c( leftIsClear c) i( turnLeft move i) IFELSE c( rightIsClear c) i( turnRight move pickMarker i) ELSE e( turnLeft move e) REPEAT R=3 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move i) m)

Reward: 0.0

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( rightIsClear c) w( turnRight move pickMarker w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnRight e) REPEAT R=3 r( move turnLeft r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move i) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnRight move e) REPEAT R=4 r( turnLeft move turnLeft r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move i) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( rightIsClear c) w( turnRight move pickMarker w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnLeft e) REPEAT R=3 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnRight move e) REPEAT R=2 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) IF c( rightIsClear c) i( turnRight move pickMarker i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) REPEAT R=3 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move i) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) REPEAT R=2 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( rightIsClear c) w( turnRight move pickMarker w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnLeft e) REPEAT R=2 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) WHILE c( frontIsClear c) w( move w) m)

Reward: 0.027778

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) REPEAT R=2 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) REPEAT R=3 r( move r) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) IF c( rightIsClear c) i( turnRight move pickMarker i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) REPEAT R=3 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) WHILE c( frontIsClear c) w( move w) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) REPEAT R=3 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) REPEAT R=2 r( move r) m)

Reward: 0.111111

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) IF c( rightIsClear c) i( turnRight move pickMarker i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) REPEAT R=2 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) WHILE c( frontIsClear c) w( move w) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) REPEAT R=4 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) REPEAT R=2 r( move r) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( rightIsClear c) w( turnRight move pickMarker w) IFELSE c( leftIsClear c) i( turnLeft move i) ELSE e( turnLeft e) REPEAT R=4 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) WHILE c( frontIsClear c) w( move w) m)

Reward: 0.0

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) IF c( leftIsClear c) i( turnLeft move putMarker i) IFELSE c( rightIsClear c) i( turnRight move i) ELSE e( turnLeft move e) REPEAT R=5 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( rightIsClear c) i( turnRight move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) REPEAT R=3 r( move r) m)

Reward: 0.138889

DEF run m( IF c( frontIsClear c) i( move putMarker i) WHILE c( leftIsClear c) w( turnLeft move w) IF c( rightIsClear c) i( turnRight move pickMarker i) IFELSE c( frontIsClear c) i( move i) ELSE e( turnLeft move e) REPEAT R=5 r( turnRight move r) IF c( frontIsClear c) i( move i) IF c( leftIsClear c) i( turnLeft move pickMarker i) WHILE c( frontIsClear c) w( move w) m)

Reward: 0.0
###### Attempt: change 'a hidden task' to 'harvester'

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) IF c( rightIsClear c) i( turnRight i) ELSE e( turnLeft e) w) IF c( leftIsClear c) i( turnLeft move i) ELSE e( turnRight move e) m)

Reward: -0.001

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight r) m)

Reward: 0.444444

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight r) turnLeft move turnLeft m)

Reward: 0.444444

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight r) move turnLeft move turnLeft m)

Reward: 0.444444

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight r) move turnLeft move turnLeft turnLeft m)

Reward: 0.444444

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight r) move turnLeft move turnRight turnLeft m)

Reward: 0.444444
...

###### Attempt: Use text to represent score + 'harvester'

# Prompt
The program accomplished [] percent of the task. Please generate a program to try to solve the task better.

DEF run m( WHILE c( frontIsClear c) w( move WHILE c( markersPresent c) w( pickMarker w) IF c( rightIsClear c) i( turnRight i) IF c( leftIsClear c) i( turnLeft i) move w) IF c( rightIsClear c) i( turnRight move i) IF c( leftIsClear c) i( turnLeft move i) m)

Reward: 0.083333

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) m)

Reward: 0.694444

DEF run m( REPEAT R=6 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) m)

> KeyError: 'R=6'

My apologies for the error. The REPEAT token only works with R=2 to R=5. Here's a modified version of the program:

DEF run m( REPEAT R=3 r( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) m) m)

> SyntaxError

Apologies for the syntax error. Here's the corrected version of the program:

DEF run m( REPEAT R=3 r( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) r) m)

Reward: 0.833333

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft w) m)

> SyntaxError

Apologies for the syntax error. Here's the corrected version of the program:

DEF run m( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft w) w) m)

> SyntaxError

I apologize for the repeated syntax errors. Here's another attempt at generating a program to solve the task better:

DEF run m( REPEAT R=5 r( IF c( frontIsClear c) i( move IF c( markersPresent c) i( pickMarker i) i) i) turnRight move turnRight IF c( frontIsClear c) i( move i) turnLeft move turnLeft IF c( frontIsClear c) i( move i) turnRight move turnRight IF c( frontIsClear c) i( move i) turnLeft move turnLeft r) m)

> SyntaxError

I apologize for the syntax errors. Here's another attempt at generating a program to solve the task better:

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft w) r) m)

> SyntaxError

...

> Delete the SyntaxError programs

DEF run m( REPEAT R=4 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) m)

Reward: 0.555556

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=2 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.444444

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=3 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.333333

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) IF c( markersPresent c) i( pickMarker move i) IF c( frontIsClear c) i( move i) m)

Reward: 0.444444

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight IF c( frontIsClear c) i( move i) turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) IF c( markersPresent c) i( pickMarker i) IF c( frontIsClear c) i( move i) m)

Reward: 0.305556

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=4 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.361111

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight r) IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) m)

Reward: 0.416667

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=5 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.388889

DEF run m( REPEAT R=3 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move w) turnRight move turnRight r) IF c( markersPresent c) i( pickMarker i) IF c( frontIsClear c) i( move i) m)

Reward: 0.416667

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=3 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) REPEAT R=2 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.388889

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=3 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) REPEAT R=4 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.388889

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=5 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) REPEAT R=3 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.388889

DEF run m( REPEAT R=2 r( WHILE c( frontIsClear c) w( move IF c( markersPresent c) i( pickMarker i) w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft move turnLeft r) REPEAT R=4 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) REPEAT R=5 r( IF c( frontIsClear c) i( move i) IF c( markersPresent c) i( pickMarker i) r) m)

Reward: 0.388889
...

###### Attempt: Use binary + Use text to represent score + 'harvester'

# Prompt
The program performs better/worse/the same. The program (only) accomplished [] percent of the task. Please generate a program to try to solve the task better.

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft turnLeft r) m)

Reward: 0.583333

DEF run m( WHILE c( markersPresent c) w( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft turnLeft w) m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft turnLeft WHILE c( noMarkersPresent c) w( move w) turnLeft turnRight r) m)

Reward: 0.555556

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft turnLeft WHILE c( markersPresent c) w( move w) turnLeft turnRight r) m)

Reward: 0.166667

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( frontIsClear c) w( move w) turnLeft turnLeft WHILE c( noMarkersPresent c) w( move w) turnLeft turnLeft r) m)

Reward: 0.138889

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight turnRight move turnRight r) m)

Reward: 0.166667

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight r) m)

Reward: 0.833333

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft r) m)

Reward: 0.861111

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft r) m)

Reward: 0.277778

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( rightIsClear c) w( move w) turnRight r) m)

Reward: 0.277778

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft WHILE c( rightIsClear c) w( move w) turnRight r) m)

Reward: 0.277778

DEF run m( REPEAT R=5 r( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft WHILE c( rightIsClear c) w( move w) turnRight WHILE c( frontIsClear c) w( move w) r) m)

Reward: 0.277778

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft WHILE c( rightIsClear c) w( move w) turnRight WHILE c( frontIsClear c) w( move w) m)

Reward: 0.277778

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft WHILE c( rightIsClear c) w( move w) turnRight WHILE c( frontIsClear c) w( move w) IF c( markersPresent c) i( move i) m)

Reward: 0.277778

DEF run m( WHILE c( frontIsClear c) w( move pickMarker w) turnRight move turnLeft WHILE c( frontIsClear c) w( move pickMarker w) turnLeft move turnRight WHILE c( noMarkersPresent c) w( move w) turnRight turnLeft WHILE c( leftIsClear c) w( move w) turnLeft WHILE c( rightIsClear c) w( move w) turnRight WHILE c( frontIsClear c) w( move w) IF c( markersPresent c) i( move i) m)

Reward: 0.166667