# Snowcast Autograder Test Documentation

This document provides a description of what each test performs.

# Important notes

For each test, we try to note the following requirements for each test to run:
- **Stdin**: What commands the tester will send to your program on stdin (as if the user entered a key on the keyboard). (For example, the `q` command.)
- **Stdout**: What output the tester expects you to print on stdout (eg. `The server has N stations`)
- What Snowcast packets the server will send to your program, and what packets the tester expects to receive

In all cases, the formats for what the tester expects are defined by the [**Snowcast specification**](https://hackmd.io/@csci1680/snowcast-spec).

# Control Tests

## Startup And Connecting

### TestConnectsToServer

Checks that the control can connect to the server and send a `HELLO` message to
the server.

### TestCompletesHandshake

**Stdin**: None
**Stdout**: Reads `The server has N stations` on stdout

Checks that the the control receives and handles the `WELCOME` reply from the
server after the control sends a `HELLO` message. This test passes if the
control is able to print out the number of stations.

### TestMultipleWelcomesFails

**Stdin**: None
**Stdout**: None

Checks that the control disconnects from the server if the server sends
multiple `WELCOME` replies.

### TestNoWelcomeFails

**Stdin**: None
**Stdout**: None

Checks that the control closes the connection to the server if the server
does not respond with a `WELCOME` 100ms after the control sends a `HELLO`.

## Handling Partial Messages

### TestPartialWelcomeTimeoutFails

**Stdin**: None
**Stdout**: None

Checks that control terminates its connection to the server if the server
does not receive a complete `WELCOME` message.

This test waits for the control to send a `HELLO`, and then responds with
only the first few bytes a of a `WELCOME` reply. The control is expected to
to terminate its connection to server after 100ms and not receiving the
complete `WELCOME`.

### TestSplitWelcomeSucceeds1

**Stdin**: None
**Stdout**: Expects `The server has N stations`

Checks that the control can successfully receive a `WELCOME` message from the
server that is sent across two packets.

This test waits for the control to send a `HELLO`, and then the test server
responds with the first half of the `WELCOME` message, waits some time, and
then sends the final half of the `WELCOME` message to the control.

### TestSplitWelcomeSucceeds2

**Stdin**: None
**Stdout**: Expects `The server has N stations`

Checks that the control can successfully receive a `WELCOME` message from the
server that is sent across multiple packets.

This test waits for the control to send a `HELLO`, and then the test server
responds with packets each containing one byte of the `WELCOME` reply, with
some delay between each packet.

### TestPartialAnnounceTimeoutFails

**Stdin**: Sets a station
**Stdout**: None

Checks that the control closes its connection to the server if the control
receives part of the `ANNOUNCE` message and does not receive the remainder of
the `ANNOUNCE` message within 100ms of receiving the first part.

### TestSplitAnnounceSucceeds1

**Stdin**: Sets a station
**Stdout**: Expects `New song announced: <song name>`

Checks that the control can successfully receive an `ANNOUNCE` message from the
server that is sent across two packets.

The test server responds with the first half of the `ANNOUNCE` message,
waits some time, and then sends the final half of the `ANNOUNCE` message to
the control.

### TestSplitAnnounceSucceeds2

**Stdin**: Sets a station
**Stdout**: Expects `New song announced: <song name>`

Checks that the control can successfully receive a `ANNOUNCE` message from the
server that is sent across multiple packets.

The test server responds with packets each containing one byte of the
`ANNOUNCE` reply, with some delay between each packet.

## Message Formatting

### TestSetStationFormat

**Stdin**: Sets a station
**Stdout**: None

Checks that the control sends `SET_STATION` commands in the correct format.

### TestExitsOnInvalidCommand

**Stdin**: None
**Stdout**: Expects `The server has N stations` on to verify startup

Checks that the control terminates if the server sends an invalid command
message to the control.

### TestExitsOnInvalidReplyType

**Stdin**: None
**Stdout**: None

Checks that the control terminates its connection with the server if the
server responds with an invalid reply type to the control (i.e some unknown
reply type).

## Miscellaneous Functionality

### TestPrintsAnnounce1

**Stdin**: Sets a station
**Stdout**: Expects `New song announced: <song name>`

Checks that the control can print a single `ANNOUNCE` message received from the
server.

### TestPrintsAnnounce2

**Stdin**: Sets a station
**Stdout**: Expects `New song announced: <song name>`

Checks that the control can continuously print a `ANNOUNCE` messages received
from the server. This test sends 10 `ANNOUNCE` messages to the control and
checks that the control can print all 10 `ANNOUNCE` messages.

### TestAnnounceBeforeSetStationFails

**Stdin**: None
**Stdout**: None

Checks that the control terminates its connection with the server if the
control receives an `ANNOUNCE` message from the server before sending a
`SET_STATION` message to the server.

### TestSetStationInvalidCommandResponseFails

**Stdin**: Sets station
**Stdout**: None

Checks that the control terminates its connection to the server if the server
responds with the incorrect response type (i.e responding with a `WELCOME`
instead of an `ANNOUNCE`)

### TestQuitsCleanly

**Stdin**: `q` command
**Stdout**: None

Checks that the control can exit cleanly when the user inputs the quit command to the control CLI.

## Server Tests

### TestNoStationsFails

**Stdin**: None
**Stdout**: None

Checks that the server fails to start if no stations were provided

### TestAcceptsClientConnection

**Stdin**: None
**Stdout**: None

Checks that the server does not fail when accepting a single connection.

### TestTimesOutConnectionWithNoHello

**Stdin**: None
**Stdout**: None

Checks that the server closes a client connection when the client does not
send a `HELLO` message within 100ms of connecting to the server.

### TestCompletesHandshake

**Stdin**: None
**Stdout**: None

Checks that the server responds with a `WELCOME` response, completing the
handshake, when the client sends a `HELLO` message.

### TestMultipleHellosFails

**Stdin**: None
**Stdout**: None

Checks that the server terminate a client connection if the client sends
multiple `HELLO` messages.

### TestServerSurvivesClientDisconnect

**Stdin**: `p <file>` command
**Stdout**: None, but must read station list from file

Checks that the server does not terminate if a client disconnects.

### TestPartialHelloTimeoutFails

**Stdin**: None
**Stdout**: None

Checks that server terminates a client connection if the server does not receive a complete `HELLO` message.

This test sends only the first few bytes of the `HELLO` message to the server and the server is expected to terminate the client after not receiving the full `HELLO` message with 100ms to receiving the first few bytes.

### TestSplitHelloSucceeds1

**Stdin**: None
**Stdout**: None

Checks that the server can successfully accept a `HELLO` message from the client that is sent across multiple packets.

This test sends the first few bytes of the `HELLO` message to the server, waits some time, and then sends the final bytes of the `HELLO` message to the server.

### TestSplitHelloSucceeds2

**Stdin**: None
**Stdout**: None

Checks that the server can successfully accept a `HELLO` message from the client that is sent across multiple packets.

This test sends each byte of the `HELLO` message to the server in a separate
packet, with some delay between each of the sends.

### TestPartialSetStationTimeoutFails

**Stdin**: None
**Stdout**: None

Checks that server terminates a client connection if the server does not
receive a complete `SET_STATION` message.

This test sends only the first few bytes of the `SET_STATION` message to the
server and the server is expected to terminate the client after not
receiving the full `SET_STATION` message with 100ms to receiving the first few
bytes.

### TestSplitSetStationSucceeds1

**Stdin**: None
**Stdout**: None

Checks that the server can successfully accept a `SET_STATION` message from the
client that is sent across multiple packets.

This test sends the first few bytes of the `SET_STATION` message to the server,
waits some time, and then sends the final bytes of the `SET_STATION` message
to the server.

### TestSplitSetStationSucceeds2

**Stdin**: None
**Stdout**: None

Checks that the server can successfully accept a `SET_STATION` message from
the client that is sent across multiple packets.

This test sends each byte of the `SET_STATION` message to the server in a
separate packet, with some delay between each of the sends.

### TestSetStationBeforeHelloFails

**Stdin**: None
**Stdout**: None

Checks that the server terminates a client connection if the client sends a `SET_STATION` message before a `HELLO` message.

### TestSetStationSucceeds

**Stdin**: None
**Stdout**: None

Sends `SetStation` message to the server and checks to make sure the correct `Announce` is received.

### TestSupportsMultipleStations

**Stdin**: `p <file>` command
**Stdout**: None, but must read station list from file

Checks that the server can support multiple stations. This test sets up the
server with multiple stations and then iterates over each file provided for a
station. For each file, the test does the following:

- send a `SET_STATION` message to the server for each station and assert that
the server responds with a corresponding `ANNOUNCE`.
- checks that the test client is placed with the correct station in the
server.

### TestStreamsWithoutReadingEntireFile

**Stdin**: `p <file>` command
**Stdout**: None, but must read station list from file

Checks that file data is streamed by reading a fixed number of bytes at a time instead of reading the entire file into memory at once.

To ensure this behavior, this test uses `/dev/urandom` and `/dev/zero` as files for the stations.

### TestInvalidStationFails

**Stdin**: None
**Stdout**: None

Checks that the server terminates a client connection if the client sends a `SET_STATION` message to the server that contains an invalid station number.

### TestAcceptsMultipleConnections

**Stdin**: `p <file>` command
**Stdout**: None, but must read station list from file

Checks that the server can handle multiple connections with each connection
have different stations. This test makes several connections from different client, prints the station list to a file, and verifies that each client is listed on the correct station.

### TestAcceptsMultipleConnectionsRegardlessOfStation

**Stdin**: None
**Stdout**: None

Checks that the server can handle multiple connections with each connection
have different stations. This test performs the same check as `TestAcceptsMultipleConnections`, except the connections are verified only by looking at the set of sockets used by the `snowcast_server` process--there is no verification that clients are joined to the correct station.

**Note**: This test is designed to test a strict subset of the functionality in `TestAcceptsMultipleConnections`. If you are failing this test but passing `TestAcceptsMultipleConnections`, don't worry--we'll adjust manually.

### TestStreamRateSingle

**Stdin**: None
**Stdout**: None

Checks that the server's stream rate is 16KiB/s for a single connection listening to one station.

### TestStreamRateMultipleOneStation

**Stdin**: None
**Stdout**: None

Checks that the server's stream rate is 16KiB/s for a multiple connections listening to one station.

### TestStreamRateMultipleStations

**Stdin**: None
**Stdout**: None

Checks that the server's stream rate is 16KiB/s for a multiple connections listening to different stations.

### TestStreamsWithNoListeners

**Stdin**: None
**Stdout**: None

This test attempts to verify that the station continues playing music when no clients are joined to the station.

To do this, the test starts the server and waits >1s before joining a client. When the client first receives listener data, the data must not match the beginning of the file.

**Note**: The functionality tested here is generally straightforward manually, but quite difficult to test programmatically. We will manually review test failures to check for false negatives, and adjust accordingly. If you are failing this test, ask yourself: "Is the station still reading the file when no clients are connected?" If your design for stations does this, you will get credit.

### TestServerClosesConnectionOnInvalidCommandType

**Stdin**: None
**Stdout**: None

Checks that the server terminates a client connection if the client sends a
command with an unrecognized command type.