Try   HackMD

Snowcast Specification

The following specifications describe the Snowcast protocol. We've written this specification as a separate document from our handout to give you practice reading specifications that define how an application behave, which is how real-world networking protocols are defined! In later projects, you'll read and interpret real network specification, so this is an important skill for this course and beyond!

The specification for Snowcast has two components:

  • The Implementation specification describes the functional requirements for your client and server binaries (eg. command line arguments, how to handle user input, etc.)
  • The Protocol specification describes the requirements for the control and song data protocols (ie, what messages should be sent and how they should be formatted)

What to expect: If you're reading this for the first time, please skim over this part get a sense of: 1) what you need to build, and 2) the kind of things we are specifying before you start writing code.

You don't need to internalize all the details yet, we just want you to get a sense of the structure of the specifications so you know how to go back to them.

We give you broad freedom on how you design your program and write your code, but we need to be very specific on what you're implementing. This is important in order to make sure that your program inter-operates with other implementations (a critical part of building network programs!), and to make sure that your program works with our reference implementation and the automated tests.

How we specify protocols

We've written these specifications in the style of real-world networking standards (based on the IETF format from RFC2119). In later projects, you'll be reading real standards using this language, so we want you to practice with it now.

A key part of specifying network standards is identifying which features are required and which are optional and perhaps helpful or recommended. To make this clear, the standards use some specific keywords for this. Here are ours:

  • MUST: This is a required feature to make the protocol work and interoperate with our reference version and our autograder. For this assignment, you'll lose points if you don't have it. (eg. "your server MUST use a certain format for command-line arguments")
  • MUST NOT: This is the opposite of MUST. If you do this thing, you will lose points. (ie. Your program MUST NOT segfault during normal operation.)
  • SHOULD: This feature is probably a good idea. We use this if we think that doing something a certain way will make your life easier, but we don't require it.
  • SHOULD NOT: Opposite of SHOULD. We don't recommend this–doing something this way may make things harder.
  • MAY: This feature is optional. You can do it if you want, but it's not required.

We're not shouting!: The convention is to write these keywords in CAPS to signal that they carry the meanings listed here. Please don't interpret this as us using a negative tone–it's simply the convention, from a time when there weren't other good ways to format text.

Implementation specifications

This section specifies how the server (snowcast_server) and client (snowcast_control and snowcast_listener) should operate.

Client Specification

You will write two separate programs that act as the "client".

UDP Client (snowcast_listener)

The listener client receives song data as UDP packets and writes the data to stdout. The executable must be called snowcast_listener. Its command line must be:

./snowcast_listener <udp_port>

The UDP client must print all data received on the specified UDP port to stdout.

TCP Client (snowcast_control)

The TCP client handles the control data. The executable must be called snowcast_control. Its command line must be:

./snowcast_control <server_name> <server_port> <udp_port>

<server_name> represents the IP address (e.g. 128.148.38.158) or hostname (e.g. localhost, cslab6c) to which the control client should connect, and <serverport> is the port to connect to. <udpport> is the port on which the local UDP client is watching for song data.

The control client MUST connect to the server and communicate with it according to the protocol.

After the handshake, it should show a prompt and wait for input from stdin.

  • If the user types in q followed by a newline, the client should quit.
  • If the user types in a number followed by a newline, the control should send a SetStation command with the user-provided station number, unless that station number is outside the range given by the server; you may choose how to handle this situation.

If the client gets an invalid reply from the server (one whose replyType is not 2, 3, or 4), then it MUST close the connection and exit.

The client MUST print whatever information the server sends it (i.e. the numStations in a Welcome) in real time. The printed lines MUST adhere to the following formats:

  • In response to a Welcome reply, the client MUST print to stdout the following message (followed by a newline):
Welcome to Snowcast! The server has N stations

where N is the value in the Welcome reply.

  • In response to an Announce reply, the client MUST print to stdout the following message (followed by a newline):
New song announced: <song name>

where <song name> is the value in the Announce reply.

Handling user input

Your client MUST attempt to sanitize user input to prevent the user from sending a message not allowed by the protocol. You can decide what happens if the user does something wrong–eg. you MAY exit the program, or you MAY display error. However, you MUST NOT allow user input to send any packets that would otherwise violate the protocol specification.

Server Specification

The server executable must be called snowcast_server. Its command line MUST be

./snowcast_server <tcpport> <file0> [file 1] [file 2] ...

That is, a port number on which the server will listen, followed by a list of files; the list MUST be at least one file long. Each station contains only one song. Station 0 should play file0, Station 1 should play file1, etc. Each station MUST loop its song indefinitely. Additionally, the multiple stations can play the same song. Station 0 and Station 1 can both play file0 if desired.

When the server starts, it MUST begin listening for connections. When a client connects, it MUST interact with it as specified by the protocol. Additionally, it MUST send an Announce whenever a song repeats.

Streaming Music

To stream music, you should send data to each client at a fixed rate, rather than sending the song data as fast as possible. To do this, assume that all mp3 files are 128kbps, i.e. the server MUST send data at a rate of 128Kibps, or 16KiB/s.[1]

If multiple clients are connected to one station, they MUST all be listening to the same part of the song, even if they connected at different times. If no clients are connected to a station, the current position in the song MUST still progress, without sending any data. In other words, the radio doesn't stop when no one is listening.

Finally, the server MUST NOT read the entire song file into memory at once. It MAY read the entire file in for some sizes, but there must be a size beyond which it will read data in chunks. For example, you should be able to have /dev/urandom (which is an infinite stream of random bytes) as a station.

Server CLI

When the server receives a command, it MUST print out a log of the command receives and any replies it sends to stdout.

It will also have a simple command-line interface:

  • p followed by a newline MUST cause the server to print to stdout a list of its stations along with the listeners that are connected to each one.
  • p <filename> (eg. p stations.txt) followed by a newline MUST cause the server to write the list of stations to the specified file. Any existing contents of the file MUST be overwritten.
  • q followed by a newline MUST cause the server to close all connections and exit gracefully. An exit will be considered graceful if the server process returns 0.

The printed stations MUST follow the following format:

id,station_name[,connection1,connection2,...]

where connectionN has the form <IPAddr>:<UDPPort>.

Example output

If the server is run with

./snowcast_server <port> mp3/Beethoven-SymphonyNo5.mp3 ../DukeEllington-Caravan.mp3 VanillaIce-IceIceBaby.mp3

and the server stations have the following client connections (i.e. listeners):

  • Station 0 playing mp3/Beethoven-SymphonyNo5.mp3 with listeners 127.0.0.1:9000 and 127.0.0.1:9001
  • Station 1 playing ../DukeEllington-Caravan.mp3 with no listeners
  • Station 2 playing VanillaIce-IceIceBaby.mp3 with listeners 127.0.0.1:10000 and 127.0.0.1:10001

the output should be:

0,mp3/Beethoven-SymphonyNo5.mp3,127.0.0.1:9000,127.0.0.1:9001
1,../DukeEllington-Caravan.mp3
2,VanillaIce-IceIceBaby.mp3,127.0.0.1:10000,127.0.0.1:10001

Design requirements

Part of this project is thinking about how to build a robust server. As you implement your server, you should pay attention to the following design requirements:

  • The server MUST support multiple clients simultaneously.
  • There MUST be no hard-coded limit to the number of stations your server can support or to the number of clients connected to a station.
  • Invalid control messages received from clients should be handled by sending an InvalidCommand message (see Invalid Conditions for the specification).
  • In other words, in the event of an error handling a client, the server MUST gracefully handle the error by terminating the connection to that client, rather than crashing the whole server.
  • When your server exits (ie, with the q command, or on some unrecoverable error), your server MUST close any open TCP sockets. You do not need any special handling for UDP sockets.

Protocol Specifications

There are two kinds of data being sent between the server and the client that use two different protocols:

  • The control protocol is used by the client to issue commands (e.g. specifying which station to listen to), and the server uses it to reply to commands (e.g. giving the client song information). This protocol uses TCP.
  • The song data protocol is used by the server to send data to the listener client. This protocol uses UDP.

Control protocol

The control protocol is a TCP protocol between the control client and the server.

The server will typically listen for connections on port 16800 (though this is configurable on server startup).

Client to Server Commands

The client sends the server messages called commands. There are two commands the client can send the server, in the following format:

Hello:
    uint8 commandType = 0;
    uint16 udpPort;

SetStation:
    uint8 commandType = 1;
    uint16 stationNumber;

A uint8 is an unsigned 8-bit integer; a uint16 is an unsigned 16-bit integer. Your programs MUST use network byte order. So, to send a Hello command, your client would send exactly three bytes to the server: one for the command type and two for the port.

The Hello command MUST be sent as soon as the client connects to the server. The field udpPort gives the port number on which the client's listener program (ie snowcast_listener is listening. The server will use this when sending song data.

The SetStation command is sent to pick an initial station or to change stations. stationNumber identifies the station.

Server to Client Replies

There are three possible messages called replies the server may send to the client:

Welcome:
    uint8 replyType = 2;
    uint16 numStations;

Announce:
    uint8 replyType = 3;
    uint8 songnameSize;
    char  songname[songnameSize];

InvalidCommand:
    uint8 replyType = 4;
    uint8 replyStringSize;
    char  replyString[replyStringSize];

A Welcome reply MUST be sent in response to a Hello command. Stations are numbered sequentially from 0, so a numStations of 30 means 0 through 29 are valid. A Hello command, followed by a Welcome reply, is called a handshake.

An Announce reply MUST be sent on two occasions: after a client sends a SetStation command, or when the station restarts playing from the beginning of the file. songnameSize represents the length, in bytes, of the filename, while songname contains the filename itself. The string must be formatted in ASCII and MUST NOT be null-terminated. So, to announce a song called Gimme Shelter, your client must send the replyType byte, followed by a byte whose value is 13, followed by the 13 bytes whose values are the ASCII character values of Gimme Shelter.

Invalid Conditions

Since neither the client nor the server may assume that the program with which it is communicating is compliant with this specification, they must both be able to behave correctly when the protocol is used incorrectly.

On the Server

On the server side, an InvalidCommand reply is sent in response to any invalid command. replyString should contain a brief error message explaining what went wrong.

Where possible, you should provide helpful strings stating the reason for failure. If a SetStation command was sent with 1729 as the stationNumber, a bad replyString is Error, closing connection., while a good one is Station 1729 does not exist. We won't test this, but it will be useful for you when debugging.

To simplify the protocol, whenever the server receives an invalid command, it MUST reply with an InvalidCommand and then close the connection to the client that sent it. Invalid commands happen in the following situations:

Click to view
  • SetStation
    • The station given does not exist.
    • The command was sent by a client before they've sent a Hello command. Clients must send a Hello command before sending any other commands.
    • If the command was sent by a client before the server responded to a previous SetStation from the client with an Announce reply, then your server MAY reply to this with an InvalidCommand. This means your clients should be careful and wait for an Announce before sending another SetStation, but your server can be lax about this.
  • Hello
    • More than one Hello command was sent by the same client. Only one should be sent per client, at the very beginning of their connection to the server.
  • An unknown command was sent (one whose commandType was not 0 or 1).

On the Client

On the client side, invalid uses of the protocol MUST be handled simply by disconnecting. This happens in the following situations:

Click to view
  • Announce
    • The server sends an Announce before the client has sent a SetStation
  • Welcome
    • The server sends a Welcome before the client has sent a Hello
    • The server sends more than one Welcome at any point.
  • InvalidCommand
    • The server sends an InvalidCommand. This may indicate that the client itself is incorrect, or the server may have sent it out of error. In either case, the client MUST disconnect.
  • An unknown response was sent (one whose replyType was not 2, 3, or 4).

Timeouts

Sometimes, a host you’re connected to may misbehave in such a way that it simply doesn’t send any data. In such cases, it’s imperative that you are able to detect such errors and reclaim the resources consumed by that connection. In light of this, there are a few cases in which you will be required to time out a connection if data isn’t received after a certain amount of time.

These timeouts should be treated as errors just like any other I/O error you might have, and handled accordingly. In particular, on the server, a timeout must only affect the connection in question, and not unrelated connections. If the client encounters an error, it should close its connection to the server and exit.

You are required to handle three timeout conditions:

Click to view
  1. If a client or server receives some of the bytes of a message within 100ms, if it does not receive all of the remaining bytes of the message within the next 100 milliseconds, the client or server MUST time out that connection. See the FAQs section for details.
    • For instance, if the server receives one byte of a SetStation command, it MUST receive the remaining bytes within the next 100ms, or else close the connection.
  2. If a client connects to a server, and the server does not receive a Hello command within 100ms, the server MUST time out that connection.
    • After the first byte of the message is received, the remaining bytes MUST arrive within 100ms of the first byte (similar to case 1).
  3. If a client connects to a server and sends a Hello command, and the server does not respond with a Welcome reply within 100ms, the client MUST time out that connection. If the client receives some, but not all, bytes of the Hello message, the timeout is reset.

There is another condition that MAY trigger a timeout. Handling this is optional:

  1. If a client has completed a handshake with a server, and has sent a SetStation command, and the server does not respond with an Announce reply within some preset amount of time, the client MAY time out that connection. If this happens, the timeout MUST NOT be less than 100 milliseconds.

A timeout MUST NOT occur in any circumstance not listed above.


Note: while we specify precise times for these timeouts, we don’t expect your program to behave with absolute precision. Processing delays and constraints of running in a multi-threaded environment make exact guarantees impossible.

Song data protocol

The song data protocol is much less structured than the control protocol. When a client is subscribed to a station, the server MUST send song data to the the UDP port specified in the client's Hello message.

In our protocol, we assume that the control client and listener client run on the same machine and have the same IP address. Thus, if a control client connects from IP 1.2.3.4 and sends a Hello listing its UDP port as 5445, song data packets should be sent to 1.2.3.4:5445.

Each individual song data packet MUST NOT exceed 1500 bytes of song data. This is a standard limit for a size of single packets forwarded over the Internet. Otherwise, you may choose any packet size that best fits how you stream data. Each packet SHOULD be roughly the same size, though it's okay if this isn't always the case (for example, when the song loops).

Frequently Asked Questions

This section has been moved here.

  1. See the Conventions for more information on data sizes. ↩︎

Last changed by 
 
CSCI1680
0
1612
Published on HackMD