« Back to the main CSCI1680 website
Milestone due: September 19, 2023 at 11:59pm EDT
Final submission due: September 25, 2023 at 11:59PM EDT
In this project, you will implement an Internet radio station. This assignment is designed to introduce you to socket programming, get you used to thinking about network protocols, and re-familiarize you with systems programming and building concurrent applications.
You will develop and submit your work using Github Classroom, which will create a repository you can use to develop your code.
What's an Internet radio station, anyway? Internet radio is a somewhat older method of listening to streaming music over the network, similar to an AM/FM radio station you might use use in person. Basically, a server is continuously playing music on a discrete number of channels (or "stations"), and clients can join any station whenever they want and listen in.
Unlike modern, on-demand streaming audio platforms (Spotify, Youtube, podcasts, etc.), the server is always sending out the same data to all connected clients at the same time–so everyone should always be listening to (roughly) the same thing. If you've ever used Pandora or iHeartRadio, these are probably the closest modern analogs.
In Snowcast, you'll build a simplified version of an Internet radio station that includes a server and client. The server is responsible for keeping track of subscribed clients and sending them streaming data at a precise rate (16KiB/s). Clients provide the user's interface and are responsible for joining/switching/leaving stations, and "playing" the music.
In our version, we don't need to be concerned with streaming music and listening to it. Since the goal of this project is get you familiar with network protocols and implementing them, we'll just think about streaming arbitrary data to clients, which will write the data they receive to stdout
. To check your work, we'll simply test that your implementation streams the correct data at a rate of 16KiB/s. (In other words, you don't need to worry about anything related to processing/encoding actual audio data, which would actually be a challenging problem!)
You will implement Snowcast by building three programs: one for the server, and two programs that make up the client. To make these programs communicate, you'll implement two different protocols (one TCP, one UDP) that handle different parts of the application. Here's an overview of how these components will interact (with details below):
The most important parts are:
snowcast_server
) is responsible for reading a set of "song" files. Each song becomes
one "station". Clients connect to the server and select which station they want to hear. After a client selects a station, the server begins sending it song data at a rate of 16KiB/s. The server uses two different protocols to communicate with the client: a control protocol to handle selecting stations and metadata, and a song data protocol for sending the actual song data.snowcast_control
): Communicates with the server using the control protocol. This program takes in input from the user to set the station, and receives announcements from the server.snowcast_listener
): Receives song data from the server using the data protocol and just writes it to stdout
(making it easy to check/measure with another program).As part of this assignment, we want you to learn about how network protocols are specified and what it's like to implement them. You will build your client and server according to our specifications–-these cover how the control and data protocols work, as well as how to run and interact with each program. A big part of networking is building systems that interoperate well with other systems that "speak" the same protocol–you'll be graded based on how well you conform to the specification.
A big part of building network systems, and systems programming in general, is learning how to design programs to build a larger system. In this project, and future projects, you will build your work from the ground up (ie, no stencil code) to give you the opportunity to design and structure your own program.
While this may seem like a lot at first, we have structured the project (and the course) to help with this process and give you feedback. Before the final deadline, you will submit two things that serve as a "milestone" for this project:
The milestone portion of this project is due September 19 by 11:59pm EDT. For more details, see here.
We also have a lot of our resources to help you get started. In Lectures 2–3, we will introduce socket programming in detail by building some programs during class. From there, please see the The warmup and Implementation guide for helpful tutorials and implementation-level details (like how to run the tests).
The remainder of this document is structured as follows:
You may implement Snowcast in Go, C, C++, or Rust. If you are unsure, we recommend using Go, even if it is new to you, as class examples this year will use it. We have curated a list of resources for each language here.
If you want to use Rust, note that only some members of the course staff can provide language support (though we are all happy to discuss conceptual questions!). For details, see the course staff list.
You may implement this project in C, C++, Go, or Rust. Generally, you should be working with your language's socket API, eg. the Berkeley socket API from sys/socket.h
in C/C++, net
in Go, or std::net
in Rust. You may not use RPC libraries or libraries for mashalling/unmarshalling structures to/from bytes such as protobuf. If you want to use any other libraries related to networking or byte-packing, please ask on Edstem first.
If you are using C, we provide a small library that with a linked list and hash table implementation in this repository. You do not need to use this, but it is available if you want–feel free to just copy the code directly into your repository. For examples on how to use the library, see the code demos in the
examples
directory.
We've provided multiple ways to help test your code (see links for details on how to run each one):
Warning: When using the built-in tester, please take care to make sure your programs exactly follow our specifications on command line arguments and printing to stdout
. Extra print/logging statements can break the tests if you write these to stdout
.
For more details and recommendations on how to make sure your code follows the requirements, see here.
When you are ready to submit, please push your code to GitHub and upload it to the appropriate Gradescope submission using the "GitHub" method. If the GitHub method doesn't work, you may also upload a zip file of your work instead, but we highly recommend submitting via GitHub.
Regardless of what language you use, please include a Makefile such that running make
will compile all of your programs and place them in the base directory, and that make clean
will clean up any compiled programs and object files.
Specifically, make
should produce three executables, snowcast_server
, snowcast_control
, and snowcast_listener
, in the main directory of your repository.
If you are using Go or Rust and your program requires external dependencies, make sure that make
also installs these. If you are using C/C++ and are considering external dependencies, please contact the course staff first to check if we can accommodate them.
If you have any questions about how your project should be packaged for grading, please make a post on Edstem.
To make sure you're on the right track, you will submit two things before the final deadline that serve as a "milestone" point. Both components are due on September 20, by 11:59pm EDT.
You should submit your work for each part by uploading your repository to the Snowast (Milestone) submission on Gradescope.
The warmup will help provide an introduction to socket programming to help kickoff your implementation and ensure your programs can correctly interface with our autograder environment.
Please see the warmup document for details on what you need to implement and how to submit your work for this part.
In addition to the warmup, you will also submit a design document describing your planned server design. Your design should answer at least the following questions:
If you're having trouble with the design, please come to our hours, or post on Ed. If you would like to talk in real-time (in person or remote) and are unable to attend any scheduled TA hours, please let us know and we will do our best to accommodate you.
When you submit your milestone work, please submit your design document as a plain text file (milestone.md
or milestone.txt
) or PDF (milestone.pdf
) in the top-level directory of your repository.
Most of your grade will be based on how well your program conforms to the protocol and specification, including how well it interacts with our automated tests and the reference implementations. We'll release more details on weightings for each autograder test shortly.
When grading, we will review your code and test output to verify that your work meets the requirements for each test and may apply partial credit as applicable.
For our later projects, we will do most of our grading interactively, ie, by meeting with you and talking about your work (more on this later). Automated testing is designed to help you test your code out before the deadline and check for common issues, in addition to reducing some of the grading workload.
Please include a README file with your program named README.md
or README.txt
. In your readme, you should describe your major design decisions and how your implementation meets the design requirements, such as how your server is structured in terms of concurrency, how it handles announces, how it handles multiple clients, etc.
If you know any bugs or limitations in your design that do not meet the requirements, please list them and suggested how you would correct them if you had the time. We will deduct fewer points for any bugs or design issues that you have documented.
The protocol we've defined is quite limited. We will consider any addition to the protocol for extra credit. You can also augment the server or client in a non-trivial way. Here are some ideas:
InvalidCommand
).StationShutdown
packet, or something along those lines, to inform them. If a new station is added, you could send a NewStation
packet to all currently connected clients to inform them.Feel free to ask what we think about your addition.
When implementing extra credit, your implementation MUST still adhere to the protocol interoperate with the reference implementation. It is acceptable for the reference implementation to return an InvalidCommand
in response to your added commands/replies, but the existing commands/replies must still work as specified by the protocol.
The following specifications describe our requirements for your implementation. There are two parts:
What to expect: If you're reading this for the first time, please skim over this part get a sense of: 1) what you need to build, and 2) the kind of things we are specifying before you start writing code.
You don't need to internalize all the details yet, we just want you to get a sense of the structure of the specifications so you know how to go back to them.
We give you broad freedom on how you design your program and write your code, but we need to be very specific on what you're implementing. This is important in order to make sure that your program interoperates with other programs (a critical part of building network programs!) and to make sure that your program works with our autograder.
This section specifies how the server (snowcast_server
) and client (snowcast_control
and snowcast_listener
) should operate.
You will write two separate programs that act as the "client".
snowcast_listener
)The listener client receives song data as UDP packets and writes the data to stdout. The executable must be called snowcast_listener
. Its command line must be:
The UDP client must print all data received on the specified UDP port to stdout
.
snowcast_control
)The TCP client handles the control data. The executable must be called snowcast_control
. Its command line must be:
<server_name>
represents the IP address (e.g. 128.148.38.158
) or hostname (e.g. localhost
, cslab6c
) to which the control client should connect, and <serverport>
is the port to connect to. <udpport>
is the port on which the local UDP client is watching for song data.
The control client MUST connect to the server and communicate with it according to the protocol.
After the handshake, it should show a prompt and wait for input from stdin.
q
followed by a newline, the client should quit.SetStation
command with the user-provided station number, unless that station number is outside the range given by the server; you may choose how to handle this situation.If the client gets an invalid reply from the server (one whose replyType
is not 2, 3, or 4), then it MUST close the connection and exit.
The client MUST print whatever information the server sends it (i.e. the numStations
in a Welcome
) in real time. The printed lines MUST adhere to the following formats:
Welcome
reply, the client MUST print to stdout
the following:where N
is the value in the Welcome
reply.
Announce
reply, the client MUST print to stdout
the following:where <song name>
is the value in the Announce
reply.
Your client MUST attempt to sanitize user input to prevent the user from sending a message not allowed by the protocol. You can decide what happens if the user does something wrong–eg. you MAY exit the program, or you MAY display error. However, you MUST NOT allow user input to send any packets that would otherwise violate the protocol specification.
The server executable must be called snowcast_server
. Its command line MUST be
That is, a port number on which the server will listen, followed by a list of files; the list MUST be at least one file long. Each station contains only one song. Station 0 should play file0
, Station 1 should play file1
, etc. Each station MUST
loop its song indefinitely. Additionally, the multiple stations can play the same song. Station 0 and Station 1 can both play file0
if desired.
When the server starts, it MUST
begin listening for connections. When a client connects, it MUST interact with it as specified by the protocol. Additionally, it MUST send an Announce
whenever a song repeats.
To stream music, you should send data to each client at a fixed rate, rather than sending the song data as fast as possible. To do this, assume that all mp3 files are 128kbps
, i.e. the server MUST send data at a rate of 128Kibps
, or 16KiB/s
.[1]
If multiple clients are connected to one station, they MUST all be listening to the same part of the song, even if they connected at different times. If no clients are connected to a station, the current position in the song MUST still progress, without sending any data. In other words, the radio doesn't stop when no one is listening.
Finally, the server MUST NOT read the entire song file into memory at once. It MAY read the entire file in for some sizes, but there must be a size beyond which it will read data in chunks. For example, you should be able to have /dev/urandom
(which is an infinite stream of random bytes) as a station.
When the server receives a command, it MUST print out a log of the command receives and any replies it sends to stdout.
It will also have a simple command-line interface:
p
followed by a newline MUST cause the server to print to stdout
a list of its stations along with the listeners that are connected to each one.p <filename>
(eg. p stations.txt
) followed by a newline MUST cause the server to write the list of stations to the specified file. Any existing contents of the file MUST be overwritten.q
followed by a newline MUST cause the server to close all connections and exit gracefully. An exit will be considered graceful if the server process returns 0.The printed stations MUST follow the following format:
where connectionN
has the form <IPAddr>:<UDPPort>
.
If the server is run with
and the server stations have the following client connections (i.e. listeners):
mp3/Beethoven-SymphonyNo5.mp3
with listeners 127.0.0.1:9000
and 127.0.0.1:9001
../DukeEllington-Caravan.mp3
with no listenersVanillaIce-IceIceBaby.mp3
with listeners 127.0.0.1:10000
and 127.0.0.1:10001
the output should be:
Part of this project is thinking about how to build a robust server. As you implement your server, you should pay attention to the following design requirements:
InvalidCommand
message (see Invalid Conditions for the specification).MUST
gracefully handle the error by terminating the connection to that client, rather than crashing the whole server.q
command, or on some unrecoverable error), your server MUST close any open TCP sockets. You do not need any special handling for UDP sockets.There are two kinds of data being sent between the server and the client that use two different protocols:
The following sections describe the control and data protocols the client and server use to communicate. We've written these specifications in the style of real-world networking standards (based on the IETF format from RFC2119). In later projects, you'll be reading real standards using this language, so we want you to practice with it now.
A key part of specifying network standards is identifying which features are required and which are optional and perhaps helpful or recommended. To make this clear, the standards use some specific keywords for this. Here are ours:
MUST
: This is a required feature to make the protocol work and interoperate with our reference version and our autograder. For this assignment, you'll lose points if you don't have it. (eg. "your server MUST
use a certain format for command-line arguments")MUST NOT
: This is the opposite of MUST
. If you do this thing, you will lose points. (ie. Your program MUST NOT
segfault during normal operation.)SHOULD
: This feature is probably a good idea. We use this if we think that doing something a certain way will make your life easier, but we don't require it.SHOULD NOT
: Opposite of SHOULD
. We don't recommend this–doing something this way may make things harder.MAY
: This feature is optional. You can do it if you want, but it's not required.We're not shouting!: The convention is to write these keywords in
CAPS
to signal that they are keywords. Please don't interpret this as us using a negative tone–it's simply the convention, from a time when there wasn't a good way to do text formatting.
The control protocol is a TCP protocol between the control client and the server.
The server will typically listen for connections on port 16800 (though this is configurable on server startup).
The client sends the server messages called commands. There are two commands the client can send the server, in the following format:
A uint8
is an unsigned 8-bit integer; a uint16
is an unsigned 16-bit integer. Your programs MUST
use network byte order. So, to send a Hello
command, your client would send exactly three bytes to the server: one for the command type and two for the port.
The Hello
command MUST
be sent as soon as the client connects to the server. The field udpPort
gives the port number on which the client's listener program (ie snowcast_listener
is listening. The server will use this when sending song data.
The SetStation
command is sent to pick an initial station or to change stations. stationNumber
identifies the station.
There are three possible messages called replies the server may send to the client:
A Welcome
reply MUST
be sent in response to a Hello
command. Stations are numbered sequentially from 0, so a numStations of 30 means 0 through 29 are valid. A Hello
command, followed by a Welcome
reply, is called a handshake.
An Announce
reply MUST be sent on two occasions: after a client sends a SetStation command, or when the station restarts playing from the beginning of the file. songnameSize
represents the length, in bytes, of the filename, while songname
contains the filename itself. The string must be formatted in ASCII and MUST NOT
be null-terminated. So, to announce a song called Gimme Shelter
, your client must send the replyType
byte, followed by a byte whose value is 13, followed by the 13 bytes whose values are the ASCII character values of Gimme Shelter
.
Since neither the client nor the server may assume that the program with which it is communicating is compliant with this specification, they must both be able to behave correctly when the protocol is used incorrectly.
On the server side, an InvalidCommand
reply is sent in response to any invalid command. replyString
should contain a brief error message explaining what went wrong.
Where possible, you should provide helpful strings stating the reason for failure. If a
SetStation
command was sent with1729
as thestationNumber
, a badreplyString
isError, closing connection.
, while a good one isStation 1729 does not exist.
We won't test this, but it will be useful for you when debugging.
To simplify the protocol, whenever the server receives an invalid command, it MUST reply with an InvalidCommand
and then close the connection to the client that sent it. Invalid commands happen in the following situations:
SetStation
Hello
command. Clients must send a Hello
command before sending any other commands.SetStation
from the client with an Announce
reply, then your server MAY reply to this with an InvalidCommand
. This means your clients should be careful and wait for an Announce
before sending another SetStation
, but your server can be lax about this.Hello
Hello
command was sent by the same client. Only one should be sent per client, at the very beginning of their connection to the server.commandType
was not 0 or 1).On the client side, invalid uses of the protocol MUST be handled simply by disconnecting. This happens in the following situations:
Announce
Announce
before the client has sent a SetStation
Welcome
Welcome
before the client has sent a Hello
Welcome
at any point.InvalidCommand
InvalidCommand
. This may indicate that the client itself is incorrect, or the server may have sent it out of error. In either case, the client MUST disconnect.replyType
was not 2, 3, or 4).Sometimes, a host you’re connected to may misbehave in such a way that it simply doesn’t send any data. In such cases, it’s imperative that you are able to detect such errors and reclaim the resources consumed by that connection. In light of this, there are a few cases in which you will be required to time out a connection if data isn’t received after a certain amount of time.
These timeouts should be treated as errors just like any other I/O error you might have, and handled accordingly. In particular, on the server, a timeout must only affect the connection in question, and not unrelated connections. If the client encounters an error, it should close its connection to the server and exit.
You are required to handle three timeout conditions:
SetStation
command, it MUST receive the remaining bytes within the next 100ms, or else close the connection.Hello
command within 100ms, the server MUST time out that connection. If the server receives some, but not all, bytes of the Hello
message, the timeout is reset.
Hello
message within 100ms, the client has another 100ms to send the remaining bytes. If the remaining bytes are not received within 100ms following the first byte, the server must close the connection; the timeout will NOT reset again.Hello
command, and the server does not respond with a Welcome
reply within 100ms, the client MUST time out that connection. If the client receives some, but not all, bytes of the Hello
message, the timeout is reset.There is another condition that MAY
trigger a timeout. Handling this is optional:
SetStation
command, and the server does not respond with an Announce
reply within some preset amount of time, the client MAY time out that connection. If this happens, the timeout MUST NOT be less than 100 milliseconds.A timeout MUST NOT
occur in any circumstance not listed above.
Note: while we specify precise times for these timeouts, we don’t expect your program to behave with absolute precision. Processing delays and constraints of running in a multi-threaded environment make exact guarantees impossible.
The song data protocol is much less structured than the control protocol. When a client is subscribed to a station, the server MUST send song data to the the UDP port specified in the client's Hello
message.
In our protocol, we assume that the control client and listener client run on the same machine and have the same IP address. Thus, if a control client connects from IP 1.2.3.4
and sends a Hello
listing its UDP port as 5445, song data packets should be sent to 1.2.3.4:5445
.
Each individual song data packet MUST NOT exceed 1500 bytes of song data. This is a standard limit for a size of single packets forwarded over the Internet. Otherwise, you may choose any packet size that best fits how you stream data. Each packet SHOULD be roughly the same size, though it's okay if this isn't always the case (for example, when the song loops).
See here.
See the Conventions for more information on data sizes. ↩︎