Project 1: Snowcast (F24) :snowflake: - HackMD

<style> .markdown-body h1, .markdown-body h2, .markdown-body h3, .markdown-body h4, .markdown-body h5, .markdown-body h6, .markdown-body { font-family: "Helvetica", "Helvetica Neue", sans-serif; } h3 { border-bottom: 1px solid #ccc; } section { margin-bottom: 2rem; padding: 0em 1em; padding-bottom: 1em; border-radius: 4px; background-color: #f7f7f7; border: 1px solid #ccc; } center { text-align: center; } .alert details { background-color: transparent !important; } code { background-color: rgb(244,244,245) !important; color: #ff0000; } .alert-warning { background-color: #fcf8e3 !important; color: #8a6d3b !important; } .alert-success { background-color: #dff0d8 !important; color: #3c763d !important; } summary { font-weight: bolder; } summary:hover { text-decoration: underline; } </style> **[« Back to the main CSCI1680 website](https://brown-csci1680.github.io)** # Project 1: Snowcast (F24) :snowflake: **Milestone due: Monday, September 16 at 11:59pm EDT** **Final submission due: Thursday, September 26 at 11:59PM EDT** --- ## Introduction In this project, you will implement an Internet radio station. This assignment is designed to introduce you to socket programming, get you used to thinking about network protocols, and re-familiarize you with systems programming and building concurrent applications. You will develop and submit your work using Github Classroom, which will create a repository you can use to develop your code. ## Overview ### Internet radio What's an Internet radio station, anyway? Internet radio is a somewhat older method of listening to streaming music over the network, similar to an AM/FM radio station you might use use in person. Basically, a server is continuously playing music on a discrete number of channels (or "stations"), and clients can join any station whenever they want and listen in. Unlike modern, on-demand streaming audio platforms (Spotify, Youtube, podcasts, etc.), **the server is always sending out the same data to all connected clients at the same time**, so everyone should always be listening to (roughly) the same thing--for a modern equivalent, think of a livestream on a platform like Twitch, but only for audio. ### Our version: Snowcast In Snowcast, you'll build a simplified version of an Internet radio station that includes a server and client. The server is responsible for keeping track of subscribed clients and sending them streaming data at a precise rate (16KiB/s). Clients provide the user's interface and are responsible for joining/switching/leaving stations, and "playing" the music. In our version, we don't need to be concerned with streaming *music* and listening to it. Since the goal of this project is get you familiar with network protocols and implementing them, we'll just think about streaming arbitrary data to clients, which will write the data they receive to `stdout`. To check your work, we'll simply test that your implementation streams the correct data at a rate of 16KiB/s. (In other words, you don't need to worry about anything related to processing/encoding actual audio data, which would actually be a challenging problem!) ## What you will build: the big picture You will implement Snowcast by building three programs: one for the server, and two programs that make up the client. To make these programs communicate, you'll implement two different protocols (one TCP, one UDP) that handle different parts of the application. Here's an overview of how these components will interact (with details below): ![](https://hackmd.io/_uploads/SyqxoZCAh.png) The most important parts are: - **The server** (`snowcast_server`) is responsible for reading a set of "song" files. Each song becomes one "station". Clients connect to the server and select which station they want to hear. After a client selects a station, the server begins sending it song data at a rate of 16KiB/s. The server uses two different protocols to communicate with the client: a **control protocol** to handle selecting stations and metadata, and a **song data protocol** for sending the actual song data. - The client is composed of two programs. All clients run both programs at the same time: - **The control client** (`snowcast_control`): Communicates with the server using the control protocol. This program takes in input from the user to set the station, and receives announcements from the server. - **The listener client** (`snowcast_listener`): Receives song data from the server using the data protocol and just writes it to `stdout` (making it easy to check/measure with another program). ### Essential resource: the spec (skim now, come back to it later) We've defined the details of the Snowcast protocol in a separate document, the **[Snowcast Specification](https://hackmd.io/@csci1680/snowcast-spec)**. As part of this assignment, we want you to learn about how network protocols are specified and what it's like to implement them, so we've defined Snowcast similar to a real specification to give you practice with reading them. You will build your client and server according to our specifications, which cover how the control and data protocols work, as well as how to run and interact with each program. A big part of networking is building systems that interoperate well with other systems that "speak" the same protocol--you'll be graded based on how well you conform to the specification. :::success **How to use the spec**: The specification contains all of the fine details on how the Snowcast protocol should be implemented. If this is your first time reading the handout, feel free to skim over it now, but you should follow the tasks in the warmup (described next) to actually get started. Once you're acquainted with the tools and the basics of socket programming, return to the spec for the details! ::: ## How to get started A big part of building network systems, and systems programming in general, is learning how to design programs to build a larger system. In this project, and future projects, you will build your work from the ground up (ie, no stencil code) to give you the opportunity to design and structure your own program. While this may seem like a lot at first, we have structured the project (and the course) to help with this process and give you feedback. Before the final deadline, you will submit two things that serve as a "milestone" for this project: - **[The snowcast warmup](https://hackmd.io/@csci1680/snowcast-setup-guide)** will provide a starting point for socket programming and an introduction to debugging with Wireshark. **Look here for instructions on how to clone the assignment, and a link to the stencil code.** - In addition, you will submit a **[design document](#Design-Document-—-15)** that describes a plan for the rest of your implementation The milestone portion of this project is due **Monday, September 16 by 11:59pm EDT**. For more details, see [here](#Milestone---20). We also have a lot of our resources to help you get started. In Lectures 2--3, we will introduce socket programming in detail by building some programs during class. From there, please see the **[The warmup and Implementation guide](https://hackmd.io/@csci1680/snowcast-setup-guide)** for helpful tutorials and implementation-level details (like how to run the tests). The remainder of this document is structured as follows: - **[Assignment details](#Assignment-details)**: logistics on how you'll implement Snowcast and how to submit your work - **[Grading](#Grading)**: How we'll grade it - **[Specifications](#Snowcast-Specifications)** (**skim now, return as you need them**): The full details on our requirements for how the protocols work and how we expect you should implement the server and client ## Assignment details ### Languages **You may implement Snowcast in Go, C, C++, or Rust**. If you are unsure, we recommend using Go, even if it is new to you, as class examples this year will use it. We have curated a list of resources for each language [here](https://brown-csci1680.github.io/documents/). **If you want to use Rust**, note that only some members of the course staff can provide language support (though we are all happy to discuss conceptual questions!). For details, see the [course staff list](https://brown-csci1680.github.io/staff/). You may implement this project in C, C++, Go, or Rust. Generally, you should be working with your language's socket API, eg. `net` in Go, the Berkeley socket API from `sys/socket.h` in C/C++, or `std::net` in Rust. You may *not* use RPC libraries or libraries for mashalling/unmarshalling structures to/from bytes such as protobuf. If you want to use any other libraries related to networking or byte-packing, please ask on Edstem first. <details><summary>Read this if you're using C</summary> If you are using C, we provide a small library that with a linked list and hash table implementation in [this repository](https://github.com/brown-csci1680/c-utils). You do not need to use this, but it is available if you want--feel free to just copy the code directly into your repository. For examples on how to use the library, see the code demos in the `examples` directory. </details> ### Testing We've provided multiple ways to help test your code (see links for details on how to run each one): - **[Reference implementation](https://hackmd.io/@csci1680/snowcast-setup-guide#Final-step-running-the-reference-version) and testing manually in Wireshark**: We've included completed versions of all Snowcast programs that implement all of the requirements in your stencil repo. You can use this to get a feel for how the program should work. - **[Built-in tester](https://hackmd.io/@csci1680/snowcast-setup-guide#Running-the-built-in-tester)**: we've also provided a test suite you can use to check your server and control client's functionality. We recommend trying this *after* you get things working manually. :::danger **Warning**: When using the built-in tester, please take care to make sure your programs exactly follow [our specifications](https://hackmd.io/@csci1680/snowcast-spec#Implementation-specifications) on command line arguments and printing to `stdout`. Extra print/logging statements can break the tests if you write these to `stdout`. For more details and recommendations on how to make sure your code follows the requirements, see [here](https://hackmd.io/@csci1680/snowcast-setup-guide#If-you-have-failing-tests). :::  ### Submitting your work  When you are ready to submit, please push your code to GitHub and upload it to the appropriate Gradescope submission using the "GitHub" method. If the GitHub method doesn't work, you may also upload a zip file of your work instead, but we highly recommend submitting via GitHub. #### Autograder setup: how we will build your code Regardless of what language you use, please include a Makefile such that running `make` will compile all of your programs and place them in the *base* directory, and that `make clean` will clean up any compiled programs and object files. Specifically, `make` should produce three executables, `snowcast_server`, `snowcast_control`, and `snowcast_listener`, in the main directory of your repository. If you are using Go or Rust and your program requires external dependencies, make sure that `make` also installs these. If you are using C/C++ and are considering external dependencies, please contact the course staff first to check if we can accommodate them. If you have any questions about how your project should be packaged for grading, please make a post on Edstem. ## Grading ### Milestone - 20% To make sure you're on the right track, you will submit an initial "warmup" part of your implementation, and write a brief design document to sketch your plan for your larger design. The milestone components are due on **Monday, September 16, by 11:59pm EDT.** You should submit your work for each part by uploading your repository to the **Snowcast (Milestone)** submission on Gradescope. :::warning **Note**: We're still setting up our Gradescope submission for this--we'll post an announcement on Edstem when it's ready. Thanks for your patience! ::: :::danger ~~**What do I need to get an override?** In order to receive an override, you need to submit both parts of the milestone (your warmup implementation and design document, described below) before the deadline. Don't worry, though, your implementation does not need to be perfect, but you do need a complete submission that includes all components.~~ (**Update 9/13**: Overrides for all currently-waiting students have been sent; please see EdStem for details.) If you're having trouble working on the milestone, please come see us in office hours and we can help point you in the right direction. If you're concerned, please also feel free to talk to Nick. It's best if you reach out to us sooner rather than later! ::: #### Milestone part 1: Warmup [The warmup](https://hackmd.io/@csci1680/snowcast-setup-guide) will help provide an introduction to socket programming to help kickoff your implementation and ensure your programs can correctly interface with our autograder environment. Please see the warmup document for details on what you need to implement and how to submit your work for this part. Your submission will include a start of your implementation and a Wireshark screenshot to demonstrate you can run the tools. #### Milestone part 2: Design Document In addition to the warmup, you will also submit a design document describing your planned server design. Your design should answer at least the following questions: 1. How will your server use threads/goroutines? In other words, when are they created, and for what purpose? 2. What data does your server need to store for each *client*? 3. What data does your server need to store for each *station*? 4. What synchronization primitives (channels, mutexes, condition variables, etc.) will you need to have threads/goroutines communicate with each other and protect shared data? 5. **What happens on the server (in terms of the threads/data structures/etc you described in (1)-(4)) when a client changes stations?** Thinking about this should help you answer the above questions. If you're having trouble with the design, please come to our hours, or post on Ed. If you would like to talk in real-time (in person or remote) and are unable to attend any scheduled TA hours, reach out and we'll do our best to accommodate. When you submit your milestone work, please submit your design document as a plain text file (`milestone.md` or `milestone.txt`) or PDF (`milestone.pdf`) in the top-level directory of your repository. ### Implementation — 60% Most of your grade will be based on how well your program conforms to the protocol and specification, including how well it interacts with our automated tests and the reference implementations. Our starting point for our implementation-grading process is the results of your autograder tests, but we will also review your code and test output to verify that your work meets the requirements for each test and may apply partial credit as applicable. Automated testing is designed to help you test your code and check for common issues, in addition to reducing some of the grading workload. However, **please note that your grade is always determined by a human, not a program**--this is especially important for this course, since testing networked applications is actually quite difficult! Our setup guide provides more detail about how our autograder tests work, and ways to test your code manually--we urge you to do this to help you understand the results of the automated tests and autograder, and to help you narrow down any issues. **In particular, if you have failing tests but are convinced your implementation is working** (via your own manual testing and analysis of your design with respect to the requirements), please explain this in your readme and we'll consider it while grading. For our later projects, we will do most of our grading interactively, ie, by meeting with you and talking about your work (more on this later). ### Readme and Design — 20% Please include a README file with your program named `README.md` or `README.txt`. In your readme, you should describe your major design decisions and how your implementation meets the design requirements, such as how your server is structured in terms of concurrency, how it handles announces, how it handles multiple clients, how you manage streaming, etc. If you know any bugs or limitations in your design that do not meet the requirements, please list them and suggested how you *would* correct them if you had the time. We will deduct fewer points for any bugs or design issues that you have documented. ### Extra Credit — Up to 5% The protocol we've defined is quite limited. We will consider any addition to the protocol for extra credit. You can also augment the server or client in a non-trivial way. Here are some ideas: <details><summary><b>Click to expand</b></summary> - Add a command which requests a listing of what each of the stations is currently playing (it is acceptable for the TA binary to respond to this with `InvalidCommand`). - Add support for multiple songs per station. - Add a command to retrieve a station's playlist (maybe the next 5 items or so). - Add support for adding and removing stations while the server is running through the command line interface. If you remove a station while a client is listening to it, send a `StationShutdown` packet, or something along those lines, to inform them. If a new station is added, you could send a `NewStation` packet to all currently connected clients to inform them. </details> Feel free to ask what we think about your addition. When implementing extra credit, your implementation MUST still adhere to the protocol interoperate with the reference implementation. It is acceptable for the reference implementation to return an `InvalidCommand` in response to your added commands/replies, but the existing commands/replies must still work as specified by the protocol. # Frequently Asked Questions See [here](https://hackmd.io/@csci1680/snowcast-setup-guide#Frequently-Asked-Questions).